A couple of thoughts from one who turned his back on flying for a career and came to "enjoy" years of regret.
Spend a bit of time standing outside the fence watching the activity at your local major airport. Imagine being no longer a part of it. How do you feel?
If you honestly feel you wouldn't miss it, by all means find something else to do. But make sure it's something that will improve your lot in life and not be a backward step.
Otherwise, start focussing on the positives of what you have rather than the negatives.
Remember there are plenty of people out there working to live and not enjoying their work either. It's not just an aviation thing.