Baldy,
My Two Cents worth of advice.
Stay at your miserable job but find yourself a really nice Glider, get stuck in doing that kind of flying, enjoy every minute of it, do your PPL and rent an airplane when you want, maybe even get to towing Gliders when you are able to do so.
At age 51, you would be throwing good money after bad to start a career flying Helicopters for a Living.
If you think your current job is stressful and requires you to deal with Idiots, I can assure you launching off into a new career flying helicopters professionally will just be the same but with far less pay and benefits than you have now.
Live your passion but do it in a manner that you fly when you want to, where you want to, with the people you want to do it with, in the weather you want to do it in, be your own Boss while doing it. You will be much happier in the long run.