I feel much the same, CSD - when you actually make the move and get out, you see things in a slightly different light.
The reasons most of us probably join in the first place (to fly good military aircraft in challenging roles etc) are legitimate, but get clouded in time by the things that piss us off about the job. I think the main thing that made me jump the first time was not having sufficient control over my own destiny.
Having ducked and weaved to get where I wanted has worked well for me over the years, in that I think I'm happier than I would have been had I stayed in the normal career progression and climbed the tree to my level of incompetence - quite low some would say
- and it's mainly that aspect of taking steps to 'massage' my career to my own liking whilst still hopefully contributing pretty fully to the organisation which I find satisfying now.