Who says we are supposed to be flying out of passion?
Where does this bizarre notion comes from that you should only be a pilot if you have a burning passion for all things aviation? I'm doing it because I get (got) paid well over £100k a year while having more time off than I would in the majority of other jobs. I wouldn't change career if I didn't have to because it means a hit to my family's lifestyle, at least in the short term and possibly permanently. It looks like circumstances will dictate a career change is necessary. I won't miss flying, but I will certainly miss the salary and the time off.
Unless you're lucky enough to do something you really enjoy, which doesn't apply to the majority of people, you work to give you the lifestyle you want and nothing more than that.