I stayed flying only in my time off.
I see no need to up my fun sparetime experience to a job, I enjoy it, but some of the things that comes with selecting it as a carreer are not for me.
So if you love both your current job and flying, then that would be my advice.... keep both, and be happy
