GG has beaten me to the punch here. You're either fully in it or you may as well walk away. You can hardly earn yourself a decent job in aviation if you're only going to do it part-time.
You've obviously got more to consider than just yourself, but unless you make the required leap of faith and jump in the deep end with both feet, it can't be done.
With your time in the industry off and on you should already know what career prospects are available to you and what you can expect to earn; its not that hard to find that out.
And sure, aviation has its down sides, but looking around at the tough times people are suffering in other industries, show me one where the good times are guaranteed.
To my way of thinking, the current state of the industry is largely irrelevant, that's not where your decision has to come from.
Pandanus Palm.