GO for it
Ask your self. . when i'm retired and i look back on my life will i be smiling and happy ? if you stick to IT you will maybe be smiling in your nice house etc, but always casting your eyes sky ward and thinking "What if"... I would go and do it....do IT on the side... I'd rather be left with nothing but be smiling of the 60 years of flying,blue skys, etc. .
but if you only want to fly for fun.....stick with IT and use that money to fly for fun...if you want to do it as a professional... Go for it, don't doubt your self and enjoy the ride....
"Anything is possible if it's everything to you"