I didn't justify it, just set a budget for year 1 of what I could afford. I know that the things I regret most are things I didn't do, opportunities I didn't take rather than things I did which perhaps didn't work out. I have wanted to learn fly for 20+ years but never had the combination of time and money until recently, I do wish I had started learning a few years ago even though I knew I couldn't commit much time to it.
If it helps it is much easier to learn when you are younger than say when retired which for many folks is when they finally have the time and the money available.
I dont think you can justify it as such and if you are looking to maybe its the wrong approach. If its something you want to do and you have the money to do it then go for it. You can always stop if needed but just make sure its something you are doing because you want to and enjoy it.
Good luck whatever you decide.