I've had this conversation several times with the 'you must have a few bob to be able to afford flying' mob. I don't earn a lot and I'm always pretty sure that most folk earn more than me so I sit them down and work out what they spend on booze, fags, foreign holidays, new car in the drive etc. It's always far more than I spend a year on flying.
Depends how badly you want to fly.