I think the worst part is really just the airport experience. Buying a ticket online is quite easy, and if you print a boarding pass at home, it's not so bad. But if you have to arrive early at the airport, far from the city centre, wait in long queues, be humiliated by security, gate changes, etc, etc...
WHBM has it right. If you want to pay civilized first class fairs, flying can still be rather civil. If you want to get from A to B for the cheapest possible cost, it's bring your own caviar I'm afraid.