It's about matching your cababilities and desires with that of the aircraft and the more money you are prepared to spend the less compromises you have to make - just like anything else in life really.
1 - You just want to be up there - try powered parachutes doodlebugs or similar. They will get you up there but you ain't going anywhere
2 - Minimalist but want to actually go places - very occasionally. Lower end microlights and PFA Permit aircraft are slow and maybe old or cumbersome - or both. May have weight limitations and lacking in avionics. Strictly fair weather VFR
3 - Cheap to run, good performance but with weather limitations. Most 'hot' BMAA factory built microlights and PFA kits have benefit of Permit affordability but cannot fly at night or in IMC - strictly daytime VFR and factory built micros have 'legal' weight issues. VLA/LSA types in this category soon (we hope!)
4 - 'Fair' weather tourers - C of A aircraft are certified and can be used at night and in IMC. Two seaters such as TommyHawks and C150's can be picked up cheap but on going maintenance and running costs could be alarming
5 - 'Fair' weather family tourers - 4 seat C of A aircraft such as PA-28 and C172 can take you and passengers all over Europe and beyond in IMC conditions and at night. On going costs need serious consideration !!
6 - ALL weather tourers. Turbocharged singles and/or twins that can take you over the clouds and through icing. Normally retractable and very expensive to operate you will be able to take the aircraft almost anywhere at any time.
OK - So this is an overly simplified list but you get the idea that to do (1) is going to cost you minimal money but there are operational limitations with what you can do. (6) on the other hand will cost you mega bucks to own but should rarely compromise you. Of course the other part of this equation is your personal abilities and limitations and matching them to the right aircraft.
Personally speaking I would go for Permit aircraft all day long as I have no desire to fly in clag and will happily walk away from a planned cross country in 'iffy' weather. I fly for the pure fun of it but you may have different idea's which is great because life would be very boring if we were all the same.