I know plenty of pilots who have had full careers in the military, trades and started flight training in their late 20s and early 30s. I have found theses pilots are more motivated (generally) and have done very well for themselves.
GA can be very rewarding both financially and for some new life experiences. Accept you may have to travel and do some dirty work for a few years, but if you approach it with a good attitude you'll end up having the time of your life (much like my short career to date).
Bottom line, don't let the old GA grinches on pprune put you off! If you truly have the passion to follow through, go for it!