Your not the only one making that sort of decision. I myself although in Australia was wondering whether to undertake an Aerospace Engineering Degree or an Aviation/Business Degree. I ended up choosing the later, I just wanted to get on with the flying. I know of a couple of people who chose the aerospace path, and will be doing their flying part time; however realistically, its hard to integrate a full time university course and flying up to a profressional level simultaneously. I'm a firm believer that when you do flying it is better to do it intensely like a cadet program or the like.
However having that extra qulification is important as it is something to fall back on; say one day you broke a leg, or got blind in one eye.
I know of someone who got a degree, wasn't aerospace engineering, it was a differing field whilst doing flying at the same time. He was in his 20s and ended up getting into the our flag carriers cadet program.
On the bright side of things, your still young and have plenty of time ahead of you; with motivation and a hard work, you might prevail in the end.
do what you want to do
cheers