I wouldn't say that it's a silly dream. Just make sure you know what it's actually all about before you make any irreversible decisions. I'd say most of our pilots spend at least 90% of their time flying a desk. It's a great job for some, some prefer to be crew dogs.
I'm a flight test engineer, not a professional pilot, so I can tell you that it'd be much easier for someone with the required flight experience to get a degree and become a test pilot than an engineer gathering enough flight time on his own to become qualified. But I'd be very leary of getting an "Aerospace Technology" degree. Most of those just prepare you to be a maintenance manager. Any type of engineering degree would be preferred by most employers with Aerospace obviously being the most helpful.