Getting a degree is never a mistake as long as it is a decent course, from a decent establishment, and you achieve a decent result. Many of my friends who went straight into flying, seriously regret not going to university first. You have your whole life to be a pilot; the degree will simply mean you start three or four years later, and yet it will give you something to fall back on if you lose your medical or whatever.
I do not recommend doing anything aviation related, unless that is the kind of job that you would like to do if you end up without a career as a commercial pilot.
As for worrying about being too specialised in one subject - that is the whole idea of a degree. Personally, I did close to six years of Physics. I do not use those qualifications at all now, and I have probably forgotten more than I knew in the first place. However, I would not miss out any of my university time even if I had the chance to go back and make my decisions all over again with the advantage of hindsight.