It's a difficult decision. Most parents would like their kids to have a degree aka fall back option. Trouble is, as you know, degrees aren't cheap. Aviation degrees are very narrow in focus. I would only consider one if there were management/safety modules so if the worst happens within the first 5 or so years into your career, there's a ground based option. (It does happen, diabetes has nuked a few guys I know). To that end, consider an instructor rating etc (sim instructing etc). However, as you move through the career, the degree may or not be of use.
If you're set on being a pilot, then an aviation degree may be not cost effective. I would be researching what employers are requiring for employment. A degree might be there, but an aviation degree mightn't be as attractive (ironically) as a degree in another subject.
The biggest hassle with a degree is that is becomes "out of date" in a few years. So, as others have mentioned, gaining a degree later on in life is achievable. (I just completed a BSc(hons) in Psych -> and I'm going to train up as a CRM teacher).