I've known at-least one very competent FTE with a degree in physics, and I'm sure that there must be others.
You have a father prepared to pay for a degree and flying training? You're a very lucky chap!
Personally if flying was my first passion (personally I have equal passions for engineering and flying so did it otherwise) I'd suggest you look to get on with the flying training then to finish a physics or similarly technical degree through something like the OU when either in a flying job, or hunting for one post-qualification.
But if you are being told that you need a degree to succeed in aviation, get them to go and talk to some experienced professional pilots for an honest opinion.
G