Depends on what your Physics syllabus is like. My Physics topics were "Space", "Waves and particles", a very small amount of "material structure" and a small amount of "mechanics".
I did both Physics and Maths A Level and I went on to do Aerospace Engineering. The Maths A level I did covering Pure, Stats and Mech helped me about 10 times as much as my Physics A level did. All the mechanics I did in Physics was covered by Maths course anyway. Most Universities will only ask for Maths or Physics, but the good ones will want an A!
If you're not thinking about University then I'd say choose whichever one you find interesting.