Finishing school in '98, we did simultaneous equations aged 12. Quadratics came a bit later - maybe 14 or 15. We did some elementary calculus at 16, but didn't study it properly until A-level.
I studied for my physics S-level using old A-level papers. In my GCSE year, only two kids in my county got straight-As at GCSE. Fast forward a few years and most schools seem to get at least that number of straight-A candidates each year.
Going back to the original question... I don't think you need to be a whizz at maths in order to understand all the concepts for the PPL, but you do need to be able to solve (relatively) simple problems accurately and quickly. I think an intuitive grasp of physics is important too, but it doesn't need to be too formal - I doubt many people work out stall speed equations in their head whilst they're flying - most of us just avoid extremes of bank at low speed, and keep the speed up when it's bumpy.
Speaking personally, I tend to find things simpler when they have an obvious application - as in working out piloting problems. A good motivation like wanting to learn to fly will help a lot too.