Mathematically, having a knowledge (and by no means an extensive knowledge) of basic transposition of formula, trigonometry, converting units and manipulating data can be an advantage to you when you are in groundschool
In terms of physics, you will see its use when you study aerodymamics, engines, structures and electrics. Don't panic, none of its rocket science, but an understanding WILL help.
When I started my training I had a similar worry, however before you commit maybe buy a secondary school maths and physics book or even the Oxford Aviation Training range of Maths and Physics interactive CD ROMs which I have heard are good
I would say as well that brushing up on your mental maths is most essential for life on the flightdeck.