Never had any major problems with the maths or physics involved at ATPL level or CPL level for that matter and I have a very limited background in these areas. I understand this is cause for concern at the moment for you because I was exactly the same but trust me when I say that with the right school (BGS worked for me) you shouldn't struggle much at all.
Brush over some basic trigonometry, algebra and learn what the sin, cos and tan buttons on your calculator are for and you shouldn't have too much to worry about. For physics just get hold of an old GCSE medium level book if you're really worried but my opinion is that you'll be fine.
Hope this puts your mind at ease.
VFE.