It's possible, there are even test pilots around with less hours than that (but usually a lot of technical qualifications).
However, the only way it's likely to happen is if you get a job with a company who are also involved in flight testing, and serve a form of apprenticeship. This does happen, but being in the right place at the right time is very important - there's no standard job to apply for.
You will need the maths in the job, unless you are lucky enough to be paired with a very generous minded FTE (not many of us meet that description). But, there's no rule that says you have to have done it at school; self-teach is often the order of the day.
The other route is that a lot of light aircraft test flying for people like PFA / BMAA / EAA / USUA / AUF is done by professional pilots in their spare time. There's no money in it, but lots of satisfaction. The thing to do is join the appropriate association, and contact their Chief Engineer about their requirements.
BMAA publish quite a nice civil test-pilots syllabus stating what you need to know, at
http://www.bmaa.org/tech2.htm (one of their technical information leaflets) which is worth a look. I doubt other associations will be much different.
G