Adam,
I took up private flying to get over my fear of sitting in the back of airliners. I have, from time to time, been terrified by turbulence (bumps) in the PA28 I have been learning on, but my instructor is a retired airline pilot and has worked with me to overcome my fears (I don't like steep turns or spiral dives very much either!). I have 40+ hours, done qualifying cross-country and am doing revision for skills test - still a bit nervous on occasions - though it does get better.
One of the most useful pieces of advice my instructor gives me from time to time is, if the aircraft seems to be bouncing around a lot - let go of the controls for a moment; it's amazing how it all suddenly smoothes out.
Remember the causes of turb. Air is constantly moving - this feels more exaggerated in a light aircraft - but if it was going to be too rough for the plane - or your capabilities, your instructor shouldn't let you fly. It will almost always be choppy near the ground - as a result of ground features breaking up the airflow; and thermals are normal - otherwise our gliding friends wouldn't go very far.
Remember your aircraft is built with inherent stability. The only kind of bumps that are really likely to cause control problems are: i. wake turbulence - stay away from helicopter wash and slow moving large aircraft; and, ii. CB - thunderstorms - again, you wouldn't fly in/near them - would you?
Mountains cause their own problems - seek some more specialist instruction if you're going to fly near/over them. If you feel you have been bounced around too much on approach, i.e. so the approach is no longer stable - don't forget to go around - you may get the same bumps the next time - but you'll know where they'll come from and be expecting it.
Getting some aeros in might help too - as Mad Girl suggests - but pick an instructor you feel you can really trust and explain the problem.
Hope this helps
HH