You can fly in France without transponder ( as mode A only is not valid anyway , you need mode C to enter class D airspace and TMAs.)
Advice :
Just avoid class D and stay below 3000 feet.
But check the new ICAO VFR charts due out on april 1st 07, there is a lot of new Class D TMAs ( i.e airports ). So check your destinations carefully.
And do not forget many small airfields are French only on R/T.