As some wise person has already said, a brief pause will pass unremarked and, indeed, even an actual stammer or stutter is not a problem to ATC. As with every walk of life, ATC has its share of those thus afflicted and they manage to get recruited, qualify and work successfully for years - so why not pilots?
As to that comment about our French friends not obtaining a PPL if they have such a problem - I find it hard to believe. They let them have PPLs when they can't read maps or learn the rules of the air, after all!!!!
Seriously, just relax and stop worrying, it doesn't bother us in ATC.