Go too fast and the turn rate reduces, go too slow and you stall before you can pull the required g to get a higher rate of turn.
Each aircraft has a "cornering speed" which gives you best turn rate. Best turn rate occurs at the airspeed which just gives you maximum structural g (permitted) without going any faster. This is a well defined point on the V/n diagram where the stall speed curve intercepts the maximum permitted g limit line.
You can read on the topic of energy maneuverability on this paper:
http://dynlab.mpe.nus.edu.sg/mpelsb/dts5121/Perf4n.pdf
The topic gets even more interesting when carried into three dimensions.