"Why do you believe that it "stops working"? This situation is different from the wheels of a car 'breaking loose' from the road."
True, but if you pull collective (like take-off and landing) the torque becomes imbalanced, unless you want to constrain the pilot to have to accelerate every time he wants to manouvre...
"The servomotor does not contribute to the driving of the rotor or propeller. It is only required to overcome a small portion of the friction between the worm and wheel. The majority of the friction is handled by the angle of repose of the worm thread."
But for reasons given above system as drawn won't work, although a torsen diff will greatly improve matters. Don't forget that whenever the diff is locking up it is effectively wasting power, whether torsen or viscous. At any rate i don't understand why you want the prop to run slow in hover but fast at speed - compressibility/noise could become a problem. If any thing the higher mass fow rate through the prop at high speed suggests slowing it down would be more effective - ie direct drive from rotor to prop...
Mart