So, am I right thinking that that the remaining engines operate at higher efficiency (even on a lower cruising FL) and the decrease in performance is due to windmilling and asymmetry drag?
Several things come into play: The remaining engine is typically operating well above it's optimal efficiency range and hence is
less efficient than during a normal cruise. Cruise altitude is much lower, meaning more drag. Windmilling drag and lots of rudder to compensate for the thrust asymmetry adds more drag.