To be absolutely correct:
It depends on whether the aircraft has positive or negative weathercock stability......
On the 4-engined jet I fly, which has POSITIVE Weathercock, the AFM states that the Critical Engine is the OUTER Engine on the UPWIND side.
