There may be many reasons to use reduced flaps when control is degraded. Might be performance, or pitch authority during landing, or G/A, system considerations, or something else altogether.
Only Airbus test pilots and engineers know actual reasons, which may be very obscure, and/or affect only a tiny part of the flight envelope covered during flight testing and certification. However, if the manual says F3, I would strongly advocate to follow this advice.
Also, this is not unique to A320. A330 procedures dictate using F2 a lot when in degraded modes. On the 787 most (if not all) F/CTL, or hydraulic failures call for F20 instead of the usual F30.