FW has inherent stability so the aircraft will keep flying with no flight control inputs, eg whilst the system is rebooted following catastrophic software crash, glitch caused by cosmic rays, lightning etc. Helis will typically be upside down within a couple of seconds of swashplate freeze at cruise speed. You couldn't do a Soux City in a helicopter!
I think you might be surprised if you look at modern airliner designs with regard to inherent stability. Im sure of course that you are aware that most military fighter types that are FBW are inherently unstable.
Not that it is possible but if you were to turn off the FBW then a fixed wing aircraft will be in a similar position to your hell!
With regard to your swashplate freeze. I take it that the swashplate is hydraulically powered? What happens if the hydraulics lock up? What happens if your control rod linkage separates?
FBW is only a part of the whole system. It replaces parts that are inherently less reliable than the replacement. It reduces costs, build and maintenance, meaning we as pilots are more in demand for the increase in work that can be done
It also means a joe average pilot can hand fly to a higher standard, which Im more than happy to take advantage of!