FBW just means how the control surfaces are controlled, i.e. no direct physical link.
The important detail is how the particular flight control law is implemented!
If an aircraft abruptly changes which law it is in, then we start opening up the possibility of man-machine interface mis-matches.
There is no abrubt change in the behaviour of the aicraft, just some protections are lost. It remains fully controlable as a non-FBW A/C (+ reduced protections).
The problem lies more on the way to apprehend the problem/urgency to the crew. Flying endlessly in the comfort of a quite cruising environment for years and being faced with a sudden serious incident is a challenge for every crew.
Warnings/procedures should be more canalized, facilitating crew's choices and decisions. Nowadays we are more on the "overflow" side.
Of course you can consider more automation, less crew decision, just management... it's a debate for decades! Or less...?