For me, flaps stuck down also raises the question of how this will affect my ability to cope with an engine failure (or other additional problem).
Deployed flap not only removes significantly one of the most controlled methods of increasing rate of descent at a given airspeed (I find side slipping always takes me by surprise r.o.d. wise) but will give me a glide angle that is steeper & not as familiar from PFLs increasing the undershoot risk.
Similiarly, as well as concern about the cause, other types of failure also need to be thought about in terms of cumulative effect & the individuals ability & confidence in being able to cope with failure 1 + failure 2.
Personally, as I'm sure is the same with most on here, I try & enjoy a type of flying that stretches me but does not leave me feeling vaguely uneasy for the entire flight.