Originally Posted by
Neo380
I've reread #74 and concur! We are not trying to test every combination of variables, like the U2 flight plan (with no altitude data!) and it's impact on the FAA system.
I agree that task is never ending. But you say it yourself "failure testing is often limited to defined alternate path (within the software) testing" that path CAN'T be the already failed path, because it's bound to fail again. Especially if the circumstances are more operators than the system was stress tested for, many in new (military) roles. This is the smoking gun, and the cover up (or at least not being discussed) the lack of alternate paths.
You go on "critical systems like this should ALWAYS [my emphasis] fail safe [that's what I've been saying!], ie reject any invalid input , or input which causes invalid output, rather than fail catastrophically, which appears to be the case this time'. EXACTLY. All this talk about edge cases, and French data etc etc is really just BS...
"Similarly for hardware and connectivity of critical systems, no one failure should cause a system wide crash'. But it has, repeatedly now. I wonder about BC testing too!
There are cases were one invalid or rejected input means subsequent inputs cannot be processed properly, EG running totals or counts may ne inaccurate. Certainly in the case of a control system it's generally better to keep going, but from the developers point of view it isn't always clear whether it's a 'keep running regardless' scenario or a 'once you're on the wrong line every station is likely to be the wrong station' scenario.