Hello tdracer, I envy you, I'm still slaving away at the keyboard...
The requirements are indeed the weak link. It's possible to completely set out the requirements for a wing, FADEC, fly by wire. We can't do that with machine learning and AI.
And if we can't do that, then the only thing to do is to put them into service and see how many crash over the following decades. Given the excellent state of aviation safety, there's precious little room for improvement.
That's the moral question, and I think that we shouldn't entertain that at all, not simply for the sake of eliminating a company cost.
The fear I have is that the money power will try and brush the morals under the carpet (just as they are trying to do in the self driving car industry).
However if someone can prove in advance of the first revenue flight that it will be a safety improvement (a truly narrow window of opportunity) that would be harder to argue against. But that's impossible to do.
I work with a bunch of guys who do machine learning algorithms, and the one permanent fact that remains is that they need training data. And if that's incomplete, or just plain wrong, you end up with a junk machine. Moores observation doesn't come into it; more transistors does not make our training data set complete, or right. And there's precious little sign of these things exhibitting inspired imagination or adaptability like Sully did.