MAX’s Return Delayed by FAA Reevaluation of 737 Safety Procedures

[PerPurumTonantes]

Originally Posted by CurtainTwitcher

Their strategy was "we will always blame the crew" should, what they convinced themselves was a remote possibility of an accident. I also believe that the outstanding & expert investigation by PPRuNe contributors was, and continues to be a thorn in their side for this strategy.

A thorn in their side to the point where it was suspected (by user wonkazoo) that Boeing had got someone to flood this forum with posts blaming the pilots ("JUST FLY THE PLANE").

[GordonR_Cape]

Originally Posted by Speed of Sound

What has always bothered me right from the beginning of this whole thing is that at some point in the design process the one, two, three or synthetic AoA input question must have come up. And when it did, surely someone put their hand up and said “With only one AoA vane feeding the FCC, an early failure of the sensor will only come to light when either the AP is switched off and/or the flaps are retracted. This really is not the time/speed/altitude/phase of flight to have large amounts of AND.”

Given that it is unthinkable that all potential outcomes weren’t modelled, who on earth decided that this was an acceptable risk to take? 😵

A profound question, answered almost immediately by the next post:

Originally Posted by brak

To quote myself from a previous thread:

And now this from the news:

"Increasingly, the iconic American planemaker and its subcontractors have relied on temporary workers making as little as $9 an hour to develop and test software, often from countries lacking a deep background in aerospace -- notably India.In offices across from Seattle’s Boeing Field, recent college graduates employed by the Indian software developer HCL Technologies Ltd. occupied several rows of desks, said Mark Rabin, a former Boeing software engineer who worked in a flight-test group that supported the Max."

When reality is stranger than an ironic post.

Further down in that article:

“Boeing was doing all kinds of things, everything you can imagine, to reduce cost, including moving work from Puget Sound, because we’d become very expensive here,” said Rick Ludtke, a former Boeing flight controls engineer laid off in 2017. “All that’s very understandable if you think of it from a business perspective. Slowly over time it appears that’s eroded the ability for Puget Sound designers to design.”Rabin, the former software engineer, recalled one manager saying at an all-hands meeting that Boeing didn’t need senior engineers because its products were mature. “I was shocked that in a room full of a couple hundred mostly senior engineers we were being told that we weren’t needed,” said Rabin, who was laid off in 2015.

The Max became Boeing’s top seller soon after it was offered in 2011. But for ambitious engineers, it was something of a “backwater,” said Peter Lemme, who designed the 767’s automated flight controls and is now a consultant. The Max was an update of a 50-year-old design, and the changes needed to be limited enough that Boeing could produce the new planes like cookie cutters, with few changes for either the assembly line or airlines. “As an engineer that’s not the greatest job,” he said.

Short answer: There was no-one in charge! The hot-potato was passed around in circles, until the design worked (sort-of). No-one thought through all the implications, and tested all of the systems, until it was already airborne.

[CurtainTwitcher]

Originally Posted by GordonR_Cape

A profound question, answered almost immediately by the next post:



Further down in that article:


Short answer: There was no-one in charge! The hot-potato was passed around in circles, until the design worked (sort-of). No-one thought through all the implications, and tested all of the systems, until it was already airborne.

If the answer is there weren't any adults in charge, that is an even more terrifying explanation. If what you say is correct, where was the FAA, EASA and the rest of the global regulators in all this? The FAA certified under the guidance of Boeing's own Authorized Representatives and everyone else just accepted at face value? I still contend that ultimately, the finacialization of industry drives this end result.

If this is what we observe in this instance, it opens up a Pandora's box of questions about every aspect of aviation safety, and every future accident will focus much more scrutiny on the manufacturer and the regulators as well. Another obvious problem area is going to be fatigue, where regulators have progressively allowed margins to be eroded to benefit commercial objectives.

[KRUSTY 34]

Originally Posted by edmundronald

I think management was simply so dumb that they didn't realise a bad design was bound to fail. It's something you don't learn in business school, to think like an engineer.
The FAA was there to keep Boeing honest but got captured.

This is a case which demonstrates how money will circumvent regulation in a democracy until some incident creates an outcry. You can see the same thing happening in construction standards eg. the Greenfell Tower fire in the UK.

Edmund

Oh they were dumb alright.

Remember, a senior Boeing Engineer was first sidelined and then his employment was terminated, because he questioned the integrity of the program.

I think it’s safe to say this act of Corporate insanity is what led to this collective “Dumbness”.

[fergusd]

Originally Posted by brak

To quote myself from a previous thread:

And now this from the news:

"Increasingly, the iconic American planemaker and its subcontractors have relied on temporary workers making as little as $9 an hour to develop and test software, often from countries lacking a deep background in aerospace -- notably India.In offices across from Seattle’s Boeing Field, recent college graduates employed by the Indian software developer HCL Technologies Ltd. occupied several rows of desks, said Mark Rabin, a former Boeing software engineer who worked in a flight-test group that supported the Max."

When reality is stranger than an ironic post.

So the software quality will be off the bottom of the scale. Ultra low cost, offshore, software creation almost universally results in junk.

The deeper they dig into this the worse it will become, if anybody with true safety critical software knowledge gets their hands on the code for this the MAX will remain a paperweight for many years.

This is not unique to the aviation industry either . . . how depressing.

[BDAttitude]

Originally Posted by fergusd

So the software quality will be off the bottom of the scale. Ultra low cost, offshore, software creation almost universally results in junk.

The deeper they dig into this the worse it will become, if anybody with true safety critical software knowledge gets their hands on the code for this the MAX will remain a paperweight for many years.

This is not unique to the aviation industry either . . . how depressing.

How true ... although my most recent experience with this matter is that it is no longer a financial thing. They have become expensive there in the recent years as well. IMHO it is more a cultural thing. Management enjoys the servility lack of resistance to bad decision and being pampered and being hommaged when on site there. Ain't no grey bearded pig headed wester engineers over there.

[fergusd]

Originally Posted by BDAttitude

How true ... although my most recent experience with this matter is that it is no longer a financial thing. They have become expensive there in the recent years as well. IMHO it is more a cultural thing. Management enjoys the servility lack of resistance to bad decision and being pampered and being hommaged when on site there. Ain't no grey bearded pig headed wester engineers over there.

Yup, but you're correct that it's not just financial, a large part of this is lack of availability of suitably qualified software engineers . . . many people 'think' they can 'write code', very, very few can actually do it (in actuality) and even fewer work in environments which can create the required quality repeatably . . . so you cannot just go out and buy a high quality capable team, at all . . . the bean counters 'think' that 'anybody' can do software and so why pay more than 1/10th of that which is actually necessary . . . and boom . . .

If I had $0.01 for every time I've seen junk software, created offshore, which is killing the business that paid for it (and paid plenty for it, the unit cost may be $9/hr, but when you end up needing 10x the staff to meet the deadlines . . . - this is the offshore business model - sell people, not quality) I would be significantly wealthy.

Fundamentally people do not value software, it is very difficult for the largely ignorant layman (anybody outside of the 1% of the software industry that does high quality work) to understand why high quality software costs so much.

If I were the FDA I'd be pulling every single piece of software development evidence and documentation and reviewing it to see if any of it complies with the required processes . . . I'd bet good money that it doesn't . . .

How many times have I seen that ? - more times than I care to think about.

[BDAttitude]

Originally Posted by fergusd

Yup, but you're correct that it's not just financial, a large part of this is lack of availability of suitably qualified software engineers . . . many people 'think' they can 'write code', very, very few can actually do it (in actuality) and even fewer work in environments which can create the required quality repeatably . . . so you cannot just go out and buy a high quality capable team, at all . . . the bean counters 'think' that 'anybody' can do software and so why pay more than 1/10th of that which is actually necessary . . . and boom . . .

And those who do have the ability to code head for other ventures because they are generally intelligent knowledgeable and studious people and can't be bothered with this sort of competition and obnoxious working environment.
Similar things happen with professional airmen I guess.
In fact I was having two Starfighter jockeys as supervisors during my career. I'm sure they would head to different industries and businesses today after leaving the air force.

[Speed of Sound]

Maybe the politicians need to take an interest along the lines of the Rogers Commission into the Challenger disaster. That way they would have the power to seize evidence, subpoena witnesses, provide legal protection for whistleblowers and potentially jail anyone found to have deliberately facilitated the situation.

This could be done once the MAX is back flying.

[Kerosene Kraut]

Could this whole affair end the grandfathering of aircraft certifications?

[73qanda]

Fergusd can you give some brief ( easily understood) examples of how high quality code is different from most code? Or how the process of writing/developing it is different? I have never been involved in computing at all so have little understanding of what the differences might be.
Cheers

[Speed of Sound]

Originally Posted by 73qanda

Fergusd can you give some brief ( easily understood) examples of how high quality code is different from most code? Or how the process of writing/developing it is different?

High quality code is code which has been written, tested, tweaked, tested, tweaked again, tested again until all foreseeable outcomes and possible conflicts have been examined and the code is still doing what it was initially designed to do.

This takes time to do properly. The difference between good code and bad code is simply the point at which you stop testing and declare that it is ‘good enough’.

[Mac the Knife]

73qanda, almost impossible to explain to a non-programmer (and some programmers!)

Take a read of some of the things in The Daily WTF - The Daily WTF: Curious Perversions in Information Technology - and things may get a little clearer for you.

Mac

[BDAttitude]

The amount of iterations you need in that business is strongly dependent on the system knowledge of coders involved. Specs are usually not 100% sound especially regarding unforseen conditions and abnormal conditions. You can have people involved who will do either the right thing by themselves or ask if they don't know. And you can have people involved which will do just something or nothing.

[PaxBritannica]

Doesn't sound to me as if the MAX problem is a coding issue. Coders work to a specification. The specification comes from a design. The design is Boeing's responsibilty. As design parameters changed, it should have been someone's job to take a high-level holistic view of the overall design and spec and identify everything that would be affected. It's not OK to just redesign, write new specifications and give them to the cheap offshore coding subcontractors.

From all accounts, Boeing got rid of the people whose job this would have been. I understand why. Those people are expensive, and by the very nature of their work they slow things down.

Boeing saved money on coders, but also saved money on the experts who were needed to compensate for the cheap coding.

[Pearly White]

Originally Posted by Kerosene Kraut

Could this whole affair end the grandfathering of aircraft certifications?

You pretty much need a paternity test.
The problem is that what is being grandfathered is starting to look much more like a third cousin twice removed.

[GordonR_Cape]

Originally Posted by 73qanda

Fergusd can you give some brief ( easily understood) examples of how high quality code is different from most code? Or how the process of writing/developing it is different? I have never been involved in computing at all so have little understanding of what the differences might be.
Cheers

In my past life I wrote code (albeit in the insurance industry). The differences include that I worked in the business for 20 years, knew what the requirements meant, and was able to relate face-to-face to the actuaries who designed the products. This helped because even if they were badly specified, I was involved in an iterative process of clarifying the specifications.

Such a job is best defined as an 'analyst-programmer', not just a pure coder. It helps the speed and accuracy of the process. And preferably not someone working on contract.

After all of that I was able to do testing (not my own code), because I knew the products, and knew the code. Obviously you need independent testers, but the deepest level of testing must specified in advance by people with knowledge of the systems and the code.

P.S. I came from an engineering background, but many of the principles of clear thinking are the same.

[Speed of Sound]

Originally Posted by PaxBritannica

Doesn't sound to me as if the MAX problem is a coding issue.

You are right.

Ultimately this is a management decision to accept a solution to the changed flight characteristics problem which had little redundancy built in and very serious consequences if it went wrong close to the ground.

It was further aggravated by another management decision to ‘sell’ the aircraft to customers as an ‘iPad only’ conversion which was almost certainly why MCAS was left out of all training and ops literature.

[DaveReidUK]

Originally Posted by Kerosene Kraut

Could this whole affair end the grandfathering of aircraft certifications?

The scope for grandfathering was beginning to run out, anyway.

Once the 777X is certificated, it's hard to see what other candidates there might be - Airbus have already applied the Neo treatment to all of their legacy narrow- and wide-body family, there clearly aren't going to be any further 747 deelopments, so what does that leave ?

[doxter]

Originally Posted by 73qanda

Fergusd can you give some brief ( easily understood) examples of how high quality code is different from most code? Or how the process of writing/developing it is different? I have never been involved in computing at all so have little understanding of what the differences might be.
Cheers

what i would offer here Sir is that in many ways its the same principle all through commerce . how do you get quality when you outsource ? providers will have a contract that defines exactly what there obligations to the client are , and now more . otherwise they would be giving them services for free . yes you can of course get tailored / adhoc services but very often that is where the sting in contracts are , they become expensive and also this translates to delay and further cost on the buyer . the impossible challenge that is outsourcing .

agree with this by @GordonR_Cape- it is the vital everyday approach that can help deliver quality .

"This helped because even if they were badly specified, I was involved in an iterative process of clarifying the specifications."