FADEC issues - are there any?

[Lead Balloon]

Re RADALT, there is the 2009 B737-800 incident at Schiphol – actually 1.5 kms away, where the aircraft crashed in a field during approach. All three pilots, one member of the cabin crew and five of the 128 passengers were killed, 10 passengers and two cabin crew were seriously injured and 107 passengers and one of the cabin crew sustained minor injuries. Six passengers were uninjured.

The investigation ascertained that one of the RADALT systems suddenly sent an erroneous minus 8’ height reading to the automatic throttle control system. Quote from here:

As this false radio altimeter height was lower than the height (27 feet agl) at which the A/T automatically enters the RETARD mode, the A/T immediately began to reduce thrust to flight idle because all the other conditions required for this to happen (flaps >12.5 degrees, A/T speed mode is selected, a climb or descent to a selected altitude is not in progress and the aircraft is not maintaining a selected altitude) were already met.

[mustafagander]

tdracer I would think that the stock B747 air/ground sense of 2 MLG untilt would be very robust and reliable. After all you are counting 2 of 4 to give the signal.

[pax2908]

May I ask one more question, please, specifically referring to the extensive testing (as mentioned by tdracer in the context of the certification phase). What is the threshold where this flight test must be repeated (re-certification), following a change in the rest of the FADEC "firmware" (for performance improvement, or for mitigation of some observed problem, etc)?

[TURIN]

Originally Posted by Lead Balloon

Re RADALT, there is the 2009 B737-800 incident at Schiphol – actually 1.5 kms away, where the aircraft crashed in a field during approach. All three pilots, one member of the cabin crew and five of the 128 passengers were killed, 10 passengers and two cabin crew were seriously injured and 107 passengers and one of the cabin crew sustained minor injuries. Six passengers were uninjured.

The investigation ascertained that one of the RADALT systems suddenly sent an erroneous minus 8’ height reading to the automatic throttle control system. Quote from here:

Having just read that report, blaming the RadAlt fault for this event is stretching it a bit.
From the report.

Why It Happened

As will be evident from the account above, the flight crew collectively failed to detect once finally fully established on the ILS that the steadily increasing pitch attitude was sufficiently abnormal to require that a reason for it needed to be urgently established. Thirty-seven seconds elapsed between the flap 40 landing flap selection and the onset of the stick shaker during which time the PFD indications and the engine thrust set

[Lead Balloon]

Who “blamed” the RADALT? In the tragedy in question the RADALT was, in fact, a single point of failure which resulted in the thrust levers automatically moving to idle.

I don’t comment on pilot competence issues because I don’t think it’s appropriate for me to do so.

[tdracer]

Originally Posted by mustafagander

tdracer I would think that the stock B747 air/ground sense of 2 MLG untilt would be very robust and reliable. After all you are counting 2 of 4 to give the signal.

The air/ground indication was originally envisioned as a single analog wire (similar to how min/approach idle was selected on 747s prior to the -8) - so a broken or shorted wire would result in an incorrect indication. Yes, we could have made it potentially more robust by using two discretes - one from each prox sensor unit (at the cost of additional wire weight), but we never really got that far into the implementation. I'd already formulated an idea for getting additional aircraft information to the FADECs by using EICAS as a 'data concentrator' - replacing the ARINC 429 busses from the Air Data Computers with ARINC busses from the EIUs (EICAS Interface Units) - going so far as feasibility discussions with my EICAS counterparts, but knew it would be a hard sell due to the increased EICAS work statements.

But then, we ran into a big (unrelated) issue which I could easily solve with the change that I'd already looked into of letting EICAS send the information to the FADECs - and I was able to justify the additional costs by solving that unrelated issue. Ultimately made for a much better FADEC installation, and I was a bit of a hero with the Propulsion Integration team because I was able to save ~200 lbs. of FADEC wire weight with the changes I'd come up with

[tdracer]

Originally Posted by pax2908

May I ask one more question, please, specifically referring to the extensive testing (as mentioned by tdracer in the context of the certification phase). What is the threshold where this flight test must be repeated (re-certification), following a change in the rest of the FADEC "firmware" (for performance improvement, or for mitigation of some observed problem, etc)?

Level A software has a prescribed process for certification of any software changes - as documented in DO-178. Even very minor "one line of code" changes have to be completely recertified. This involves a whole bunch of testing and documentation by the engine company to obtain Part 33 certification - not just testing the 'changed' portions but doing 'regression' testing to make sure there are no unintended changes to other parts of the software. Boeing then does a defined set of testing in our FADEC integration lab, followed by a certification flight test with the s/w installed on a single engine - there is a defined set of test conditions for the flight test that has been agreed to between the FAA and Boeing, with additional conditions added as needed to demonstrate whatever changes are in that particular s/w change.

My understand was that the cost of doing the entire cert process for even a 'simple' Level A s/w change is around $1 million. On occasion, it was faster and cheaper to modify the hardware to compensate for an issue with the s/w that it was to go back and modify/recertify the software to correct the issue.

[Citabria40X]

Originally Posted by tdracer

The design intent is TCMA will never activate if the engine is operating and responding properly to thrust lever inputs.

I see this in the context of certain systems, in this case the fuel shutoff valves. Isn't this an admission of the intent to bypass the flight crew in determining when to turn off the engines? Instead, shouldn't "TCMA" be a Thrust Control Malfunction WARNING, i.e., an alert to the folks with actual skin in the game at that moment? It occurs to me that software designers are taking on too much responsibility without the associated risk. Designing automatic reactions that affect hundreds of lives shouldn't be equivalent to software for a glorified autonomous pizza delivering vehicle with no pilot aboard, should it? Airline captains, since Crew Resource Management (CRM) was invented, nowadays solicit input from crewmembers who in turn assert themselves with their own wisdom born of experience. For engineers to draw irrevocable battle plans well in advance seems to be a regression from the principle that "two heads are better than one."

Btw, I've been lurking over the Air India 171 crash and appreciate your time given to explain these systems and your lifelong experience in related software design. Thank you very much.

[Lead Balloon]

Originally Posted by Citabria40X

I see this in the context of certain systems, in this case the fuel shutoff valves. Isn't this an admission of the intent to bypass the flight crew in determining when to turn off the engines? Instead, shouldn't "TCMA" be a Thrust Control Malfunction WARNING, i.e., an alert to the folks with actual skin in the game at that moment? It occurs to me that software designers are taking on too much responsibility without the associated risk. Designing automatic reactions that affect hundreds of lives shouldn't be equivalent to software for a glorified autonomous pizza delivering vehicle with no pilot aboard, should it? Airline captains, since Crew Resource Management (CRM) was invented, nowadays solicit input from crewmembers who in turn assert themselves with their own wisdom born of experience. For engineers to draw irrevocable battle plans well in advance seems to be a regression from the principle that "two heads are better than one."

Btw, I've been lurking over the Air India 171 crash and appreciate your time given to explain these systems and your lifelong experience in related software design. Thank you very much.

The FAA mandated a system that would operate - shut off the engine when the delta between the thrust lever position and thrust output on the ground was sufficiently large for a sufficiently long period - autonomously of the pilots.

[TURIN]

Originally Posted by Lead Balloon

Who “blamed” the RADALT? In the tragedy in question the RADALT was, in fact, a single point of failure which resulted in the thrust levers automatically moving to idle.

I don’t comment on pilot competence issues because I don’t think it’s appropriate for me to do so.

My apologies, as you brought it up, in the context of RadAlt reliability, I mistook your intent.
We digress, but the failure of the RadAlt isn't what I would describe as a single point of failure.

[Lead Balloon]

No need to apologise, as my hastily drafted posts sometimes don’t convey, well, what I intend.

In the context of this thread and the ‘other’ event, I thought it was relevant to point out that a single erroneous sensor output could cause the automatic movement of both thrust levers to idle, in the air (but not ‘very high’), in a ‘modern’ transport category-certified Boeing jet.

Perhaps the correct description is not ‘single point of failure’, but that a single erroneous sensor output put a big hole in one of the pieces of Swiss cheese.

[Musician]

Originally Posted by Citabria40X

I see this in the context of certain systems, in this case the fuel shutoff valves. Isn't this an admission of the intent to bypass the flight crew in determining when to turn off the engines? Instead, shouldn't "TCMA" be a Thrust Control Malfunction WARNING, i.e., an alert to the folks with actual skin in the game at that moment?

The risk that TCMA is meant to address is thrust imbalance while landing or aborting take-off. The NTSB was involved in an uncommanded high thrust (UHT) event in Saudi Arabia 1997; the aircraft left the runway and eventually burned out (after it was successfully evacuated). Basically, the pilots expect the engines to be at idle, and if one of the engines still delivers significant thrust, that interferes with braking, and also wants to steer the aircraft to the side, while at the same time rudder effectivity diminishes as the aircraft slows down and airflow across the rudder decreases.
At the same time, this is a high-workload situation where the pilots' attention is focused elsewhere, and things are really happening fast. You are assuming that when the pilots notice the aircraft yawing off the centerline, one of them will always check the display for a potential alert, confirm that it is valid, and then shut the engine off immediately. How much time is that supposed to take?
And if the pilot sees the alert and shuts the engine off without thinking, to save time, wouldn't it be better to have the engine off automatically in the first place?

Please also remember that, until today, TCMA activation has always been safe even when it was unwarranted, i.e. the aircraft had touched down and the pilots were able to successfully stop the plane.

There is speculation that TCMA is somehow involved in the Air India crash, but at this point there is zero evidence for it. If it was involved, then the contributing factors would need to be analysed, and conclusions would be forthcoming; but we don't even know that it is involved. For comparison, with MCAS, the contributing factors were a maintenance issue with the AOA sensor; MCAS relied on a single sensor when the aircraft has two; the aircraft did not indicate the AOA sensor failure; and crew training was inadequate. All of these issues have been addressed, and MCAS is still there. For TCMA, we don't even know of any contributing factor yet; if it did fail (remember, we have no evidence it did), it's a complete mystery why it malfunctioned.

Your scenario of the pilots having a leisurely pow-wow troubleshooting why the aircraft drags off center as they're slowing down seems to be a tad unrealistic to me.

[Lead Balloon]

TCMA is meant to address the risk of “thrust imbalance while landing” (or RTO)? Reference for the quoted bit please, with a definition of the period and height ranges that constitute “while landing”.

”Zero evidence” for some TCMA-related double engine shut down? Zero is a big call. There is some circumstantial evidence of both engines ceasing to deliver any significant thrust, simultaneously or almost simultaneously. It’s ‘in’ until some more credible evidence - like EADR data - moves it from infinitesimally remote to zero.

[Someone Somewhere]

These seems to be one of the most relevant posts:

Originally Posted by tdracer

Assuming TCMA works as intended, it does eliminate the risk of uncommanded/uncontrolled high thrust (UHT) on the ground. There is still some risk that UHT could happen in the air during final approach - if you're less than ~100 ft. it could be pretty exciting. It's shown to be controllable, but the pilot does need to be paying attention and fly the aircraft or it could end badly. The good news is that the exposure is maybe 30 seconds per flight, and the probability of UHT is something between one in 10 million and one in 100 million flight hours, so the odds of it happening during final are astronomically high. However UHT can be caused by a single failure so we need to show it's controllable (ref 25.901(c) and 25.1309 - you can't use probability arguments for single failures that are catastrophic).

Originally Posted by Musician

I linked the examption in my previous post.

First, TCMA was not a problem in this incident, because even if it caused both engines to shut off, the pilots controlled the aircraft, and it never left the runway. It didn't make the aircraft unsafe.

Secondly, for TCMA to fail in the air, there must be two failures: the air/ground logic must fail, and TCMA must erroneously detect an UHT condition: that's two improbable failures.

The exemption is about this:

Description of Issue

Historically, propulsion control systems on large commercial airplanes have been designed with single elements controlling fuel flow. Industry practice has provided design features to protect the structural integrity of the engine, but it is still possible for single failures or malfunctions of the propulsion control system to result in uncommanded high thrust (UHT). Industry design practice provides a means for flight crews to accommodate such failures by shutting down the engine. The effectiveness of this design practice has been demonstrated in today's fleet of large commercial transport airplanes, as there has never been a report of serious injury resulting from a case of UHT.

In the past, compliance to 14 CFR 25.901(c) has been found based on the assertion that the flight crew can recognize and accommodate UHT. However, following a 1997 Saudi Arabian Airlines Boeing 737-200 accident, engineering studies showed that for some airplane designs the traditionally accepted assertion may not always be valid. In response, the FAA has begun to evaluate type designs with far greater scrutiny regarding the flight crew's ability to recognize and safely accommodate single failures that can lead to UHT.

A committee consisting of representatives from the FAA, the Joint Aviation Authorities (JAA), airplane manufacturers, and engine manufacturers was formed in 1998 to study strategies for providing additional protection from thrust control malfunctions resulting in UHT. The committee found that for the existing in-service airplanes whose propulsion systems have demonstrated a level of reliability on the order of one UHT event per 10 million flight hours, it would not be in the public interest to mandate major and novel design changes in an attempt to eliminate the already small potential exposure to UHT malfunctions resulting from single failures. The committee's recommended approach to ensure continued high levels of reliability for all presently certified models is to monitor in-service performance and if any unacceptable failure modes are identified, to take prompt corrective action by introducing focused design improvements using proven technology.

The 787 airplane design minimizes the number of single failures that can lead to UHT, and has a design feature which is intended to detect UHT and automatically accommodate it when the failure is detected while the airplane is on the ground. Previous engineering simulations have shown that the 787 airplane is controllable for detected failures that cause UHT; however, it was recently observed that a combination of a high crosswind and UHT may not be controllable for operations on or very near the ground. Given the very low failure rate of UHT failures, the very limited exposure time when the failure is potentially uncontrollable, and the additional environmental factor of high crosswind, a catastrophic event caused by UHT is not anticipated during the life of the 787 fleet. However, strict compliance to § 25.901(c) cannot be shown; since the regulation does not allow single failures that jeopardize continued safe operation, no matter how improbable.




The part that I bolded is the Thrust Control Malfunction Accomodation system. TCMA does not work in the seconds before touchdown, when an UHT-type failure could make the aircraft fail the landing. That's what the exemption is for.

See also from an early 787 FCOM:

The EEC provides protection against thrust control malfunctions that can result in an idle thrust asymmetry condition on the ground.

The EEC commands shut down of the affected engine when:
• airplane is on the ground,
• thrust lever is approximately at idle, and
• engine is above command N1 speed and not decelerating normally

[Lead Balloon]

I think we already knew all that?

[Someone Somewhere]

You were asking for a reference as to what TCMA is for. The certification exemption seems to summarise the situation nicely, even if Musician was being slightly loose with words (UHT is a hazard during landings and RTOs, TCMA only protects during the on-ground portions of them).

(edit: depending on the situation, the hazard could be thrust imbalance, excess total thrust, or both. I have not seen significant discussion of this and I don't think it's really necessary)

[Lead Balloon]

That clears everything up then.

[Musician]

Thank you, Someone Somewhere !

Originally Posted by Lead Balloon

TCMA is meant to address the risk of “thrust imbalance while landing” (or RTO)? Reference for the quoted bit please, with a definition of the period and height ranges that constitute “while landing”.

If you really want to know more, look at section 2 of https://www.researchgate.net/publica...Civil_Airplane for a list of scenarios. If you can find the FAA memorandom that's based on, I'd be grateful if you shared a link to it.

"Zero evidence” for some TCMA-related double engine shut down? Zero is a big call.

We have ample evidence that AI171 suffered a loss of thrust on both engines. That's not the point.
My point is that we shouldn't assume TCMA is a serious hazard until we have evidence that it is.

[Citabria40X]

Originally Posted by Lead Balloon

The FAA mandated a system that would operate - shut off the engine when the delta between the thrust lever position and thrust output on the ground was sufficiently large for a sufficiently long period - autonomously of the pilots.

Agreed, but my point was that they shouldn't have. A mechanical solution to limit the thrust possible would be better.

[Lead Balloon]

Originally Posted by Citabria40X

Agreed, but my point was that they shouldn't have. A mechanical solution to limit the thrust possible would be better.

You’re not alone in having that opinion. But our opinions are now academic. It’s been mandated, designed, implemented and certified.