FADEC failure

[TO MEMO]

For an A320, what happens if both fadec fail for an engine.
According to FCOM you will have no engine indications, but engine runs.
The question is: Do you still have engine control?

[lomapaseo]

Describe the failure condition.

Typically the FADECs are failed safe. i.e. if they don't trust themselves and their inputs any longer they latch the FCU at the last known trustworthy condition of flight. The pilot always has the option of turning off the fuel if the thrust is too much for conditions.

[TO MEMO]

According with FCOM 3, FADEC fail you will have no engine indications (all amber crosses) but engine runs. One must use the ECAM system pages, such as hydraulic and bleed to confirm that engine is operating. It also states that if the engine is not operating, you should set master switch off.
But it has no further gidance for "if the engine is running".

So lets suppose that you have ENG 2 FADEC A FAULT. Later inflight fadec B fails also. So you end up with ENG 2 FADEC FAIL. You have all amber crosses and you know the engine will not quit just because you lost FADEC.

The question is: Is the engine responsive to Throttle lever movement?
Is there any relationship between TLA and the engine HMU? If not possible, that would mean cutting the engine for descent.

Cheers

[lomapaseo]

My understanding is that the engine will not be responsive to throttle (a schematic would probably show this). Anybody?

If it's a single engine failure then yes to cutting the engine for descent.

[Neupielot]

i think yes. Just no indications and no protections?
Anyone with solid definite answers plz :O

[PBL]

The original question "what happens what FADECs fail?" is ill-posed.

They can fail because the units fall off the airplane. In that case, there is unlikely to be anything resembling engine control.

I am guessing that TO MEMO really meant something like the following: what can you expect in the way of functioning engine control if you see such-and-such an ECAM message?

Could one maybe rephrase the question in this form? That way, it is likely to have a definitive answer. If not, then not.

PBL

[lomapaseo]

PBL

Can we work with the questions in post # 3? or is more information needed to clarify?

[A321COBI]

you are still in control of the plane but have not readings and are unaware of position and altitude
the plane can be controlled if you are a good guesser
notify the ATC immediatly and wait for further instuctions for gliding to an airport

[TyroPicard]

If both FADEC channels on one engine fail, the FADEC can no longer control the thrust. Nothing else will either.
ECAM:
T/L.... IDLE
ENG PARAMETERS...CHECK (as described above) I suggest also look at the rudder trim position?
IF ABNORMAL shut it down (which I assume means not at idle?).

If ECAM presents FADEC FAULT caution it might just be spurious - which you can determine from the presence/absence of indications.

[PBL]

Can we work with the questions in post # 3? or is more information needed to clarify?

Much more information is needed to clarify, probably more than anyone here has.

There are, say, about 100,000 lines of source code (LOC) in typical FADECs on a modern engine. The highest quality level known to have been attained in code for a critical system is about one error in 25,000 LOC, and more typical quality levels are of the order of 1 in 1,000 LOC. So you can expect a few to about a hundred errors in FADEC code. Now, you can't tell what executing code through those errors is going to do to the behavior of the FADEC. The manufacturer has to have performed an appropriate a hazard analysis. That will presumably demarcate what kit behavior the FADEC can possibly influence and what not.

Such a failure may or may not raise an ECAM message. Tyropicard says what happens if you lose both channels, assuming a fail-stop scenario. With a fail-live scenario it could be different.

That is the glory of digital electronics. It makes hazard analysis extremely hard, indeed in many cases combinatorially infeasible.

PBL

[lomapaseo]

PBL

Such a failure may or may not raise an ECAM message. Tyropicard says what happens if you lose both channels, assuming a fail-stop scenario. With a fail-live scenario it could be different.


well now that you have introduced the term fail-stop scenario and fail-live scenario could you tell us the difference relative to what gets displayed on the screen to the pilot? because that's all he knows regardless of what you call it.

[tubby linton]

I had a dual fadec channel fault in the sim a number of years ago shortly after a colleague had experienced this on the line. The engine failed with no ecam activation.The other unforeseen consequence was the beta target was not available so it was back to dead foot dead engine.

[PBL]

well now that you have introduced the term fail-stop scenario and fail-live scenario could you tell us the difference relative to what gets displayed on the screen to the pilot?

No, I can't. There is no oracle which is telling you in reality how the system failed. There are only failure behaviors. The issues are way too complex for a short message on a general-interest bulletin board.

Besides the issues of correctness of code, there are others. Here is a fundamental paper describing some of them: Driscoll et al on Byzantine Failures (http://www.cs.indiana.edu/classes/p545-sjoh/read/Driscoll-Hall-Sivencrona-Xumsteg-03.pdf). Kevin and Ken Hoyme (who no longer works in the industry) are the designers of the Boeing 777 AIMS.

Notice that the authors say that a major commercial aircraft was within days of losing its airworthiness certificate. Indeed, when one knows which aircraft this was, this would have been very serious indeed.

PBL

[A321COBI]

PBL is correct far to complex for a general-interest bulletin board
the inner workings are confusing and cannot be simply explained and often dont even make sense
This is something you learn of in pilot training [which I did]

[Neupielot]

Oh man ....im so confused now.......
I had a dual fadec channel fault in the sim a number of years ago shortly after a colleague had experienced this on the line. The engine failed with no ecam activation

and ECAM:
T/L.... IDLE
ENG PARAMETERS...CHECK

2 conflicting answers. Will a sim instructor plz stand up ?! :)

[rudderrudderrat]

Hi Neupielot,

This problem highlights one of the drawbacks of being a "slave to ECAM".

If you have an engine failure with a FADEC 1 (2) Fault, then you have to diagnose the problem by being a pilot and look at how the aircraft is performing, investigate other systems pages, recognise that you have an engine failure and I would suggest the simplest solution is call for the QRH "ENG 1(2) STALL" which say:
T/L.... IDLE
ENG PARAMETERS...CHECK
If Abnormal
- ENG MASTER (affected engine) ....Off

Initially there is no ECAM procedure for this problem, but once the Engine Master is put to off, the Rest of the ECAM procedure will be displayed.

[PBL]

Contrary to what I suggested above, let me indeed try to put a little more meat on this issue. For it seems to me that discussants still imagine they can figure out definitively what happens from some entry in some manual somewhere.

Fail-stop means the channel goes dead. No traffic. Tyropicard's contribution said what happens then.

Let's consider fail-live.

If your FADEC sends the signal to reduce fuel flow, your engine is going to idle back.

If your FADEC sends the signal to send fuel flow to max, your engine is going to go to max thrust.

If the FADEC sends oscillating signals to put fuel flow at min, then at max, then at min, then at max,.... with a frequency of X Hertz, then ..... well, somebody who designed the system can maybe tell us what your engine is going to do. Does it depend on the value of X?

Then there are the infinitely varied other signal behaviors that can occur.

PBL

[Mad (Flt) Scientist]

Let's consider fail-live.

If your FADEC sends the signal to reduce fuel flow, your engine is going to idle back.

If your FADEC sends the signal to send fuel flow to max, your engine is going to go to max thrust.

If the FADEC sends oscillating signals to put fuel flow at min, then at max, then at min, then at max,.... with a frequency of X Hertz, then ..... well, somebody who designed the system can maybe tell us what your engine is going to do. Does it depend on the value of X?

Then there are the infinitely varied other signal behaviors that can occur.

PBL

Well, for the first two, that somewhat assumes that this strangely failed FADEC is going to continue to respect the scheduled limitations for idle and max fuel flows respectively. I wouldn't put a whole lot of hope into that, personally. A run down to stop or a runaway to firewall and beyond sound just as likely to me.

The oscillatory behaviour sounds a lot like a bode test, something done to certify engine response. usually that's constrained by input dynamics (how fast can you slam the throttle back and forward?) and by input signal filtering. With neither a constraint here, there's no guarantee anything good might happen. And, again, who says it'll stop at idle and max thrust levels.

I agree 100% with the last sentence, and the sentiment you expressed earlier regarding the difficulty of even the designers understanding failure modes in complex systems, never mind trying to reverse engineer it from a very cursory descripotion in an AFM or operating manual.

[PBL]

Well, for the first two, that somewhat assumes that this strangely failed FADEC is going to continue to respect the scheduled limitations for idle and max fuel flows respectively. I wouldn't put a whole lot of hope into that, personally.


MFS, I wouldn't suggest this is likely. I don't think any failure behavior is likely, otherwise the FADEC wouldn't be certified.

The point is that the FADEC is driven by program code. One output is a max-it! signal. Another output is an idle-it! signal. They are generated by execution sequences (paths through the program) which come from max-it! inputs and idle-it! inputs respectively. But how does one know they only come from those inputs? The third example is of a behavior which likely cannot be produced by a valid (intended) execution path through the program. But there are those invalid paths as well, very likely.

For programs of, say, 100,000 LOC, one mostly doesn't know that for sure, even if one has done a lot of checking. The company which produces the demonstrably highest-quality code in critical systems has never produced a completely error-free program of a size over about 10,000 LOC, although they regularly produce larger programs that are demonstrably free of run-time errors (data going out of bounds, and so on).

And as for a likelihood that things will be OK or not, there are no statistical techniques which are trustworthy and which will enable you to predict it, when the likelihoods are as low (for failure) as required for critical systems.

My example was only intended to show those who think that computers are like car brakes: if things don't work right, there are only a small number of ways it can have gone wrong, so you check all those out and sure'nuff you find the problem. With computer code of 100,000 LOC, if things don't work right, there are more ways it can have gone wrong than there are atoms in the universe.

And that is only code errors. If one worries about so-called Byzantine failures (see the Driscoll reference), then the outputs cannot be necessarily regarded as the output of any execution sequence through the FADEC program at all.

In other words, people who answer this is going to happen when your FADEC fails are either giving very incomplete and thereby not necessarily very useful information, or are giving precise answers under restricted conditions (Tyropicard), or just don't understand issues well enough to hedge their answers with appropriately restrictive conditions. I'll leave it to others to classify the answers so far on this thread!

It may be that certain restricted conditions cover a majority of actual failure cases. If so, may I suggest it would be wise to say "your FADEC failure is likely to be of this-and-this type, and such failures may be indicated/handled this way......"


I agree 100% with the last sentence, and the sentiment you expressed earlier regarding the difficulty of even the designers understanding failure modes in complex systems, never mind trying to reverse engineer it from a very cursory descripotion in an AFM or operating manual.


I would hope you would :)

PBL

[lomapaseo]

PBL

Let's consider fail-live.

If your FADEC sends the signal to reduce fuel flow, your engine is going to idle back.

If your FADEC sends the signal to send fuel flow to max, your engine is going to go to max thrust.

If the FADEC sends oscillating signals to put fuel flow at min, then at max, then at min, then at max,.... with a frequency of X Hertz, then ..... well, somebody who designed the system can maybe tell us what your engine is going to do. Does it depend on the value of X?

Then there are the infinitely varied other signal behaviors that can occur.

PBL

OK, now you have explained what you mean by the live and and failure modes :)

So in the dead mode the results and expected response by the crew are then obvious.

The fail-live mode sounds like a horror story for the crew so we really don't want that.

But there are various parameters that need to be satsfied for the FADEC to issue a command to the engine and suposedly these parameters will never line up in an acceptable range when the FADEC loses all sense of reason, thus the FADEC looks to its other channel for help and if that is also fecked up the FADEC just drops off the line and you are back to fail-dead.

I don't think that we can imagine a computer response named "Hal" in any case

[A321COBI]

I agree, sounds very dangerous and something I never want to experience in my future a a pilot
The fail-live mode sounds like a horror story for the crew so we really don't want that.

[PBL]

The fail-live mode sounds like a horror story for the crew so we really don't want that.

Indeed not. One of my pals made his name a few decades ago by trying to show when and how one could reduce fail-live to fail-stop (as it is called).


But there are various parameters that need to be satsfied for the FADEC to issue a command to the engine and suposedly these parameters will never line up in an acceptable range when the FADEC loses all sense of reason, thus the FADEC looks to its other channel for help and if that is also fecked up the FADEC just drops off the line and you are back to fail-dead.

Sorry, reasoning like that just doesn't work. Let's take it step by step.

(a) various values of parameters must cohere for the FADEC to issue a command.

Indeed so, conditions very often satisfied. One can even imagine that the code is such that the FADEC has been proved to have this property when the code retains its integrity and has executed properly (two big assumptions).

(b) these parameters will never line up in an acceptable range when the FADEC [fails in some way].

An enormous amount of effort goes into attempting to demonstrate properties such as this. In practical terms, for real code of size about 100,000 LOC, one cannot do it, although we are getting closer than we were twenty years ago.

(c) thus the FADEC looks to its other channel for help

You hope. Or maybe not, because it doesn't "know" anything is wrong. The two channels may even "believe" they agree with each other, even though both have different real values.

(d) if that is also screwed up, the FADEC drops off-line and you have reduced to fail-stop.

Except when the FADEC "believes" (wrongly) that it is correct and functioning and agreeing, and continues on as before.

I'll make you an offer. Let us carry on with this division of failure into steps. You make suggestions (such as above; four steps); I indicate the difficulties with those steps. You then address these difficulties; I indicate further difficulties with your solutions. You then ..... and so on. Let us suppose we stop after, say, 1000 pages. I have at that point no further difficulties to bring up.

I sell this document to an engine manufacturer. If it really does what it seems to do, I imagine it would be worth something in the order of an eight-figure sum. After all, that is only what an engine costs. You can have 90% of it (you will have done the real work; I will only have run interference).

Will you go for it?

PBL

[lomapaseo]

I sell this document to an engine manufacturer. If it really does what it seems to do, I imagine it would be worth something in the order of an eight-figure sum. After all, that is only what an engine costs. You can have 90% of it (you will have done the real work; I will only have run interference).

Will you go for it?

PBL

Of course I wouldn't pay for it if I can already convince the regulators that what I already offer is extremely improbable to screw up.

The numbers don't lie :)

[PBL]

Of course I wouldn't pay for it if I can already convince the regulators that what I already offer is extremely improbable to screw up.

The numbers don't lie

Two points.

First, no one has "convinced the regulators" with any numbers that FADEC SW is "extremely improbable to screw up". "Extremely improbable" is a technical term in certification and is taken to be a probability less than or equal to 10^(-9) per operational hour. The highest you can validate through testing, practically is O(10^(-5)) per op hour, four orders of magnitude too coarse.

This is of course known (although not as well known as it should be). The reliability requirements in certification only apply to HW. DO178B is a "process standard", not a "product standard". It doesn't specify quality of product, but only what steps shall be taken during a development process. It doesn't look as if DO178C will be much different in this regard.

The issue of how one can validate HW-run-by-SW to a high-reliability level when one does not - cannot - validate the SW to anything like that level is a contradiction which people working in the industry just have to ignore. There are even quite a few people working actively on the standards who don't understand the contradiction, let alone take it seriously. It's quite a problem.

Second, you missed my point. Developing a verification suite for any piece of used SW of that size is the Holy Grail for the critical-SW industry, and that is what I described. There is no doubt that a major engine manufacturer would be prepared to fork out such sums of money for a guaranteed solution to the SW verification problem. They have already forked out many times this amount in very partial work along these lines which they believe is somewhat helpful.

But solving the SW verification problem won't solve everything. It won't solve the Byzantine problems described by Driscoll et al. Things won't be perfect, even if the code is guaranteed to be so semantically.

PBL

[Checkboard]

Hmmm Hardware run by software. To illustrate the point about unpredictable errors:

On November 3rd, 1973 National Airlines DC-10-10 N60NA "Barbara" was flying from San Francisco to Miami. The experienced crew began discussing how the auto-throttle would respond if the circuit breakers for the N1 Tachometers were pulled. They decided to try it, and see what happened.

They pulled the circuit breakers on the N1 tachs for all three engines while in auto-throttle speed mode. Nothing happened.

They then commanded a 5 kt speed reduction, and the throttles reduced slightly, and that was all.

They then disconnected the auto-throttle, in preparation for resetting the circuit breakers, disconnected the auto-throttle and reset the circuit breakers.

On doing this, the right hand engine underwent an uncontrolled overspeed, disintegrating the fan, removing the entire right cowl, severing the hydraulics, electrics and control cables, and penetrating the right hand fuel tank. The debris also impacted the fuselage and the centre engine, punctured the 6th window forward of the S3 door and the passenger seated next to this window was ejected in the subsequent explosive decompression. :eek: :sad:

[lomapaseo]

There are even quite a few people working actively on the standards who don't understand the contradiction, let alone take it seriously. It's quite a problem.


I see that even the use of smileys in my posts don't show up in quotes :)

Actually the certification standards take consideration of past historical experience in rates.

When new technology comes along like FADECS Then it is reviewed for its likley effect both good and bad and the regulation modified accordingly. Areas of interest in modification are those that are expected to be no better than 10-7 in freedom from risk of catastrophe. Other areas of postulated risk in your fail soft scenarios don't typically show up in the regulated basis. Else the regulatory change process would never get a modification updated in our lifetime.

[PBL]

I see that even the use of smileys in my posts don't show up in quotes :)

Sorry, and thanks for the heads-up. It didn't get picked up by my editor, and neither did this one, so I added it by hand.

Your comment is apt, but I can't tell if it is meant to be an addition to the comment of mine you quoted, or whether you are expressing mild scepticism. I'll just say that I am actively involved in certification standards for critical SW and my quoted comment states a problem I have been tasked to rectify in guidelines for the general standard, for its German application. I hope my colleagues will have comparative success with DO178 also.

PBL

[lomapaseo]

Originally Posted by lomapaseo
I see that even the use of smileys in my posts don't show up in quotes

Sorry, and thanks for the heads-up. It didn't get picked up by my editor, and neither did this one, so I added it by hand.

Your comment is apt, but I can't tell if it is meant to be an addition to the comment of mine you quoted, or whether you are expressing mild scepticism. I'll just say that I am actively involved in certification standards for critical SW and my quoted comment states a problem I have been tasked to rectify in guidelines for the general standard, for its German application. I hope my colleagues will have comparative success with DO178 also.

PBL

Fair enough on the smiley not remaining through a quote (damn software glitch) :)

I tend not to use a lot of words but the smiley was only intended to indicate a devlil's avocate approach for balance to the discussion. The same with my follow up comments.

I also have spent countless hours, days, years in drafting updated regulations, In the end all gets considered, but the time spent in consideration is balanced against about where we think it contributes in overall risk. In spite of this attempt at prioritization it takes years to achieve any kind of consensus.

Hopefully somewhere in all the posts above the orginal question has garnered enough from the discussion to be answered.

[PBL]

Not showing smileys is an artifact of the rendering. The usual characters expressing smileys are suppressed, including their spacing, and a glyph is substituted. Putting the cursor around the glyph doesn't pick up the original (invisible) characters, so they don't get pasted into the quote. And the glyph doesn't get picked up because it is not a character. I don't understand the phenomenon completely but it is close to unavoidable, I would think.

PBL

[Elvis Priestley]

I saw this failure , Fadec A+B fault when I was positioning a while back .
First I heard a loud banging from the engine and the Captain called me into the F/D to give my thoughts .
ECAM keep pinging Fadec A , Fadec B , Fadec A+B ...it was abit confused .
All parameters for the engine were spurious but thrust symmetric and constant as confirmed by Flight controls page , and responded normally to TL's .

[CHAPARRAL]

Some years ago a captain performed a cautionary in flight shut down (IFSD) of an engine on a 320, in result of an ENG 1+2 FADEC FAULT spurious message.

As a fleet manager, I asked him why did he do that, instead of operating normally or at least kept the engine running at idle, as apparently, full engine control was available.

The reply was: “could not be sure of engine condition”.

To me, shutting down an engine when not called by ECAM or QRH procedure, must be based on very solid grounds, and substantiated on erratic or loss of governing capability.

I’ve always interpreted the “lack” of precise procedural action on this case, related to the multitude of possibilities left over for the extend possibilities of the failure itself.

So, to resume my point: if the engine speed can be commanded and no erratic behaviour is present, continue normal operation. If not, evaluate the possibility to operate at idle, if any abnormal is noted, then perform the precautionary IFST,in accordance with the manufactor procedure.

Cheers, C.