YYZjim, re the sister ship ADIRU mishap. Investigation: 200503722 - In-flight upset; Boeing 777-200, 9M-MRG, 240 km NW Perth, WA

First off, terrific post. I'm struggling to fathom how this is the first time this incident has been mentioned. Maybe it isn't. It seems a number of parallels can be already drawn between the 2 flights. Could someone with more knowledge than me provide an answer as to whether or not a faulty ADIRU could correspond to ACARS + transponder being disabled?

Well considering it took place in 2005:
One would hope, being as there was an AD, that the present day software version would preclude a repeat, especially as the has not been another event in 9 years.

@ampclamp
wxcept posibly as another comment on how useless the CVR is as it only had 5 minutes of relevent information because it had not been switched off on the ground.

If Boeing thought that this incident was similar in nature and they had no answers then the 777 fleet would be grounded ( a situation which very nearly occurred after the 2005 incident). The fact that Boeing have not issued any AD's to operators (to my knowledge) suggests that they are not concerned that the aircraft has an inherent fault that could cause another 777 to disappear. The crew in the 2005 incident were able to override the automatics and recover the aircraft. For something similar to have occurred there would have to be another undetected software failure followed by a double incapacitation. Something which IMHO would be an order of magnitude beyond 10-9.

Mistakeology
 

Logic errors can remain undetected in programmed systems for a long time.

The protocol of flying on with 'redundant' units defective is such a 'program' that by definition does not create an accident but equally obviously does erode safety margins.

When it is the integrity of the 'intelligence' between pilot and aircraft that is jeopardised by such a program it then puts at risk the strategy of having a human in ultimate control.

Programmer humility deficiency might be a common root cause.

A week to finish current search?
 

A report here that the current search will take 5-7 days to complete if the weather and the bluefin 21 holds up.

Malaysia Airlines MH370: Underwater search at 'very critical juncture', could be completed this week - ABC News (Australian Broadcasting Corporation)

I know that there is a wish to find an answer but this is not it.

Logic errors can remain undetected - but this one was detected the quotes are from an investigation into an event that was caused and an AD was very publicly issued to return to the previous version of the software.

So now are you really suggesting that Honeywell, having been told of the fault in their software in unequivocal terms, forgot about it? Then over the 9 years since the incident that they have not updated the ADIRU software to fix the fault? To use a quote from tennis - You cannot be serious.

And of course this ADIRU software fault would need to also disconnect ACARS and switch off all three redundant VHF radios incapacitate the crew and then recover itself and fly the aircraft in uneventful cruise to the southern Indian Ocean.

Perhaps you would like to revisit your logic?

Mistakeology
 

Your post kindly emphasised 'was' making the point that the logic error has been detected,

My point is the logic error of flying on with a tolerated defect in a system with the danger that a second defect could mislead the pilot is a critical vulnerability.

The vulnerability does not go away now that this one has been detected.

I tend to agree with this, but I have to admit that my first thought upon learning of this incident was ADIRU failure and/or an EE bay fire. The latter still explains everything known about the incident except for one important issue -- how the plane wound up headed in the general direction of Perth and the supposed track around Indonesia (still not entirely convinced of that as established fact given the source).

As for Boeing, in the absence of evidence that there is a fault in the aircraft (and theories aren't evidence), there are ample economic and liability/legal reasons to do nothing unless/until concrete evidence of a fault is discovered. Grounding the 777 fleet would be an enormous hardship for a number of carriers for which this aircraft type is the backbone of their long-haul intercontinental fleets, a group that includes the three legacy US carriers.

You don't ground a fleet of aircraft in the absence of specific evidence of a design problem. Prior groundings such as the Comet I (c. 1952), Lockheed Electra (c. 1959), the DC-10 (1979) and 787 were based on physical evidence of a potentially catastrophic problem with the aircraft. In this case, such physical evidence is, to date, completely lacking.

You have obviously not worked developing safety critical software.

The software in the ADIRU is not developed as if it were a video game or a university project: it is developed in line with RTCA DO-178 and ARINC 653. These are very strict standards with a lot of testing. However, despite all the testing some faults may/will be found and in most cases the system is designed that a fault in one module will be contained as part of a Failure Mode Effects Analysis. It would appear that a fault was successfully contained and then unmasked when another module was updated.

Now at that stage with safety critical software the FAA and Honeywell reverted back to the previous version - which had worked without a problem using an AD. Honeywell would then have had a 'MUST FIX' top emergency software fix to carry out. In many organizations that means NO new software version can be delivered unless that fault is fixed.

Your attitude that they would have left it on the old version as that was 'good enough' is just not the way the industry works.

I would expect that the fault was fixed within days and then after recertification testing with the FAA and Boeing, Honeywell would have delivered a new ADIRU software build with all known bugs including this one fixed. The longest part of that effort will have been testing, and the particular issue that caused the ADIRU to fail would be included in the new acceptance test suite. Almost certainly there would also have been some effort to defend against ADIRU faults in the FMC software as part of the FMEA work.

High availability safety critical software development demands getting things right, designing systems to be resilient to subsystem faults, and rapid resolution of any faults found.

@Ian W,
Unfortunately the best testing can only test for what you are looking for.
Test scripts will only be based on what are considered possible scenarios.
Furthermore, the IT industry is now full of people who have been taught to program, not learnt to program.
One would like to think an operating system like windows would be fully tested, yet a host of bug-fixes are released every month, for years.

CVR question
 

One of the many frustrating and ironic twists in this baffling accident is that, if and when the CVR is finally found (and recovered), it will in all probability, just contain two hours of silence - because only the last two hours of cockpit audio is retained on the CVR. All preceding audio is over-written thus denying investigators arguably their best clues to this bizarre tragedy.

My question is this - How complete is that over-writing?

We all know that deleted data from a PC hard drive can still be read - if you know the right (maybe that should be 'wrong') people. We have already seen this demonstrated with regard to the Captain's flight sim. data.

Could this also be made easier by being over-written by continuous silence? Could there be recoverable 'soft' data under that pure white over-write? Or are there some subtle technical differences between delete, erase and over-write?

Sudden climb may cause loss of windscreen
 

I have speculated that the ADIRU failure experienced by 9M-MRG, which led to a sudden climb, was experienced by 9M-MRO (on route MH370) on March 8, 2014. The sudden increase in the pressure differential across the hull may have led to another problem recently experienced by two other B777-200s.

On April 13, 2012, an Alitalia B777-200 (EI-ISB on route AZ-8320) flying from Rome to Dubai at FL370 declared an emergency near Athens. The first officer's windscreen had cracked. The crew descended rapidly to 6000 feet and diverted to Athens.

On July 3, 2012, an Air France B777-200 (F-GSPL on route AF-85) flying from San Francisco to Paris at FL370 declared an emergency over Hudson's Bay. The windscreen had cracked and the crew reported problems maintaining pressurization in the cabin. The crew descended to 10,000 feet and diverted to Montreal.

All three aircraft are of pretty much the same vintage:
Alitalia EI-ISB first flight December 18, 2002
Air France F-GPSL first flight June 12, 2000
Malaysian 9M-MRO first flight on May 14, 2002

One would need to know the number of cycles, rather than simply the calendar age, to determine if windscreen problems are fatigue-related, and possibly represent a systemic problem which is just now coming to light in the B777 fleet.

Mistakeology
 

My concern is a systems concern,

The software engineer works in an environment that makes assumptions about its upstream inputs (eg, a sensor might fail - et al,) and downstream consequences.

Within that is the acknowledgement that the downstream resources (eg. fault condition SOPs - et al.) cannot cater for all eventualities and so must rely on pilot professional competence & expertise.

The assumptions (eg middle value is safe - et al.) exploiting multiple redundancy can render the remaining two of three working transducers worse than a singleto, since failure of either would give an erroneoous result - the Australian 777 episode exemplified this,

Iin that case, flying with a faulty third channel was worse than the system having no redundancy, Has this now been built into all triple redundancy middle value systems?

In both that case and the ill fated Air France episode, incorrect transducer readings were not sufficiently visible to the pilots - the last resort safety sytem - for it to be obvious to them just what was happening. Even the last resort 'hand fly the beast' option has to be negotiatedwith a software system that is already percieved - at lest partially - as working otherwise than as intended,

It is the combination of reliance on the pilot and being unable to guarantee to present the information the pilot needs that give the total system a level of vulnerability that can make a safely redundant system dangerous in the presence of a known failure.

An MEL that says a defective component can be tolerated must demonstrate a safe system (including a suitably informed pilot) in the event of ANY subsequent failure.

And when the statisticians do their sums, making standard 'independence' assumptions, they must be obsessive about them, as must everybody from people buying components to the authors of safety procedures,

Windscreen failure.

Accepted. At FIR handover ? Divert then to Penang.

Northern route ? Then change to Southern route at 180/182 ?
We forget "no trans", "no acars/satcom".

Actually they were, the rest of your stuff is so convolutely written, it is impossible to follow.

A small point of order
 

"Prior groundings such as the Comet I (c. 1952), Lockheed Electra (c. 1959), the DC-10 (1979) and 787 were based on physical evidence of a potentially catastrophic problem with the aircraft. In this case, such physical evidence is, to date, completely lacking."

The L-188 Electra fleet was never grounded. Restrictions on max IAS were imposed until the flutter problem was worked out, but unlike the DC-10s and Comets, they continued to operate. That was a decision based on the economics of an airline whose sole equipment was the L-188, and also because no-one knew how the two mid-air breakups they suffered had related causes.

Green

I think part of the reason it is "boring" is the JACC / Angus Houston and others are not prone or see any need to having unwarranted news conferences and have succeeded in damping down media speculation.

And apart from the basic info on the search each day, not much else.

That will of course all change when they find something !

holdatcharlie asks the CVR question.
Question asked and answered much earlier in the thread. The answer (as I recall) was that analogye CVRs might have some trace left - although two hours of continues erase means probably not. However, the CVR unit on this aircraft was digital so the answer is Zero. (I sit to be corrected).

Ocean Shield - SSS search
 

AMSA have recently defined the search area as within a circle of 10km radius centered on the 2nd ping detection. That area is defined by the White circle on the graphic below. The rectangular area on a major axis of 030°/210° represents what previous AIS position reports indicate the area the Bluefin 21 AUV is working in. Each days positions are a different color, and the first position for today (20/04:03z) is shown in a Cerise color.

The Red stars represent when pings were acquired, and the sequence is numbered anticlockwise starting at the top. The White star is the position that the pings associated with the first ping were lost (LOS).

http://oi57.tinypic.com/2uo2eds.jpg

NOTE: Use Ctrl+ as many times as needed to enlarge the image, and Ctrl 0 will return the page size to normal.

The speculation about possible ADIRU software and windscreen problems being an issue in this case begs the question:

<< IF >> one or both of these had been a contributory factor, and assuming substantial recovery of wreckage, would the investigators have any chance of finding evidence of it? Particularly as a windscreen cracked by altitude/pressure changes might resemble one cracked on contact with water.

Just on the issue of windshield failure, there is an interesting digression on the manufacture of the above, in the TSB report pertaining to


AIR FRANCE
BOEING 777-228ER F-GSPZ
CHURCHILL, MANITOBA 290 nm NE
17 OCTOBER 2002


where an arc resulting from overheating in the J5 terminal caused a small fire and the windshield to crack.


I won't link as I can't get it through moderation,

but anyway comms were not affected (obviously)

so if this were a contributing factor in the present case we would need to find a reason for lack of comms.


ETA (from the report)


'Boeing has undertaken a program to redesign the window terminal block to eliminate the screw connection. The new window blocks were scheduled to be incorporated into Boeing 777 aircraft, Line Number 471 (delivery date February 2004). The new design incorporates a locking pin/socket, which will address issues concerning loose or cross-threaded screws and inset ferrules. All Boeing 747, 757, 767 and 777 windows delivered thereafter, either on new aeroplanes or as spares, will have the new terminals installed. Boeing intends to deliver spares in kit form with the new wire end terminals included. The operator will have to remove the existing wire end terminal and splice in the new one when replacing windows on existing aircraft. The intent is to eliminate concerns with arcing at the window power terminals.


Boeing released a Fleet Team Digest article to B757 operators in May 2003, discussing terminal arcing and overheating. The article detailed actions to incorporate re-designed terminals into the affected cockpit windows.'




I'm not sure how relevant this might be, but it looks like there was no retro-fitting of the new system so as far as I can figure out, 9M-MRO will have had the original sort.

CVR Question
 

Holdatcharlie,

As other people above have stated, when data on a PC or MAC has been deleted, the sectors on which the data resides is only flagged as deleted in the file allocation table. On the old 30 minute analogue tape CVR, it may still have been possible to recover over written audio, because the erasure head isn't always perfectly aligned with the recording head. or may not reset every magnetic fibre on the tape But on modern solid state digital CVR it is impossible because the memory modules only has 30 minutes or 2 hours of storage space depending on which model is on the plane, so the oldest data is constantly being over written by new data. The older data is completely erased.

I hope this helps.

Finding more cabin recordings
 

The hope is on finding more cabin recordings based on passengers' personal devices being used during those dramatic final hours (i.e. smartphones, videocameras, etc.). Their memories might still be readable if sea water has not degraded its components seriously.

Yawn ! :ugh:

You've already been told above not to compare the generic IT industry to the niche part of the IT industry that deals with safety-critical systems.

Windows and such like are not subjected to the same formal design process that safety critical systems are. Thus of course you will get a greater number of bugs, some of which may cause substantial issues to the stability of the system. Bleeding edge generic IT projects these days can use agile development.....no such thing in safety critical systems.... in safety critical systems there is a traceability requirement for you to be able to show that a given line of source code fulfils given requirement specifications, the level of detail is truly excruciatingly tedious... but it all serves a very important purpose !

Sure there's always scope for issues, but the whole point of the almost obsessive-compulsive development methodology for safety critical systems is that failure or bugs won't kill anyone !

On another note....

"Malaysian authorities are considering issuing death certificates for the missing passengers of MH370.
It's part of the plan to provide financial assistance to the families of passengers.
Deputy Foreign Minister Hamzah Zainudin, head of the next of kin committee, said no decision had been made on how much each family might receive".

Er ? The Montreal Convention.
Or is the carrier going to prove that they are totally "without fault".

It's quite a mute point to be fair, because given the advances in technology if you're going to revisit the black box (and I think AF447 and MH370 prove there's at least a need for an updated spec, even if the cost-benefit analysis doesn't include retrofitting) then you'd include enough non-volatile storage per box to take many many hours of recording (along with other things such as an extendable, floating additional antenna).

Oil slick
 

I think I have read every forum page but have heard no news of the analysis of the two oil slicks, did I miss it ?

Review the JACC media release recently. Confirmed oil not from a plane.

It strikes me that these days (a duplicate copy of) the recorded flight data from both black boxes could in fact be stored in something physically tiny that was entirely self contained and external to the plane. There have been demands for a transponder to be carried that cannot be disabled by the crew, under duress or otherwise, so how about a little unit, externally mounted, on the tail cone perhaps, top of the fuselage or tip of the fin. No physical connection to aircraft power or systems, powered by an on board battery, which in turn is charged by a miniature RAM air turbine when the aircraft is moving. Include a transponder that will respond on demand to an internationally standardised IFF style interrogation challenge, or maybe regularly just pings co-ordinates to a satellite. If the same device also contains a non-volatile memory module which is written to from inside the plane via a bluetooth interface from the _real_ black boxes, with a facimile copy of both CVR & FDR data, all in a package designed to survive a water landing and end up on the surface amongst the debris field, if there is one, perhaps with the on-board satellite pinger previously mentioned still operating, then this situation literally can't happen again...

joy ride:-
Without going into the microscopic details, I would reckon a fracture attributed to internal forces (failure with cabin pressurised) and one due to external forces (impact with water) will be pretty evident on close examination.

I really hope I'm wrong but it's not looking good for finding the aircraft in that 10km radius zone. It seems more than 2/3rd has been searched to no avail.

The Malaysians have long labelled the investigation as being a criminal one. One can only assume, it wasn't a knee jerk reaction and that they have solid reason to do so.

"Bluefin-21 has searched approximately 50 per cent of the focused underwater search area to date."
Search and recovery continues for Malaysian flight MH370

The JACC were a little more non committal than that:

"Preliminary analysis of the sample collected by ADV Ocean Shield has confirmed that it is not aircraft engine oil or hydraulic fluid". (source: JACC website)

Howard Hughes,
Are you suggesting that the oil is not totally discounted as being from the aircraft and that it could be from cargo? E.g. an electrical transformer ruptured in the crash?

I notice in the JACC Press release that 11 aircraft are still being used
and this seems to be the case on a daily basis.

Not all the aircraft can drop buoys but are they still searching for debris, pings ????

Any comments ?

Nevertheless, there is a good chance the non-volatile memory would retain its data which could potentially be recovered. If you'd broken your phone by going for a swim you wouldn't bother because the phone is not economically recoverable - but if it's a question of recovering data for a crash investigation that's another matter.

I am just pointing out that they have given themselves a little room to move! :ok:

Latest JACC media release
 

Media Release
21 April 2014—am

Up to 10 military aircraft and 11 ships will assist in today's search for missing Malaysia Airlines flight MH370.

Today the Australian Maritime Safety Authority has planned a visual search area totaling approximately 49,491 square kilometres. The centre of the search area lies approximately 1741 kilometres north west of Perth.

This morning, Bluefin-21 AUV completed mission eight in the underwater search area. Bluefin-21 has searched approximately two thirds of the focused underwater search area to date. No contacts of interest have been found to date.

The focused underwater search area is defined as a circle of 10km radius around the second Towed Pinger Locator detection which occurred on 8 April.

Bluefin-21 AUV's ninth mission will commence later this morning.

The weather forecast for today has conditions deteriorating, particularly in the north of the search area, as Tropical Cyclone Jack continues its track southwards. Wide spread showers are developing with isolated thunderstorms to the north and east south-easterly winds.