Chinook - Still Hitting Back 3 (Merged)

[Squidlord]

Airborne Aircrew wrote:

Code is tested, retested ad infinitum and released. After release, it is tested in the real world. That's where the things programmers can't see, test for, think of come to light. Then amendments, (updates), are made to the original code and disseminated. Until that process has occurred through numerous cycles it cannot be considered "safe".

I think that's a very odd characterisation of how "safe" code is arrived at. In particular, it seems to imply that the software that is first flown can not be safe until it has been flown for long enough to allow all the amendments. Bit of a bummer for the poor sods who have to fly the software in the early days! It's also peculiar to compare with Windows since it is generally considered to be unsuitable for use in a safety-related context. Indeed the constant amendments to it are part of the problem!

I have read several times in this thread that the FADEC code is "un-auditable". There are just two reasons why the code could be in such a state:-

1. The company/contractor who wrote the initial code doesn't have/will not release the source code. [...]

This is all too common, at least regarding release, and still happens. You might say that MoD should never let a contract without ensuring access to any safety-related source code and I wouldn't necessarily disagree but it does still happen (sometimes, the source code is profoundly difficult to get hold of due to, e.g., ITAR regulation).

I agree with Thor Nogson that ZH875 gave a plausible description of the situation with the FADEC software (though it does seem that Boscombe actually made a positive assertion that the software was unsafe). I think ZH875 is just describing Airborne Aircrew's fourth option:

4. Those charged with auditing the code find such an unholy mess they throw up their hands and give up.

I have seen lots of examples of this. I.e., appallingly written safety-related code that is just a spaghetti morass that is essentially unanalysable and untestable. I remember one particular awful example where, working from the spec., I could see thirty (it might have been one or two more, I don't remember) scenarios that needed to be dealt with. So the spec. could have been implemented as a thirty-way case statement and would have been very easy to comprehend accordingly. Instead, the code was a mass of nested and overlapping "if" statements that had well over 60 million distinct paths. Completely impossible to verify by analysis or test.

Thor Nogsen:

It is possible however, to prove that [software] is unsafe - If the code doesn't match the specification for example.

That would prove the software is unreliable, not that it is unsafe. It also requires that the spec. exists and is appropriate and complete. So, now's a good time to mention that I have also seen lots of safety-related software variously:

1. without any sort of decent specification
2. with a specification that has clearly been reverse-engineered from the software
3. with a blatently wrong specification

For 1., there is essentially no spec. to compare the software to, for 2., showing the software meets the spec. is kinda redundant and for 3. you'd rather the software didn't meet the spec!

tucumseh says that source code access was catered for explicitly in MoD regulations in the early-mid 90s. I'm a bit surprised to hear that. It was a bugger's job getting hold of the source code for C-130J in the mid-90s and it would have been much easier if it had been contracted in the way tucumseh says regulation demanded. Perhaps the regulation was there but routinely ignored. There's no such regulation these days.

chinook240 wrote:

Approx 221,000 flying hrs later (average AFT OF 13,000 X 17 years) the code has not been rewritten. Is it still grossly unairworthy

I'm not sure what chinook240 is getting at since (s)he seems to concede the code may have been amended (presumably in response to some of the deficiencies identified in it). It's perfectly possible to turn unairworthy (unsafe) code into airworthy (safe) code by amending it to remove deficiencies.

I'll also note that 221,000 problem-free flying hours is not enough to demonstrate the airworthiness (safety) of code as critical as the Chinook FADEC software (in the absence of any other forms of evidence). You'd want an absolute minimum of a million hours and very probably a lot more. (It would depend on how "safe" it had to be and how much confidence in its safety you required).

chinook240 again:

As simple aircrew, the idea that understanding the design can change the status of an aircraft from unairworthy to airworthy seems counter-intuitive. Or am I confusing the terms 'airworthy' and 'safe'?

Maybe. I think airworthiness is an inherent property of an aircraft because it is concerned with the potential for it to be operated safely. Safety is a property of an aircraft (or some other equipment, system, etc.) in the context of how it is used and maintained. An aircraft can be airworthy but operated unsafely because, for example, it is flown upside-down or flown by untrained aircrew or ...

I don't think airworthiness or safty changes just because one understands the aircraft design. But what will almost certainly change drastically is the ability to demonstrate airworthiness. It is not enough for an aircraft to be airworthy (or safe, for that matter). It must be demonstrated airworthy (or safe, for that matter). And if the FADEC software was unverifiable by Boscombe because they did not understand it and/or the context of its use in the aircraft, it is conceivable it was airworthy (safe) but without their verification evidence no one could (legitimately) claim to have demonstrated it airworthy (or safe).

So when chinook240 writes:

My question is simple, the code hasn't been rewritten, if it was grossly unairworthy then, surely it must be now, you can't have it both ways.

If the code genuinely has not been amended in any way then (imo):

1. it's airworthiness status can not have changed.
2. it's safety status may have changed drasticly becasue it may be used in a different way (chinook240 alludes to changed procedures in another post).
3. the ability to demonstrate the software airworthy or safe may have changed drasticly (when tucumseh writes, "it can be seen that with no further modifications it may have become airworthy, as understanding matured", I disagree but I do think that improved understanding could well have allowed demonstration of airworthiness).

(Oh, and strictly, I disagree with Chugalug2 when (s)he suggests that if regulation is not followed, it necessarily means the software is unairworthy. However, I do believe that if regulation is not followed then it is very unlikely that the software could be satsifactorily demonstrated airworthy (as required) - a subtle distinction.)