Stats, Maths, Probability.

[Oxford1G]

Can any one please help me with the following:

If you carry out a survey, say on 100 pilots, on one fleet who have completed a manual load sheet. Over what length of time would you need to collect the data, to make this an accurate survey? What would be the % error in the results? Could you then apply that same error of probability to another fleet in your airline? Would it be accurate to say for example that if we found a 20% error in loadsheets with one fleet, then the same could be applied to another fleet?

Thanks,

Someone with a project who needs the results yesterday!

[Genghis the Engineer]

Not enough information old chap.

First, are you looking for quantified error (normal or skewed normal distribution) or simple right/wrong (either binomial or more hopefully Poisson distribution).

In reading across, are the levels of pilot qualification and experience the same? Are the load sheets of comparable complexity?

There are standard methods for working out the tolerance of accuracy on your result based upon sample size. I personally haven't done that sort of analysis for 10 years, so honestly can't remember how you do it, but my gut feeling is that if you're looking at a normal distribution, and say 100 pilots x 10 load sheets each, the confidence will be pretty high ON THAT POPULATION. Proving whether you can read across to another population is another thing.

My personal experience is that most days, about 30% of pilots are capable of ******ing up a load sheet, this is one of many reasons why airlines employ dispatchers.

G

[NorthernSky]

You don't say whether you are carrying this survey out for interest or in order to assess a safety hazard.

Might I suggest that if your interest is safety, minor loadsheet errors which cannot affect the safety of the aircraft and cannot affect security adversely, may be reasonably discounted?

For example, bags calculated at 13Kgs on a domestic flight (when the figure should be 11Kgs) will have no impact. Showing all the bags in the forward hold when they were loaded in the rear, may, depending upon the aircraft type (in an ATR-72 you would be in danger of running out of trim).

Correct total numbers of passengers must be important, as this is a security issue, though you must then assess probability of an error occurring on the flight which is subject to a terrorist threat - this will be low, though these days we must dot all our 'i's and cross our 't's.

You may employ a risk matrix (probability versus severity) in order to get a handle on these issues.

If you're doing your survey in the interest os safety, good luck. I hope you're not doing it so that management can call people who have erred into the office for a chat without coffee. If this latter is the case, ask your managers who it was who recruited, trained, and tested these pilots, and who it was who placed them in positions of responsibility...?

[FlyingForFun]

I think the questions which have been asked in answer to your question illustrate why this quote is so true!

FFF
----------

[john_tullamarine]

A lot has to do with the design of a trimsheet system. A well thought out design, with due consideration given to Murphy's Law, can go a long way to reducing the incidence of error. One of the neatest procedures I have come across involves the determination of loading by conventional trimsheet (by a load controller type) with an independent crosscheck using a mechanical trimwheel by the operating crew on the flight deck. This check is very quick to use in practice and reduces the risk of untrapped errors getting through quite dramatically.

Reminds me of an audit in which I was involved some years ago. A quick look at the (FAR25 aircraft) trimsheet indicated that it had been printed incorrectly ... which followed from the original being printed incorrectly in the AFM ... can you believe it ? The sheet could be used .. but with some nuisance difficulty. I took the opportunity to ask (over a coffee) a number of operating pilots how they used the sheet. I can't remember how many chaps were in on the discussion but I got a great many innovative (but all wrong) ways to run the sheet ... that aspect of the audit rapidly turned into a training exercise ....

[twistedenginestarter]

Over what length of time would you need to collect the data, to make this an accurate survey?

You need to choose the test period to gain the most from the survey. Statistics cannot help you here as it simply says "Give me a model of your system and I can tell you the pattern that model will exhibit, and thus I can tell you for example the significance of a slightly differrent pattern". So you need to devise a model which of course must be acceptable to your audience.

Normally you would design sampling to avoid systemmatic error. So if one individual did all the load sheets for one shift then doing one shift would be less predictive of things in general than spreading the readings to cover several shifts. Unless of course you were interested in predicting the performance of that shift, in which case the reverse applies.


What would be the % error in the results?

You are simply categorizing them as right or wrong. So you would first have to check your checking. After that I would take a simplistic view. If you did 1 reading and it was 'wrong' clearly you wouldn't want to say the error was 100%. Similarly if you do 100 and 1 is wrong, you don't want to go for 1%. However if 20 were wrong you would be safe to say 'approx 20%'.


Could you then apply that same error of probability to another fleet in your airline? Would it be accurate to say for example that if we found a 20% error in loadsheets with one fleet, then the same could be applied to another fleet?

Yes - providing you think the same model applies. Most likely it doesn't so you would be best to compare your results with some others. Try a third fleet either within your company or externally

Sorry if this doesn't help but statistics tend to deal with things that vary but conform to a known distribution function. Your problem is you can't see variation (unless you did experiments on lots of fleets) so you can't easily build a model of variability.

[Oktas8]

Speaking of trimsheet design...

The Cessna 172 "S" AFM, as published in 2000 (or maybe 1999) has completely different values for imperial & metric weights on the same chart.

Read down the left hand side with lbs and you get one scale. Read the right hand side of the same graph and you get a completely different scale. One assumes the lbs side is correct, as most American manufacters are still stuck firmly in the 1800's when it comes to measurement units.

Makes me wonder how many other aircraft have similar errors.

[john_tullamarine]

Oktas8,

If you care to scan and email me a copy of the relevant sheet I can sort out the confusion in a minute or so.

[OverRun]

Oxford1G

This might be too late for your project, but here goes:

1. If the pilots are filling in a few loadsheets a day, then 100 pilots * a few * 365 = many load sheets per year. So we can use 'large population' statistics.
2. For an accuracy level of plus or minus 5 percentage points, the number of loadsheets needed to be checked is 384. Survey the 384 load sheets, and (assuming you get the figure of 20% errors), you can then say that you are 95% sure that the true percentage of inaccurate load sheets is between 15% and 25%.
3. For an accuracy level of plus or minus 3 percentage points, the number needed is 1067. You can then say that you are 95% sure that the true percentage of inaccurate load sheets is between 17% and 23%. 2 percentage points is 2401 samples.
4. Go to http://www.geocities.com/profemery/samplesize.htm for further calculations.

It would only be accurate to transfer the results if the other aircraft had very similar loadsheets and computations and pilots. For example, I guess that a Navajo Chieftain and a Cessna 441 would be fairly similar, but a Navajo and a 737NG would be rather different.

[Oxford1G]

Thank you all very much for your reply' s and help. :)

[john_tullamarine]

OverRun,

Curiously, well designed trimsheets for, say, a Navajo and 737, will look remarkably similar and have a remarkably similar method of usage.

Hopefully Steve Emery will condone my pinching the URL reference you provided ... which is now included in our tech log URL sticky thread.

[NorthernSky]

JT,

Shame on you for your use of 'hopefully'!!!

A chink in your armour!!??

All the best,

NS

[john_tullamarine]

... just being my normal polite self ....

[NorthernSky]

JT,

There's no doubt that your politeness is well known and highly regarded!

Forgive my 'picky' attitude, but 'hopefully' is so much misused these days that it has all but lost its correct meaning. It doesn't mean 'I hope that...', but does describe that which is done in a hopeful manner. Thus, someone travelling to a second job interview might do so 'hopefully', but would be wrong to say 'Hopefully, I will get the job'.

'Thankfully' suffers the same trouble, alas!

[john_tullamarine]

ah - etymological exactitude. Cringing and cowering before the stinging torment of my accusers I am charged, convicted, and despatched to my hideous and terrifying fate within the pages of the largest dictionary in the library ......

In one's defence, one could argue that an historically based academic approach to the etymology of this word has been superseded over the past 40-50 years by modern popular usage into which sloth I fell in my previous post.

As many would say, language is the one true democracy (the not altogether successful efforts of l'Alliance Française de Paris in respect of 'French like she should be spoke' notwithstanding) ....

Regretably we must accept that usage changes ....

My own favourite is the historical misuse of 'eating humble pie' and, no doubt, you concur with my view in this matter.

We both would acknowledge outright horror at the failure to use apostrophe when appropriate and the ludicrous frequency with which the device is used when it is quite inappropriate. When combined in the one sentence, typically on advertising billboards (doesn't the advertising industry place any importance on presentation these days ?), the effect on one's mental state is profound ......

Ought I return to the scene of my malfeasance and destroy the evidence of my illiteracy ?



At the end of the day the limerick remains the one and only true artform ........ peace, brother ....

['%MAC']

Wow, I’m intimidated to post anything after that exquisite example of prose.

Back to the thread, if you do this study you’ll need to ensure that your sample is random, and sufficiently robust, as was previously exhausted. You can draw an inference that errors on one type of load manifest will correspond with errors on other equipment, but until you perform a comparative study, you cannot provide a degree of accuracy. If you are an undergraduate, a sampling of load sheets with a chi-square analysis may be all that is required, assuming the data points are all ordinal. If you’re involved in industry, farm it out and go back to the crossword. If you’re a grad student my condolences you’ll be worrying about confounding variables, stratification, reliability, validity, yadda, yadda.

I imagine the most difficult part of this experiment would not be the methodology but having access to the load manifests. Don’t know of a company that would be willing to part with them so quickly, especially if you’re looking for errors.

[john_tullamarine]

Pieces be upon you, my son ....

NorthernSky and I were being quite tongue in cheek ...... pay no attention to our mock prattling ......

[NorthernSky]

I believe I may have met my match... :) :) :)