Dealing with Difficult Designers

[Shawn Coyle]

I'm sure this is something that has happened to everyone in flight testing - dealing with designers or other engineers or managers who don't want to hear 'bad' news.
We get enough trouble in flight test by putting pet theories or designs up against reality, but how have you handled it?
I'm trying to find some advice to give to those coming into the flight test business so that they can learn to be both diplomatic and effective in making sure the message gets accepted.
Any examples (suitably sanitized) would be most welcome.

[Straight Up]

In my experience (only a few years so far), the designers and management who are getting the bad news don't like it (obviously). I find that if I listen to them, let them get it off their chest, then point out 'that the data gathered is what it is, and yes the test was conducted correctly, so what can we do to improve the item?'

This normally takes a couple of goes to get the info into them. The hardest part is convincing them that the test was done correctly (lots of parameters recorded and compared to the intended test conditions usually does it).

In the case of crew evaluations (HMI etc), the designer has to be convinced that the crew know what they are talking about, which can be quite difficult. Sometimes it even pays to get the designer to the aircraft, dress him up (gloves, helmet etc) and get him to try and use the items in question (not always practical).

I would say the best way to deliver the bad news is clearly, concisely (as with any data), have enough facts/data to back you up, and, if possible, try to think of ways to improve the bad news. Ideas for the redesign, relocation or further testing etc to try to end on a positive note and get them thinking positively (and save time), not all black and negative.

Even so, there are some designers/managers who will over ride us, and say 'too late to redesign, too costly' etc. Then the customer comes along and says the same thing and suddenly they all start panicking like it was a new thing. I've tried to get the info across, which I think is part of my job, but some people just don't listen.

Above all, always remain calm and professional, designers don't like bad news about 'their baby' (who does), and can sometimes get a little arsey, try not to get aggravated when someone looks like they're doubting your work/abilities, they're just looking for an excuse to save time/money/their jobs.

Patience and perseverance usually work for me.


Overall, I prefer to work with computers than managers/designers, as you only have to punch the information into a computer once. (though, as said above, try not to punch the designer!)

[Mad (Flt) Scientist]

Try not to forget, however, that infallability is not granted to pilots and FTEs either; nothing will get the engineering organisation determined to override FT input than a test organisation which thinks it can do everyone else's jobs better than they can.

The key is to work as a team, not as FT vs design/engineering. Everyone - even project management (blasphemy :)) has something they can contribute.

[Genghis the Engineer]

Without doubt conflict is avoided best by all concerned feeling that they are part of the same team. I recall working with a design team here in the UK where they and the FT team were deliberately sitting down and debriefing everything at the end of each days testing - usually in a quite informal environment. The result was that everybody felt very much to be part of the same team and all recommendations and technical discussions were very much directed at a combined desire to get the product absolutely right.

Less satisfactory is where the design and assessment teams are geographically separate (sometimes in different countries) and this sort of close relationship becomes very hard. In that case, what becomes important is that all negative findings are kept as private as possible. If deficiencies are not in the public domain, then ego and face saving is not too much of an issue. Often then it is possible to become public about the problems after they've been solved, because everybody involved feels good about being part of producing a good aircraft - if problems become too public before they're solved people start digging in and a solution acceptable to everybody becomes increasingly hard to solve.

Where it really goes wrong is where the project management system doesn't allow close liaison with the design team. Some years ago I was busy testing a British training aircraft with an American engine. We found a quite serious problem with the engine installation. The system we were bound by required me to report my problems to my management, they then reported them to project management office on another site, they then went via a regional office in the US and finally the American engine manufacturer was informed. Unsurprisingly, the Americans dag their heels in and denied there was any problem - with half the industry knowing about the problem before them one can hardly blame them. It got solved eventually, but took months longer than it should have done and caused a fair bit of ill-will and unnecessary 3rd party testing in between.

So yes, even project management have a part to play - in large part by ensuring that FT and design are talking regularly without delays or political filtering of the information. It's hard for a project manager to grit their teeth and realise that the best way to manage is by allowing relevant experts to communicate directly and monitor what they are doing, rather than trying to act as intermediaries - but that is the best way particularly in this particularly thorny scenario.

Finally there is a tendency amongst flight testers towards a rather arrogant "we are the experts and know best" attitude. When dealing with some project managers this is understandable (it does in some organisations tend to be a "Peter Principle" sideways move position). However, we must be absolutely prepared to justify, in whatever terms our audience needs, all of our conclusions and recommendations. Practices such as omitting data that contradicts the conclusions, changing graph scales to make a case seem stronger than it is, trying to blinding an audience with science, have no place - but I'm willing to bet we can all think of instances of any of them.

G

[John Farley]

Straight Up

Well said sir. I completely agree with your post.

A test pilot is a critic and critics are seldom popular. So your techniques are essential to getting the job done.

But, on the other hand we must not forget that test flying for a good design team that has the right culture is about as good as anything can get.

Regards

John

[Weight and Balance]

An interesting topic Shawn, with some interesting replies. Here's what I can add from personal experience with delivering bad news.

You should expect to have your results questioned, and rightly so. Don't take it personally, and be prepared to back up your statements. If you can't back them up, and you're not ready to answer questions about test techniques and data reduction and accuracy, maybe it's not yet time to speak up.

I've found it helps greatly to give "good news and bad news". For example, tell the team we have a problem here, but we can fix it by .... Don't get hung up on using your solution, the idea is to get everybody thinking about solutions.

Taking the boss for a ride can work wonders, but it can be a little scary too. Many years ago while testing engine oil system mods at the North American facility of a European manufacturer, we encountered compressor stall at max certified ceiling. The highest ranking European engineer on site was highly critical and skeptical. He stated that we weren't supposed to be looking for compressor stall (true), the Europeans had never had compressor stall (not true, we later found out), and we Canucks probably didn't even know what compressor stall was (that hurt).

At that point we convinced the old guy to suit up and ride with us to the very repeatable test point. We got both engines banging quite loudly. I really thought the European was going to have a heart attack. He looked distinctly ill all the way back down. However, he listened to us from that point on.

[djpil]

Genghis hit it on the head with this statement:
Where it really goes wrong is where the project management system doesn't allow close liaison with the design team If that's not set up properly then personality and ego clashes inevitably follow.
From the engineering side we note the old joke about fighter pilots' ego - enhanced at test pilot school from what I've seen in a number of cases.
The last major military project I was involved in had the flight test and project management at loggerheads well before the prototype got on its wheels (which was when the project was cancelled).
I've had more success with smaller civil programmes and worked with a number of well known (and lesser known) TP's in the UK and USA - all marvellous to work with - they understood what it was to work in a team. In Australia, I've experienced mixed success - had a lot to do with the trust that the test crew held in the engineering group at the time. If I start on the Nomad project this could be a long story but some useful lessons amongst it.

[Genghis the Engineer]

One thing troubles me a little about some answers above. It shouldn't, in my opinion, be the FT department's job to go into too much detail in suggesting fixes. Inevitably we'll be asked what we think might work, and controlled "suggesting" is entirely in order - but flight testers shouldn't get too deeply into redesign work otherwise we'll subsequently find ourselves in the difficult position of assessing our own design work.

Whatever else I've said previously, it does remain our job to be independent assessors.

G

[Straight Up]

Genghis,

I entirely agree, I only make an initial suggestion (if I think of one when testing) when delivering the bad news, then leave the designers to get on with it (unless they have any questions etc). I try to keep a balance between the ‘tester separation’ and the ‘Teamwork’ ideas, as I don’t want to get to estranged from the rest of the process, but don’t want to ‘contaminate’ the test side either.

In my previous incarnation (avionics integration and test), my team was merged with the design team and put under one manager, with idea being that everyone gets to do a bit of everything (I think the designers were bored rigid with their part). We pointed out the flaw in this idea, but were merged anyway, luckily everyone kept doing what they had been before anyway (but we had the same manager and were co-located, which was a good thing), so there wasn’t a problem.

I would also like to stress (as others have before me) that teamwork is essential, my original post probably didn’t mention this too strongly (concentrating on the ‘delivering bad news’ part), and comes across a bit ‘them and us’. It can be difficult, especially with large firms being spread out, or two or more firms involved in the project. Currently I am in my office, which is handily located over 800 km from the aircraft, which needs an hour or so flight and a 2.5 hour drive to get to. Previously I worked on a project where the two firms were in different countries, which added the extra obstacles of differing outlook, culture and, of course, language. However, all of these things can be overcome (or minimised) with a bit of effort, and, of course, teamwork.

[Shawn Coyle]

The discussion so far has been most helpful.
I'm considering putting this into our syllabus to help those just starting in flight test to avoid problems we've all faced.
As far as suggesting fixes goes, that was one of the things I was taught at ETPS many years ago - don't tell them how to fix the problem exactly - tell them what you want as an outcome and see what they come up with.
Teamwork and a clearly defined view of where the project is going is essential.

[John Farley]

Mmmmm. I have a bit of trouble with the FTE should not design a fix (if he/she can)

I think the case that G is talking may (possibly) apply to a certification FTE but there are plenty of other FTEs working for firms

Think CRM chaps. Company resource management. The biggest and best fix to the original RAF Hawk T1 was designed by an FTE. They even called it Fred’s back end because that was his name.

If in a company the FTE voice is not heard at all levels then I detect the presence of an undesirable company culture

[Genghis the Engineer]

As an FTE I have designed quite a number of fixes, and in one recent case quite a substantial portion of an aeroplane. It doesn't present too big a problem IF you ensure that somebody else is there to oversee what you're doing. The problem is, particularly as a certification FTE, if you designed the fix you can get too close to the solution and subconciously refuse to see it's own faults.

So it really comes down to good practice. If, as the certification FTE who also happens to be the design signatory (which I often am) I were to design and approve something without somebody else having sight and veto over what I've done then frankly I deserve to be shot. Sometimes as JF says I do find myself in the best place to design the fix and have been seen sat in the middle of an aircraft factory with a drawing board. But, it is absolutely vital that I then ensure somebody competent to do so has overseen my own solution.

G

[Mad (Flt) Scientist]

JF & G

One of the problems in a large organisation when the Flight test organisation (not just FTEs, pilots and others too) starts to "design fixes" is that the Design part of the organisation may be both offended and/or feel threatened. That will often be followed by a lock-down on communications, for fear of the "other side" stealing away "our job" - even if it's not a loss of employment issue, people get very attached to their jobs and for someone else to start doing it causes a lot of grief.

It gets worse when the FT organisation is designing fixes for things which are not, in the design organisation's mind, a problem. The course of action should be to prove the problem, not to say "oh, ok, we'll fix it then".

And before I'm accused of bias, let me say that the other side of the coin, which is basically design picking and choosing the test results it wants and basically circumventing the testing process thereby is just as bad. That way lies the FT organisation jealously guarding all the data, for fear someone will look at it "the wrong way".

[ICT_SLB]

Another, more recent, developement that compounds the problem is that often nowadays design and flight test are not in the same country let alone airfield. This can mean that not only is everything is filtered down a phone line - more possibility of misunderstanding and getting things wrong - but another group gets in on the process.
I find an increasing amount of my time is taken up explaining to a Project Engineer (a) the testing required (b) what is needed to accomplish that efficiently and finally (c) what the results mean & what we need to do about them. Good or bad Project guys can make or break programs and some of the arbitrary decisions made by them surpass all understanding.

[idle stop]

I'm very much with JF on his idea of Company Resourse Management. Design and Flight Test must work together from the outset of a project. Such a doctrine should be a culture and not merely enshrined in company procedures. As an aside, I remember a case when Design and Commercial departments didn't think to ask Flight Test whether certain modifications would require Flight Test work. Result, when surprise, surprise, regulatory authority says 'What about the test flying?': inadequate price quoted/contracted and thus significant loss of profit margin.
I agree the problems when there is significant geographical separation: not easy to pursue a working relationship of mutual trustful harmony.
With as expensive a business as ours, we should not have time for petty jealousies: it matters not who comes up with the fix, if it works to prevent costly additional resources and time. That way we should keep the CEO/Shareholders and, most importantly, our customers, happy.

[VP959]

To just add another flavour to this discussion, which I suspect will gel with most of you, why not persuade your organisation to adopt an Integrated Test, Evaluation and Acceptance strategy?

I am doing just this on my current project (and yes, I am an ex-FTE, but currently a project manager.....). The result is that we have the designers, end users, TP's and FTE's all working together throughout the test and acceptance phase, from the initial static tests on structure through to the final flight tests. Problem fixes will result from the concerted efforts of all, including, most importantly, the end users of the platform.