What's wrong with "Off The Shelf"?
 

The phrase OTS comes up in debates here about all sorts of equipment, very often applied in a derogatory manner. This has always puzzled me. It wouldn't occur to most of us to purchase anything other than OTS; the only person I know who had a car built to spec (a TVR Tuscan) it was a complete disaster and sat immobile under a thick layer of dust for several years. The OTS airliners I've flown for the last twenty something years have all worked fine. The only instance of an "off-piste" spec, for an unusual toilet fit, caused years of problems. Surely there's huge opportunity for big savings. Why is OTS a swear-word in military aviation?

We're SO SPECIAL- and not just Mil types either...
 

An inbred arrogance that says "we are so totally different, unique, and yes SPECIAL". I did an MOD office IT installation project 15 years ago and the level of customisation of standard Microsoft products was eye-watering. And it didn't work, was late, and cost a bomb. No doubt experts will be able to argue otherwise. COTS (commercial off the shelf) is a 4 letter word. Civil ATC was as bad, only now they buy COTS and then modify the hell out of it- making it expensive, unreliable (as it doesn't do what it was designed to do) and unsupportable. The same thinking applies across the board. It's a cultural thing...

Because some stuff such as avionics is especially hardened for military use.

Though the bog valve in the VC-10 was a squash ball that BAe were charging the RAF something like £80 a pop for.

Didn't the Nimrod AEW 3 suffer a similar fate, as there was no fixed price on the contract, so a lot of money was spent trying to compress down the software for it onto the hard drive in the system, where the simple expediency of increasing the size of the drive would have cured it? but of course BAe wouldn't be getting paid then.

Ohh and by the way, not everything is off the shelf in civi street, take the golf shelters at Woodford :E

http://thumbsnap.com/sc/I6sRIft5.jpg

Nimrod AEW was not a viable concept for many reasons - it was bought because the Government of the day could not afford to upset the British Unions and therefore could not buy the much preferred AWACS option at the time. The cost plus contract that was given also meant that the more the company got things wrong then the more money they made. In the end - "jobs for the boys" cost the Country nearly £1Bn in 1980s prices. (Jeremy please note).


On a different subject - if you buy "off the shelf" then you have to ensure that the build standard of the components used is up to scratch and that they will be available throughout the lifetime of the product. If you replace kit using COTS then you change the drawing for each individual aircraft and you may end up with a mixed fleet and each aircraft different - very complicated for certification and the safety case when you update kit.

Nutloose, you are correct about the squash ball for the VCR-10 toilet flush but the ARC IPT Supply Manager kept them in his locked desk drawer at Wyton lest some sports mad type half inch them! They were Dunl@ps finest after all.

Yep some bright spark changed the spec to a hard ball that took ages to warm up..... Ermm to fit in the bog cough cough.. Mind you the plastic flap guide thingies were cracking windscreen scrapers, they apparently worked quite well on the VC-10 too.

The other problem with COTS is that they are often different to what has been used previously and a fortune is spent on changing it to fit SOPs, rather than changing the way it was always done to something that is probably more efficient.

Nothing inherently wrong with COTS provided the decision is fully informed. It has some inherent limitations:

1. A COTS product was developed to meet some else's requirement, and you may never get to see the detailed requirement it was developed to meet.

2. You don't own the design of a COTS product, so if they modify it you have to accept the changes if you want any more spares

3. A COTS product was developed to align with someone else's operating and support procedures - if you do bay maintenance and they didn't you either have to adopt their procedures or pay for the extra development of bay test equipment, procedures, spares etc

4. If you choose a COTS product you must take it as-is. If it falls short of your requirement you just accept the shortfall because it's a mature product at the wrong end of the product lifecycle and any changes will be hideously expensive, especially once you've requalified the changes. "Modified COTS" is also known as "Suicide Acquisition"

5. A COTS product is already fully mature at the start of *your* design/integration process, so it will start to see obsolescence problems sooner than a developmental item.

6. A COTS product will require a *custom* integration, and all the integration will ahve to be done on your side of the interface (hardware AND software)

7. Always remember that choosing COTS is a design solution - not a system requirement. It's depressing how many times a project starts saying "we need to use a COTS solution" when they cannot possibly know that at that stage of the project.

If all the above issues and risks can be identified, quantified and managed then COTS approach may well be a viable solution.

PDR

COTS is great if it does exactly what you want it to do. For the sake of argument, lets suggest we want to buy a dune buggy that comes with with a 8.9mm machine gun and a specific engine and a specific comms system.

If we want it to work in isolation then thats great - we can buy it. But if we want to add a UK weapon (strip out the machine gun as we don't have said calibre, and put the GPMG on it), and we want it to have a secure radio that we use already, rather than buy lots of kit with different radios, and we want the engine to run on the same fuel type as we already use, and by the way we need it to be airportable and we'd also like to use it to go on amphibs etc.

The moment you start looking about you realise COTS reflects the opening point of negotiations, not the finished product. What it is then about is modifying and integrating it to be something that actually useful to your military, and not a botch job collection of random kit that doesnt actually work together. That would cost far more and acheive far less.

...unless its the NATO Mid-Term upgrade for the E-3A.

COTS - Often confused with "One size fits all" - It never does, even COTS is configurable off the shelf and consequently will contain code or modules which can be enabled or disabled. Often clients pay mega bucks for a wire link or the simple enabling of code already written and installed. Check out any phone or laptop...multiple versions from basic all the way up to all singing and dancing..15 versions but only one or two pieces of hardware...all enabled by hidden configuration or inaccessible links.

Airlines customise/optimise extensively to specific requirements and national standards. Interiors, Comms, etc.

The old adage....

Vanity = I can squeeze the supplier to get my solution at a "COTS" price.

Sanity = I may have to customise a few things to get it to work as required

Reality = I cannot make it work without it becoming virtually bespoke.

Whoops: Budget blown.

Litigation, Nightmare :E

Imagegear

Perhaps because it can't be dragged out beyond 30 years and still be labelled as "cutting edged"...

Agreed - then there's also the punter who has attempted to create a new widget three times, each with a different supplier and failed spectacularly - perhaps the task was just too "before it's time" :ok:

Imagegear

So whats the news with the Lakota/EC145 usage by the US Army - suitable or not?
This is/was the largest publicised COTS purchase that I know of...

It's worth looking a bit closer at some of these objections; "the product is mature from the start....". Translation: "it works right away". How is this a bad thing? It certainly doesn't follow that it becomes obsolete any quicker.

"Airlines customise/optimise extensively.." Actually mostly not the case other than for cabin detail and galley fit. Even there, the only time in my own experience a big change was demanded, it was a shambles. Cathay originally demanded their 747-400 with analogue instruments for commonality. That was swiftly canned when Boeing told them how much more it would cost!

What would an iPad look like built to a military specification? Obviously it would have to withstand nuclear attack and work underwater and would probably be about the size of a piano.

Some good observations. Common sense says many requirements cannot be satisfied by COTS. Others clearly can. The trick is to avoid years of tendering only to find out that no-one makes a commercial product that can withstand +50 to -45C, is nuclear hardened with support guaranteed for 15 years. Very often, beancounters aren't actually interested in getting "value for money", the main motivation being to place obstacles in the way to delay expenditure. One obvious argument is that some technical and performance requirements are mandated upon MoD by, for example, the Home Office, who would refuse to release the necessary specification to the likes of Halfords - and even MoD itself! I recall just before the Personal Role Radio contract was awarded, a certain unit wanted the capability for an imminent deployment. An enthusiastic project officer bought them £35-a-pair kiddies walkie talkies from Argos. Well, they didn't last the journey to the airport. Sounds hilarious, but not if you're dug in a ditch 48 hours later with no comms.

The components and technologies in products are only in production for finite periods of time. A mature product will use components/technologies that were available when it was being designed, and they therefore have a much higher probability of needing some obsolescence mitigation or modification in *your* product lifecycle than something which is currently still developmental.

This isn't theory - it's a piece of engineering science which is matched by observation every day.

PDR

An example of a modification of a perfectly acceptable COTS solution from history was the Belgian FN which was procured to replace the Lee Enfield.

The Army thought to fully automatic burst would encourage profligate expenditure of ammunition. The SLR had that feature removed. Obviously had to pay more for less.

OTOH some OTS, as opposed to COTS was evident on the Vulcan and later Nimrod, that was the pillar lamp as fitted to the Lancaster and stamped GVI. Of course it could have been that Avro had a shed load left over from Anson/Lancaster/Lincoln etc

The best thing the military could buy off the shelf would be airworthiness oversight, if EASA 145 is good enough for all the airlines in Europe it should be good enough for the UK military.

Most of your posts, PDR, seem to be very logical but it simply doesn't follow that an expensive procurement process is necessarily linked to technical advance. On the contrary, there are instances of the procurement process taking so long that the equipment is obsolete before it even enters service!

Nobody is suggesting that buying kids toys is a good idea, as in the walkie-talkie example. But there are many robust and capable handheld radios in widespread use. Insisting that ours be designed to order would seem to be a good example of wasteful procurement.

In the case of "big procurement" one typically aims for a requirement/technology capability target, some 3, 4 or even 10 years beyond conceptual design phase.

Suppliers will have to base their design on a projected level of technical development which will not initially be understood, and to complicate matters, the original business case may subsequently change beyond all expectations to accommodate new threats, requirements, etc.

In my experience, Clients generally tie down the window of acceptability to a very small area consequently it becomes very difficult to pass.

COTS rapidly becomes unworkable for anything except the smallest projects.

Imagegear

Agreed, but as long as one differentiates between Commercial and Military off the shelf. Another factor is interoperability. And Joe Bloggs Backstreet Comms Ltd isn't likely to understand a typical MoD SIMOPS, Transec or Comsec requirement, never mind have the wherewithal to implement a cunning plan to demonstrate interoperability with various allies. (Not that many co-operate with us, or that we ever seek true interoperability!) A good example of a perfectly valid reason for a made to measure radio is the 1980s multi-mode fitted to RN Sea Kings, Lynx and Merlins. (And Nimrod R after the RAF nicked them). Eye-wateringly expensive due to the harmonic rejection spec and non-standard frequencies. Most companies would laugh at the former.

Who said anything about an "expensive procurement process"? I'm simply talking about the trade-off decision to meet a particular need with off-the-shelf vs developmental items. Nor am I suggesting that COTS (or more often "MOTS") is inherently a "bad" idea. I'm just pointing out that when looking at things to integrate into an aircraft system there are risks and issues which are often not full appreciated, especially by project managers and procurement authorities (hence the final line of my main post "If all the above issues and risks can be identified, quantified and managed then COTS approach may well be a viable solution").

You can find examples all over the place - possibly one of the most obvious being when the UK decided not to buy the "OTS" P-40 (which was totally unsuited to our mission) and instead contracted for the development of the P-51 (so it could be designed AROUND our mission).

If you are really, really lucky you may find a piece of OTS equipment that meets every single line of your requirement spec. In theory it can happen, but in >30years in the military aircraft industry I've never seen it. So when you buy COTS/MOTS you either trade the requirements it *doesn't* meet or you have it modified until it does. Modified COTS is just fundamentally a bad idea which is bound to be expensive, risky and unsupportable - it can be done and sometimes you strike it lucky but it's rare. Traded requirements should always restrict some aspect of the mission or increase some aspect of the ownership risk/cost (assuming the requirements were properly established) - otherwise the requirement didn't need to be there. These things are self-evident.

In the walkie-talkie example - if there are OTS items that meet the requirement then you can obviously buy it. Walkie talkies don't have much of an integration need at the technical level. But of course these days they will all include microprocessor-based systems, and these microprocessors will be obsolescent within 5 years (you won't be able to buy any spares). The manufacturer may offer an upgrade to a new processor, but that will need different test systems and test code, and will invalidate (say) the EMC or APEX qualification. It also means that you will have a mixture of configurations in service so you either have to bin the old ones or manage the different spare parts, repair procedures, test systems etc.

Have you tried getting a spare screen or battery for (say) a five-year-old smartphone? Have you tried to replace the processor in a 5-year-old laptop?

You will get similar problems with developmental kit, but it will be longer before it happens because the components used will be the latest at the time of procurement, not "already several years old" at that time.

James V Jones (Texan former tank officer who became one of the gurus in through-life engineering) used to lecture on this sort of thing, and he said that Rand had analysed procurement histories of all US government procurements <above some value that I can't remember> and one of their findings was that they couldn't find a SINGLE "modified COTS" procurement which ended up being cheaper than the developmental alternative. I don't have the study reference (it may be in the boxes of study notes in my loft, but I'm not looking for it!), but thousands of people attended those lectures over the years, so there should be others who remember it.

PDR

The thing I have observed is the apparent inability of the 'system' to adapt traditional procurement models to the reality of COTS/MOTS procurement. We are still spending an inordinate amount of time gathering User Requirements and converting these to System Requirements before putting out ITNs or ITTs, marking bids, selecting the preferred bidder, awarding contracts and then Testing and Evaluating the delivered 'solution. Very slow and cumbersome in the extreme.

To my way of thinking, in some areas we'd be better off submitting an invitation to industry to deliver a particular capability and see what comes in. For most people the most complex and significant technical thing we buy is a car. The URD is not 127 pages long, and does not contain statements such as 'must be able to go forward and reverse' or 'must have a means of making the vehicle change direction whilst in motion', because these are assumed capabilities. That's not to say that we don't have any requirements, but these tend to be mature and based on capabilities we know are available. Personally, mine now include Cruise Control, Climate Control and enough seats/space to move the kids to and from school/uni. There are about 6 realistic options and I pick the one that offers the best balance between cost and capability or perhaps the best one I can afford that is available - I find it had to believe that this model can't work more efficiently than it does in procurement.

I agree with PDR1
In my experience trying to manage a large new, IT heavy, aviation project, it is obsolescence of COTS software that will come back to bite you. Literally hundreds of separate but interdependent software products may be required.
If you own the overall product then you need an in-house organisation to track obsolescence and manage the impacts this does not come cheap.
IMHO best buy the entire product, with the software maintenance task,and let industry do the dirty work of keeping it all together and functioning.

STH

As I said earlier, very often this slowing down is a political decision to delay expenditure. Otherwise, you are spot on. One of the practical problems is the URD is often physically impossible to attain (never mind sustain) and after some years fannying about the project manager is forced to issue a clarification paper telling London (a) what is possible, (b) what is affordable within the endorsed funding, and (c) what they'll get. A solution emerges.


M2N

Unsurprisingly, mandated policy until the department responsible for management and oversight was disbanded without replacement in June 1993. In July 1996, two generations later in posting terms, EFA (Typhoon) convened an urgent meeting at which one of their very expensive consultants declared that they had uncovered a phenomenon called electronic component obsolescence, and would wish to open dialogue with attendees in an effort to work out how to deal with it. We gave him the mandated Def Stan and left. It doesn't take long for corporate knowledge to disappear.

Which is why MoD stopped doing it! But this ignored the fact that the consequential cost was measured in lost aircraft and lives. The same solution emerges.....

-re the walkie-talkie example (which you've got to admit is quite funny) there is a key integration issue, with the man. It is what the Army refer(red?) to as the left shoulder problem, whereby a slack handful of kit is allocated a spot on the PLCE left shoulder harness/strap. Aforementioned PRR, LCAD, PRC349 (in turn, stressing the antenna) and so on. The PRR battery compartment, as well as the Argos kiddie toys, leaked like a sieve, which tends to make electronics fizz. ("What, you wanted it splashproof?") As the trials included a jaunt round the Warminster obstacle course, including watery holes and ditches, a few questions were asked about why the product selected even passed, never mind "won". Well, actually, it didn't, coming a resounding last in many trials elements. A possible explanation emerged when, the next day, a picture of Lord Willie Bach appeared in the press, having just had a "winning" PRR thrust into his mits by a sharp company marketing chap. By no means the strangest procurement selection tale!

Contrary to what many might believe, every commercial passenger jet produced is actually a custom product. For example, every 737 that Boeing produces is custom built to customer requirements, and is assigned its own end item number. No true COTS product available in this regard.

I have no experience of military procurement. However, I did spend many years in the offshore industry, part of which being managing contracts for various services.

The most important thing we found was that the scope of work for the service had to be accurate, measurable and apply to the job in hand. From reading earlier posts, and past comments on procurement, in a lot of cases the people buying didn't do this.

For example, the engines on the Navy's new frigates were COTS items, absolutely fit for purpose for cruise ships, but not the right thing for warships. Who wrote the scope of work / specification, then who let the items be bought and installed and didn't o r wasn't allowed to say 'Hang on a minute....'

Waiting for incoming on this ......

riff raff:-
Exactly right! I was once employed by a charter airline that flew "pre-owned" BAC 1-11's. Our fleet included 200, 300, 400, and 500 series aircraft, but within those different series were different designations, ie 207, 301, 401, 414, 509, 518, etc, each one indicating an original airline's bespoke model (no doubt the cognoscenti will guess the airline in question ;-). The variations could even extend to the performance parameters, ie the 500's had "new" or "old " wing leading edges and thus differing ODMs.

These discussions often degenerate into using cars as examples of generic "off the shelf". I'm not sure that can be true either. I once bought a UK spec VW Polo in Berlin (when such practice saved considerable sums of money, as perhaps it may well do again in a few years time). I could specify any engine and any trim that I wished, and had to avoid the temptation to stray from the very restricted range then on offer within the UK (with eventual sale/trade-in in mind). In other words, the shelf has now many more possibilities than it perhaps once did...

Let's be clear what we're saying. Nobody is suggesting that airliners are literally plucked off the shelf like a tin of baked beans. But when an airline wants 200 people taking to Spain fifty times a week they don't invite a contractor to commence a design process.

PS, chugalug, I don't hold up the 50's/60's UK airlines as a model for optimal procurement.

Airliners these days have an extremely short lifespan compared to military platforms. The risk of software obsolescence, due to COTS products, is therefore considerably less than a military platform using COTS,which often has to survive for up to 50 years. Look at many ISTAR platforms like AWACS for example.

A nice simple statement of requirement, indeed. Replace "Spain" with "countries of possible foreign policy interest between 2020 and 2060" and it starts to get a bit trickier. This highlights another problem with military procurement - the requirement often draws heavily upon intelligence and futurology, disciplines which make business forecasting appear a paragon of accuracy.

If we could be better at knowing precisely what we need our kit to do, we might be better at accepting OTS stuff that is "good enough" for the purpose. Unfortunately we have not proven very good at forecasting events over the years and I don't see any particular reason why that should change, so we'll probably just keep pushing the boundaries to insure against all those Rumsfeldian "unknowns".

Actually I think in a way they do. Boeing asked industry is they wanted an SST or a wide-body. Airline companies want sub-sonic and had their requirement fed in hence 787

One member of MOD's Governance Team expressed surprise regarding a project with which I was involved recently, because the life was in the order of 20 years. It was/is quite heavily IT focused, though relatively specialist in nature. He was reassured that there was expected to be a need for a 'Tech Refresh' at or about the 10-year point. Ten years is about the maximum he would expect an 'IT Capability' to last in service.

Wars and superior military capabilities are all about inventing and possessing and producing cutting edge technology. You don't get that off-the-shelf.

In WW2, it was only the newest and latest virtually unproven designs that guaranteed a winning edge.
Unfortunately, many paid the ultimate price when that new technology or design had flaws that produced early and unexpected failures. That's the price of winning wars.

OTS technology works just fine for commercial applications and peacetime military activities, but it won't win wars. It will help when non-critical equipment is involved.
I can recall when a U.S. military study found 187 different makes and models of road vehicles were utilised by the military during WW2, resulting in a plethora of different parts and vastly increased logistical requirements.
As a result, it was determined that reducing the numbers of makes and models and utilising common componentry - and proven OTS componentry - was advantageous.
However, very rarely is a road vehicle a critical item, unlike aviation items.

onetrack, excellent points.

Some very highly paid people in MoD (so the wrong people!) constantly wrestle with these problems but occasionally the likes of the Public Accounts Committee ask a good question that demands a back to basics assessment. This happened in 1999 when they issued a report "Modifying Defence Equipment". There were 6 test cases, only one of which delivered to time, cost and performance. Senior staff didn't know how to reply, because the procedures and regulations governing the subject had been cancelled, and money chopped.

But a short paper was submitted from the viewpoint of the one successful programme, breaking down the acquisition process into functions. Two things stood out. First, there were two key posts whose roles kept cropping up through-life. Every project needed them, constantly, but as a matter of policy they had been disestablished without being replaced. The one successful test case had, purely by chance, a programme manager who had carried out these roles in a previous life. The failures identified in the other five were directly attributable to this work not being done. Nothing was done because, as ever, for senior staff to endorse the recommendations would mean criticising their own past decisions.

The second was more an observation. The author thought Service personnel very pragmatic and tolerant of procurement problems (witnessed by many of the above posts) and opined that, when one broke down the serious moans and groans, the key was getting Ranging, Scaling, Documentation and Packaging right first time. (So much else falls out of this). It then pointed out that, unsurprisingly, one the the first things the abandoned Service HQ role (above) does is raise an RSD&P form. A simple form, the only real thought being the Maintenance Policy Statement, and a whole process kicks off. But, by disbanding one post and throwing the forms in the back of a 6x4, everything ground to a halt. Support staff throughout MoD and Industry suddenly had no tasking, this was construed as nothing to do, and posts were cut never to be replaced. You had stupid things happen, like very complex equipment bought, but no spares, test equipment, training or tech pubs. (Recognise this, front line?) Radios bought, with no antenna, because the postholder paid to spot such howlers no longer had a job.

Shortly before I retired I saw the effect of the last first hand. When they know you're going, you get the trouble shooting jobs which require you to upset people. (The MAA should learn this lesson). A unit, shortly to deploy, had been given their new comms kit, but no antennae. "Not in the URD" apparently, but it is rather implied when you buy a radio, as you have to test and trial it before acceptance. Or so you'd think. I was standing with the long haired CO and he took a call on his mobile. His Yeoman of Signals, on a day off, was at a boot sale. He'd spotted a Rhode and Schwartz broadband HF antenna, in good nick, for two grand. If he flashed his credit card, would mess funds cover it? No said the CO, but he personally would pay. (He knew he was going to lose men without it, another thing lost on the BCs). The next day YofS pitched up, took 10 minutes to erect the antenna, and all was well. A good example of COTS, because the Mil Spec was the same as the Civ spec. They returned 6 months later, no losses.

Interesting arguments.

My own take on things is that too much of the argument for specialised equipment is based upon the dysfunctional procurement processes and overly extended operational lives.

Wars are won with cutting edge equipment? Possibly, there is no doubt that conflict gives innovation and advancement a real kick - but how does that square with keeping equipment in service for 30 years. Obviously after 5 years most of this stuff belongs in a museum.

Buying equipment which is not yet developed is a hugely risky prospect - and leads to 'concrete' radar systems and the like - which obviously are not going to be any use.

Speccing the equipment is obviously critical and the military has a hugely unfortunate tendency to make it all far to too difficult. The best example I can think of is the coffee machine on a certain US aircraft, capable of operating at +6,-3g, massive temperature margins and can withstand a 27g impact. Not surprisingly an extraordinarily expensive coffee maker - based on a completely flawed premise. Nearer home look at military LandRovers - different really just for the sake of being different.

Far too much of this seems to be driven by the idea that the equipment must last 25 years. During a conflict nothing lasts that long - it is obsolete in 2 years. But is has too in peacetime because it takes the procurement process 3 to 5 years to buy anything. As noted in many of the posts above nothing IT can be made that future proof, accept that and change it out every 5 years - just like the rest of the world does!