$9 Per Hour Code in Boeing Jets

[SysDude]

I haven't seen this posted here yet: https://finance.yahoo.com/news/boeings-737-max-software-outsourced-204657048.html

In short, they felt that they didn't need senior engineers on mature products, so they outsourced aircraft software development to companies using $9 per hour coders.

[B2N2]

That’s pretty shocking...

[Bend alot]

Been posted a day or two back in the MAX thread - not much comment.

Guess no-one is shocked anymore on what gets uncovered on a regular basis.

[Coochycool]

This rings a distant bell in my mind.....from about 12 years ago. When I came across a guy sat on a beach in Goa doing precsely that to make a crust.

He was a druggie who clearly had a history of self-harming.

The chain always breaks at the weakest link....

Fly safe folks

Cooch

[futurama]

This is a bull**** misleading article, like the "Boeing falsified records" one.

The number of outsourced Indian engineers involved in MCAS: big fat zero.

All of the major aerospace manufacturers have Indian outsourced suppliers & developers. Boeing, Airbus, Bombardier, etc.

Airbus might have the largest Indian presence, with > 5,000 Indian employees/contractors in the supply chain including over 1,000 engineers last time I looked. Probably much more now. And I doubt their pay structure is any different from Boeing's.

Are we all going to stop flying Airbus now?

[Bend alot]

This is a bull**** misleading article, like the "Boeing falsified records" one.

The number of outsourced Indian engineers involved in MCAS: big fat zero.

All of the major aerospace manufacturers have Indian outsourced suppliers & developers. Boeing, Airbus, Bombardier, etc.

Airbus might have the largest Indian presence, with > 5,000 Indian employees/contractors including over 1,000 engineers last time I looked. Probably much more now.
Airbus have been very quiet! But that makes it a good thing then - lets outsource management also.

[ThreeThreeMike]

It's a typical "news" story, produced to attract mouse clicks. There are heaping shovels full of innuendo and misleading statements, while facts are in short supply.

The ex-Boeing employees quoted offered little of substance. Vague generalities are poor substitutes for specific and compelling detail.

[SysDude]

Futurama: Slow down.... the article clearly states that there were no Indian engineers in the MCAS development.

You have also stated, with numbers, that this is a potential systemic problem across the industry; I never made a point in my OP, but this is one.

I have no dog in this hunt.

[ThreeThreeMike]

I haven't seen this posted here yet: https://finance.yahoo.com/news/boeings-737-max-software-outsourced-204657048.html

In short, they felt that they didn't need senior engineers on mature products, so they outsourced aircraft software development to companies using $9 per hour coders.

The Yahoo article, your thread title, and your post above falsely assert that Boeing outsourced software development to employees that were paid $9 per hour.

The reference to that wage was made by a consultant who has no idea how much Boeing's subcontract employees are paid. Your careless and inaccurate pronouncement illustrates how such inflammatory claims are spread.

“Engineering started becoming a commodity,”,” said Vance Hilderman, who co-founded a company called TekSci that supplied aerospace contract engineers and began losing work to overseas competitors in the early 2000s.

U.S.-based avionics companies in particular moved aggressively, shifting more than 30% of their software engineering offshore versus 10% for European-based firms in recent years, said Hilderman, an avionics safety consultant with three decades of experience whose recent clients include most of the major Boeing suppliers.

With a strong dollar, a big part of the attraction was price. Engineers in India made around $5 an hour; it’s now $9 or $10, compared with $35 to $40 for those in the U.S. on an H1B visa, he said. But he’d tell clients the cheaper hourly wage equated to more like $80 because of the need for supervision, and he said his firm won back some business to fix mistakes.

[SysDude]

ThreeThreeMike: Please provide finer granularity on your wage assertions, rather than throwing darts. I summarized the article in my view, perhaps incorrectly. I am the messenger. I would love it if someone could refute the article with solid numbers.

[ThreeThreeMike]

ThreeThreeMike: Please provide finer granularity on your wage assertions, rather than throwing darts. I summarized the article in my view, perhaps incorrectly. I am the messenger. I would love it if someone could refute the article with solid numbers.

Refutation is readily apparent. All you need to do is read the portion of the story I quoted in my post. There is no link between Boeing, its software, and coders being paid $9 per hour.

Other than that, you are asking me to provide proof they aren't being paid that wage, which is the reverse of how such things work.

[SysDude]

Refutation is readily apparent. All you need to do is read the portion of the story I quoted in my post. There is no link between Boeing, its software, and coders being paid $9 per hour.

Other than that, you are asking me to provide proof they aren't being paid that wage, which is the reverse of how such things work.

This analysis breaks down wages and other factors by country:https://towardsdatascience.com/comparison-of-software-developers-in-india-with-us-uk-germany-and-the-entire-world-8d2a1ba3218a

I ran across many similar statistics through other sources. Seems to me that $9 per hour is a decent wage in India for software developers.

[BDAttitude]

This analysis breaks down wages and other factors by country:https://towardsdatascience.com/comparison-of-software-developers-in-india-with-us-uk-germany-and-the-entire-world-8d2a1ba3218a

I ran across many similar statistics through other sources. Seems to me that $9 per hour is a decent wage in India for software developers.
I suppose aerospace is equal or better than automotive. Automotive pays about 20.000 - 35.000 $. Which would make it 12 - 20$ hourly.
Low wager (driver, cleaner etc.) will get typically about 100$/month.

[ThreeThreeMike]

This analysis breaks down wages and other factors by country:https://towardsdatascience.com/comparison-of-software-developers-in-india-with-us-uk-germany-and-the-entire-world-8d2a1ba3218a

I ran across many similar statistics through other sources. Seems to me that $9 per hour is a decent wage in India for software developers.

Again you fail to account for your assertion Boeing is using Indian labor that's paid $9 per hour. The deflection in your latest post does nothing to prove your false premise.

Where in your Google searches did you find proof Boeing is using outsourcing for software development that pays their employees $9 per hour?

That's the specific claim you and Yahoo made.

[Longtimer]

Again you fail to account for your assertion Boeing is using Indian labor that's paid $9 per hour. The deflection in your latest post does nothing to prove your false premise.

Where in your Google searches did you find proof Boeing is using outsourcing for software development that pays their employees $9 per hour?

That's the specific claim you and Yahoo made.

https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-737-max-software-outsourced-to-9-an-hour-engineers

[SysDude]

Again you fail to account for your assertion Boeing is using Indian labor that's paid $9 per hour. The deflection in your latest post does nothing to prove your false premise.

Where in your Google searches did you find proof Boeing is using outsourcing for software development that pays their employees $9 per hour?

That's the specific claim you and Yahoo made.

I pointed to the article in the OP, but I can't read it for you. HCL was mentioned extensively in the article. The article states that Indian SW engineers are used in the supply chains and quotes references by name and profession.

Per payscale.com, HCL Software Engineers salary in India: https://www.payscale.com/research/IN/Job=Software_Engineer/Salary/b54bf575/HCL-Technologies-Ltd.

[tdracer]

It's worth noting, there are five classifications of software in aviation - Level A, B, C, D, & E. Level A is for flight critical software - e.g. FBW and FADEC, while Level E is pretty much 'don't care' if it's not 100% correct since it has no impact on flight safety - e.g. IFE and some maintenance tasks. Naturally B, C, and D fall in between those extremes. This is all documented in DO-178, which also outlines the level of testing and documentation required for the various integrity levels.
Outsourcing Level D and E software is pretty common, since it's really not that important.
As for summer interns, the engine company was responsible for creating the FADEC s/w (although we had significant input into the requirements), so we didn't write code. However we had a lab, where we tested the FADEC software to make sure it properly interfaced with the various aircraft systems, and that our requirements were incorporated correctly. During the 747-8 development and flight testing, our FADEC lab was staffed with several summer interns, running tests that had been developed by our engineering group, and by all accounts they did a first class job of it.

BTW, most software problems in Level A, B, and C s/w were related to the requirements - either wrong, incomplete, or confusing - not the actual implementation.

[ve3id]

Ladies and Gentlemen:

All comments about the ethnicity of the $9 an hour programmers aside, I have to sit back and think about how complex systems are on a modern aeroplane for anybody, regardless of pay grade, to be able to be fully conversant with all the systems interactions that can occur.

If you remember back in the Reagan era, there was a program called the Strategic Defence Initiative, more commonly referred to as 'Star Wars.

Amongst all the scientific commentary of the time, I cannot forget that of David Parnas, whose lucid insight may have turned the tide against relying on technology to overcome a threat created by technology.

His position was that the SDI was such a complex system, that a counter-system would be so commensurately complex that it would not be possible to test it until the actual threat arose, and then it would be too late to fix the inevitable bugs.

I wonder if we now have achieved that level of complexity in some aircraft systems, where, in our race to relieve the crew of responsibilities and workload, we have created systems that can actually make life more difficult for them if systems are not fully debugged and their operation fully understood?

Those two switches on the engine quadrant controlling trim are so convenient, I have to wonder why their access is not the first thing anybody thinks of when such a problem occurs.

[SysDude]

ve3id brought up an excellent point about ethnicity. To be clear, the article is about locality, not ethnicity; employees within the supply chain from India are producing "some" aircraft code cheaply. Yourdon's book is classic reading for Software Engineers.

178 testability weaves into this nicely. The OP article raises many of these issues in a low-tech manner.

[ThreeThreeMike]

Increasingly, the iconic American planemaker and its subcontractors have relied on temporary workers making as little as $9 an hour to develop and test software, often from countries lacking a deep background in aerospace -- notably India.

In offices across from Seattle’s Boeing Field, recent college graduates employed by the Indian software developer HCL Technologies Ltd. occupied several rows of desks, said Mark Rabin, a former Boeing software engineer who worked in a flight-test group that supported the Max.

https://www.bloomberg.com/news/articles/2019-06-28/boeing-s-737-max-software-outsourced-to-9-an-hour-engineers

Nope. Still no cigar.

There is no proof whatsoever that Boeing has employed subcontractors whose employees make $9 per hour to write code for the MAX. Same with SysDude's dead link to what is supposedly pay scales in India, which has no relevance at all to this subject.

The Bloomberg writer's assertion that India has no aerospace culture is laughable. Notable German aircraft designer Kurt Tank (perhaps you've heard of the Focke-Wulf 190) became one of the founders of the modern Indian aircraft industry, beginning a long career there in 1955, just eight years after India won its independence from Great Britain:

Tank worked as Director of the Madras Institute of Technology, where one of his students was Abdul Kalam (later Kalam became President of India and designed indigenous Satellite Launch Vehicle(SLV) and Integrated Guided Missile Development Programme). Kurt Tank later joined Hindustan Aeronautics, where he designed the Hindustan Marut fighter-bomber, the first military aircraft constructed in India. The first prototype flew in 1961; the Marut was retired from active service in 1985.

The above-mentioned Dr. A P J Abdul Kalam was an instrumental contributor to Indian aerospace industry as well as president of India between 2002 and 2007. As noted, Dr. Kalam did pioneering work on rocketry and missile programs for the government.

Dr. A P J Abdul Kalam :: BrahMos.com (http://www.brahmos.com/people.php)

BrahMos Aerospace is a capable and respected firm that builds missiles and satellites.

BrahMos Aerospace - An India Russia Joint Venture (http://www.brahmos.com/index.php)

But all I read in these articles regarding the 737MAX and its software is innuendo the semi-ignorant Indians work for Ebenezer Scrooge levels of recompense, while producing work that is filled with errors and must be corrected by real engineers in America. To the horror of Boeing ex-employee Mr. Rabin, we find the recently graduated Indians are at desks just across the street from Boeing. My God!

​​​​​​It appears you both have been fed supposition and innuendo dressed up as news for so long, you cannot ascertain the difference between imputation and factual statements.

[Paul Lupp]

What matters the most, surely, is the experience and competence of the software engineers, not their nationality or how much they are paid.

Adequate integration, testing and validation of the software should ensure its integrity and effectiveness; even experienced coders can make mistakes but the software development process should ensure that such incidents are caught and rectified before they get a chance to be released

[bill fly]

Those two switches on the engine quadrant controlling trim are so convenient, I have to wonder why their access is not the first thing anybody thinks of when such a problem occurs.

Well now, that has been discussed uphill and downhill since these accidents.
“when such a problem occurs” is a bit too woolly a term here.
If the problem is just an uncommanded trim runaway, the convenient switches will likely be off pretty damn quick.
If the problem is AoA error-caused: Stall Warning, Speed Disagree, Stick Shaker, and upon flap retraction MCAS massive forward trim input , while already trying to sort out the preceding mess - and pretty soon augmented by ground proximity warnings, then it may well be, that the convenient switches are not the first thing anybody thinks of.
Even something as routine as reducing power might be delayed / missed due to all the distraction, as we see in the last case.
Not condoning the actions / lack of actions but trying to explain them.

[aterpster]

What matters the most, surely, is the experience and competence of the software engineers, not their nationality or how much they are paid.

Adequate integration, testing and validation of the software should ensure its integrity and effectiveness; even experienced coders can make mistakes but the software development process should ensure that such incidents are caught and rectified before they get a chance to be released
Of course nationality is irrelevant. Low pay, some relevancy.

Most important is loss of tight control over coding when the programming is partially farmed out.

[.Scott]

The article does not specify what Boeing code was written by HCL at rates as low as $9/hour - other than to say it wasn't MCAS.
The "work-around" quote is not specifically attributed to a student worker. Though anyone is allowed to have an idea - and if the system architect and review panels approve it, a $9 worker could appropriately put that claim on his or her resume.

By standards, all life-safety code is reviewed and methodically tested. This includes module "unit testing". The unit testing is an exercise in proving each separated piece of the code that will end up on the product (aircraft) is working as it should and contains nothing more or less than it should. Unit testing itself requires programming and those programs are then checked automatically (by generating code coverage reports) and manually reviewed. In many cases, I would not have a problem with a $9/hour Indian programmer taking a shot at that unit testing chore. Chances are that it would take him or her many tries before getting their first assignment through the review process - but it would be a safe thing to do.

So, as a software engineer, I do not find that article disturbing or even news-worthy.

[EEngr]

In offices across from Seattle’s Boeing Field
$9.00 per hour? Not once Seattle's minimum wage enforcement folks get wind of it. It's either $15.00 or $16.00 per hour, depending on the organization's size.
Washington State's minimum wage is $12.00/hr. So finding an office building just outside city limits isn't going to help much either.

[Longtimer]

The hourly pay of someone doing coding does not necessarily mean that the coding is inferior or deficient.

[WingSlinger]

We have been building roads, bridges, buildings, and various structures for 6000 years or so. The engineers and architects have a very well established procedures, tools, materials, and experience to do so. The same goes for vehicles, starting with ships, simple carts, then motorized trains, cars, and in the last 100 years even flying machines.

Yet, we have been developing software, for commercial use, for only the last 50 years or so, and that’s being generous. We have no concrete knowledge on how to build it properly. The tools change every couple of years, the methods and technologies change every decade. We are deluding ourselves if we think that we are "software engineers". We are organized and supervised hackers, at best.

Handing a bunch of specs to a coder, be it in India, or in a building across the street forum Everett field is not going to yield good outcome if he is not intimately familiar with what actual situation he is going to solve. I have seen, time and time again, code written that did exactly what the spec said and completely missed the mark in what it was intended to do. Furthermore the test cases, written for QA guys, only specified actions and expected results, which in most parts it met. Very rarely are QA people told to "go at it" and try to break the code by doing things in order or in combination with other functions, it was not designed to do.

I can can get a piece of new code and usually find a fatal flaw in a few minutes. The programmers would say "but you weren’t supposed to do that". Yeah, right I wasn’t but I did, now what did you do in your code to make sure if I do I don’t succeed?

So the question is not whether someone is paid $9/hr or 90. It boils down on his experience and aptitude towards the industry he is working for. The problem with experience is that we usually promote programmers into higher levels of management just about the time when they start to be competent.

The other problem is that "one size does not fit all" when it comes to programming. When constructing a building we can use a plumber or an electrician who works on different types of buildings. Apartment buildings, single dwellings, office buildings, hospitals, whatever. Soldering a pipe and installing the fittings is the same. Running wires and installing light fixtures and breaker panels is the same. The only thing different is the size and scale. But when it comes to programming, just because a programmer is Java coder, that does not mean that he deals with financial software the same way as he would with hospital HVAC software, or aircraft monitoring and control software.

So I think we have at least few hundred years of experience to collect before we can call ourselves true "software engineers". Until then, we are just glorified hackers wandering aimlessly in search of meaning.

[Luc Lion]

Thank you for that pinch of humour.
Just wondering what is that place where databases don't crash.

I believe the 3 major sins of software engineers (or whatever you want to call them) are:
- not trying to understand IN DETAILS what the piece of software will be used for,
- not mapping the full vector space of the input parameters, or assuming a limitation in this space that will be a responsibility of another piece of software,
- designing strongly coupled complex systems and failing to understand that resilience can only be achieved through loose coupling.

Actually, there is a 4th, but it is a venial sin:
- continuously reinventing the wheel

[tomuchwork]

This is a bull**** misleading article, like the "Boeing falsified records" one.

The number of outsourced Indian engineers involved in MCAS: big fat zero.

All of the major aerospace manufacturers have Indian outsourced suppliers & developers. Boeing, Airbus, Bombardier, etc.

Airbus might have the largest Indian presence, with > 5,000 Indian employees/contractors in the supply chain including over 1,000 engineers last time I looked. Probably much more now. And I doubt their pay structure is any different from Boeing's.

Are we all going to stop flying Airbus now?

Well. Airbus jets do not divebomb on a regular basis.....enough said. Flying a B737NG myself, would trade any second to fly an A320/330.....

[.Scott]

Yet, we have been developing software, for commercial use, for only the last 50 years or so, and that’s being generous.
I began programming "commercially", but as a volunteer in 1968. I have been programming professionally since 1971. And there were professional commercial programmers well before I started. So you aren't being too generous.

First, although you dislike the term "Software Engineer", when it comes to mission-critical and life-critical systems, in my experience, it has always been a significant engineering effort. Unlike the kind of "hacking" you can have with less-critical systems, actual "programming", that is writing lines of code, takes up a small portion of the effort when applied to critical systems in the fields of medicine, transportation, and the military. The bulk of the time is consumed with determining and documenting the requirements, the system design, the software design, and the detailed designs. Then you have coding. Then you have testing. All of that under a set of process safety and development standards that are specific to the industry and the customer (USAF, Navy, automotive, FDA, etc.).

Salary aside, a truly junior programmer has a very limited role in such projects. They can code - but they will be spoon-fed.

[Chas2019]

The hourly pay of someone doing coding does not necessarily mean that the coding is inferior or deficient.

Not necessarily but it may mean that the responsibility does not fal on them either when something goes wrong.