787 Batteries and Chargers - Part 1

[saptzae]

Redesign - remove all batteries & use super capacitors. End of story!!
Cute idea, but the capacity and load capability would be too low.

And, anything that has the capacity, can make fireworks too, NiCad included.

[no-hoper]

The voltage might be slightly higher.MEL maintenance action after APU bat charger fault:
"Prior to each departure, verify on the ELECTRICAL maintenance page 1
that the APU battery voltage is 31 DC-V or greater"

[RR_NDB]

Hi,

FullWings:

If the batteries in question were installed so that they could be isolated,
(http://www.pprune.org/tech-log/505695-787-batteries-chargers-38.html#post7703916)
This comment triggered an idea will post on a Fault tolerant battery. Less risk of "uncontained thermal runaway" and capable to be "isolated". Will comment on that later.

No pilot that I know would be happy with a piece of equipment in an inaccessible area that was a demonstrated fire risk, "contained" or not. What's the point of all the dangerous goods legislation if you carry something similar to a prohibited item around with you all the time?


:}

saptzae:

32.2V is end of charge voltage (4.025 x 8). 29.7V is rated voltage.
I'd go for 25V being available when battery low and at a high current load.

Perfect.

I estimate that BDM drops less than 1V (0.8V typ) at 150A if based on Schottky diodes and less than 0.5V (0.3V typ) if based on MOSFets.



Some days ago i read and saved a .pdf from Sandia Labs telling on a device for 787 program. IIRC MOSFet. It´s lost in one HD :)


My best bet, as outlined in earlier posts, Primary failure (short) due to gradual cell deterioration after one or more (in no particular order of preference) of :
Mismanagement by BMS.
Mishandling (assembly, rework, repair, maintenance, storage)
Mishandling deep discharge and reset BMU lockout
Mishandling external over voltage, such as bus over voltage, also during maintenance or storage (recharging - dunno they do that).
Tested to overstress - Boeing says they test a lot and hard (unlikely IMHO)
Bad cells (unlikely IMHO)



Fully agree including sequence (hierarchy)

Apparently, current battery tests do not detect cell deterioration. I do not claim that tests possibly could detect it.

That´s another concern i have with these cells. Lack of knowledge. "uncharted waters" :E

bill good:

Redesign - remove all batteries & use super capacitors. End of story!!

Yes, end of history: payload of the redesigned plane will be zero. :mad:

saptzae:

can make fireworks too, NiCad included.

:{

no-hoper:

...verify on the ELECTRICAL maintenance page 1
that the APU battery voltage is 31 DC-V or greater"

This is an interesting indication. Conservative approach. Tight tolerance on batt. charge condition. Pre flight :ok:

Makes me remember the laptops battery range (empty to full of just one Volts)

[saptzae]

@RR_NDB

Yeah, the poor battery integrators and BMU/BMS guys feel the heat for sure.

Making a positive assumption in view of the fact that 150 batteries were replaced, according to Boeing mostly due to being discharged until lockout.....

Mishandling deep discharge and reset BMU lockout

It could explain why nothing happened for several years, and then two in a row...

Unfortiunately, that just being another easy answer, it still would leave the Secondary (multi-cell failure/runaway) and Tertiary failure (fire) to deal with.

Gosh, mis-drilled oil pipes leading to wing perforation and environmentally friendly coatings leading to fan mid shaft failure and torched grass seem to be so much easier to deal with.

[RR_NDB]

Hi,

saptzae:

Yeah, the poor battery integrators and BMU/BMS guys feel the heat for sure.

High rocks always are the ultimate responsible. And management failed: BMU/BMS guys feel the heat :mad:

It could explain why nothing happened for several years, and then two in a row...


Good point!

Gosh, mis-drilled oil pipes leading to wing perforation and environmentally friendly coatings leading to fan mid shaft failure and torched grass seem to be so much easier to deal with.



:)

[cockney steve]

Mishandling deep discharge and reset BMU lockout

It could explain why nothing happened for several years, and then two in a row

A long time ago, I suggested that a system which allowed a deep discharge,was a fundamentally flawed design....I was shouted-down on the basis that, under emergency conditions , every last amp might be needed.

The fact, apparently, is that under NORMAL usage-conditions, these batteries are being driven into an unsafe area....IMO they should load-shed/lock out with sufficient reserve to start the APU and then be able to safely recharge from on-board generation.

Under the present regime (grounding excepted), what happens if the fuel-flaps/nav.lights/whatever drain either or both batteries to lockout . the aircraft is presumably then held up until replacements can be sourced/fitted.
A far from ideal situation and entirely preventable with proper engineering.

@ EENG...I was trying to highlight just how ridiculously short the life currently is.

An average lead-acid battery for a diesel car would cost about 0.6 of a CENT per hour of it's life !!!
Oh, dear, I really must get out more. :8

[RR_NDB]

Hi,

cockney steve:

these batteries are being driven into an unsafe area

So, let´s reduce this problem by half:

787 batt. redesign, K.I.S.S. version 1.0 (http://www.pprune.org/tech-log/508176-design-review-787-plan-b-a350-xwb-triggered-lithium-ion-batteries.html#post7704300)

Please, bombard this first version. Objective: Fast and safer return to the skies.

[saptzae]

@cockney steve
A long time ago, I suggested that a system which allowed a deep discharge,was a fundamentally flawed design....I was shouted-down on the basis that, under emergency conditions , every last amp might be needed.Yes, but it is needed, as much as the pilot, to "fly the plane to the end".

A far from ideal situation and entirely preventable with proper engineeringYes, together with management.

[amicus]

CFRP, the epoxy component, is full of over 90 chemicals and this is why it was banned by the FAA finally from A/C interiors due to FST (Fire, Smoke and Toxicity) hazards back in the 70's due to numerous aircrew and passenger deaths deaths from FST. I worked on the problem between 1960's and 1970's
A recent CFRP crash example was in Guam a few years ago involving the B-2A. The fuel fire was extinguished within less than 30 minutes, but the wreckage continued to burn for over two days in spite of the efforts of over 60 trained USAF fire personnel due to continuing hot spots and flare ups. This is fully documented by USAF in their incident report, but FAA and Boeing ignored my protests as they had my earlier inputs and protests concerning 787 crashworthiness proposed SC's.

[Speed of Sound]

An average lead-acid battery for a diesel car would cost about 0.6 of a CENT per hour of it's life !!!

Or in the case of the factory fitted Bosch battery in my Mercedes, now in its ninth year of service, 0.04 cents per hour.

And with 110Ah capacity and 1100amp cold cranking, it would almost certainly fire up a soft start APU. Don't know about the weight but is certainly smaller than the Li-on unit installed on the 787! ;-)

And, anything that has the capacity, can make fireworks too, NiCad included.

Exactly.

At the end of the day it is a box of energy and the more energy in that box the more potential for harm.

[RR_NDB]

Hi,

JTSB report (http://www.mlit.go.jp/jtsb/flash/JA804A_130116-130220.pdf)

In japanese

[Lyman]

From the diagrams:

image #3 : showing each battery containing three cells.

image #4 : showing each individual cell, unrolled, at ~ 10 meters in length

image #9 : showing the "breaker" twixt charger and BATT @ 75amps. (?)

RR: Arr EE Got oh...

[FlightPathOBN]

from BoeingLand...

Boeing close to fixing Dreamliner battery

Boeing close to fixing Dreamliner battery: source - Yahoo! Finance (http://finance.yahoo.com/news/boeing-close-fixing-dreamliner-battery-085548211.html)

(close is relative....)

I did a quick Google translate on the JTB report..

while very crude, it does give an idea of what happened...

Slide 14 text:
With a record of DFDR, the battery voltage, the sense of a nasty smell in the cockpit
Was the same time, reduced from 31V to 11V in about 10 seconds, and then, on the 10V and name was finally under Repeat , eventually became a 10V
• was recorded every 2 seconds per 1V, the battery voltage is the battery voltage was recorded every 2 seconds per 1V
• In order to be measured by the voltage drop of about 1V lower BDM, DFDR
Converted from the voltage readings, and was originally about 32V conceivable
• Indicates battery voltage, 32V before the occurrence of cases about, the battery Lee Lee believed that almost was almost fully charged state
• For data after the voltage drops, the external device Because it is connected, still ongoing analysis

The overall flavor of all of the slides was they were not able to re-create the failure mechanism, and are going to do much more testing and analysis, including electron scan microscope, further research, and will set up a testing and evaluation methodology in the future...

[RR_NDB]

Hi,

Finally precise information (http://www.pprune.org/tech-log/508176-design-review-787-plan-b-a350-xwb-triggered-lithium-ion-batteries.html#post7705490)

Observe the charging current below 75 Amps.

[FlightPathOBN]

This appears to show 150A startup...

http://787updates.newairplane.com/Boeing787Updates/media/Boeing787Updates/Batteries%20and%20Advanced%20Aircraft/newBatteries_large.jpg?width=900&height=674&ext=.jpg

[cwatters]

The 100 replacements were due to the battery shutting down due to being over-discharged. As over-discharging can induce thermal runway, the system is designed to shut the battery down when it reaches a specific level of charge. So in these cases, the battery safety systems were operating as designed.

I wondered how the minimum safe voltage was determined? There might well be statistics involved. eg if the minimum voltage is set to X then the MTBF is Y.

That MTBF calculation might be pretty complicated. It might make assumptions about how frequently the minimum voltage is approached? Perhaps if you approach minimum voltage too frequently the MTBF reduces dramatically?

[RR_NDB]

Hi,

Short to ground (that fused ground wire) "developed" through "hole" at cell side?

http://oi48.tinypic.com/a17p8h.jpg

See at pg 2 of JTSB report (http://www.mlit.go.jp/jtsb/flash/JA804A_130116-130220.pdf)

The opening of the ground wire is showed at pg 10 but not emphasized?

[RR_NDB]

Hi,

cwatters:

approach minimum voltage too frequently the MTBF reduces dramatically?
(http://www.pprune.org/tech-log/505695-787-batteries-chargers-39.html#post7705574)

Teething phase of batteries in a new aircraft :} :{ Uncharted waters.

FlightPathOBN:

"Aggressive" marketing specs.: 150 A compared to 16 A of competition. :)

Where is this page? Source?

I did a quick Google translate on the JTB report..

:{

The overall flavor of all of the slides was they were not able to re-create the failure mechanism, and are going to do much more testing and analysis, including electron scan microscope, further research, and will set up a testing and evaluation methodology in the future...


Confirming my concerns :{

On Voltages recorded THERE IS A PROBLEM:

If the negative tip of the remaining cells were not SAFELY connected to ground, (AS HAPPENED) the voltage WILL BE LOWER. Will post the equivalent circuit on that problem ASAP.

Fact is:

DFDR is not capable (as configured in 787) to show VALUABLE and NECESSARY information on how battery failed. IMO in teething phase of an a/c, complimentary means should be required for.

[saptzae]

@RR_NDB
We may never understand WHY (BOS and TAK)Yes, as we see only effects of Tertiary failures, which destroyed all evidence of degradation leading to the Primary failure.



6 Tertiary failure symptoms (heat, fire)

5 Tertiary failure sequence (multi shorts, arcing)

4 Secondary failure symptoms (multi-cell runaway)

3 Secondary failure sequence (electrical or thermal)

2 Primary failure sequence (cell short)

1 Cell degradation











It's impossible to trace back from 6 to 1.

There seem to be no detailed records of per cell voltages around the time of Primary/Secondary failure, which would be the only way to conclusively determine sequence of Primary failure and cause of Secondary failure.

Looking at other in-service, or replaced batteries to find degraded cells is the only way.

Edit
Secondary failure should be reproducible by failing a Cell.

Tertiary failures should be mitigated by better electrical and thermal insulation of individual cells.

[FlightPathOBN]

RR,

That battery spec was from the website that Boeing has set up for the issue...

Batteries and Advanced Airplanes - Boeing 787 Updates (http://787updates.newairplane.com/787-Electrical-Systems/Batteries-and-Advanced-Airplanes)

If the negative tip of the remaining cells were not SAFELY connected to ground, (AS HAPPENED) the voltage WILL BE LOWER. Will post the equivalent circuit on that problem ASAP.

serial bus bars...so if there is a single cell failure.... :uhoh:

The opening of the ground wire is showed at pg 10 but not emphasized?

it just states "disconnection"

[FlightPathOBN]

rough translations:
slide 6
Can be seen mark and damage the high temperature in the vicinity of the positive terminal of the cell element 3
○ (arrow), aluminum mosquito melted by heat attached white part Potential can be considered, I need to investigate further

slide 8
Observed by optical microscope and scanning electron microscope inward from the outside of the cell z
Likely open hole investigate the case of cell 6 scheduled to face z
Situation that is transmitted suggests the heat element is diffused into the fan through the hole in the case, I thought the energy was transmitted from the outside to the inside

Slide 11
January 26, on the 27th, the main manufacturer in Fujisawa Survey of the BMU that had been attached to the battery
Damage to the substrate
(1) Intense operation test in the state of being energized I was not able to test
(2)Record the damage condition
(3)Further analysis is being found to damage from the situation, but there is a big difficulty for damage

slide 13
Damage to one of the electrodes. Cells 3
• I considered damaged from heat.
• Why has occurred a trace of damage including damage surveys continue
2 blowout of the current collector.
• blown found in the positive electrode current collector of the cell 1, 2, 3 and 8
• aluminum is high as the positive electrode current collector material is melted, I thought the temperature reached, and was blown
• that a large current flows in the current collector, the temperature reached the cell Allowed, such as the positive electrode current collector that has been heated by thermal runaway in Be considered potential Blown ground wire out of the box
3. Batteries
• is considered likely that the blown current flows through the outer box. In addition,
There was no record of the history of a lightning strike to the aircraft

[FlightPathOBN]

http://operationsbasednavigation.com/wordpress/wp-content/uploads/2013/02/ScreenHunter_23-Feb.-20-11.17-e1361387910822.jpg

Red 'star' indicates: The positive electrode current collector (Blown)

Red text at upper right states: Safety valve on 7 and 2 open

[RR_NDB]

Hi,

FlightPathOBN:

rough translations: (http://www.pprune.org/tech-log/505695-787-batteries-chargers-39.html#post7705842)

:ok: Thank you!

Red 'star' indicates: The positive electrode current collector (Blown)
Red text at upper right states: Safety valve on 7 and 2 open

The model i constructed in my mind in last weeks is being reviewed in light of this findings.

Anyway:

Can be seen mark and damage the high temperature in the vicinity of the positive terminal of the cell element 3

Damage to one of the electrodes. Cells 3
• I considered damaged from heat.
• Why has occurred a trace of damage including damage surveys continue
2 blowout of the current collector.

that a large current flows in the current collector, the temperature reached the cell Allowed, such as the positive electrode current collector that has been heated by thermal runaway in Be considered potential Blown ground wire out of the box

All this this reinforces my model. A short of cell # 3 tip to battery case (and to a/c ground).

blown found in the positive electrode current collector of the cell 1, 2, 3 and 8

No idea on why blown in # 8 :confused:

is considered likely that the blown current flows through the outer box.

In addition, there was no record of the history of a lightning strike to the aircraft

Investigators will conclude current from internal short circuit to case blowed aircraft harness ground wire of MAIN battery.

I expected this conclusion from them before. Explanation may be they concentrated on cells analysis.

[mm43]

My interpretation of a Google translation:-

Slide 15 of the current JTSB report indicates that regardless of switches/CBs in the Nav light circuits being isolated, the Nav lights were on and drawing current from the APU battery. The investigator indicates the subject needs further research.

[RR_NDB]

Hi,

mm43

...regardless of switches/CBs in the Nav light circuits being isolated, the Nav lights were on and drawing current from the APU battery.

This seems to be a problem not "connected" to the MAIN battery issue.

I did think if the destruction of the MAIN battery ground wire could damage this circuitry or control. Fact (probable) is a high, very high current flowed from battery (MAIN) to ground. Thousand(s) Amps.

They are puzzled and even mentioned lightning strike possibility. And discarded.

A thick ground wire from a battery case doesn´t open without a reason. :)

[FlightPathOBN]

RR,

Is the fwd EE battery being used as a UPS for the nav system?

EDIT:

News today is saying that because the nav lights were flickering, there was a wiring issue...

What about the IFE re-booting?

[RR_NDB]

Hi,

FlightPathOBN

What about the IFE re-booting?
(http://www.pprune.org/tech-log/505695-787-batteries-chargers-39.html#post7706063)
VERY GOOD QUESTION! :ok:

Problem now seems much different than in BOS

Wiring issue? :E

[RR_NDB]

Hi,

FlightPathOBN:

it just states "disconnection" (http://www.pprune.org/tech-log/505695-787-batteries-chargers-39.html#post7705788)

Certainly the investigation will look to this fact. They had other priorities.

The things will become more interesting: Circuit analysis. :8 :cool:

[sb_sfo]

Guys, you will recall from another discussion that the IFE will reboot when there is an EE bay smoke alarm. Do try to keep up, will you?:=

[FlightPathOBN]

sb,

Why would the IFE reboot on the EE bay smoke alarm? Does that make sense to you?
The IFE is not even run from the fwd EE bay. :mad:

Much the same as the running lights disco...should not be from the fwd EE systems...

Since the issue is being worked on 24/7 by entities all over the world, with NO SOLUTION AND today, the JTSB is stating that the ac was not wired correctly...

So, who is keeping up with what? :mad:

[RR_NDB]

Hi,

sb_sfo

the IFE will reboot when there is an EE bay smoke alarm.

This is new for me. With multiple threads on the issue this may occur. :)

Let me understand: The smoke alarm in the FWD EE bay resets IFE?

:confused:

Do try to keep up, will you?

Considering electricity and electronics are my degree and considering the importance of the 787 grounding issue for aviation i am trying to "keep up". Actually trying to be proactive. (http://www.pprune.org/tech-log/508176-design-review-787-plan-b-a350-xwb-triggered-lithium-ion-batteries.html) But the info we have is scarce. Today for the first time i saw the block diagram of the MAIN battery circuit. I had it in my mind only by some info presented by TURIN and today confirmed it was correct.

:ok: for the info on reboting. I was trying to image a possible "connection" to the main issue.

Question:

Do you have an idea why flashers were draining APU battery current?

[FlightPathOBN]

watch this video of the evac...(the seatback IFE)

Video: Evacuation of Boeing 787 Dreamliner after emergency landing in Japan - YouTube

[Lyman]

The a/c was half full, the aisle IFE screens were on, very few of the rest, and the POS lights are powered by APUBatt.

So Captain selected the lights in EMER descent, and left the screens on...

What am I missing?

[RR_NDB]

Hi,

More precise information from NHK: (http://k.nhk.jp/daily/index7.html)

Japanese aviation authorities have uncovered a design fault in the electric wiring of a Boeing Dreamliner that made an emergency landing last month.
But officials with the Transport Safety Board say the fault has no direct connection with the aircraft's battery trouble.The 787 aircraft, operated by All Nippon Airways, landed after a battery overheated, sending out smoke.
Experts have focused on electricity flow aboard the plane because some of its lights remained on after their switches were turned off.They found that wiring was installed according to the aircraft's blueprints, but that electricity flowed differently from its intended design.They say current flowed from one battery into a circuit whose switch was supposed to be off. The battery was reportedly not the one that overheated.The officials say they will investigate the problem more closely.

So, another issue. :E

787 uses high current solid state switches. Same modules are used in F35. These switches (solid state relays) could fail (short) and create "strange" behavior on circuits. For example, the diode module in MAIN battery circuit if shorted needs prompt (protective) reaction from battery management or even crew members in extreme cases. A new project requires IMHO to be more alert than other mature planes. The degree of innovation in 787 is a factor to be considered with attention. Some (due components) failures could be very difficult to quick understand even for a (circuit) designer. The above news may be related to a shorted solid state relay.

[saptzae]

@RR_NDB
So, another issueA direct connection, feeding current into a battery with a failed cell, would drive good cells into runaway by overvoltage.

There should be no paralleling of battery to battery or powered bus to battery without protective means such as BDM.

A MOSFet based BDM would have to be switched off quickly to prevent current reversal into the battery.

[fizz57]

I've been thinking about the APU battery feeding the nav lights during towing and such.... presumably the nav lights are normally fed from another source. This raises the possibility of current from that other source finding its way to the APU battery - without a diode - if the switching fails...

[saptzae]

@fizz57

Concur, by connecting to NAV lights from two sources, such as main bat bus and APU battery, there may be a parasitic current path into APU battery, bypassing the intent of the design.

When any cell is fully charged, a very small net current (mA!) into the cell, would lead to overvoltage of the cell, unless the BMU balancing manages to act against it and compensates it out, to keep cell voltage down. I am not sure if parasitic bypass current would be high enough to cause overvoltage. Depends on the bypass voltage, which would have to be well above 32V, and also on how good BMU is at balancing.

Any overvoltage will deteriorate cell, thus lead up to Primary failure.

Once a cell shorts (Primary failure), and battery voltage drops, the parasitic current path allows larger current to flow (a few A), which also would be too large to be balanced out by the BMU.

That would set battery up for Secondary failure of multiple cells by overvoltage.

When multiple cells fail, Tertiary failure is just a matter of time.

It may be a lead, its not all of it. A parasitic current path into main battery remains to be found.

[Speed of Sound]

Once a cell shorts (Primary failure), and battery voltage drops, the parasitic current path allows larger current to flow (a few A), which also would be too large to be balanced out by the BMU.

Once a cell shorts, the input to the BMU for that particular cell will show a short circuit. How the BMU responds to that will depend on what it thinks is the cause of the short circuit.

[saptzae]

@SoS
Once a cell shorts, the input to the BMU for that particular cell will show a short circuit. How the BMU responds to that will depend on what it thinks is the cause of the short circuit.
Right, and the BMU will hopefully disable the charger.

For this scenario, the parasitic source - main bat bus -> switch -> nav light -> switch -> APU bat - remains, feeding the APU bat.

Another possible parasitic connection reported is direct connection of Main bat and APU bat. If so, scenario applies to Main bat as well.

Scenario is


APU bat cell short
BMU detects cell short
BMU shuts charger down and alerts
APU bat voltage is reduced by up to 4V
But parasitic source remains at above 28V (4V x 7)
Source voltage to be taken up by the remaining 7 cells
Ideally (if BMU not already disabled), the BMU trys to balance against parasitic charge current to prevent 7 cell overvoltage.
BMU fails because current too large (> 3 A)
Multi-cell overvoltage
Cascading cell failures
Runaway

Edit
As I explained in earlier posts, short may be transient, and hard to detect, unless BMU sees cell voltage transients.

[sb_sfo]

I was a little hasty in my reply last night, and I couldn't get the right smiley to stick to my post. The info on the IFE was a month old(!) now.

When there is an EE bay smoke alarm, the IFE load is shed automatically. The Panasonic IFE will take about 20 minutes to spool back up, and you will get uneven screens through the cabin depending on the pathways in the system.

[FlightPathOBN]

sb,

no worries..this thread is very long, and the other thread has parallel, same, and different information...

Concur, by connecting to NAV lights from two sources, such as main bat bus and APU battery, there may be a parasitic current path into APU battery, bypassing the intent of the design.

I think in wiring we call that a double fault...

[bill good]

don't know the solid state relays used but the units I've seen all have a protection diode which will pass current in the reverse & care is always needed where they are used. i.e. useful only where one DC source is involved or any other component or wiring failure cannot profide a DC backfeed source.

Regards

[FlightPathOBN]

IF the ground was blown as shown in the JTSB report...there is a very deep rooted problem with the electrical flow diagram in the Firebird...

[mm43]

IF the ground was blown as shown in the JTSB report...Kind of looks as if there is some unforeseen commonality via the DC buss and the Main and APU battery packs. The on ground services supplied by each battery may well be the link.

[RR_NDB]

Hi,

The ground wire (MAIN batt. ANA) was fused by internal short. (model)

How without a (internal) short to ground fuse this thick wire?

Posted earlier the equivalent circuit. Will think if there is another way to blow the ground wire.

fact is, lack of continuous fuselage ground return pose some difficulties and a broken return path (seems not the case) creates very strange behaviors difficult to understand quickly. Even with DC.

[blackhand]

testing 123

[Lyman]

I am not allowed to pull quotes from the R/N thread (I am banned), so I will paraphrase a recent post.

The "cells" are in series. Lose one "cell" lose the "battery".

It is not for nothing the emphasis in the press has been on "cells", instead of "batteries".

Perception is everything. Fire is the dramatic issue, as it should be. The base line is the regulations. Any problem with one "cell" fails the entire SYSTEM.

It is an odd thing to emphasize, but important. Fire is not the problem.

The problem is the certificate, and loss of a safety critical system more often than the regs allow.

Eight batteries in SERIES, Not eight cells.

In protecting against "FIRE", the eyeballs are off the problem.

How many of the replaced batteries had one bad cell? Does it matter? The question is about dispatch, and reliability.

Cannot launch without both batteries at a specific SOC. The rest is spilt milk.

[FlightPathOBN]

Todays news is saying that the Japanese investigation found the APU had been mis-wired to the forward EE battery....

perhaps that is why even with the forward battery ground gone, the IFE was still powered up

[Lyman]

How did they start the APU? Ground Fault?

[TURIN]

How did they start the APU? Ground Fault?

One would assume that the main busses were still powered, therefore APU would be started from power available and not the APU bat. :ok:

How many of the replaced batteries had one bad cell? Does it matter? The question is about dispatch, and reliability.

Cannot launch without both batteries at a specific SOC. The rest is spilt milk.

Didn't we cover this already? MEL allows dispatch with APU inop. I assume that includes APU bat inop.

[EEngr]

TURIN (http://www.pprune.org/members/46562-turin)

MEL allows dispatch with APU inop. I assume that includes APU bat inop. But not on fire. Even if it's in a titanium box, I doubt they'll push back and go.:uhoh:

What will the procedures be for a fire in flight (in the new box again, of course). In all probability, immediate diversion, landing and evacuation. The battery box just improves the survivability of an in-flight fire.

[FlightPathOBN]

Lyman,

JTSB said this week it had found the ANA APU had been erroneously wired to the main battery that overheated.

[syseng68k]

From this:

Boeing readies short-term battery fix, facing uncertainty | Business & Technology | The Seattle Times (http://seattletimes.com/html/businesstechnology/2020373450_boeing787xml.html)


The battery control system will have sensors to monitor the temperature and
voltage of each individual cell rather than the battery as a whole, one source said.
...and this:

FAA Says Boeing Needs to Address Battery Risks Before Dreamliners Will Fly Again | Frequent Business Traveler (http://www.frequentbusinesstraveler.com/2013/02/faa-says-boeing-needs-to-address-battery-risks-before-dreamliners-will-fly-again)


Boeing also said it plans to develop a new battery design that will measure
the temperature and any voltage changes in individual cells.
Looks like the original design did in fact only have a single temp sensor,
even if they were managing individual cell voltages. Beats me how such a
design ever got past engineering.

I wonder if the group here will be able to send bills for consultancy fees
to Boeing ? :E...

[FlightPathOBN]

I wonder if the group here will be able to send bills for consultancy fees
to Boeing ?

at the same time they pay the airlines for the parked aircraft! :sad:

[Smilin_Ed]

Looks like the original design did in fact only have a single temp sensor, even if they were managing individual cell voltages. Beats me how such a design ever got past engineering.

I don't understand why the airlines bought off on this Rube Goldberg arrangement. This hugely complex battery system clearly is unsuitable for the intended use and is an obvious hazard to safe flight. They threw KISS out the window and sacrificed safety on the altar of weight reduction. :ugh:

[syseng68k]

smilin ed:

The way I see it, it's inadequate oversight of the complete battery
subsystem by any sufficiently qualified and experienced engineer. Perhaps
that's what happens when you hand over the company to beancounters, who
then fire all the older engineers who really knew what they were doing
and understood the meaning of the expression "due diligence".

What is interesting is why Yuasa, who know how these cells need to be
managed, didn't raise questions about the design, but perhaps they did
and were overuled. Hopefully the final report will be more enlightening.

Glass is half full though, right ?. It will be fixed and be a better
a/craft because of it...

[Lyman]

As complex as it is, it is not complex enough. If they wire each individual battery for a heat signature, they are acknowledging how unreliable the system is. Any outlier temperature above the threshold would cause a shutdown of the entire assembly.

An admission that fire is possible in one cell is an admission that the design is unreliable, per se. And not just because of fire.

"Separation"... Ostensibly to prevent heat transfer. This system is eight batteries, in series. Lose one, lose them all. It does not matter how well the offending cell is isolated, isolation for overtemp fails the system and takes it off line.

There will be fire, that is the purpose of the upgraded box. No one installs a heavy fireproof enclosure if the chance of fire is manageable above its predicted rate of failure. And the rate of failure is not acceptable, box or no box....

Fire or no fire. Why fireproof a system that doesn't work even when it is not burning? The LiIon technology is cargo, in the hold, or as equipment.

Fire is prohibited, not mitigated.

IanW.... Functionality, (Does it "Work"?) is not required to own a Patent.

New? check.... Unique? check..... Does it work? no check..... :ok:

[Ian W]

Syseng68k
Looks like the original design did in fact only have a single temp sensor,
even if they were managing individual cell voltages. Beats me how such a
design ever got past engineering.Back at the beginning of the thread there was detail about the BMS using a patented system that managed the charging of the cells using software that predicted the temperatures from the voltage across the cells. It would appear that 'patented' is not the equivalent to validated as working.

What is Thales saying about this?- They provided the entire battery assembly and BMS.

[FlightPathOBN]

from Boeing Land....

Another person familiar with the engineering work said the new box would be made of stainless steel nearly half an inch thick. It would be capable of containing an explosion, and would have a tube to vent smoke and flame outside the jet.
However, the source said engineers have raised questions about the safety of venting flames outside the plane, especially if it is on the ground and being fueled. The effect could be something like a flamethrower, this person said.

final solution? :mad:

[Chu Chu]

Add a flame arrestor?

[syseng68k]

Back at the beginning of the thread there was detail about the BMS using a
patented system that managed the charging of the cells using software that
predicted the temperatures from the voltage across the cells. It would appear
that 'patented' is not the equivalent to validated as working.
Actually, the patented method doesn't look at temperature, but fwir, tracks
the voltage curve towards the inflexion point. Then when it gets a match
with an internal model of a cell's charging characteristics, predicts end of
charge from that. Might be a very good idea, but is a bit simplistic in terms
of what else needs to be done to properly manage the cells. While temperature
and voltage are loosly related, you can't predict temperature from voltage
alone. Edit: In fact, you can't predict temperature with any degree of accuracy
at all.

It's much easier and more complete to measure each cell's temperature and
voltage, since they are the critical limiting factors in terms of maximum
charge or discharge currents...

[FlightPathOBN]

Add a flame arrestor? :}

mid-air fueling :ok:

edit: maybe they looked up how Titanium burns!

[cockney steve]

I'll just reiterate what i posted l ooo ooo ng ago.

place each sub-cell in an insulating ceramic pot (which could have a screw-on lid in the same material.

Make the cell-tails from heat- sensitive alloy
fit each one with a temp-probe and voltage-monitoring wire.
place all three in a second fireproof ceramic pot.

The main terminals each have a busbar to which the tails are attached.
each subcell individually monitored,so much tighter control. any high current or overheating will melt the tail (fusible link) thus disconnecting the individual sub-cell. THE BATTERY WOULD MAINTAIN IT'S AVAILABILITY (unless all 3 sub-cells in a single-cell failed O.C.

Thermistors cost buttons-even "certified "ones should be well affordable within the current 16,000 dollar postulated ripoff price.

Controller/memory/discrete components are all pennies and there is no earthly reason why 48 temp and 25 voltage wires should present a problem to the control/monitoring/charging unit.

It seems no-one is willing to lose face by admitting this fitment was ill-thought-out and piss-poorly executed.

PROPERLY engineered, I'm sure the initial objectives could be met and the fusing/monitoring/insulating (heat, as well as electrical) regime would probably allow a thinner,lighter containment-vessel (sorry, "container")
Fingers crossed they rejig the charging/monitoring wherein I feel the problem lies. They're determined to stick with the technology, they HAVE to respect it's limitations and keep within the SAFE operating envelope.
@Smilin' Ed.They threw KISS out the window and sacrificed safety on the altar of weight reduction.

Unfortunately, the sodding great tin fire- box added back all the weight .

not to mention the cost of this fiasco ,to both finances and reputation.

Perhaps, the engineers KNEW but were overruled by arrogant :mad: "suits" "know it all :mad: all. Perhaps it was a genuine oversight or omission and nobody recognised the importance of a building burning.......et al.

[FlightPathOBN]

CS,

How much of the rest of the aircraft was designed and implemented the same way?

Perhaps, the engineers KNEW but were overruled

fitting a square peg in a round hole has NEVER worked....the batteries need to be round, the 'engineers' blew it on the design..like many of the other assemblies that have gone wrong...

finite element computer engineering disconnected with manufacturing capability.
:{

[MurphyWasRight]

Controller/memory/discrete components are all pennies and there is no earthly reason why 48 temp and 25 voltage wires should present a problem to the control/monitoring/charging unit.


At least for the temp sensors there is no need for more than 3-6 wires (depending on any redundancy desire) since there are plenty of serial bus (more than one flavor) temp sensors available.

Some of these also have voltage monitor inputs but then things get a bit complex due to common mode issues etc.

Accuracy is not a major factor, +- 1 degree C would suffice, rates are probably as interesting as absolute.

[cockney steve]

@ Murphy...I was trying to make the point that, even doing it on the cheap with discrete components and maximum redundancy, it wouldn't exactly be overwhelmingly complex. Every single sub-cell could be monitored and contained thus giving a hugely enhanced safety-cushion.


@flightpathOBN I still think that GS-Yuasa's construction is beyond reproach....yes, I take on board the issue of folds, expansion and contraction.

WOUND CONSTRUCTION WILL NOT STOP THAT THERMAL MOVEMENT
It is arguable that, given the binding -effect of the spiral-wrap, the crushing-pressure in the core will be far higher than in the flat-folded construction. Not to mention the heat buildup in the core....this could be mitigated by using very tall electrodes with fewer turns...then you're on to 24 long-slim, cigar-shaped cells wired in series-parallell.
Ever wound a ball of wool or string over your fingers? coiled rope round hand and elbow? Gets tight, doesn't it! :p
the batteries in service(and changed) have been abused!
other discussion in R&N, but not of the quality of this one:8

[Ian W]

Cockney Steve
the batteries in service(and changed) have been abused!

This was my feeling as well - perhaps not deliberate abuse but slavishly following the laid down procedures which perhaps have some instructions that 'assume' that nobody-would-do-that. Or perhaps procedures that assume that the aircraft will be in a particular state when something is done (e.g. all electrics powered down for a battery change.) but don't actually require it.

What abuse do you think could cause a single cell to be on the edge so recharge or even use while parked would cause a problem?

[Lyman]

Ian W

How about...
1. Overcharging
In general, lithium ion batteries are significantly more susceptible to internal failures that can result in self- sustaining increases in temperature and pressure (thermal runaway) than their nickel-cadmium or lead-acid counterparts. This is especially true for overcharging, which causes heating and destabilization of the components of the cell, leading to formation (by plating) of highly unstable metallic lithium. The metallic lithium can ignite, resulting in a self-sustaining fire or explosion. Finally, the severity of thermal runaway from overcharging increases with increasing battery capacity, because of the higher amount of electrolytes in large batteries.

"We may never know the cause..."

The cause is part of the problem, known since the advent of the technology. Given "mystery" how can anyone in their right mind propose a "fix"?

Without eliminating each possible cause, one is left with a known fire, of unknown origin...

So at least in service to the regs, and the English language, absent a known cause, there is no "fix", interim, permanent, or otherwise.

Only a "method to control spread of fire".... :}

[MurphyWasRight]

CS:
@ Murphy...I was trying to make the point that, even doing it on the cheap with discrete components and maximum redundancy, it wouldn't exactly be overwhelmingly complex. Every single sub-cell could be monitored and contained thus giving a hugely enhanced safety-cushion.



I understand your were providing a "proof of existance" not a necesarily a final dessign.

Depending on location of monitor PCB, preferably outside the "fire box" so logging would survive an 'incident', serial bus sensors would almost certainly be cheaper than individual wires given the cost of certified connectors and wire harnesses etc.

Have to say I am very puzzled that the cells are not individually monitored in the current design since that would give the best early warning of problems.

[inetdog]

cockney steve said:
WOUND CONSTRUCTION WILL NOT STOP THAT THERMAL MOVEMENT

Equally important as, or perhaps more important than, the thermal expansion is the roughly 10% change in volume of the combined electrodes, active material and separator between fully discharged and fully charged state. This is a characteristic of the LiCo electrode chemistry.
Repeated wide range State of Charge cycling may potentially be more damaging to the electrode structure than thermal cycling, especially if there are other contributing factors such as debris from manufacturing or dendrite growth.

[Lyman]

Murph...

Have to say I am very puzzled that the cells are not individually monitored in the current design since that would give the best early warning of problems.

There are at least two reasons Boeing is not puzzled. (in my opinion).

They were overconfident in the single Temp system, or they knew multiplying the sensors would result in a higher number of Battery failures.

I lean toward the latter; Boeing knew the more data they had regarding the system performance, the more failures they would incur.

This is not sinister, necessarily, it is a judgment call within the rather loose confines of their self inflicted Regulatory restrictions.....

Their decisions were based on on over reliance on optimism....

No harm, no foul.....But Hell to pay if it goes up in smoke....

[FlightPathOBN]

from Boeing Land...

Reports of Boeing 787 test flights "completely inaccurate": FAA

SEATTLE (Reuters) - The Federal Aviation Administration said it is not close to approving test flights of Boeing's 787 Dreamliner with a proposed fix for the plane's troubled batteries, denying news reports that such tests could start as early as next week.
"Reports that we are close to allowing 787 test flights are completely inaccurate," spokeswoman Laura Brown said on Tuesday in an email to Reuters.


http://finance.yahoo.com/news/reports-boeing-787-test-flights-001132267.html?desktop_view_default=true

[deptrai]

Here's an excellent overview of triggers of thermal runaway, mechanisms of internal shorting, and implications for managing field failures (particularly page 5 onwards, "Triggers for Li-Ion thermal runaway" from the annotated book facsimile) :

Lattice Energy LLC- Field Failures and LENRs in Lithium-based Batte... (http://www.slideshare.net/lewisglarsen/lattice-energy-llc-field-failures-and-lenrs-in-lithiumbased-batteriesjan-23-2013)

I would be surprised if the 787 battery incidents were triggered by external "abuse", ie the batteries were operated outside limits, although it is not unthinkable. If this was the case, the investigation should be able to find out sooner rather than later.

It seems clear that cascading was an issue - due to inadequate spacing/thermal insulation, a failed cell would release heat to neighbouring cells. This will be adressed with the proposed new design (but as many have already pointed out, it won't prevent the battery from failing, it will just fail with less smoldering).

Boeing states that clean room technology is used to manufacture the cells. This is to minimize contaminations which could later trigger an internal short (such contaminations can not yet be detected with cell screening). I would speculate that cleanliness and manufacturing QC are well managed and not an issue (it shouldn't at least :), and if it is, the investigations should be able to find out).

According to Boeing, the 2.2 Million cell hours before January 2013 were without incidents. If the 787 incidents, or one of them, are true "field failures", as described in the book, neither triggered by external influence, nor triggered by contamination introduced during manufacturing, the investigations will never be able to know with certainty what exactly happened.

Now if Boeing wants to contain the fire, and hope it's a 1:1'000'000 thing, this leads me to believe they're leaning towards field failure as a probable cause?

[saptzae]

LiCoO2 is mature technology. These cells are derated to 4.025V EOC and ultra reliable.

There is too much damage to the cells. It is improbable, that the cause of the Primary failure can be conclusively established by forensics.

Further analysis of the electrical system and of reported wiring errors will provide clues.

Now if Boeing wants to contain the fire, and hope it's a 1:1'000'000 thing, this leads me to believe they're leaning towards field failure as a probable cause?
I doubt that. They just want to make sure.

[deptrai]

LiCoO2 is mature technology. These cells are derated to 4.025V EOC and ultra reliable.

Agreed, the cells should be very reliable. According to GS Yuasa Technologies,

"[GYT] fixed their chemical design at 1999 and did not change. GYT large Li-ion cells [...] have never showed internal short accident through more than 10 years production. (>5000 cells)"

(from a sales pitch to NASA https://batteryworkshop.msfc.nasa.gov/presentations/08_Safety%20Design%20Features%20of%20GS%20Yuasa%20Lg%20Li-Ion_TInoue.pdf - which also shows the differences between "aerospace-grade" and typical "commercial" Sony cells. These GS Yuasa cells are certainly well engineered).

[FlightPathOBN]

"The battery, made by Japan’s GS Yuasa, was Boeing’s choice for the 787’s design in 2005, which Laslau says is unfortunate because its cathode was based on cobalt oxide. “The cobalt oxide chemistry that Boeing chose has a fantastic energy density, but it’s not the safest. Lithium iron phosphate would be safer,” he says. But the iron phosphate alternative wasn’t well developed seven years ago."


Boeing Battery Blues (http://spectrum.ieee.org/aerospace/aviation/boeings-battery-blues)

edit...

I still feel that a circular wind is needed..the thermal expansion would be radial and transferred along the length. Hopefully, they would be smart enough to wind it with the proper thermal spacing.
With a square wind, you have the ability to create a bend during manufacture, but even if you dont, the thermal expansion will keep hitting the corners and eventually may degrade the winding.
There is also the compression/decomp of normal aircraft ops on the battery itself

[FlightPathOBN]

From Boeing land..

Battery maker GS Yuasa believes the fix for the battery should include a voltage regulator that could stop electricity from entering the battery, the Journal said.

Boeing proposed its fix to the FAA on Friday, but on Thursday, Yuasa told the FAA that its laboratory tests indicated that a power surge outside the battery, or other external problem, started the failures on two batteries, according to the newspaper.

Boeing, battery maker at odds over 787 fix - WSJ | Reuters (http://www.reuters.com/article/2013/02/27/boeing-787-report-idUSL1N0BRFC820130227?feedType=RSS&feedName=technologySector&rpc=43)

[inetdog]

From the Reuters article:

Boeing spokesman Marc Birtel said that the investigation has not showed that overcharging was a factor and that the 787 had quadruple-redundant protection against overcharging in any case.

The admitted wiring error or component failure that allowed the APU battery to power loads that should only have been on the bus from the main battery leads me to conjecture that in the ANA case the quadruply redundant protection may have been at most singly or doubly redundant. :=

[bill good]

There has never been any sense in keeping voltage across a Li-Ion battery when 100% SOC. It is like trying to overfill a fuel tank. But this is only one hole in the swiss cheese model.

Regards

[inetdog]

bill good said:
There has never been any sense in keeping voltage across a Li-Ion battery when 100% SOC. It is like trying to overfill a fuel tank. But this is only one hole in the swiss cheese model.
Not the same Li chemistry, but I believe that at least some plug-in EV manufacturers recommend that for maximum life their battery should not be left idle for long periods with an SOC above 80%. They advise to top off the charge shortly before using it if full range is needed.
Since under expected usage, the 787 batteries will be sitting without loads almost all of their lives, this could be an issue.

[saptzae]

Cells are derated to 4.025V EOC, or about 70%-80% of max charge at 4.2V.

Consumer or EV batteries will charge to 4.1 - 4.2V.

The difference in reliability is huge. 0.1V reduction quintuples cell life.

[Lyman]

ANA: Dreamliner Battery Solution (http://www.frequentbusinesstraveler.com/2013/03/ana-dreamliner-battery-solution-quite-advanced-japanese-regulators-remain-cautious/)

[FlightPathOBN]

"A Boeing presentation in February described "baking the battery to induce overheating, crush testing and puncturing a cell with nail to induce short circuit."

At the same time the FAA approved the special conditions in 2007, FAA staff and the aircraft manufacturing industry, including Boeing, were devising lithium-ion battery tests that included all the details the special conditions lacked.

Published in 2008 and adopted by the FAA three years later, the standard known as RTCA DO-311 gave precise instructions for tests. The worst-case-scenario test required turning off all failsafe electronics, short-circuiting the battery and watching for flames for three hours.

Boeing did not run those tests. "The RTCA standards were not designed for the 787," and Boeing provided extensive testing to show the 787 met the special conditions, spokesman Marc Birtel said.

The FAA acknowledged the batteries were potentially flammable in the special conditions approved. Said former Inspector General Schiavo, "They knew they had problems. They just said 'OK.'""

Insight: Will Dreamliner drama affect industry self-inspection? | Reuters (http://www.reuters.com/article/2013/03/02/us-boeing-787-oversight-idUSBRE92104W20130302)

[FlightPathOBN]

ANA says it had Dreamliner power distribution panel trouble three times

ANA says it had Dreamliner power distribution panel trouble three times - Yahoo! News (http://news.yahoo.com/ana-says-had-dreamliner-power-190343000.html)

Japanese union uneasy with 787 electrical power panels | Business & Technology | The Seattle Times (http://seattletimes.com/html/businesstechnology/2020496541_787japancircuitsxml.html)

At least the circuit panels are made in Mexico, not Japan...:mad:

[fizz57]

I think we might be getting closer to the root cause of the battery problems. Anyone have any idea how solid state isolators and diodes (and lithium batteries!) react to conditions that burn out circuit boards? Even if described by Boeing as "a low energy arc that lasted milliseconds, very small", what happens to the bus voltage during the incident? How does the battery isolation/charging circuitry react? Has this been tested for on the bench?

[toffeez]

“The risk to the company is not this battery, even though this is really bad right now,” said one 787 electrical
engineer, who asked not to be identified. “The real problem is the power panels." Unlike earlier Boeing jets, he said,
the innards of the 787 power distribution panels — which control the flow of electricity to the plane’s many
systems — are “like Radio Shack,” with parts that are “cheap, plastic and prone to failure.”
Seattle Times.

http://imageshack.us/a/img705/5265/2013409801.jpg (http://www.pprune.org//api.viglink.com/api/click?format=go&key=04ae6db96709241d69f389137ea2abf9&loc=http%3A%2F%2Fwww.pprune.org%2Frumours-news%2F505455-faa-grounds-787s-55.html%23post7719637&v=1&libid=1362644275242&out=http%3A%2F%2Fapicdn.viglink.com%2Fapi%2Fclick%3Fformat%3 Dgo%26key%3D04ae6db96709241d69f389137ea2abf9%26loc%3Dhttp%25 3A%252F%252Fwww.pprune.org%252Frumours-news%252F505455-faa-grounds-787s-55.html%26out%3Dhttp%253A%252F%252Fimageshack.us%252Fa%252Fi mg705%252F5265%252F2013409801.jpg%26ref%3Dhttp%253A%252F%252 Fwww.pprune.org%252Fsearch.php%253Fsearchid%253D1053618&ref=http%3A%2F%2Fwww.pprune.org%2Fsearch.php%3Fsearchid%3D10 53618&title=FAA%20Grounds%20787s%20-%20Page%2055%20-%20PPRuNe%20Forums&txt=%3CIMG%20border%3D0%20alt%3D%22%22%20src%3D%22http%3A%2F %2Fimageshack.us%2Fa%2Fimg705%2F5265%2F2013409801.jpg%22%3E&jsonp=vglnk_jsonp_13626442820932)

[cwatters]

Anyone have any idea how solid state isolators and diodes (and lithium batteries!) react to conditions that burn out circuit boards?

I'm an electronics engineer. There is nothing like enough information in those articles to comment on how one part of the system might respond to a fault in another. Common sense at the design stage would require each system to isolate itself from others in the event of an internal fault (eg breakers/fuses used so that a short circuit doesn't have consequences outside that part of the system). I've not seen any reports that the aircraft that had power panel fauts was the same aircraft that had battery faults.

[fizz57]

Agree 100%. My only point is that this new information about failing panels does indicate that something in the 787's electrical systems is not behaving as intended, with collateral effects that may not be limited to shorting distribution panels.

I don't buy the argument that the panels themselves (or the batteries, or the chargers) are under-engineered. Today's components may have more silicon than copper but are just as robust. And in any case bench testing will have shown up any obvious shortcomings.

Funny how this information about line failures is leaking out slowly (and not from Boeing). First the battery replacements, now this. What's going to be next?

[FlightPathOBN]

cwatters, fizz,

Exactly...

As noted in the voluminous amount of article(s), most of these issues were not reported. The 150 batteries, the panels, and who knows what else...we only find out in small controlled batches of disconnected information. I cant really believe that there have been 150 batteries returned for a single issue with the battery alone. There arent that many aircraft flying, and only what, 16 batteries per ac...

The articles note that everyone is deliberately vague, and in all cases, wont identify the aircraft the issue was found on.

Like many other directives, inspections will be ordered on all aircraft, so you may see many other aircraft with these issues, once people start looking.

[Lyman]

Shall we at least try to determine these "circuit boards" problems are NOT the printed circuit boards in the battery case?

Power Panels could easily be confused with "circuit boards".

Perhaps the 150 battery replacements had to do with failures in the battery's own "circuit boards"?

BLOOMBERG is reporting ANA leaks as "circuit boards".

Flight Path.... are you on board with the "Eight Battery" Battery? Not the "Eight Cell" "Battery"?

Boeing used the "Eight Cell" concept to inflate their line experience.

Eight Batteries, in series. IMO.

[Chris Scott]

It's time for another statement from Boeing/FAA/NTSB. This rumour mill is getting out of hand. Surely the customer airlines cannot be expected to do it. The credibility of the joint world-leader in airliner manufacture is slowly bleeding. Not a prettty sight.

Lyman

Quote:
.... are you on board with the "Eight Battery" Battery? Not the "Eight Cell" "Battery"?
(unquote)

I presume you, like me, have at least one conventional automobile, using recent - but not cutting-edge - technology. They each have a 12V DC electrical system, whose en-route power supply is some kind of engine-driven generator, backed up by a battery to cater for short-term contigencies. However, the main task of that battery is to provide a very high current for a few seconds to turn the engine's starter motor. (With me so far?)

In most cases, the 12V battery will be of the lead-acid type, consisting of eight 1.5V cells in series.

Are you telling me now that my car has 8 batteries? If not, please explain the difference in plain, simple English, if that's possible.

Quote:
Boeing used the "Eight Cell" concept to inflate their line experience.
(unquote)

Can you decode that for me?

[rogerg]

In most cases, the 12V battery will be of the lead-acid type, consisting of eight 1.5V cells in series.
In a car the battery is six 2V cells ie 12 V

[Lyman]

Hi Chris...

I use the concept "eight batteries" and have from the OP.

A cell, is "wet", a "component" of a battery. A battery is an enclosed, individual unit capable of installation as a self contained power storage system, in connection with other similar components, or solitary.

Simply, a "Battery" means "collection of".

Each battery in the Yuasa container is capable of being isolated, and performing a storage function, sealed from the others.

So I consider each battery as composed of three separate electrodes, or cells, producing a typical voltage when discharging.

Boeing used the experience of each separate battery in its claim of fleet hours....

As they should, but they also did not rely on that description in the Press.

In the Yuasa application, any battery failure fails all eight.... The series is broken.

That is why it was important for Boeing to allow both descriptions in the Press (imo). They could refer to a cell failure, when in fact, a cell failure meant the entire system was failing. They did not want to portray the whole battery system as that vulnerable to failure, it trashed their predictability....

Your car battery has six cells. It is a battery, unto itself.

The blue box in the Boeing contains eight three cell batteries, in series.

[mm43]

NTSB - Boeing 787 - Interim Report - March 7, 2013 (http://www.ntsb.gov/investigations/2013/boeing_787/interim_report_B787_3-7-13.pdf)

Detailed Analysis.

[FlightPathOBN]

From Boeing Land...wow...

Boston airport firefighters encountered sizzling liquid and a hissing, “exploding” battery when they entered the 787 at the center of a two-month-long National Transportation Safety Board investigation, according to documents released Thursday.

The NTSB said Thursday it plans two public hearings next month, one to explore lithium-ion battery technology in general and another to discuss the design and certification of the Boeing 787 battery system. (Emphasis added...at least another month before hearings)

The safety agency announced the hearings as it released an interim factual report and 499 pages of related documents on its investigation of the Japan Airlines 787 fire at the Boston airport on January 7.

Among the findings in the documents released Thursday:

• On the day of the Boston fire, the battery did not behave as Boeing or subcontractor Thales predicted.

The battery’s power discharge was “not at the constant rate described by the Boeing or Thales documents and included large changes and reversals of power within short periods of time,” according to the Airworthiness Group Chairman Report.

• Sitting on a rack above the battery that burned was a smaller lithium ion battery, also supplied by Japanese manufacturer GS Yuasa, that is used to provide emergency power for the jet’s flight controls “for a minimum of 10 minutes when no other electrical power is available.”

Investigators found the exterior of this battery had been “lightly scorched” by the fire below and noted that its case had openings at the corners.

• No heat damage was found to any primary airplane structure.

However, the floor panel and carbon fiber floor support material, which are considered to be secondary structure, “were found to be heat damaged beneath where the APU battery had been installed.”

• The firefighters who were called to put out the fire did not know they were dealing with a lithium-ion battery, and had great difficulty putting out the intense fire.

When Capt. Mark Munroe of the airport’s aircraft rescue and firefighting (ARFF) unit entered the plane, he “saw heavy white smoke billowing through the floor” of the passenger cabin.

After locating the fire inside the electronics bay in the belly of the airplane, firefighters entered the compartment through dense smoke and applied shots of Halotron fire extinguisher to the battery.

Lt. David Hoadley of the ARFF unit reported that “It seemed like the fire did not want to go out, it kept rekindling.”

Then the battery “exploded,” according to Capt. Monroe.

“Capt. Munroe heard the battery hissing still and pushing white smoke or steam. There was liquid sizzling over the sides of the battery and still heavy smoke conditions. ... The battery continued to hiss before exploding.”

Monroe related that “he felt something hit him in the neck while he was in the airplane,” and he was sent out for medical treatment. “Something had burned his neck.”

Firefighters attempted to remove the battery from the jet, but found that the “quick disconnect” mechanism Boeing had included to allow mechanics to take out the battery for maintenance was “melted and un-recognizable” and a metal plate was preventing access.

The battery had to be cut out from the rack where it sat.

“With a hot battery and a gloved hand (Lt. Hoadley) could not access the bolts on the lower rails with tools. They attempted with pliers to remove the bolts for maybe 20 minutes. What looked like Teflon slides were burnt away and the battery would not move. There were 3 more screws that could not be removed.”

Firefighters cut away the metal plate, severed the battery wire, then “pried the battery loose with hydraulic spreaders and removed it.”

The battery was passed down to a firefighter and placed on the tarmac about 50 feet from the airplane.

The fire was declared under control an hour and forty minutes after the initial notification.

Boeing’s entire fleet of 787s has been unable to fly since two battery incidents in January prompted a Federal Aviation Administration order grounding the planes.

The company’s proposed tests of improvements to its battery system are currently under review by the FAA, which is expected to make an initial recommendation on those plans next week.


NTSB describes sizzling, hissing 787 battery in Boston fire | Business & Technology | The Seattle Times (http://seattletimes.com/html/businesstechnology/2020505762_ntsb787reportxml.html)

[Chris Scott]

rogerg,

Thanks for spotting my "deliberate" (!) mistake. :{

Hi Lyman,

I now understand the thrust of your argument - thanks. Don't accept all of it, however. It's true that the six 2V cells in a 12V lead-acid car battery can't be separated without destroying the battery (AFAIK).

You are also right to point out the cascade effect from one short-circuited cell to its neighbours, which doesn't seem to happen in lead-acid batteries (and perhaps not in NiCds, but I'm not sure).

I infer you are saying that Boeing has claimed that an hour of battery time represents eight hours' experience. If so, can you direct me to a quote?

You argue that, because each 3.7V LVP-65 cell (which you refer to as a "battery") has three electrodes, that equates to having 3 cells within each. That seems a non-sequitur to me, and the Yuasu LVP-65 data sheet (http://www.pprune.org/It's true that the six 2V cells in a 12V lead-acid car battery can't be separated without destroying the battery (AFAIK).)suggests otherwise. The LI-Ion battery in my mobile phone also has a nominal voltage of 3.7V, so does it have three cells? Or are your three cells wired in parallel?

[Chris Scott]

Today's NTSB Interim Factual Report indeed makes interesting reading.

APU started at 1004 EST, battery failed at 1021 (only 1 minute after the flight crew had disembarked).

So the failure occurred about 16 mins after the recharge cycle had begun. Must admit I had expected a longer time-frame. I wonder what the temperature of the battery was before and after APU start.

[Lyman]

Hi Chris.

Didn't APU start off one of the donks?

Time
Event
1000:24
Airplane touched down.
1004:10
APU started.
1006:15
Airplane completed turn into parking location.

1006:48
Parking brake set.

1006:52
Engine 1 shut down.
1006:54
Engine 2 shut down.

1021:01
APU battery bus voltage decreased from 32 to 31 volts.

1021:04
APU battery current increased to between 44 and 45 amperes for 4 seconds, indicating current flowing into the battery.

1021:07
APU battery bus voltage decreased to 30 volts.

1021:09
APU battery bus voltage decreased to 29 volts.

1021:10
APU battery bus voltage increased to 31 volts.


I am certain of Boeings claim to have an hours experience for each "Cell". I could look, but am not good in that sort of thing, I am computer marginal.

Put it on hold, I'll try.

[archae86]

The March 7 released materials lab report has considerable detail on the battery covering some points which have been considered open on this form.

It can currently be found at:

http://dms.ntsb.gov/pubdms/search/document.cfm?docID=388418&docketID=54251&mkey=85973

On my browser (firefox), clicking the download button five minutes ago generated an exception, but clicking the view button downloaded the pdf to my browser, from whence I was able to download the full pdf to my disc.

Among other interesting points, the "Description of the battery" section covers in more detail than I've previously seen the temperature monitoring technique, and responses, and other battery management details. Also mentioned are the container material and thickness, cell manufacturing dates and serial numbers, and so on.

All this in addition to considerable additional detailed observations from the event battery itself.

I think the materials lab report may contain more material of interest to participants in this thread than does the interim factual report also released today. However the NTSB server is currently responding very slowly, so some patience or retries may be needed.

[syseng68k]

archae86:

I get a few lines of javascript from the link, which says access denied.
Have tried using firefox and IE with the same result, do you have a link
that works ?.

Was able to find the other report from the NTSB website and some things
I did notice from an initial scan are that there are only two temp sensors,
but also that the isolation contactor in the battery case has normally
closed contacts. It's driven by logic on the BMU boards, also inside the case,
so if they fail for whatever reason, there is no way that the battery can be
disconnected from load or charger.

It would be much safer to make the contacts normally open, so if the power
or system fails for any reason, the battery is automatically disconnected...

[archae86]

The Pprune site inserts considerable overhead on links, presumably as part of a scheme to get paid for forwarding traffic in some cases.

If you can get to the NTSB site, you may find it helpful that the docket ID for this matter is 54251.

Before PPrune tampers with it, the following is a direct link to the docket page, and this time I've asked PPrune not to "automatically parse links in text". Possibly it will work better for some.

http://dms.ntsb.gov/pubdms/search/hitlist.cfm?docketID=54251&CFID=2871&CFTOKEN=67900912

I did test the previous one before posting--and aside from the mentioned long delays an intermittency, it did work in both FireFox and Chrome. For this one, you would need to copy and paste the URL I provided directly to a browser address bar, then wait a while, then click on the March 7 Materials lab report link, wait some more, then click on the "View" button. This all worked in both Firefox and Chrome for my own testing just a few minutes before finalizing this post.

A more direct link to the actual March 7 materials lab report page, if it works for you, should only require an initial wait, then clicking the View button, then a download wait--but won't work well if your browser is not configured for convenient pdf viewing.

http://dms.ntsb.gov/pubdms/search/document.cfm?docID=388418&docketID=54251&mkey=85973

That last link I tested in Internet Explorer as well. Assuming the PPrune site does not hide these (they were plainly visible and able to be copied in the preview I could see) they should work.

[FlightPathOBN]

arch,

What are you talking about, the direct link has been posted several time in this thread and others...

[inetdog]

Lyman:
Your car battery has six cells. It is a battery, unto itself.
The blue box in the Boeing contains eight three cell batteries, in series.
If you look inside a lead-acid battery, you find that there are often multiple positive and negative plates, with individual separators, layered inside a single electrolyte-containing envelope constituting one cell. The same appears to apply to the Boeing "cells", although the physical separation of the sets of electrodes is admittedly a lot greater.
So, I am willing to consider that some aspects of the cell can be interpreted more easily by thinking of them as three unit cells in parallel, I am not willing to go all the way and say that they should be categorized that way.
I will accept the description of the blue box as a battery bank rather than a battery because it does have individually removable components.

[inetdog]

I hope you do not mind "reprints", for this is based on my comments from a thread on the same topic in another forum:

From the Interim Report:
The APU controller (discussed in section 1.6.5) monitors the parameters that are needed to operate the APU. The APU controller is powered by the APU battery bus, which receives its power from the APU battery. If the APU battery fails, then the APU battery bus will no longer receive power, and the APU will shut down.
And the ventilation fans which will remove any smoke resulting from the event will shut down too.
The ventilation fans in question are probably powered by multiple sources, including the total of 4 engine driven generators, and during flight the fans would therefore likely work OK.
The problem here is definitely that when the battery is both the only power source for the fans and the battery is also the source of the smoke (or perhaps there is NO battery power for the fans) you have an unsurmountable logical problem.
It is much less likely that the ventilation would have failed had the plane been in flight, especially as there would, at least in the new design, have been some ventilation from cabin pressurization and low pressure on the outside of the flame thrower -- I mean vent duct.

This seems even worse design to me though:
Firefighters reported that removing the battery was difficult because a metal kick shield installed in front of the battery prevented them from accessing the battery’s quarter-turn quick disconnect knob. Also, the quick disconnect knob could not be turned because it was charred and melted.
!!!:
The stainless steel sleeve and signal wires had damage consistent with excessive electrical current where they attached to the connectors at each end; at the battery case, the damage was also consistent with fire.
Excessive current on the signaling wires to the battery charger? Maybe just ground current in the signal grounds and the shield because of internal short circuit inside the battery case? At any rate, it does not appear to have damaged the charger.

Only two temperature sensors in the box (thermistors), possibly one for each of two BMU card? The 8 wires from each BMU card were described as voltage sensing wires, but probably doubled as balancing current conductors. The two cards did not contain two independent Battery Management Systems (BMSs) but were rather parts of one unit Battery Management System.

FDR data showed that, at 1021:01, a 1-volt decrease from the designed voltage of the APU battery (32 volts) was recorded. Three seconds later, the data showed a change
in current flow to 44 to 45 amperes into the battery. The battery voltage continued to decrease, and, at 1021:08, the current flow returned to 3 amperes out of the battery. At 1021:30, the battery voltage decreased to 28 volts, and the APU shut down 7 seconds later.30 Table 2 shows selected events recorded before and after the APU shutdown. The FDR did not record any data indicating that the APU battery voltage had exceeded 32 volts.
Not that it necessarily contributed to the event, but the initial sudden 1 volt anomaly did not cause the charger to shut down! This continued for 29 seconds. The steady decrease in battery voltage while charging also was not seen as significant!! The charge rate indicates that for whatever reason (probably the routine APU startup 15 minutes earlier) the battery was still substantially discharged at the time of the event. It was also being rapidly recharged after the substantial load of the APU start. No temperature information seems to have been recorded (?). I am also open to speculation that if the original voltage drop represented an internal short circuit in one cell, then the remaining cells could easily have been overcharged without the charger realizing it from the series voltage. Have to look at the BMU data to be sure.

OOPS:
The BMU main circuit card and sub-circuit card do not contain nonvolatile memory (NVM), and none of the BMU data are recorded on the FDR.

I think that the current flow reported is from the hall effect sensor, positioned like a shunt, in the battery itself and so represents net charging current to the battery independent of what the charge controller may have been delivering to the bus at the same time.

Aft EAFR stopped recording. Forward EAFR continued recording for about 9 minutes
58 seconds.
When the battery went offline and the APU spun down, there was no longer a source of power to the rear EAFR.



And, the last of my stream of consciousness commentary for now:
The battery shall be designed to prevent spilling flammable fluid, a hazardous event with occurrence with a probability of less than 10-9.
Much has been made in the press with the fact that two battery events have occurred within the first 50,000 flight hours that should have had a probability of 1 in 10,000,000 hours or less. If you think that the evidence of either of the two events supports that flaming electrolyte was in fact discharged, we have now hit the 1 in 1,000,000,000 condition too! Note that it was not just flammable but actually burning fluid. Maybe that's safer?

[Chris Scott]

Been out all evening since downloading it, but now see some have been struggling to locate it. Seems to be slightly different from the one posted above by archae86, but If his or others don't work, readers might like to try this one for the
NTSB Docket Management System:

Document 7 Airports/ARFF 16 - Attachment 3: BOS ARFF Incident Report Filing Date February 12, 2013 5 page(s) of Image (PDF or TIFF) 0 Photos (http://dms.ntsb.gov/pubdms/search/document.cfm?docID=388479&docketID=54251&mkey=85973)

I commented earlier:
"So the failure occurred about 16 mins after the recharge cycle had begun. Must admit I had expected a longer time-frame. I wonder what the temperature of the battery was before and after APU start."

Quote from Lyman:
"Didn't APU start off one of the donks?"

Yes and (maybe) no. I'm not sure our experts in PPRuNe know enough about the system to establish what proportion of the APU starter-generator load is taken by the main electrical system (when available, as in this case), and how much (if any) is taken by the APU battery. On my last a/c type, the battery is used to supplement the main-system TRs in supplying the 700-1000 amps initial load. However, that system architecture may not be necessary or applicable on the B787.

If the APU battery took no significant part in the APU start, it seems an odd coincidence that the failure should have taken place so soon after APU start, at the end of a 12-hour flight.

To indulge in speculation, the only other possible triggers that spring to mind might be the changes in pressure associated with the recent descent and landing, or the very recent temperature changes caused by the opening of the rear cargo doors adjacent to the Aft E&E Bay. Does anyone know the temp at BOS?

PS
The link I posted above doesn't work for me right now, and neither does the manual one posted by archae86. Part of the problem may be at the NTSB end, I guess.

[inetdog]

Chris Scott:
If the APU battery took no significant part in the APU start, it seems an odd coincidence that the failure should have taken place so soon after APU start, at the end of a 12-hour flight.

For whatever reason, the battery was still being charged a rate of ~45 amps 15 minutes after the APU start. Since there should be no other loads in flight (unless the BOS plane had the same cross-wiring problem as the other), we can safely assume that it at least contributed to the APU starting current.

[bjm_bi]

Lyman,
I don't know if this is what you saw, but there's a reference here to the 787's "cell hours".
Inside the 787 (http://boeingblogs.com/randy/archives/2013/02/inside_the_787_1.html)

"The 787 main and APU batteries, each with eight cells, have logged more than 2.2 million cell-hours on the ground and in the air since the airplane entered revenue service, including more than 50,000 revenue service flight hours."

This inflates experience IMO.

[saptzae]

Confirmed no NVM / no logging of details.

Scenario:


10:21:01 Primary failure - cell short of #3



10:21:04 Secondary failure - Large current (45A) for 3 seconds overcharging and damaging remaining 7 cells and driving them into _electrically_ induced runaway. TBD whether this is due to charger not shutting down or parasitic connection to a power source (Main Bat).
10:21:27 Tertiary failure commences. Electrically induced runaway of #1, #2, #4, short of cell #6 along the way feeding more heat into #5, #7, #8. Smoke and fire.

TBD what caused Primary failure.

I suspect that NTSB schematic is simplified. It appears that there is no cell balancing. This would be ridiculous because it would lead to cell overvoltage and to Primary failure.

[archae86]

I suspect that NTSB schematic is simplified. It appears that there is no cell balancing. This would be ridiculous because it would lead to Primary failure.

I agree that the schematic obviously does not show all connections or components. Regarding balancing, there is a small explicit reference in the Materials Laboratory Factual Report released on March 7.
Page 4 of the lab report includes the text "The BMU performs cell balancing on cells over 4.00 V and monitors overall battery voltage for under-voltage conditions."

[TURIN]

The charge rate indicates that for whatever reason (probably the routine APU startup 15 minutes earlier) the battery was still substantially discharged at the time of the event. It was also being rapidly recharged after the substantial load of the APU start.

And...

Quote from Lyman:
"Didn't APU start off one of the donks?"

Yes and (maybe) no. I'm not sure our experts in PPRuNe know enough about the system to establish what proportion of the APU starter-generator load is taken by the main electrical system (when available, as in this case), and how much (if any) is taken by the APU battery. On my last a/c type, the battery is used to supplement the main-system TRs in supplying the 700-1000 amps initial load. However, that system architecture may not be necessary or applicable on the B787.

If the APU battery took no significant part in the APU start, it seems an odd coincidence that the failure should have taken place so soon after APU start, at the end of a 12-hour flight.




My understanding of the APU start system is that one Starter/Gen is powered from the main electrical busses and one from the APU battery bus.

It is an either/or scenario. Not both.

If main busses are powered then that's what is used, if not, it's over to the battery.

This concept of the APUC being powered by the APU battery however, regardless of main bus power does seem odd.
I'm not sure if this is the case on other a/c types. Research required methinks.

[hAnd]

From the Factual Report:

"These features included thermal protection devices, circuitry to monitor cell and battery voltages and temperatures, circuits to ensure that all cells in a battery are charged equally and within safe voltage limits, and components and circuitry that discontinue charging of the battery when conditions warrant this action."

I wonder how the cell balancing works, if only cell measurements are possible, but no other reaction than stop charging at overall battery level (via message from BMU to BCU or opening the contactor by BMU4).

I see no possibility to charge the cells separately, only in series connection. So I am really curious how they "ensure that the cells ... are charged equally".

Assuming a cell short circuit as primary event, it seems, the BCU manufacturer's patented "voltage monitoring only" cell diagnostics failed to detect or failed to report (to BCU) or failed to open the contactor (as a redundant charge-shutdown path, I guess).

Individual cell temperature measurement at the cost of some cents would have given a higher confidence in cell failure detection, instead of the shaky voltage monitoring, that Securaplane is so proud of.

[Chris Scott]

Quote from TURIN, responding to the sugestion that the APU may contribute some power to APU start even when main electrical power is available:

"This concept of the APUC being powered by the APU battery however, regardless of main bus power does seem odd.
I'm not sure if this is the case on other a/c types. Research required methinks."

I can't comment further on the 787 architecture. But I can repeat an ex-pilot's rough understanding of the fix used by Airbus to solve APU start problems on the A320 - in case it helps as background info.

During an early A320 demo flight in 1988, with the president of France on board, an APU start resulted in a TR failure, and a shedding of electrics. At that time, the system used only one of the two main TRs to provide the DC load of >700 amps. The TR was plainly not man enough for the task.

The chosen fix was to introduce the second main TR into the loop, and let it share the load. However, in later years, it became clear to me that the two main batteries (there's no APU battery on non-ETOPS A320s) were also sharing the load. You could see it quite clearly by selecting the ELEC page on ECAM during the APU start. Cannot detail the mechanism involved, but it seems that once the DC voltage falls below the voltage of each battery, that battery's contactor closes. So the two main batteries' contactors are not inhibited from closing during an APU start. (Whether the system has been changed since I retired, I don't know.)

The 787 has radically different architecture, with a dedicated APU battery and (presumably) much more powerful TRs.

Moving on, I wonder if the number of APU starts using the APU battery alone (on the ground before engine start and with no external power available) has been recorded. The report quotes the JAL captain as stating simply that the APU had previously been "turned on about 30 to 40 minutes before the a/c left the gate at NRT".

[saptzae]

After cell failure, 45A went into the battery for 3 seconds and triggered remainder of the mess.

Timely disconnection is key after a cell failure. Within 10-20 milliseconds that is. Faster would be better, it could avoid discarding other cells.

Detection of cell voltage or battery current transients is the only way to detect cell failure fast enough.

Cell temperature rises way too slow and can not contribute to detection of cell failure at all, it is simply too late then.

[Ian W]

Chris Scott
To indulge in speculation, the only other possible triggers that spring to mind might be the changes in pressure associated with the recent descent and landing, or the very recent temperature changes caused by the opening of the rear cargo doors adjacent to the Aft E&E Bay. Does anyone know the temp at BOS?

BOS Jan 7: Min temp 29F Max temp 39F Mean 34F - at 10:10 I would expect it to be around 32F at a guess. So nothing particularly cold.

[FullWings]

I think the BMS did detect a battery fault, as there was an EICAS alert to that effect around the time the APU shut down, according to the report.

Shame that there was nothing it could do to stop the battery bursting into flames. I think the alert was shown as: ''GOODBYE AND GOOD LUCK...''

[Kiskaloo]

In regards to the power distribution panel issues, Boeing traced them back to a bad batch from the supplier. In addition to the three NH events, a UA 787 had the same problem and diverted to MSY. QR also reported a problem with a panel on a 787 delivery flight and AI experienced issues on some of their 787s, as well.

@saptzae
Detection of cell voltage or battery current transients is the only way to detect cell failure fast enough.

Which according to an earlier post on the 787 battery subsystem is something the Battery Management System does on a per-cell basis (in addition to monitoring the cell's temperature).


@FullWings
I think the BMS did detect a battery fault, as there was an EICAS alert to that effect around the time the APU shut down, according to the report.

And I believe I recall reading that the EICAS was alerting the flight crew of NH692 to issues with the voltage and temperature of the Ship's Battery.

[Chris Scott]

Quote from inetdog:
"For whatever reason, the battery was still being charged a rate of ~45 amps 15 minutes after the APU start. Since there should be no other loads in flight (unless the BOS plane had the same cross-wiring problem as the other), we can safely assume that it at least contributed to the APU starting current."

I think the 45 amps was part of the failure sequence, and the report says it lasted only 4 seconds.

Quote from Ian W:
"BOS Jan 7: Min temp 29F Max temp 39F Mean 34F - at 10:10 I would expect it to be around 32F at a guess. So nothing particularly cold."

Thanks - call it zero Celsius, and cargo doors open 13 mins or less, with Aft E&E bay doors probably still closed. Electronics bays tend to be fairly warm in flight.

[RR_NDB]

Hi,

Events surrounding auxiliary power unit shutdown. (http://www.ntsb.gov/investigations/2013/boeing_787/interim_report_B787_3-7-13.pdf)

1000:24 Airplane touched down.
1004:10 APU started.
1006:15 Airplane completed turn into parking location.
1006:48 Parking brake set.
1006:52 Engine 1 shut down.
1006:54 Engine 2 shut down.
1021:01 APU battery bus voltage decreased from 32 to 31 volts.
1021:04 APU battery current increased to between 44 and 45 amperes for 4 seconds, indicating
current flowing into the battery.
1021:07 APU battery bus voltage decreased to 30 volts.
1021:09 APU battery bus voltage decreased to 29 volts.
1021:10 APU battery bus voltage increased to 31 volts.
1021:15 EICAS message discrete indicated that the APU battery failed.
1021:27 APU battery bus voltage decreased 1 volt per second during the next 3 seconds until
reaching 28 volts at 1021:30.
1021:37 APU battery bus voltage decreased to zero volts and returned to 28 volts three times, and
APU battery current began to move between zero and -4 to -5 amperes, indicating current
flowing out of the battery.
1021:37 APU controller went offline, and APU shut down.
1021:37 Aft EAFR stopped recording. Forward EAFR continued recording for about 9 minutes
58 seconds.
1021:40 EICAS message discretes indicated that the left and right 1 and 2AC buses became
unpowered.
1021:41 EICAS message discrete showed that the APU battery failure was no longer indicated.
1022:00 EICAS message discrete indicated that the main battery was discharging.
1022:10 APU controller was back online.
1022:53 EICAS message discrete indicated that the main battery power switch was off.
1023:16 Airplane systems providing data to the EAFR had shut down.
1031:35 Forward EAFR stopped recording.
Note: EICAS, engine indicating and crew alerting system. The APU controller is the source of 32 recorded parameters,
includingAPU shaft speed and APU battery bus voltage.

:confused::mad::suspect:

1021:37 APU battery bus voltage decreased to zero volts and returned to 28 volts three times, and APU battery current began to move between zero and -4 to -5 amperes, indicating current flowing out of the battery.


After the battery failure (1021:15 EICAS message discrete indicated that the APU battery failed. ) the (failed) battery was still in the circuit!

:mad:

1021:01 APU battery bus voltage decreased from 32 to 31 volts.
1021:04 APU battery current increased to between 44 and 45 amperes for 4 seconds, indicating current flowing into the battery.


This is another :suspect: point:

If (as it seems) batt. was out of circuit and it´s voltage suddenly drops the algorithm should verify WHY, before trying to charge at this rate.

So, in a first look (still lacks info) we may think the charging system "aggravated" the problem (one cell probably "failing=shorting")

[USMCProbe]

The 787 APU has two starter/generators each. One powered by the APU battery, one by "something else". If ships power is available, the starter that is powered by ships power starts the APU. Taxiing in the APU battery should not have been used for APU start.

I also read the 45 amps from the report. 45 amps wouldn't come close to starting a 787 APU. They presented the 45 amp flow as a fact, but I don't remember them guessing why.

[RR_NDB]

Hi,

WHy?

1021:37 APU controller went offline, and APU shut down. (http://www.ntsb.gov/investigations/2013/boeing_787/interim_report_B787_3-7-13.pdf)

Repeating:

Why?

[Lyman]

Hi RR

The APU controller is powered by the APU Batteries. If the Controller shuts down, as a result of battery failure, it would not be good to keep the (uncontrolled) APU powered up?

[RR_NDB]

Hi,

Objective should be, must be and can be:

Fault tolerance and Graceful degradation.

It´s ABSOLUTELY possible to start it (APU), keep it "well managed" even with the loss of the (APU) battery.

So, simply

I question the design!

[Kiskaloo]

With other aircraft families, if the APU controller goes offline, does the APU continue to run? Or would it shut down, as well?

The APU controller went back online at 1022:10 - in such a scenario, can the 787 APU restart if there was power available (say via the engine generators)? Or must the APU battery be energized and available in order to start the APU on the 787?

[Lyman]

Hi RR..

If one stops to consider the method of construction of the Batteries, then reviews the Failure Analysis Associates link, for spontaneous failures, there are some possibilities remaining to explain the failures.

The construction of each battery requires delicate manual handling, perfectly clean ingredients, and materials that perform within narrow limits, in the interest of performance.

Each very small "shutdown event" increases the internal resistance, and lessens the capacity of the battery. As capacity decreases, demand increases, (as a percentage of "available") exacerbating the failure path.

Ever suspicious, I return often to the "150" Field replaced Batteries. Something was wrong with them, so there was some cause that created the "problem".

Did Boeing replace three batteries per fleet unit without knowing the cause for failure? That would be most troubling.

But if they knew the cause, how is it NTSB does not know it? Even more troubling.....

And why would NTSB use the associated Battery (MAIN) from the incident aircraft as an exemplar?

Budget trouble? (:p)

cheers

[inetdog]

kiskaloo:
Or must the APU battery be energized and available in order to start the APU on the 787?
According to the description in the report, the controls for the APU are powered only from the APU battery bus.
The battery bus might also be powered from the charger (BCU) if the battery goes open circuit, but apparently when the battery fails in a short circuit mode or high enough level fault alarms go off the battery bus cannot receive external power and so the APU cannot run.

Lyman:
And why would NTSB use the associated Battery (MAIN) from the incident aircraft as an exemplar?
Similar use history and manufacturing batch, maybe?
Easier than searching other aircraft in use to find a qualified exemplar. Or is the manufacturer required to archive one or more sample batteries per batch?

The exemplar has (so far) only been subjected to non-invasive examination.

[Lyman]

inetdog

The battery bus might also be powered from the charger (BCU) if the battery goes open circuit, but apparently when the battery fails in a short circuit mode or high enough level fault alarms go off the battery bus cannot receive external power and so the APU cannot run.

So if in an emergency, the APU is powering the aircraft, and the APU BATT decides to short circuit, it takes the APU along with?

What?

Also...

The exemplar has (so far) only been subjected to non-invasive examination.

I suppose "Exemplar" could be defined in more than one way. I assume it is used as: Undamaged, unused, for comparison. The Main Battery in JAL is not strictly speaking an exemplar, it is involved in the circuitry intrinsically.

Definition of EXEMPLAR: one that serves as a model or example: as . a: an ideal model . b: a typical or standard specimen

It (JAL008 MAIN) really is not an 'exemplar'.

Right, picky.

:ok:

[RR_NDB]

Hi,

Did Boeing replace three batteries per fleet unit without knowing the cause for failure? That would be most troubling.

But if they knew the cause, how is it NTSB does not know it? Even more troubling.....


:{

[RR_NDB]

Hi,

inetdog

The battery bus might also be powered from the charger (BCU) if the battery goes open circuit, but apparently when the battery fails in a short circuit mode or high enough level fault alarms go off the battery bus cannot receive external power and so the APU cannot run.

:{

[mm43]

... apparently when the battery fails in a short circuit mode or high enough level fault alarms go off the battery bus cannot receive external power and so the APU cannot run.If that is the actual design criteria, some heads need more than banging.

The APU once online should be self sufficient, and not beholden to the state of APU battery.

[RR_NDB]

Hi,

mm43

The APU once online should be self sufficient, and not beholden to the state of APU battery.

:ok: Perfect!

[Chris Scott]

Quote from USMCProbe:
"The 787 APU has two starter/generators each. One powered by the APU battery, one by "something else". If ships power is available, the starter that is powered by ships power starts the APU. Taxiing in the APU battery should not have been used for APU start."

Thanks, that was the vital factor that was missing in my discussion. :ok:
The 45 amps was going into the battery, BTW, 16 minutes after the APU had started, for just 4 seconds; apparently just a symptom of the failure sequence. So, bearing in mind that the battery seems to be uninvolved in the APU start, what made the APU battery fail just 16 mins after the APU was started?

At NRT, was it used to start the APU, or did they have external power? Was it fully charged on APU shutdown at NRT? Does recharging continue from the main electrical system? If not, presumably it would have leaked some of its charge during the 12-hour flight, but not much. It seems possible that the recharge process might have been nearing completion when the failure sequence started.

[Chris Scott]

Hi mm43,
"The APU once online should be self sufficient, and not beholden to the state of APU battery"

If the APU and/or its 2 gennies (starter/generators) fail, perhaps catastrophically, there must remain a power supply to control the APU shutdown, and trigger the fire bottle(s) if necessary. The APU may have a FADEC generating its own power during normal running, but the APU also needs some kind of battery back-up, IMO. (Must admit I'm "winging" this one!)

[Kiskaloo]

@Lyman
Did Boeing replace three batteries per fleet unit without knowing the cause for failure? That would be most troubling.

But if they knew the cause, how is it NTSB does not know it? Even more troubling….

In the absence of external ground power, the 787 Ship's Battery can provide power for functions including refueling the plane or operating the brakes during towing. The APU battery can provide power for functions like running / anti-collision lights during towing.

Almost all of the battery failures were caused by ground personnel running these functions off the batteries for longer than Boeing's guidelines called for. This caused the batteries to discharge to a state that tripped safety systems to prevent over-discharge and/or damage to the battery. Said batteries were removed from the plane, re-charged at the airline's maintenance facility or an appropriate MRO, and then returned to service.

@inetdog
According to the description in the report, the controls for the APU are powered only from the APU battery bus.

The battery bus might also be powered from the charger (BCU) if the battery goes open circuit, but apparently when the battery fails in a short circuit mode or high enough level fault alarms go off the battery bus cannot receive external power and so the APU cannot run.

This is evidently a Boeing design philosophy. I have been informed that the 777's APU will also shut down if the APU battery fails and on the 767, turning the APU battery switch to the off position while on the ground will shut down the APU.

[Lyman]

The proposal by Boeing was that only overcharging would cause a fire....

Fire is required to be 10^-9 as a safety critical factor.

So I would not want my batteries charging during flight, that makes the regulations not pertinent. Fire on the ground is calculated to different parameters.

I also would not want to use these batteries under casual circumstance....

That is why quick charging is a requirement, the minimum V must be met prior to launch, not anytime after....

It also explains why Boeing initially pooh poohed the smoke in cabin aspect of the fire. "Inflight, the cabin Pressure prevents it..."

Pretty slick.

And why spontaneous ignition was allowed to be considered impossible by FAA.

I say dis-establish this rubber stamp agency, pronto.....

Or require Congress to speak forensic linguistics.....

[Lyman]

kiskaloo...

Almost all of the battery failures were caused by ground personnel running these functions off the batteries for longer than Boeing's guidelines called for. This caused the batteries to discharge to a state that tripped safety systems to prevent over-discharge and/or damage to the battery. Said batteries were removed from the plane, re-charged at the airline's maintenance facility or an appropriate MRO, and then returned to service.

Can you cite a source?

Also, once discharged, these batteries cannot be "recharged" according to Boeing...and Yuasa. There is no "return to service". My belief is that once discharged, the presence of dendrites makes them useless. And potentially dangerous.

Were the accident batteries in the "replacement group"?

Which frankly does not sound reasonable, just set the baseline charge state higher than FAIL. Let the lights go dark, ffs.

This is evidently a Boeing design philosophy. I have been informed that the 777's APU will also shut down if the APU battery fails and on the 767, turning the APU battery switch to the off position while on the ground will shut down the APU.

The introduction of Lithium Ion Batteries seems to have inspired another look at this "design philosophy"?

[inetdog]

Chris Scott:
Was it fully charged on APU shutdown at NRT? Does recharging continue from the main electrical system? If not, presumably it would have leaked some of its charge during the 12-hour flight, but not much.

I cannot see the logic in limiting the power to the APU BCU to come only from the APU itself. At a minimum it would have to be rechargeable from shore power, and there would be no reason not to also power it from one of the general AC power busses.

In the Narita incident, it was admitted that there was some sort of unintended cross connect between the APU battery bus and other loads including the entertainment system. Whether that included any possibility of sneak charging around the BCU has not been mentioned, but the diode between the APU battery and the APU battery bus should have prevented that.
Single Point Of Failure anyone?

[FlightPathOBN]

just set the baseline charge state higher than FAIL

exactly, why, if it is such an issue, would you design a system that allowed this to happen?

It is very easy to set the limit above fail, and have it simply shut down, not run itself into the ground...

I think there is much more to this than we have been allowed to know....

[inetdog]

Chris Scott:

If the APU and/or its 2 gennies (starter/generators) fail, perhaps catastrophically, there must remain a power supply to control the APU shutdown, and trigger the fire bottle(s) if necessary

Which also makes it interesting to see that the battery failure "triggered an APU shutdown". Possibly once the fuel was cut off, the fully functional APU managed to generate enough power while coasting down to complete any required sequence. There was no other power source on the plane at the time, but it does sound like the shutdown would have occurred regardless of what other power was available.

[Kiskaloo]

@Lyman
Also, once discharged, these batteries cannot be "recharged" according to Boeing...and Yuasa.

They cannot be recharged aboard the airplane. They can be recharged on the bench. This information comes from 787 Electrical Engineers with direct knowledge of the procedures.

Were the accident batteries in the "replacement group"?

JA829J's would not have been as the airframe had just been delivered, so it had the APU battery installed at the factory.

JA804A did have it's Ship's Battery replaced in October 2012, but as to whether it was a reconditioned battery, I cannot say.

Can you cite a source?

At least 100 batteries failed on 787 fleet | Business & Technology | The Seattle Times (http://seattletimes.com/html/businesstechnology/2020241385_787deadbatteriesxml.html)

Most of the batteries were returned because they had run down so far that a low-voltage cutout was activated.
The person on the 787 program with knowledge of the problems said that the electrical-system design makes it commonplace for airline mechanics to inadvertently run the lithium-ion batteries down too low.
Because lithium-ion batteries can be dangerously volatile if undercharged, as well as when overcharged, an automatic cutoff is built into the 787 batteries so that if the charge falls below 15 percent of full, the battery locks.

[Lyman]

Hi kiskaloo...

from the article you cite...

More than 100 of the lithium-ion batteries have failed and had to be returned to the Japanese manufacturer, according to a person inside the 787 program with direct knowledge.

my bold...

That would not be for recharging right? In other parts of the article

Because lithium-ion batteries can be dangerously volatile if undercharged, as well as when overcharged, an automatic cutoff is built into the 787 batteries so that if the charge falls below 15 percent of full, the battery locks.

So all the dead 787 batteries have been shipped back to Japan and replacements have had to be sent from there.

Pretty expensive charge, shipping, sole source tariffs, down time.

I still think, from reading the Exponent analysis, that Yuasa batteries, perhaps all batteries of this type, plate metallic Lithium at the Anode, when discharging even above 15% of FULL SOC. Is there even an industry standard? If the flow is discharge, does the dendrite process ennable even at 100% SOC?

One battery, in a series of eight, FAILS the Back Up utility of the system, by regulation, and the reliability issue steps in front of the fire.

Replacing 150 batteries for non performing reliability is more an issue than the occasional fire.

Sounds crazy, but fire is not really the fatal flaw.

[FlightPathOBN]

They cannot be recharged aboard the airplane. They can be recharged on the bench. This information comes from 787 Electrical Engineers with direct knowledge of the procedures.

According to the information, over 150 batteries have been sent back to Yuasa, because once they hit that point, the system will not allow it to be re-charged.

It goes further to say that the cost is $16K per cell...

Whos bench, YUASA's...how convienent...

If you have a direct connection with the EE at Boeing, ask them why they designed such an F'd Up system....

The A350 will be flying before the B787 will....

[TURIN]

The Main Battery in JAL is not strictly speaking an exemplar, it is involved in the circuitry intrinsically.


Erm, how so?

The Main battery and APU battery circuits are (supposed) to be completely independent.

[Lyman]

TURIN

Forensically, an exemplar represents baseline, undisturbed, unperturbed. It is not best practice to use a specimen involved in the accident in any way, it could color the results of any and all testing.

And, as we see, the ANA MAIN ship and APU had at least a "tickle" of incestuous commonality.

QED

[cwatters]

1021:01 APU battery bus voltage decreased from 32 to 31 volts.
1021:04 APU battery current increased to between 44 and 45 amperes for 4 seconds, indicating current flowing into the battery.


My reading of this is that the battery might have been charging when the voltage suddenly dropped a bit. If the voltage dropped due to a partial short circuit in a cell that would probably result in the charge current increasing.

Initially the voltage only falls 1V which is probably not enough for the BMU to see it as a fault (The battery voltage might vary that much due to normal load variation?). That might explain why it continues charging for another few seconds. Only when the voltage has fallen far enough does it realise there is a battery fault and issue the EICAS message.

The way the voltage fall sounds like one cell in trouble because it's dropped roughly one cell voltage (eg 4V).

The strange thing is that the system knew the battery had failed (EICAS message) but the contactor hadn't been commanded to disconnect the battery from the aircraft.

I notice the battery voltage falls to 0V and returns to 28V. I wonder if that was caused by the contactor trying to disconnect the battery? I hope the board controlling the contactor doesn't need a functioning battery to provide power to control the contactor?

Edit: It would be interesting to replace a cell with a box of electronics that simulates a cell. See what happens if you just slowly turn down the voltage of this dummy "cell".

[cwatters]

The person on the 787 program with knowledge of the problems said that the electrical-system design makes it commonplace for airline mechanics to inadvertently run the lithium-ion batteries down too low.
Because lithium-ion batteries can be dangerously volatile if undercharged, as well as when overcharged, an automatic cutoff is built into the 787 batteries so that if the charge falls below 15 percent of full, the battery locks.

That's all sensible but I wondered why they don't have two low voltage trips? For example if they had one set to say 20% that would prevent the battery getting low enough that it has to be replaced. Instead it would just need someone to apply ground power, start the APU and "press reset".... or have I missed something?

Obviously you would still need to retain the 15% trip for safety reasons.

[inetdog]

cwatters:
Initially the voltage only falls 1V which is probably not enough for the BMU to see it as a fault (The battery voltage might vary that much due to normal load variation?). That might explain why it continues charging for another few seconds. Only when the voltage has fallen far enough does it realise there is a battery fault and issue the EICAS message.

If the battery was charging at all, then the BCU would have seen current going into the battery. Hopefully one of the things it was looking at was the reading from the Hall Effect current sensor which is directly on one of the battery leads on the battery side of the BCU connection. And if it was checking frequently enough, it would have seen the voltage drop happen while the net current into the battery was still positive. Hard to explain that away (from its point of view) as the effect of a change in load. It may not have applied that logic, but maybe it should have?

I notice the battery voltage falls to 0V and returns to 28V. I wonder if that was caused by the contactor trying to disconnect the battery? I hope the board controlling the contactor doesn't need a functioning battery to provide power to control the contactor?

My expectation is that both the primary contactor and the backup contractor open when unpowered, and energy to close the contacts comes from either the BCU or the battery itself, under the direction of the BMU. But if the BMU circuits got fried quickly, they might not have been able to enforce their intention to open the contactor. This happened some time (a few seconds) after the initial observed event and we really don't know when the first damage was done to the BCU. Or the battery voltage dropped below the hold voltage of the contractor or the decision threshold of the BMU, causing it to open, the battery voltage could then have rebounded enough to close it again.
Not enough information. :hmm:

[Lyman]

At a guess, 98 per cent of the discussion here is 100% forensic debrief.

Were I Boeing, that is where I would like the discussion to be.

Likewise Yuasa.

The manufacture of the Yuasa Batteries requires meticulous handwork, scrupulously clean surroundings, and exquisitely pure ingredients and materials.

The CT scans of sections of even the JAL exemplar show problematic bunching, packing, and wrinkling of the electrode stack. Given the design, the geography of the enclosure, and its attendant kit, to ignore the probability that failure is merely a shortened service life due environmental impacts is not far fetched.

On the face of it, that is what Boeing determined, hence the AFR (Accelerated Field Replacement).

It's Friday evening. I'm down to the local for a pint. Maybe I'll run into W. Occam, kick this around.....

[USMCProbe]

I believe on most airliners, if the batteries are shut off (offline) one way or another, the APU shuts down due to the APU ECU being unpowered. I did it by accident on a 320 a year ago. APU's generally have auto shutdown features, at least on the ground, as well. In the air, usually the crew has to shut it down.

I can't verify this for the 787, but since they are all certified under similar standards, I would guess it would be the same. Engine Fadec's a lot of time have their own Fadec generators that power themselves under normal circumstances, and use ships power if those generator's fail.

I haven't been reading much other than the NTSB stuff. Since they are replacing the whole "battery system", I would guess that they really don't know what the causal factor(s) was. If they had found the cause, they would just fix the singular problem.

[Chris Scott]

Me #881 (http://www.pprune.org/tech-log/505695-787-batteries-chargers-45.html#post7732935),
"If the APU and/or its 2 gennies (starter/generators) fail, perhaps catastrophically, there must remain a power supply to control the APU shutdown, and trigger the fire bottle(s) if necessary. The APU may have a FADEC generating its own power during normal running, but the APU also needs some kind of battery back-up, IMO. (Must admit I'm "winging" this one!)"

inetdog #887 (http://www.pprune.org/tech-log/505695-787-batteries-chargers-45.html#post7733016),
"...Which also makes it interesting to see that the battery failure "triggered an APU shutdown". Possibly once the fuel was cut off, the fully functional APU managed to generate enough power while coasting down to complete any required sequence. There was no other power source on the plane at the time, but it does sound like the shutdown would have occurred regardless of what other power was available."

Good point. However, isn't it a sort of chicken-and-egg situation? Normally, the APU genny trips off line almost immediately. And, as USMCProbe now says:
"I believe on most airliners, if the batteries are shut off (offline) one way or another, the APU shuts down due to the APU ECU being unpowered."

So, what protects an APU if it catches fire during an auto-shutdown, with no personnel present? (Must emphasise this is not a 787-specific question.)

[TURIN]

So, what protects the APU if it catched fire during an auto-shutdown, with no personnel present? (Must emphasise this is not a 787-specific question.)

Usually the APUC stays powered for approx 2 mins after the APU shuts down. I think the 737-3/5/500 had problems with this early on as pilots/engs were shutting the battery off as soon as the APU started to shutdown. The consequence was that on the next APU start a failure would occur as the APUC was still in shutdown mode.


Back to the 787.
If Boeing insist on a battery that 'may' fail but it's 'safe' as it will be contained then the APU needs a seperate power source. IE A PMG that will continue to power the APUC/FADEC when the APU is up and running.

If the backup can't be relied upon then the backup needs a backup.

What a mess!

[Lyman]

If the Battery is integrated into the distribution network, it is not a Back Up Battery. It is a Battery that has an additional function in name only. By definition, this functional demand is outside the intent of the regulation, imo.

This battery has the elements of a reliable back up source. Too much was expected, and demanded. Is it a legacy of antiquated obsession with pounds of weight? It may be down to Fire, or reliability. Choose wisely. This situation was predictable, and was predicted. The failure in implementation is down to lack of test, and lack of in service experience. Or, worse.

The CT scans of sections of even the JAL exemplar show problematic bunching, packing, wrinkling and pinching of the electrode stacks. Given the design, the geography of the enclosure, and its attendant kit, to ignore the probability that failure is merely a shortened service life due environmental impacts and/or excessive demand is a mistake.

If this type, specific storage battery is not suitable, and cannot be made safe, then the outcome of the public hearings might be "game-changing".

The box was a stupid PR move, a non starter.....

[USMCProbe]

Every plane that I can remember flying had an auto shutdown feature on the ground in case of APU fire. It must be a certification thing. The APU can be running with no crew aboard for aircraft servicing. There is also an APU fire control panel in a wheel well usually, with a fire bell that goes off outside the aircraft. The APU fire extinguishing systems are connected to the hot battery bus. I don't know what that bus is called on the 787.

Unfortunately for the 787, there is no certification requirement for auto battery fire extinguishing, or a battery ejection seat in case of battery fire on the ground.

[syseng68k]

archae86, #850:

Thanks for that. Have had time to have a good read now and there's quite
a bit of fresh info to fill in more of the gaps.

The key points seem to be:

1) There are two temperature sensors. One to inhibit charging when the
battery is at under or over temperature, and a second "reported to the
BCU", to control charge current. Effectively, only a single sensor to cover
charge and discharge.

2) There is a single hall effect current sensor ("dc current tranformer"),
where two might have been expected for redundancy reasons, for such a
critical function.

3) Charge balancing is done for each cell, but no info as to how this is
monitored / managed in discharge by the software.

4) The voltages of individual cells are not logged, which explains why it
hasn't been possible to determine the failure timeline and primary cause.
The only logging being the charger error codes (idiot lights) and fdr
logging of bus voltage.

5) The battery isolation contactor has normally closed contacts. Trivial
item, you may think, but in fact, it has very serious implications.

I'll try to explain how a contactor functions, with apologies to those who
know all this already. A simple contactor is nothing more than a pair of
contacts arranged as an on/off switch, as is found in eg: a domestic light
switch. Whereas a light switch is operated manually, a contactor or relay
is switched by applying power to an electromagnet/coil, which is mechanically
linked to the contacts. Spring loading against the coil provides opening
force when power is removed. The contacts may be arranged to be normally
closed or normally open, ie: switch normally on or off, with no power applied
to the coil.

In most cases, for fail safe operation, normally open contacts are used so
that in the absence of power or circuit failure, the contacts open, isolating
the circuit. In the 787 system however, the contacts are normally closed and
thus need power to open the contacts and isolate the circuit. This is opposite
to what would be expected for such an application, since there really is no
way to isolate the battery from charger or load in the event of electronic,
software or other failure. In extremis, the battery will remain connected
until the fire is put out and the wires cut.

I know i've been banging on about all kinds of issues with the design for
weeks, but more info that comes to light, the more seems to be exposed and
that's without any idea of what the underlying control software is getting
up to...

[bill good]

Keep in mind there are two failures of batterys from two different locations each with very simular end results. The report on the first indicates 17 mins after the APU was started from the APU battery, the voltage dec from 32V to 30V which indicates the first time a short is present. Since the current information is being derived from the neg of the battery & is after the shorted cell it is the sum of both charge & the discharge of the faulty section of the battery & thus would not be the correct recorded value. Because the breakdown of the insulation barrier between the cells has effectively reduced the number of cells over voltage would now appear on the cells before the short resulting in damage but lesser damage than the cells with the higher circulating current involving the path via the case to the ground point of the battery case & A/C bonded points. Further just to make sure the whole thing is destroyed all the wires from every connection appears to be taken thru a stainless "protective " shield which is bonded to the case to short out all wires from each cell to the external world of the case once the heat melted the insulation. i.e. just about 100% chance of every cell being destroyed. (The local heat build up is seen in the pictures is clear & reports of flame by the fire crew confirms) I can't help thinking that if the case was not bonded this event would have not flagged so many design oversights & safety issues. Not a good idea to enclose all the protective & contol circuits & components in an area of a battery case which has known potential fire & heat issues as well. All in all how did this design get this far?? Could someone who has watched the current levels of the B787 APU battery indicate that 45Amps without reducing for 15 mins after starting the APU is a normal expected level as it seems a bit high. There are more concerns but the battery case & installation of this case is the primary cause. The swiss cheese model follows after this level with secondary oversights as time will show.

ps Looking at the specs for this cell the CC part for a cell start point 3.75V which would be degraded for a battery to 29.6V (est) & at 10A is 1 hr. Again anyone know if the battery in the B787 gets a CC to start the recharge process? Applying 32V initally seems very strange & would like to see how long a battery would last being tested on the bench by taking it to a low SOC & applying the 32V that is being stated as normal. (i would be in the next building) Best est 300-500 cycles.

Regards

[saptzae]

Further to my post 857: http://www.pprune.org/7731615-post857.html

Three possible explanations for the 45A into the battery from 10:21:04:
a) Likely: that the charger commenced fast charging at about 0.7C
b) Improbable: APU bat bus was connected to a high current source such as Main bat bus.
c) Unlikely: Parasitic connection via small load such as nav lights. Such connection could not deliver 45A.

Thus it appears likely that the BMU has not detected the cell short and that the charger commenced fast charge, initiating the conflagration.

Detecting Cell short is difficult: http://www.pprune.org/7679259-post500.html
(Note: since that post, it was confirmed that there are only 3 sub cells per cell)

What remains illusive, is the cause of the Cell short.

[Lyman]

@saptzae post #500

What has to be done, is to make the BMS more sensitive to voltage transients on individual cells and shut charging down very quickly.

A Battery that cannot be charged cannot be relied upon to perform. The purpose of the Batteries: to provide a storage of dependable power to satsify regulations.

NTSB have stated unequivocally that this incident was precipitated by a short in one of the eight 'cells'.

The short led to thermal runaway, a disqualifying condition for service. (imo)

Boeing has proposed a heavy and sturdy containment to prevent fire from spreading and other disqualifying conditions, admirable.

The failure mechanism is known. Can this battery be made safe such that containment is not a requirement?

Other large battery packs have been constructed from “large format cells” that have capacities in the range of 10 to 100 Ah. Standards for these sorts of applications are currently being written or revised to be appropriate for lithium-ion technology

(from: "Fire Research Study", Exponent Failure Analysis Associates, July, 2011

We'll have to see, but I don't think 'containment' will 'fly'.

[saptzae]

@Lyman
The failure mechanism is known.I am fairly confident about causes of cascading - Secondary and Tertiary - failures. Known?, Not yet. Lets say we are on a good track to understanding.

Can this battery be made safe such that containment is not a requirement?Yes, upon remedy after gaining understanding of the failure causes.

Containment is just an additional layer and is not critical part of safe operation.

To accomplish this


Monitor and Manage cells better
Eliminate cascading failures - Stop secondary and thereby Tertiary failure

Safe means that a cascading failure occurs less frequently than what the regulator specifies.

The grounding is better discussed in the other thread.

[Lyman]

Howdy saptzae,

Monitor and Manage cells better. Eliminate cascading failures - Stop secondary and thereby Tertiary failure

As to Monitor.... Secureaplane has their patent, a method of sensing and thereby controlling charging? For each cell? That did not happen... Patents protect frrom imitation, they do not guarantee performance.

As we see, it is not possible to control (manage) charging in an eight battery (cell) series. The "control" is to take the unit (all eight) off line. This pertains to reliability, certainly a technical issue, grounding is not pertinent here...


NTSB have eliminated overcharging as a cause, at least preliminarily....

That leaves a primary. Short circuiting in large format LiIon batteries is not well understood, as a read of the above study demonstrates. There are no industry standards as of July 2011.

The FAA regulations were written in 2007 (sic).

For a serviceable cell to short circuit, internally, the separation Anode/Cathode must be breached. As far as I can determine, that means a break down of the separator, and/or, the presence of metallic Lithium dendrites, spanning the borders of the connectors at either end.

Dendrites far and away are the more troublesome of the two, separator issues are related to diffusion of the Ions, dendrites are a mechanical connector....

Some of the microscopic photography in the materials addendum is interesting....

The hearings are open to the Public. Are you considering being present?

I am.

[HighWind]

Syseng68k (http://www.pprune.org/members/302789-syseng68k) #902


Hi, I'm an Electronics engineer working in another industry, involving MW size power electronics, UPS systems, and IEC61508 P/P/E.
2) There is a single hall effect current sensor ("dc current tranformer"), where two might have been expected for redundancy reasons, for such a critical function.I would prefer two, not only for redundancy, but also for insulation monitoring.
The battery is in a conductive casing, and is expected to be insulated from the power circuits. But one single fault in a battery could connect it to the casing, thereby creating a circuit that can't be disconnected. I would insert a HAL sensor in both the positive and the negative connection, for insulation detection.
5) The battery isolation contactor has normally closed contacts. Trivial item, you may think, but in fact, it has very serious implications.
This is serious, it seems the designers has been more concerned with the risk of the system failing so it can't provide power, than the system failing so it can't protect itself. (against high currents, and high/low charge).

I would prefer two separate contactors (or at least a two pole), with normally open contact sets, one installed in the positive and the other in the negative connection. The contactors and BMS has to be able to detect and break the highest possible short circuit current.

The contactor operation could then be a part of a daily self-test (Feedback from forcefully guided contact sets).

This would also allow the battery box to have unpowered terminals during installation, the power could be connected when the communication link is established.

This would then create a chicken and egg problem, since the contactors has to be unpowered, when the aircraft is parked in order not to drain the battery, and the system needed to wake-up the battery has to be powered from the same battery. This could be solved by providing power via. the data communication connector to the charger/DC bus controller.

If Boeing continues with the hot-box solution then the energy-density would be lower than the current solution, and more rack-space would then be required.
Such a battery box would probably have a MTBF (safe faults) in the range of 5 years, for extra availability of backup power a 2oo3 architecture could be used.

[Lyman]

Howdy HighWind...

The battery is in a conductive casing

So too, is each individual battery, ('cell').

I think... (Aluminum) Ground Failure issues?

:ok:

[RR_NDB]

Hi,

syseng68k:

Chris,

5) The battery isolation contactor has normally closed contacts. Trivial item, you may think, but in fact, it has very serious implications. (http://www.pprune.org/tech-log/505695-787-batteries-chargers-46.html#post7734079)

:mad:

A BAD BATTERY! A design capable to ... :{ :sad:

[syseng68k]

HighWind, #908:

Hi, I'm an Electronics engineer working in another industry, involving MW size
power electronics, UPS systems, and IEC61508 P/P/E.
Very welcome and judging by the name, location and work, wind energy systems
perhaps ?.


I would prefer two, not only for redundancy, but also for insulation monitoring.
The battery is in a conductive casing, and is expected to be insulated from the
power circuits. But one single fault in a battery could connect it to the casing,
thereby creating a circuit that can't be disconnected. I would insert a HAL
sensor in both the positive and the negative connection, for insulation detection.
Very good point, since any differential / imbalance between the two current sensors
are an indication of leakage to ground. Ideally, each sensor would be on the busbar
terminations at the power socket, to cover leakages right up to that point.


I would prefer two separate contactors (or at least a two pole), with normally open
contact sets, one installed in the positive and the other in the negative connection.
The contactors and BMS has to be able to detect and break the highest possible short circuit current.
Hadn't thought about about case leakage isolation, which does need contacts in
each line to work. There's stiil the problem of >1 cell shorting to case, but I guess
that scenario could be covered by fusable links between the cells, rather than the
existing copper straps. A two pole contactor would get the job done and should be
more reliable and cost effective as well. However, my point would still be that
if the cells are properly managed and run within data sheet limits, the problem
won't arise in the first place.


This would then create a chicken and egg problem, since the contactors has to be
unpowered, when the aircraft is parked in order not to drain the battery, and the
system needed to wake-up the battery has to be powered from the same battery.
This could be solved by providing power via. the data communication connector to the charger/DC bus controller.
That's just a system design issue and could perhaps be covered via a low power micro,
with uA in sleep mode, always connected to the battery, or even a small disposable
memory style backup cell. The micro is then woken up by the first comms packet down
the line.

Over the past few weeks, many people contributing here have found loads of issues
with the design and while hindsight can be a wonderfull thing, the present design
looks like half the job that should have been done. The introduction of cots tech
into aviation may have saved loads of money, but I wonder if some of the design
rigour and attention to detail have been thrown out of the window at the same time.

Having said that, I'll bet the design teams were really pleased with themselves to
start with :yuk:...

Regards,

Chris

[FlightPathOBN]

From Seattle Times...

Everyone needs to stop picking on Boeing, formerly of Seattle, for this minor 787 battery snafu.

It’s now becoming clear that the company paid a company to pay a company to hire a subcontractor to run some simulated tests on theoretical imaginary battery problems, exactly as required by the FAA.

What more could anyone expect of them?

[Lyman]

FPOBN,

The Special Regulations were composed in 2007, and approved the same year (October).

The Fire Research report, in addressing the "large format" Li technology, stated unequivocally that there were no Industry standards in place as of JULY,2011.....

Large Format seems to mean 10-100 aH. So FAA left Boeing to come up with a PROVEN system. I do not fault FAA for that. To the contrary, advancing technology requires room to advance. The special regs set limits, not specs on how to ennable the technology short of them (pardon the pun). The limits turn out to be sufficiently rigorous, NO FIRE, NO UNCONTAINED ELECTROLYTE. (One in a Billion, one in ten million)

The Airworthiness Directive is couched in similar terms and provoked a snarky rejoinder from the airframer. "Show us how to conform"....

It is up to the builder to perform, there is no room for wriggling.

Large Format seems to be simply, "scaled up". Not an engineer, but that seems like an invitation to Fail. "Nice Battery, can you make it bigger?"

Primarily as to heat control. Now Boeing can "scale back", until they can demonstrate some acceptable thermal performance, but that will take time, and it means satisfying a regulator who has been taken to the cleaners once already.
It also means packaging more and smaller batteries into an existing architecture.

The Big Box format might be fine to alleviate fire control issues, but it does not address the prevention of fire in the first place, something that was touted to be on the order of "One in a BILLION" (John Goglia). It does nothing to salvage the trashed reliability profile as to "Back Up Battery".

Have you seen the photos of the "Exemplar" battery removed from JAL 08?

The "Jelly Roll" looks to have sat in the lunch box for quite awhile, and may have seen Sally and Joey sit on it several times.

The electrode stacks are sensitive to shape retention, as to remaining safe from internal damage, and evenly distributed excess heat. (Sic).

[Ian W]

Lyman
As to Monitor.... Secureaplane has their patent, a method of sensing and thereby controlling charging? For each cell? That did not happen... Patents protect frrom imitation, they do not guarantee performance.

I think this may be the initial development management error. Did anyone ever test this Securaplane patented technology with a failed cell? Or was the assumption made that with this patented method they wouldn't fail or the failure would be detected? From what we see now the 'patented technology' allowed the charger to carry on with a dead cell. This means that the idea doesn't work or that it was incorrectly implemented.

One wonders why it was thought necessary to rely on indirect assessment of cell health and indirect temperature sensing and serial charging rather than individual cell monitoring and parallel charging which would seen a lot safer?

[Lyman]

Prior to filing document of disclosure for a patent in the early seventies, I would not have guessed that a Patent hasn't anything whatever to do with the utility of the patented process/device.

There is no requirement anywhere in the successful Patent process to demonstrate that the device actually works, or has any use whatever....

It provides "protection" from infringement, only. It is not a Warranty, nor is it a "Guarantee". Obviously, in the case of the Dreamliner.

Boeing would not be the first client to be over-impressed with a REGUSPATOFF file number....

But the only germane discussion at this point is how can the battery be made reliable, let alone safe?

VOLTAGE and TEMPERATURE numerics are useless against thermal runaway. So far as keeping this "Battery" operating in any case. They might be useful in stopping a conflagration, but for Gods sake, why is anything that can ignite spontaneously anywhere near an AIRCRAFT?

Surely Boeing knew that?

[saptzae]

A single cell (blade) failure can be contained. A multi cell failure (hub) is hard to contain.

Thus am I most concerned about fixing the Primary cause and preventing the Secondary and Tertiary cascading failures.

@Lyman
As to Monitor.... Secureaplane has their patentFrom #590 http://www.pprune.org/7685780-post590.html
It is one of those patents one gets to get a patent for making life harder for the competition to sue one. Well, and a little for marketing hype too. As to its merit, inflection, prediction, assumption is IMHO a demerit.
Edit: I do not believe that the patent, which was developed for Pb type cells, is applied here, other than in the marketing documentation.
As we see, it is not possible to control (manage) charging in an eight battery (cell) series.It has been discussed many times. It is common practice for more than 100 years. It can be managed.

Rather, by what we see, all boils down to inadequate management, on all levels.


Primarily, of the cells, so they short.
Secondarily, of the cell short, leading to the conflagration
That 100+ batteries got line replaced

The containment box worked pretty well!

For a serviceable cell to short circuit, internally, the separation Anode/Cathode must be breached. As far as I can determine, that means a break down of the separator, and/or, the presence of metallic Lithium dendrites, spanning the borders of the connectors at either end.Concur, but we do not know which, less so why, and the evidence is all but destroyed after subjected to heat sufficient to melt aluminum.

Dendrites far and away are the more troublesome of the two, separator issues are related to diffusion of the Ions, dendrites are a mechanical connector....Yes, Examining other serviceable cells from the fleet for deterioration is my best hope.

The hearings are open to the Public. Are you considering being present?
Interesting question. I wont be present, but will consider to make a submission.

@HighWind
Welcome!

Yes, two hall sensors would be good.

Yes, two pole contactor would be better.

A bistable contactor would be possible.

@syseng68k
Fusible links would not be practical. Cells should be insulated more reliably (heat resistantly) from the case (edit: and each other).

[FlightPathOBN]

All that aside..

The NTSB report showed that the battery was tested, the charging system was tested, but the battery and charging system were not tested together, and the entire system was not tested while installed in the aircraft, nor with the entire electrical system of the aircraft.

How this approach permeates throughout how the rest of the aircraft was tested, remains to be seen.

As we have all noted, the exact cause has not yet been determined.

The sum of the parts does not always equal a whole....

edit: remember back to some of the early issues with this aircraft? Remember the retro fit of titanium wires for lightening arrest. Really, a comp aircraft, and it went that far down the path, before lightening was addressed?
and of course, the associated extra weight of all the wires...

[saptzae]

@Ian W
Well, the securaplane lab building burned down once while testing (without BMU connector plugged in) :{

Seriously, they surely did not test a cell to failure inside an airframe, nor with _all_ systems operating on a test bench.

System level testing is whats lacking, both during design and in production, for example, parasitic connections where found after TAK incident.

They apparently drove a nail into a cell. But they did not deteriorate a cell to short itself "naturally" by over/undercharge.

A nail through all layers of all sub cells is 10s of times better short than a single spot of a sub cell shorting!

[saptzae]

@FlightPathOBN
Yes, there is so much to learn about this wonderful airplane.

When all this is over, the entire industry will have progressed substantially.

[Lyman]

saptzae,

Rather, by what we see, all boils down to inadequate management, on all levels.

Primarily, of the cells, so they short.
Secondarily, of the cell short, leading to the conflagration
That 100+ batteries got line replaced

Inadequate management at the internal short stratum? Primary? It must be prevented, not managed, to wit:

Management at the secondary level, post internal short, is not management, it is shutdown.

Management can not rectify the problems we see.

Internal short, leading to shutdown, takes a safety critical system "offline".

Internal short or any failure leading to FIRE, is disqualifying of the Battery's installation, let alone management......

The "FIREBOX" may work, but it is extraneous, the battery cannot be aboard, if there is even possibility of FIRE, by rule.

Do you see this?

[Lyman]

A Nail is better than....What?

A nail is not representative of a typical FAIL. Overuse, over demand, age, Pressure/Temp cycling, vibration, etc.

A Nail is a parlor joke, right? A sledgehammer would not be appropriate either....

Again, no industry standards prior to July, 2011

:ok:

[saptzae]

@Lyman
I apologize for explaining things poorly. You help me to manage my posts better. Thank you.

By "Management" I refer to action it takes, to meet the objective of reliable and safe operation, irrespective on what takes the action.

Rather, by what we see, all boils down to inadequate management, on all levels.

Primarily, of the cells, so they short. BMS fault

Management of cells includes control, monitoring and balancing to assure reliable and safe operation.
Control of battery charging
Monitoring of cell voltages and battery current and temperature
Real time determination of battery status and cell condition

Battery voltage
Battery current
Battery temperature
Cell voltage
Cell voltage transients at individual cells. This is needed for cell short detection.
As discussed earlier, cell pressure would be an interesting alternate way to detect cell shorts. This is optional.


Balancing of cells by discharging cells at or above 4.025V





Secondarily, of the cell short, leading to the conflagration BMS fault

To prevent cascading failure after a cell short

Disconnect battery
Prevent charging .
Prevent current flow into cells




Apparently, at TAK, connections were found between APU and Main batteries (via Nav lights), which were not part of the design. Production QC fault?
That 100+ batteries got line replaced Inaction?

Problem persisted after a year in service until the grounding



A Nail is better than....What?

A short by a nail is a bad test

It is not realistic

The short circuit resistance will be lower than a genuine cell short




A good test would be representative of the smallest internal short, which could lead to damage to cell.

[Lyman]

Thank you saptzae, for your patience with a layman.

I still am not tracking as to how a primary cause, an internal short, can be "managed".....

I was confusing management with "mitigation", so your response above helps in that regard. If you mean a well managed program, I see that as very helpful....

I found this interesting item in the Exponent paper...

This fact has led to a misconception that lithium-ion cells burn vigorously because they “produce their own oxygen.” This idea is incorrect. No significant amount of oxygen is found in cell vent gases.91 Any internal production of oxygen will affect cell internal reactivity,11 cell internal temperature, and cell case temperature, but plays no measurable role in the flammability of vent gases

One immediate question re: the "Materials Addendum"........

They found spherical globules of Stainless Steel in the Combusted products within the identified cell, number 6. That means 2700 degrees, and most of Exponents' text refer to 1100 degrees as typical in thermal runaway.....

What source of the metallic Stainless? Typical of cell container is Aluminum, as the enclosure represents.....A technician is holding what appears to be a damaged encasement (cell), could that be Stainless?

Have you any comment on the CT images of the exemplar cells' sections?

To me, the electrode stack is in disarray?

[syseng68k]

Not a good idea to enclose all the protective & contol circuits & components
in an area of a battery case which has known potential fire & heat issues as
well.
Agreed - we trashed this part of the design mercylessly a few weeks ago,
just as soon as pics of the enclosure became available :eek:.


Could someone who has watched the current levels of the B787 APU battery
indicate that 45Amps without reducing for 15 mins after starting the APU
is a normal expected level as it seems a bit high.
That had me thinking about an overall scenario for the failure, as follows:

1) Apu startup, very high peak current demand, which significantly warms up
the cells

2) Immediate 45 amp charge current, which warms up the cells even more, with
one or more exceeding data sheet limits. The single thermal sensor doesn't see
this in time to shut down the charger, due to thermal resistance and lag across
the package. If it does see it, it's late, and the out of limit conditions
persist for long enough to cause some permanent, though not initially fatal cell
damage.

3) Cell eventually gets tired of being abused, thermal runaway in a single cell,
cascading to others. Game over.

I think temperature is the key here. These events may have been waiting
to happen for some time, but ambient temperature and low battery cycles have
mitigated it up to now. As the cells age though, ie: more and more flights
and charge discharge cycles, their ability to withstand abuse will be degraded.
The fact that it didn't happen in flight, yet, could simply be that that the ambient
temp may have been much lower than that on the ground.

Just a theory, but seems quite feasible ?.

To add to the mix, we have no clue what the controller software is up to,
algorithms used, how various limits are decided against time and other factors
directly or indirectly influenced by the charger and/or the bmu software...

[Machinbird]

It is clear that battery temperature sensing was inadequate. To my mind, rather than an external temperature sensor, each cell needed an internal temperature sensor near its center of mass.

Rather than try to insert a thermistor type element with its conductive leads into this area of a battery, why not use a few strands of fiber optic glass down into the core of each battery cell.

The following pictures illustrate what is currently achievable.
http://cdn.freshome.com/wp-content/uploads/2009/03/temperature-sensitive-glass-tiles-2.jpg

http://automationwiki.com/images/4/47/Fiber_Optic_Temperature_Sensors_Using_Phase_Interference.jpg

[saptzae]

@Lyman
A cell can be managed to not short. BTW, a cup of coffee can be managed to not spill or burn its contents consumer.

Link to exponent paper?

I have not read the materials report yet, it was hard to download. I will asap and get back.

Edit:
Browsed through materials and CT reports.

The cell case is stainless steel, stainless steel will be migrated inwards and deposited by arcing of internal parts to case.

The CT report shows cell bulging of main battery. This indicates that something is fundamentally wrong with battery management. This evidence of cell deterioration is a further step forward to figuring it out.

@syseng68k
Again, the BMU electronics in the same box is the better solution.

@syseng68k @whoever talked about 45A during 15 minutes
Please have a look at page 8 of recorders report 520291.pdf. The current of 40+A lasted only for 3-4 seconds

@Machinbird
3-4s of high current (45A) caused the conflagration! I estimate that BMS has less than 100ms to prevent electrically induced runaway after a cell short.

Thus am I afraid that temperature sensing will be too slow. Cell pressure sensing may be an interesting option.

[Machinbird]

@Machinbird
3-4s of high current (45A) caused the conflagration! I estimate that BMS has less than 100ms to prevent electrically induced runaway after a cell short.

Thus am I afraid that temperature sensing will be too slow. Cell pressure sensing may be an interesting option. saptzae
Referring to the Boston battery failure,No one knows what temperature the cell that initially failed was at just prior to failure. The fact that the short appears to have occurred on the centerline of the plates is indicative of a temperature factor in the failure. If an overheated cell is detected early in the charge sequence (because the sensor is closely coupled to the cell), then current can be attenuated to that cell or the charge sequence aborted.

Once an actual failure begins to occur, then a pressure sensor or a rapidly responding current sensor can abort the charge and flag a battery malfunction, but that is after a failure. The main idea should be to never let the battery (as a whole and any portion of it) reach any of its limits.

My personal approach would be to charge the battery with a series of cell voltage chargers, each one floating at the potential of the actual cell it was charging.
Yes you would have more wires running into the battery, and you would have cell sensors running to the appropriate chargers, but there would be no need of a Battery monitoring unit in the battery itself. The chargers could record and control the status of each cell. If one cell required twice the time to accept a charge as its neighbor did while staying within temperature limits that fact could be accommodated and reported if appropriate.

[bill good]

I hate to think what the outcome would have been it this occured 4hrs from land over water!!:mad:

[RR_NDB]

Hi,

Mb @ #927

I fully agree with the concept of "parallel charging" you made implicit.

Actually i yet designed and yet using a prototype:

1) High current, through series config.
2) Low current, directly to each cell.

Good sugestions on ea. cell Temp and Pressure monitoring. Also agree should be considered in the redesign of 787 battery.

[saptzae]

The main battery at BOS shows cell deterioration, as per CT report.

The NTSB battery block diagram does not show cell balancing.

I was thinking that the block diagram is simplified. Perhaps block diagram is factual and battery has no cell balancing.

Without cell balancing, individual cell voltages can not be limited, and cell bulging and failure is to be expected.

Edit2:
If so, cell deterioration for lack of balancing, would be the long elusive cause of the Primary failure, the cell short.

Edit:
@Machinbird
General per cell temp monitoring will be of benefit to cell condition monitoring.

As to parallel charging. Just adds complexity at zero benefit, sorry.

@RR_NDB
Parallel low current discharge is whats needed to balance cells.

[Lyman]

There are three cells in each of eight batteries.

Y'all are talking like each cell in each battery is not susceptible to this same imbalance failure, no? Each of the three cells is a unit made of differing quantities of "paste" applied to thin poly film.

Mechanical differences among these cells would seem to be the primary cause of failure. As small as one perforation or even ion by ion, the "Battery" is an accumulation of billions of cells, the electrode must collect and evenly distribute this energy to the connectors. Imbalance is the enemy, everywhere. Not just among eight gross boxes.

The failure is primary to a location on one of the three cells. Imbalance within the cell would seem to be the fault.

Moreover, each cell has millions of perforations on its separators, each one operating at a miniscule potential, and separate from all other perforations.

Balancing the batteries amongst themselves would NOT seem to be the answer.

This is a spontaneous failure. Spontaneous not meaning "Instantly".

Spontaneous as in "self creating". The charging exacerbates the failure it is not the genesis of the problem....

Conduction and resistance is causing the problem. Charging is just one of the roles of the system.

This battery is not happy with its work...

:ok:

Copyrighted, found on the sidewalk....

http://www.nfpa.org/assets/files/pdf/research/rflithiumionbatterieshazard.pdf

[Lyman]

The use of copper as the current collector for the negative electrode has particular reliability and safety implications. At very low cell voltages (usually approximately 1 V for the cell), the potential at the copper current collector increases to the point where copper will begin to oxidize and dissolve as copper ions into the electrolyte. On subsequent recharge, the dissolved copper ions plate as copper metal onto negative electrode surfaces, reducing their permeability and making the cell susceptible to lithium plating and capacity loss. Usually, once a severe over- discharge event has occurred, cell degradation accelerates: once the negative electrode has become damaged by copper plating it will no longer be able to uptake lithium under “normal” charge rates. In such an instance, “normal” charge cycles cause lithium plating, which result in a greater loss of permeability of the surfaces. Ultimately, over-discharge of cells can lead to cell thermal runaway.

From FPRA, the ExPonent report to Fire Research....

Out of my depth... But is this a precursor to the program of replacements? What would be the metric that alerts to the need to get the battery off the a/c ASAP?

Temperature logging? Inability to "take" a charge? What was the urgency in the replacement program? Reported is "over discharge". How many times and what was rhe evidence? What was the condition of the battery at removal? Any evidence of damage, potential hazards to continued presence onboard?

Certainly not "routine"......

[saptzae]

@Lyman

I really do not understand your last two posts. All was discussed already.

There are three cells in each of eight batteries.
No, there are 8 cells containing each 3 sub-cells, or elements in NTSB speak.

Page 19
Not relevant. Cells are not to be discharged below 2.7V.

Why are you arguing against common engineering practice?

Why are we going in circles?

[Lyman]

Howdy, thanks for responding.

Let me re-post what I think is important, and not at all argumentative (?)

Whatever the nomenclature, there are three wound cells in each of eight cases.

Any primary failure must have a point of origin. In the literature, one of these potential failures is at a perforation of the separator. It is one millimeter in diameter.

My take away from the discussion is that somehow, cell "balancing" (as to recharge) would solve the problem of runaway.

So my question: What locale is served by balancing? The electrode and collector must perform this on an instantaneous basis; any imbalance across the face of each electrode creates heat in addition to that of discharge/charge.

I see two issues. One, based on quality, is the potential for isolated and microscopic hot spots due to contamination and/or lack of care in assembly.

Two, the possibility of damage expressed as deformation of the electrode stack, this due to thermal and pressure cycling, and degradation of the paste(s), substrate, or separator elements.

Because that is what I hope for. I also believe that is what Boeing would hope for. Large format Lithium Ion Batteries might be the way of the future. If they are to be, Boeing is to be recognized for taking the step, FAA for allowing it, and the travelling public for placing trust in the system......

I do not think the charging issue or any other problem is related to this failure, except to say after the fact. Perhaps especially the default "CLOSED" contactors, which may have prevented takng the batteries off line....

NTSB: Single cell fault, No evidence of overcharge...(BTW, "overcharge" is not necessary, charging in and of itself produces heat, No?)

Take it away......

[inetdog]

Lyman:
They found spherical globules of Stainless Steel in the Combusted products within the identified cell, number 6. That means 2700 degrees, and most of Exponents' text refer to 1100 degrees as typical in thermal runaway.....

It is quite possible for there to be very localized heating at the ends of a high current arc without the entire surrounding area being at the same temperature.
The predicted temperature resulting from a thermal runaway is based entirely on the exothermic reactions at the anode and cathode and do not take into account either external power input or electrical heating resulting from internal or external short circuits that may occur during the process.
Is it reasonable to model the effects of two individually improbable simultaneous failure mechanisms (thermal runaway and short circuit)? Absolutely given that either one could induce the other!

Some comments were made about the inability of a single current sensor to detect a cell to case short circuit. In the Boston case at least, there is no evidence to suggest that the primary failure was a cell to case short circuit. Instead it appears to have been an internal short circuit of one cell.
In that case the + and - lead current of the series string would still be identical and the overvoltage applied to the non-shorted cells would affect cells on either side of the failed cell.
In the Narita case, the public information is not as clear as to just when in the sequence the cell to case short circuit took place.

Regarding the apparent damage to the electrodes of the main battery, I still feel that not enough attention has been publicly directed to the large amount (~10%) of volume expansion to be expected for the entire electrode stack between low voltage cutoff and high voltage cutoff charge levels. This is simply the result of chemical changes to the active material at the electrolyte surface and is in addition to any thermal expansion or contraction. The described problems leading to repeated over-discharge during ground operations has probably led to a greatly increased amount of cycling expansion and contraction compared to the predicted service life profile. Any additional complications from charge control anomalies and thermal expansion/contraction would only add to that.

[Lyman]

inetdog

The described problems leading to repeated over-discharge during ground operations has probably led to a greatly increased amount of cycling expansion and contraction compared to the predicted service life profile. Any additional complications from charge control anomalies and thermal expansion/contraction would only add to that.

I think it would be instructive to look at the construction of each cell. Two metallic foil layers, two "paste" layers, and one "Separator", of polypropylene/polyethylene... Laid up flat, on a bench, no problem. But then it is required to "roll" this strip into a rectangular shape. What is the composition of the binders for each paste? One for Graphite, the other for LithiumCobalt Oxide?
How do these layers react to the stress of assembly?

More importantly, and given the telling photography of the MAIN Battery stacks, how does this laminated stack perform in service?

If the MAIN Battery (the "Exemplar"), shows such obvious signs of deformation, are we seeing the prelude to failure?

Poly is NOT resistant to heat, the perforations can shut, and the output can become disoriented to the electrode. That the Stainless enclosure is deformed is instructive of the stresses involved?

It is not a stretch to go here, NTSB made suggestive statements as to this avenue of the investigation, early on....

:ok:

[TURIN]

Lyman.

I take your point about stresses at the folds and the quality control required to construct a device that works as advertised.
However, as single units these batteries (cells) appear to work well in other environments.

My understanding from previous posts, is that these battery 'cells' have been used extensively as individual units for many years without too many problems.
Bearing that in mind, I don't see how the construction of these individual 'batteries' can be at fault. It must be either the way they (all eight of them) interact as a group, whether that be thermally, electrically or physically, or there is a problem with the way they are being used (charged/discharged). Or indeed a combination of all these. :confused:

[EEngr]

Its interesting to speculate about the battery failure modes and possible design improvements. But what is the service experience with the GS Yuasa LVP65 battery in other applications? Good? Bad?

If it is good, what is Boeing/Thales doing that other users are not?

In the final analysis, any technology will fail (spectacularly or otherwise) if pushed beyond its limits. If Boeing can't identify the particular conditions that exacerbate the failures, how can they be trusted to certify another design?

[FlightPathOBN]

low voltage/high amperage arc...

thats easy...an arc welder :D

(another reason why when it gets going, it is difficult to stop)

[Lyman]

I have a fair amount of experience in laminates. If I had to be restricted to one facet of this failure, I would look at the performance (in service) of the roll.

Expansion and contraction, (with and without heat) would happen at variable rates, and substrate performance would be related to the shear of the adhesive relative to the binding strength of the separate layers.

Each cycle would aggravate any weakness from the manufacture, and new and variabe failures would propagate. The system would break down over time, and penetration of electrolyte (conductor) would occur. Hotspots and potential thermal runaway would likely appear, well ahead of optimistic projections.

The evidence is in the photography of the MAIN Battery in the report. The tortuous and serpentine folds show the results of repetitive expansion, contraction, and variable "return Rates" to original.

Conclusion? Each of the layers present in the roll is fragile, in its own way. Each exhibits particular responses to the stresses of environmental and mechanical challenges. The elastic value of the laminated roll is near zero, it wants to remain in the "last" position it had. forcing it to contort in these conditions is degrading the performance of its intended use.

Layerman (Acryl Cellulosic, monolithic Alumina/acrylate, glassfibre/epoxy, two phase cellulosic/phenolic, etc.) Certified in Epon.

[FlightPathOBN]

so would the circular roll bind at corners, or would the heat radiate with the roll?

[Lyman]

Most of my experience with heat damage has to do with monolitic structure, depending on the purity of the mix, failure is propagated with variable absorption/loss of heat. Fractures are frequently complete, through the width of the material.

With a laminated application, geometry plays an important part, introducing radial differentials at adhered surfaces, and mechanical stresses that vary dependent on the expansion rate of each layer. It strikes me the rate of heat gain and loss would be different at the rolled returns of each layered mass.

How brittle is the paste? How resilient? The metal foil would behave well in mechanical stress, but conduct heat very differently than the anode cathode.
The Poly film would seem to be the most vulnerable to heat and mechanical stress.

There must be a Thermal engineer here?

[TURIN]

That's all very well Lyman, and I do not doubt your experience in this field.

However,
The original single battery (constructed of three cells if you will) appears to perform well ON IT'S OWN,

It is only when as, Eeng has alluded, Boeing/Thales bung eight of them together and attach them to an aeroplane that it all goes wrong.

[FlightPathOBN]

There must be a Thermal engineer here?

I have only worked with the thermal coeff of expansion on a large scale, such as with structures (ie GD 602, claims on the Mariners stadium, etc)

[Lyman]

TURIN

It is only when as, Eeng has alluded, Boeing/Thales bung eight of them together and attach them to an aeroplane that it all goes wrong.

Is there precedent for a single battery in service aboard an aircraft?

In "Large Format"?

All of Boeings propositions have to do with protecting seven cells from just one.

So, when was this single battery tested under CFR? Cold soak, pressure cycles, charge discharge, load, SOC over time (storage). Inert storage?

I think it has more to do with aeronautical conditions, than keeping company with other batteries.

The obvious possibility is degradation of capacity and performance, faster than was predicted on the bench. Boeing, (No one) tested this battery in the sky, integrated with all systems.

Of all the removed Batteries, did some of the locked out cells migrate back into the fleet? Disassemble, test, recharge, assemble and ship? Yuasa had a sole source contract for this battery, Take note of the "tamper proof seal" on the APU BATT......

Demands of service would have everything to do with performance, whether solitary, or grouped in eights?

[TURIN]

Is there precedent for a single battery in service aboard an aircraft?



Good point.

The ACE (Actuator Control Electronics) backup Batteries are also Li-Ion. Much smaller than a main battery and only designed to last about ten minutes. As far as I know there have been no reported problems with these, even the one that got charred by the APU battery at Boston.

The 777 has a similar ACE backup battery. I wonder if it too is Li-Ion? :suspect:

[Kiskaloo]

@EEngr
But what is the service experience with the GS Yuasa LVP65 battery in other applications? Good? Bad?

Per quotes in the media from GS Yuasa, the cell design and construction used in the LVP65 battery was first put into production batteries by them around 1999 and was considered the best choice for the LVP65 based on how well it had worked in previous batteries in other applications.

[HighWind]

Syseng68k, #911
Yes, wind energy systems.

There's stiil the problem of >1 cell shorting to case, but I guess
that scenario could be covered by fusable links between the cells, rather than the existing copper straps. This can be mitigated by ensuring the system can detect and isolate the first fault, and prevent further fault.

However, my point would still be that if the cells are properly managed and run within data sheet limits, the problem won't arise in the first place.The high number of failing batteries could indicate that either the cells are having quality issues, or the cells are operating outside limits.
I'm not sure that number of thermal runaways would be zero, even if the cells are perfect and running under perfect conditions.

Each cell has many square meter of foil, in multiple layers separated with a tin layer of metal oxide.
An imperfection in the metal oxide layer could result in a short.
If the cell is large (or multiple cells are connected in parallel), then the amount of energy that can be dissipated in a fault area is higher, and may reach the combustion/runaway temperature.

Measures sufficient for small cells, might be insufficient for large cells.

[pax2908]

Maybe simply too much current demand while the battery is still too cold ... pity there are no more details re. the Boeing tests (NTSB report page 33, end of paragraph 1.7).
On the other hand, post# 1222 in the Other thread, re. E/E cooling, mentions air drawn from the cabin (?) ; however the FDR data has things like Cooling_Sensor_Aft_Temp_EE, about 20 degree F. How does it work really?

[inetdog]

On the other hand, post# 1222 in the Other thread, re. E/E cooling, mentions air drawn from the cabin (?) ; however the FDR data has things like Cooling_Sensor_Aft_Temp_EE, about 20 degree F. How does it work really?
Possibly drawn from unheated cargo hold instead? But definitely pressurized.

[Lyman]

TURIN

The ACE (Actuator Control Electronics) backup Batteries are also Li-Ion. Much smaller than a main battery and only designed to last about ten minutes. As far as I know there have been no reported problems with these, even the one that got charred by the APU battery at Boston.

The ACE is single purpose, a true 'Backup', not integrated into the distribution system for general purpose.

If only the APU/MAIN had similar function....

This 'large format' (scaled up) battery as used in the Dreamliner suggests that LiIon is suited to standby power only...

@HighWind... I think the Yuasa program shows challenges to materials/methods as regards spontaneous failure.....also. The pastes, but especially the oxide, must be scrupulously blended, (powder/binder) without imperfection of powdery occlusions, or viscosity variation. Add to that the loss of symmetry of the roll in service, and.....poor performance.

:eek:

[HarryMann]

So true, those comments about save a pound and cost a hundredweight...

Been saying for decades that so much booze and pop being carried about the planet unused is more than foolish, its criminal when effectively its responsible for major incidents due over zealous weight saving....

I've seen a 5/32" instead of a 3/16" shear pin cause a fatal in a light aircraft, saved all of another spoonful of cornflakes at breakfast for the pilot but was in a critical position..

[John Farley]

Saving weight is what life is all about when you start the original design process of any aircraft. This applies whether the aircraft is one powered by rubber that you wind up as a kid or is the next generation airliner or combat aircraft. Unless you are totally ruthless about weight the resulting aircraft will be hopelessly out performed by any competitors.

Judging the weight benefits and penalties in the design stage is where the skill comes in. Once you have an aircraft to ground test and later flight test then we start to get the benefit of hindsight as to whether the design choices were flawed.

While it is NOW as obvious as the balls on a dog that the 787 battery choices have produced more penalties than benefits criticising the original choice on the lines of “everybody knows those things are trouble etc” is easy but does not reflect what went on at the paper design stage. I am quite sure one of the battery people might well have preferred NiCads (or whatever) but I am also sure another guy wanted more weight in the bracket that the main gear is fixed to. Detailed designers are conservative people who do not want their part of the design to let down the team.

Enter the chief designer. If he allowed every designer to have his conservative choice they would probably have to strengthen the hangar floor. So where does he draw the line? – because it is HE who has to draw it. If he does not push towards the dodgy end of the spectrum we would never have got the continually better aeroplanes that the world has seen for the last 100 years.

Working on a VTO aircraft in the early 60s I asked the chief designer why on earth the cockpit had to be so low because it was going to ruin the rearwards view and he replied “to save aluminium on the fuselage sides”. At the time I thought he was wrong. Later I understood his problem and where he was coming from.

[kenneth house]

Excessive cell case flexing due to either overcharging or reduced external pressure cycling will cause internal shorting of the electrodes, fatigue of the aluminum and copper current collector fingers, etc. This is the root cause.

When a Lithium cell is "puffy" it typically has a reduced capacity and is an indication of over-charging. i.e. it is damaged goods.

But what else can puff out a cell?

How about the 3 psi differential pressure between the inner volume of the cell case (@14.7) and the reduced pressure of the E/E bay at 6000' (@11.8 psi) -- the deflection of the 5" x 7" sheet of 0.031" thick 304 stainless steel that is the S1 and S3 cell case will experience plastic deformation. You can see it in the CT scans of the Main, Fig 22. That expansion creates voids in the windings and separates the poly separator from the foils--it is now damaged goods and its charging profile will no longer follow the sinc and exponential model assumed by the Battery Charging Unit software as described in the patent.

Is it possible that three 45-second 450Amp APU starts with 60 seconds rest in-between would pull the pack down below the 15% capacity lock-out of the BMU.

What is the 15 % lock-out? That is the condition when the BMU energizes the contactor to open-circuit the battery and render it useless unless returned to the vendor for reset/recharge.

[saptzae]

@kenneth house


Hull and eebay is at cabin altitude ~6000 feet
Cells are derated to 4.025V / ~80%SOC
BCU was not connected to charger / test rig at 2006 incident
OV/UV is being considered as part of investigation
Exemplar batteries showing deformation - CT report pg 26, img 22 - of cells are being examined

All the above was already discussed at least once in this thread.

[Kiskaloo]

The FAA has given Boeing permission to move forward with their proposed fix. (http://www.seattlepi.com/business/boeing/article/FAA-OKs-plan-for-fixing-Boeing-787-battery-4348955.php)

The Federal Aviation Administration (FAA) today approved the Boeing Commercial Airplane Company's certification plan for the redesigned 787 battery system, after thoroughly reviewing Boeing's proposed modifications and the company's plan to demonstrate that the system will meet FAA requirements...

...The FAA will approve the redesign only if the company successfully completes all required tests and analysis to demonstrate the new design complies with FAA requirements. The FAA's January 16, 2013 airworthiness directive, which required operators to temporarily cease 787 operations, is still in effect, and the FAA is continuing its comprehensive review of the 787 design, production and manufacturing process.

[TURIN]

•Hull and eebay is at cabin altitude ~6000 feet


Fixed it for ya. :ok:



The ACE is single purpose, a true 'Backup', not integrated into the distribution system for general purpose.



The APU battery has a single purpose, mainly. It is not integrated into the system. It starts the APU (occasionally) or powers the Nav lights (very occasionally).
But still failed.

The main battery has extra functions compared to say, the 777 main batteries, and that is to power emergency brakes (hardly ever) or towing without a generator (rare).
And also failed.

General purpose? No.

[FlightPathOBN]

"The FAA also granted Boeing permission to begin flight test activities on two airplanes: line number 86, which will conduct tests to demonstrate that the comprehensive set of solutions work as intended in flight and on the ground; and ZA005, which is scheduled to conduct engine improvement tests unrelated to the battery issue."

Engine improvement tests??

Here is the FAA link...(sorry, but due to budget cuts, the FAA cannot afford the word wrap in a press release)

Press Release – FAA Approves Boeing 787 Certification Plan (http://www.faa.gov/news/press_releases/news_story.cfm?newsId=14394)

edit: Waiting on the NTSB repsonse...

and of course, the last line of the news release..

"The FAA’s January 16, 2013 airworthiness directive, which required operators to temporarily cease 787 operations, is still in effect, and the FAA is continuing its comprehensive review of the 787 design, production and manufacturing process."

[Lyman]

Does the ACE perform other, occasional duty? How many discharge/charge cycles per flght? Is it enclosed, with a tamper proof seal? Is it not a true "Backup" storage battery, where the APU/MAIN batteries do extra duty?

Do you know if the ACE storage battery is part of the AD?

[Kiskaloo]

@Lyman
Does the ACE perform other, occasional duty?

As I understand it, it is only designed to provide emergency power to the ACEs in the event of a loss of other in-flight power sources.


How many discharge/charge cycles per flight?

I would expect none since it would only be used if all other in-flight power sources failed.


Is it enclosed, with a tamper proof seal?

It is enclosed.


Is it not a true "Backup" storage battery, where the APU/MAIN batteries do extra duty?

Yes, it appears to only be designed for use in an emergency.


Do (we) know if the ACE storage battery is part of the AD?

It appears to not be as I have seen no mention of it in any of the FAA, NTSB or JTSB. Same with other Li-Ion batteries aboard the 787 like the ~30 for wireless emergency light units and those that power the DFDR boxes.

[Lyman]

Kiskaloo

It occurred to me a while ago that perhaps too much was being asked of the APU/MAIN Batt system, as designed. It turns out there is a similar system one rack above the main problem. So it would be of great interest to see how Boeing addresses the AD. There are myriad problems, most of which seem to be a result of "utilizing" the Larger groups almost casually, as if they represent a time proven system, rather than what we see, a very problematic new-ish iteration of a Battery that got its credentials from the industry after the Dreamliner launched.

I am stuck on how building a bigger box solves the reliability problems. Unless the swelling, misaligned rolls, and internal shorting are remedied, the result will be safe but inert "back-ups", going off line at a rate too high to satisfy dependability. At great expense. Cannot a windmilling donk fire up the APU?

Thank you for a great response....

[bill good]

This thread is mostly related to the APU incident which is good but how much is known about the other battery involving the air return & resultant cell damage?

[EW73]

Deleted - Found that the announcement had already been posted!

[Kiskaloo]

@Lyman

To my knowledge, the Ship's and APU batteries for the 787 power the same systems as they do on other Boeing Commercial Airplane families. So I don't believe Boeing is asking more of them on the 787 than, say, the 777.

As to reliability issues, I agree that the containment box does nothing to address this. It's merely designed to ensure that if a battery enters thermal runway, it cannot affect the safety of flight of the aircraft.

To address reliability, Boeing is making changes to the battery design, increasing cell spacing and putting in insulation between the cells. The expectation is that this will reduce the chance of a single cell entering thermal runway and, if it should, reduce the chance of adjacent cells entering thermal runway. They also appear to be increasing quality control in production of the cells and assembly of those cells into batteries to reduce the chance of a short circuit that could induce a thermal runway in a cell.

Even if Boeing eliminates safety as a concern, they must also eliminate economics as a concern. As the APU battery appears to be necessary in order to run the APU, the low MTBF for the APU battery is going to affect the ability to certify the 787 for ETOPS beyond 180 minutes (the current limit for a plane with an inoperative APU). And even if a failed battery can't damage the plane, it is expensive to replace so a low MTBF will be a concern and pain point for airline customers.

[Lyman]

Here is the condition that requires the presence of the upgraded BOX.

(6) Each Lithium ion battery installation must have provisions to prevent any hazardous effect on structure or essential systems caused by the maximum amount of heat the battery can generate during a short circuit of the battery or of its individual cells.

This is why I have made such an issue of the nomenclature, "Cell" v. "Battery".

As written, the language prevents (prohibits) a single short circuit. A single short circuit in the current design will (should) take the entire eight cell "Group" off line, "loss of essential system".

Technically, then, there should be:

Eight separate "boxes" that isolate a short circuit from failing the "essential system" (the other seven "cells") and a means for keeping the power storage at usable level. OR,

""Short Circuit" must be proven to be "One in Ten Million flight hours".

This is regulatory, and hasn't anything to do with "Fire".

Again, the major problem is the 'current' design as to suitable (and "demonstrable") "Reliability", not the prevention of fire.

If the APU battery is an agglomeration of eight cells, comprising a "battery", then going off line is a major failure with or without any fire issue. If it is a group of eight individual batteries, and can be made to perform if one battery shorts out, the system is (becomes) reliable by definition...

The potential to catch fire must still be one in one BILLION flight hours, regardless the design of the enclosure.

EXCEPT. The discussion will be: "What causes the fire". The battery cannot be the source, (cause) of fire, this is fatal to its installation. It is by definition "flammable" in and of itself, and if exposed to extraneous ignition, may ignite, this is allowed.

imo....

[FlightPathOBN]

The potential to catch fire must still be one in one BILLION flight hours,

again...aaaarrrgggh! :mad:

Per FAA AC23-1309 SYSTEM SAFETY ANALYSIS AND ASSESSMENT FOR
PART 23 AIRPLANES

Step 1: Determination of the "average flight;"
Step 2: Calculation of the probability of a failure condition for a certain "average flight;"
Step 3: Calculation of the "average probability per flight" of a failure condition; and
Step 4: Calculation of the "average probability per flight hour" of a failure condition.

http://operationsbasednavigation.com/wordpress/wp-content/uploads/2013/03/ScreenHunter_27-Mar.-13-08.111.jpg

The failure mode must be statistically 10 -9 (1 in a billion) for EACH FLIGHT HOUR.


http://operationsbasednavigation.com/wordpress/wp-content/uploads/2013/03/ScreenHunter_27-Mar.-13-08.24.jpg

[Lyman]

I should have used quotes, I have no knowledge of the statistics of these failures.

The quote is from John Goglia, outspoken critic of Boeing's actions, and a former Member of NTSB...

my bad.....


BUT. Does it not mean that a fire aboard must not happen in the life of the fleet, roughly?

[FlightPathOBN]

Lyman...no worries. I have seen this quote around, in this form, but it is really not the intent of the safety case analysis. It changes the flavor significantly from 1/Billion per fight hour to 1 Billion flight hours.

It is the sum of all of the parts, so while you may have a known battery failure every thousand flight hours, it is the sum of all of the components failure modes, happening at the same time, that must be 1/Billion probability.

OT alert...

I had to provide this type of analysis for RNP procedure criteria. as an example, you have a high probability of losing a single sat, then lower probabilities of losing more to where the GPS signal is degraded enough to be useless for nav, all that combined with on-board combination of system failures...adding up to an off-track probability per RNP level.

edit:
BUT. Does it not mean that a fire aboard must not happen in the life of the fleet, roughly?

not exactly. there is the potential for fire on an ac, such as with the engines...and the Li battery fires from devices has been addressed on ac.

While the potential for the LI fires was somewhat addressed by the FAA, it doesnt appear, from the design on-board, that it was considered an issue, and mitigated by other means such as the BMS controls, in the statistical models...

So I guess the answer, and the avenue Boeing appears to be following, it that yes, with this system, we have to add the probability of a fire with the battery system, and mitigate that potential. They must also re-evaluate the entire system, and how it allowed this to happen.

As we have noted, there is all of the bench testing, but there will be the testing with the 2 aircraft. Very, very complex issue. Personally, I dont feel that they will be able to mitigate the issues that caused the batteries into failure mode in a reasonable amount of time. Testing the battery containment box, on a production aircraft?? :eek:

[Lyman]

I believe it was BOEING that stated the possibility for fire was only related to overcharging, and they optimistically pegged that at 1/billion.

If I delete all reference to statistics and probability, will you address the main thrust of my longer post, that FIRE is not the only problem here?

[FlightPathOBN]

I think we are talking about the same thing here....

Very complex, holistic, system wide issue...

edit; this just in..

Looks like it worked (for Boeing stock that is)

"Some analysts said the high-tech plane, which was grounded worldwide in January, might be flying passengers again as early as May, after the Federal Aviation Administration on Tuesday approved Boeing's plan to certify the battery system."

Boeing tests prompt analysts' upgrades, but risks still lurk - Yahoo! Finance (http://finance.yahoo.com/news/boeing-tests-prompt-analysts-upgrades-170933043.html)

[TURIN]

Do you know if the ACE storage battery is part of the AD?

No, but I do know that ALL Li-Ion batteries were disconnected by at least one airline when they were grounded in January. Which included the ACE battery modules.

To my knowledge, the Ship's and APU batteries for the 787 power the same systems as they do on other Boeing Commercial Airplane families. So I don't believe Boeing is asking more of them on the 787 than, say, the 777.

Except of course the emergency and towing brakes systems. (Main bat only).

[FlightPathOBN]

They gonna have to reboot the FMS?

[FlightPathOBN]

still have to meet or exceed this:


In lieu of the requirements of 14 CFR 25.1353(c)(1) through (c)(4), the following special conditions apply. Lithium ion batteries must be designed and installed as follows:
(1) Safe cell temperatures and pressures must be maintained during any foreseeable charging or discharging condition and during any failure of the charging or battery monitoring system not shown to be extremely remoteimprobable. The lithium ion battery installation must preclude explosion in the event of those failures. (working group to refine).
(2) Design of the lithium ion batteries must preclude the occurrence of self-sustaining, uncontrolled increases in temperature or pressure. (this may be met with requirement (1) - working group to refine).
(3) No explosive or toxic gases emitted by any lithium ion battery in normal operation, or as the result of any failure of the battery charging system, monitoring system, or battery installation not shown to be extremely remoteimprobable, may accumulate in hazardous quantities within the airplane. (this may be redundant with (1)).
(4) Installations of lithium ion batteries must meet the requirements of 14 CFR 25.863(a) through (d). (may also need to include parts 23, 27, & 29).
(5) No corrosive fluids or gases that may escape from any lithium ion battery may damage surrounding structure or any adjacent systems, equipment, or electrical wiring of the airplane in such a way as to cause a major or more severe failure condition, in accordance with 14 CFR 25.1309(b) (also 23, 27, & 29) and applicable regulatory guidance. We may want to give other means of compliance such as venting gases off-board, containment enclosure, etc.
(6) Each lithium ion battery installation must have provisions to prevent any hazardous effect on structure or essential systems caused by the maximum amount of heat the battery can generate during a short circuit of the battery or of its individual cells. NOTE: Need to add a requirement to report maximum temperature reached during the RTCA/DO-311 Short circuit test. The equivalent UN 38.3 short circuit test states that testing occur at 55 deg C. DO-311 2.3.12 – External Short Circuit With Protection Disabled: states to report maximum temperature.
Test is run at 23 deg C. (7) Lithium ion battery installations systems must have a system means to control the charging
rate of the battery automatically, so as to prevent battery overheating or overcharging, and,
(i) A battery temperature sensing and over-temperature warning system with a means for automatically disconnecting the battery from its charging source in the event of an over- temperature condition, or,
(ii) A battery failure sensing and warning system with a means for automatically disconnecting the battery from its charging source in the event of battery failure.
(8) Any lithium ion battery installationsystem whose function is required for safe operation of the airplane must incorporate a monitoring and warning feature that will provide an indication to the appropriate flight crewmembers whenever the state of charge of the batteries has fallen below levels considered acceptable for dispatch of the airplane. (this is a new requirement that is applicable to Li-ion batteries/systems).
(9) The Instructions for Continued Airworthiness required by 14 CFR 25.1529 must contain maintenance requirements for measurements of battery capacity at appropriate intervals to ensure that batteries whose function is required for safe operation of the airplane will perform their intended function as long as the battery is installed in the airplane. The Instructions for Continued Airworthiness must also contain procedures for the maintenance of lithium ion batteries in spares storage to prevent the replacement of batteries whose function is required for safe operation of the airplane with batteries that have experienced degraded charge retention ability or other damage due to prolonged storage at a low state of charge. (this requirement is already required to meet compliance to 1529).

[Lyman]

FP

As above in my post #965, the condition number six, as written, prohibits this Battery installation as designed.

Any short circuit will take the battery off line, as will any temperature that may effect sister cells.

The APU/MAIN Batteries themselves are "Essential systems", so any self induced failure qualifies as a battery caused loss of a safety critical system.

If the language is changed to "Other Essential Systems", and the FAA is satisfied that the loss of the Battery is not "essential", the Box will fly.

It looks lke they are revising the conditions to suit the Boeing Fix. Did we think they would not?

Unbelievable.

BUT. FIRE cannot be allowed, and since the Box is identified as "Fire Protection"
the whole proposal is a travesty of the first order anyway.

Curiouser and curiouser....

[pax2908]

John (#953), I believe I understand all of that. However I would say it is "inexcusable" (and not just with the benefit of what we know now) to go ahead with the new technology without having put in place a programme of battery monitoring, data logging, and periodic data analysis.

The NTSB report, unless I missed something, does not answer some question re. what data may be available. Any useful data which might have been logged by the BCU, for example?

Also the following seems not a very precise statement:
"The BMU and suppressor had passed manufacturing quality tests and would have logged a failure if the suppressor had detached before the incident"
I would read it as "The only opportunity to detect the missing TVS would have been during the QA/QC bench test of the BMU". Is this correct?

[cockney steve]

I just posted my (worthless) opinion on the R&N thread.
reiterating:
another bodge to get them earning revenue again
wound cells have same expansion/contraction/shuffling/void problems as folded cells.

Each of the 24 sub-cells should be heat and electrically insulated in a ceramic enclosure and connect to the cell's internal busbar/strap via a fusible link...thus any failed cell will automatically drop out and allow the whole battery to stay on-line at correct voltage (though capacity reduced by 1/3 due to the weakest link in the series-arrangement.

This would be a more graceful degradation than the current all or nothing scenario.
I also made a somewhat tfacetious suggestion, re- the firebox/vent.:p

[FlightPathOBN]

Lyman,

The box is simply a Red Herring to get the press/investors going.

It cannot be part of the solution, but simply a worst case scenario placate for the regulators..

I look at the extensive cooling systems on the hybrid cars/electric cars, but like a nuke reactor, in an accident, you loose cooling, shes gonna blow...

[Lyman]

Yes, the box is not allowed for the intended purpose, containment. FIRE is prohibited. So the box is moot, and has got to be a distraction only. The box can be used for containment of gases, and electrolyte, but not for fire.

Who kids who?

[FlightPathOBN]

Like I posted before, the stock analysts upgraded based on news of pending flights..

[Machaca]

Gentlemen - this is Tech Log.

Hyperbole and speculation belong in the R&N thread.

Thank you!

[Machinbird]

Each of the 24 sub-cells should be heat and electrically insulated in a ceramic enclosure and connect to the cell's internal busbar/strap via a fusible link...thus any failed cell will automatically drop out and allow the whole battery to stay on-line at correct voltage (though capacity reduced by 1/3 due to the weakest link in the series-arrangement.
Wouldn't the battery take up a considerable volume with all those additional features?
How would you propose to detect that a portion of the battery's capacity has taken a vacation? Are you sure that the weaker cell will not just deliver more current per plate as it tries to keep up with the applied load and then completely destroy itself?

Maybe it would be better to discharge a defective cell under controlled conditions and then crowbar across it with a contactor. You would have full rated amperage at ~4 volts less potential. Would the loads still run satisfactorily under those conditions? Probably, although the engineers might have to tweak the load characteristics a bit.

[RetiredF4]

The Boing solution

Battery
(http://www.boeing.com/787-media-resource/images/K1.jpeg)
Boing 3 pics HD (http://www.boeing.com/787-media-resource/787_gallery.html)

Boing briefing pdf (http://www.boeing.com/787-media-resource/docs/787-Battery-Solution.pdf)

[green granite]

The last link is broken, try:

Boeing Provides Details on 787 Battery Improvements - Mar 14, 2013 (http://boeing.mediaroom.com/index.php?s=43&item=2622)

It would appear to be a bit more than a fudge.

The improvements include enhanced production and operating processes, improved battery design features and a new battery enclosure.

"As soon as our testing is complete and we obtain regulatory approvals, we will be positioned to help our customers implement these changes and begin the process of getting their 787s back in the air," said Boeing Commercial Airplanes President and CEO Ray Conner. "Passengers can be assured that we have completed a thorough review of the battery system and made numerous improvements that we believe will make it a safer, more reliable battery system." Battery system changes include changes to the battery itself, the battery charging unit and the battery installation.

Earlier this week the FAA approved Boeing's certification plan, which lays out the discrete testing to be done to demonstrate that the battery improvements address the conditions laid out in the Airworthiness Directive that has suspended 787 commercial operations.

[RetiredF4]

The last link is broken, try:

Boeing Provides Details on 787 Battery Improvements - Mar 14, 2013


Try again, i ammended it. It is a different link to a presentation.

[green granite]

Thanks, RetiredF4 it appears to be a visual presentation of the comprehensive article I linked to. :ok:

[cockney steve]

@ Machinbird...
yes, but we're not talking huge earthenware jugs here,- lightweight modern ceramics can do the job and minimise the possibility of a short-circuit to the firebox as well as containing the heat.

We're talking a posh kludge here. Given that Boeing refuse to admit they rushed ahead with a novel emergency-supply system and no backup plan,that they refuse to abandon the current Lithium Ion technology, we have to think how this may be made safe and reliable-enough that you or I would venture forth across a wide pond in the aircraft.
Every cell insulated/isolated....thermal runaway is minimised
every cell monitored.....we've already discussed this....electronic components are cheap as chips, there are plenty of multipole Mil-spec plugs of the type currently used, which would accomodate the extra harness needed to monitor and balance every sub-cell (24 per battery in current design)
fusible connections would not be significantly bulkier than the present arrangements. "crowbaring" would add 8 contactors under your proposal and it would also reduce both the voltage and capacity of the battery.....assuming only ONE cell popped and self-isolated, you'd still have full voltage AND 66% capacity available from that cell better than being 12.5% down on both capacity and volts when in dire need!....remember, these batteries are "supposed" to be a last-resort backup...(they're not!)
the APU battery is routinely used for nav-lights at pushback, as well as APU start when main engines aren't turning.
the essential instrument and braking battery is routinely used to power a refuelling valve /flap!

Latest news seems to confirm what I said very early on.... these batteries CAN be used for high-rate applications , but both charge and discharge must be monitored and controlled rigorously.
In a situation where these batteries are used in anger, I'd think the destruction and fire risk would be pretty low on the list of worries :}

[Lyman]

From Boeing...

In Boeing's view, neither incident met the company's internal definition for the condition called "thermal runaway". That is a situation in which there "is so much energy, so much heat and so much flame that it would put the airplane at risk", Sinnett says. "We know very clearly this was not the case in the Logan event and the Takamatsu event."

Told ya' it was just a question of scale (quantity), not type (quality)....:ugh:

The problem is one of 'perception', and Boeing takes a more forgiving position; as the chief regulator, Boeing has that 'option'.

Nothing visible in the new program hasn't been broached on PPRuNe, at length, with numbers, by professionals.

So here is a prime example of Political process leaking into the industrial. When caught out, "damage control", deny, and divert attention. Bamboozle the majority, and wait things out...

Two new braided and robust ground straps, better QC in Japan, tweaks to BMU and BCU, a nifty box and tube, BYU......

(bob's yer uncle). The waiting game....

[FlightPathOBN]

Lyman,

Wasnt this approach the same used by Clinton with Monica Lewinsky affair?

The battery system did not meet the 'technical' definition of a fire?!?!?! :mad:

What about the other issues with arcing panels, burnt relay, miswired systems???

[Lyman]

FP

I was thinking more along the lines of Nixon. Because Clinton's ploy worked, but Nixon had to resign.

Sinnett (Boeing):

“This enclosure keeps us from ever having a fire to begin with,” Sinnett told the Tokyo briefing.

Confucius (Free lancer):

"A half truth is a Whole Lie..."

Lyman (Nobody):

Sinnett chose his prose carefully, for what he means is there will be no "Aircraft FIRE", not "No Battery FIRE...."

That makes his comment sleazy, sly, and at least to me, dishonest....

[Ian W]

Lyman
Sinnett chose his prose carefully, for what he means is there will be no "Aircraft FIRE", not "No Battery FIRE...."

That makes his comment sleazy, sly, and at least to me, dishonest....I think you may be getting a little overexcited for the Tech Log thread and even for the R&N thread.

From the briefing:
During engineering testing, which occurs prior to certification testing, the team demonstrated that the new housing could safely contain a battery failure that included the failure of all eight cells within the battery. The "ultimate" load is the equivalent of 1.5 times the maximum force ever expected to be encountered during a battery failure. The housing easily withstood this pressure and did not fail until the pressure was more than three times the ultimate load.
Through another test, the team demonstrated that fire cannot occur within the new enclosure. Its design eliminates oxygen, making the containment unit self-inerting. Inerting is a step above fire detection and extinguishing as it prevents a fire from ever occurring. The design also vents all vapors by venting directly outside of the airplane rather than into the equipment bay.
"We put this new design through a rigorous set of tests. We tried to find a way to introduce a fire in the containment but it just wouldn't happen. Even when we introduced a flammable gas in the presence of an ignition source, the absence of oxygen meant there was no fire.
"We drew from the new industry standard, DO311, established by RTCA, to establish our testing plan," said Sinnett. "These standards weren't available when we set the testing plan for the baseline battery and they helped us ensure the new design is robust and safe. We intend to show, during certification, that the 787 battery meets all objectives of DO-311 and only deviates from specific requirements where the 787-unique items are not covered by the standards." RTCA is a not-for-profit organization that serves as a federal advisory committee in establishing guidelines for the aviation industry.
What is it about demonstrating that fire cannot occur inside the new enclosure that you fail to grasp?

These guys who are internal and external experts on these batteries seem to have accepted that they don't know what caused the problems but have grouped potential causes for problems into categories then ensured that those causes do not cause the battery problems and -even it they did- any battery issue would be safely contained without fire.

The FAA is not going to be caught out again 'not testing enough' and nor is EASA so you can be assured that this is not a 'political' fix.

Would you prefer Boeing just left all the 787s on the ground until they can be rebuilt with canvas covered wood stringers and radial engines with propellers? Or would the danger of a gasoline fire in a wooden aircraft make you stick to ground travel?

[Lyman]

I think you have not understood the nature of the chemistry involved.

"Inerting" the enclosure, by eliminating Oxygen, accomplishes nothing. This battery will spontaneously ignite without the presence of Oxygen. The electrolyte is an organic, and burns fine without ambient O2. At one time it was believed that because some Oxygen is produced in the burning of the electroyte, and the LithiumCobaltOxide, that it is somehow self sustaining, and that is why it resists extinguishment. That is a myth.

If you would like a reference I would be happy to supply one, but instead, you may want to read the paper that was the standard in 2011.

I have linked it here before.

"Burning" is not the problem, anyway. Temperatures in a runon cell push 3000 degrees, far hotter than any Fire".

The problem remains, IGNITION. NOT Oxygenation....Boeing still have a humongous obstacle ahead, to prove that retention of a volatile Battery system is worth allowing something that has never before been allowed, Fuel, and Ignition, present on an aircraft. If some wag wants to bring up Engines, have on. Kerosene does not spontaneously ignite.

It's an interesting discussion, but the basic problem, still, is reliability, and the regulations that do not allow fire.

The specific regulations won't allow even the production of excess pressure and temperature. Boeing needs to get around that, isolation in a box does not satisfy that rule. Not yet, anyway.

I think a fair question is what happens to the ejecta from the box? The composition and construction of the vent tube must sustain temperatures approaching 3000 degrees intermixed with carbon, ash, molten aluminum, and dense toxic smoke. All this is to be dumped along the belly of a plastic aircraft possibly hours from any place to land? Into an OXYGEN rich environment for, how long? twenty minutes? Can you think of a better way to ignite CFRP? Super heated oxygen starved solvents, metals, and vapors, blown into a 400 knot jet of Oxygen? Has anyone thought that through?

Convince me. The benefit of the doubt has passed away, and was given a burial in January.

For one second, Ian, do you think this scheme would have passed original certification?

[FullWings]

Through another test, the team demonstrated that fire cannot occur within the new enclosure. Its design eliminates oxygen, making the containment unit self-inerting. Inerting is a step above fire detection and extinguishing as it prevents a fire from ever occurring.
It makes me wonder what these people are thinking. All the ingredients for "fire", i.e. rapid oxidation with release of heat, are present *inside* the cell. You could put a LiCo cell in a vacuum chamber and it would still catch fire with smoke and flames, much like a Saturn V booster still works outside the Earth's atmosphere.

The cathode material of their batteries is lithium cobalt oxide, which gives off oxygen when decomposed by heat which then merrily reacts (burns) with the organic solvents, carbon, etc. which the rest of the battery is made from.

[inetdog]

I believe that the non-technical person would have a hard time accepting the argument that even though temperatures inside the battery could still reach over 2000F, since there is no open flame there is by definition no fire.
The uncontrolled release of the chemical energy stored in the active materials of the battery is technically not fire, but it is capable of doing all of the damaging things that fire can do.
BTW, charcoal does not burn by that same narrow definition, since there is no open flame once it is started. But it sure does oxidize well and gives off a lot of heat. :)

[FlightPathOBN]

interesting factoid...

yesterday the fire sprinkler system went off in the entire engineering building, (where the fix is being designed/tested) causing about $3 million in damage...

[SaturnV]

Fullwings, the J-2 engines used on the second and third stages burned LH2 and LOx. The first stage F-1 engines burned kerosene and LOx. The NERVA engine, never operational, used only liquid hydrogen for fuel, but it was a nuclear reactor. The Apollo 13 accident was caused by an explosion in a LOx tank in the service module.

[RetiredF4]

A very explicit graph of the battery designs

battery design before mod
(http://www.boeing.com/787-media-resource/images/graphics/787%20batteryparts.jpg)

battery design after mod (http://www.boeing.com/787-media-resource/images/graphics/Infographic-787Battery-English.jpg)

not much changed? Mainly "improved for electrical isolation".


But no wonder, design change due to safety aspects seems to be of no concern

The Boing design change process (http://www.boeing.com/787-media-resource/images/graphics/DESIGN_Proc_REV4.jpg)

[mm43]

Inerting is a step above fire detection and extinguishing as it prevents a fire from ever occurring.Seems rather obvious that Boeing don't employ expertise from NASA, and NASA don't employ from Boeing for the same reason.:}

[tonytales]

Assuming that Boeing is able to sell the concept of a fire proof/heat proof containment and overboard discharge of effluents for the Lixx batteries, I wonder how long the airlines will be able to stand what was already an apparently abnormally high battery replacement rate prior to the fire/overheats. These batteries cannot be cheap and moving them to outstations may be a problem. Are you going to be able to carry a COMAT shipment of a Lixx battery on a passenger airplane?

From the maintenance aspect, changing a NiCad battery on a turnaround was only a grunt job, opening a sealed battery container and possibly finding the contents fried will probably cause some delay.

The maintenance shop where they maintain the Lixx batteries will certainly be a more interesting place too. Maintaining NiCads is a pretty demanding chore what with the necessity for depleting them completely and then charging and making sure the cells are balanced. Having cells that may erupt should keep the techies on their toes.

I also wonder if all the weight involved in the containment and venting systems plus the extra monitoring systems, etc, hasn't wiped out the theoretical weight saving of going to this new technology?

[rottenray]

Lyman writes:
I think you have not understood the nature of the chemistry involved.Alright, Lyman, I've read your engineer-bashing and your regulator-bashing through ALL of these pages.

I'm calling you on pure hubris - you think you know more than ANYONE involved in the process of designing and building these batteries and the system they are a part of.

What are your qualifications? If you don't mind, and it's not too much trouble for you, could you please drop a few airframe names or electrical systems you've had a key position in designing?

Or at least a few key projects you've worked on?

Or a few accident investigations you've assisted with?

Or...

Hell, it's not worth it.

From the very bottom of every Pprune page:

"As these are anonymous forums the origins of the contributions may be opposite to what may be apparent. In fact the press may use it, or the unscrupulous, or sciolists*, to elicit certain reactions."

Glove fit.

Lyman, you're an old man who has taken up internet forum trolling as a hobby, and I personally wish you'd find a different site to troll.

Aside from your comments I generally enjoy Pprune.

Best of luck.

RR