Just a little slip: a tragedy of errors for G-EUOE

The AAIB have released their report on the incident involving Airbus A319-131 G-EUOE at London Heathrow Airport on 24 May 2013. This is one of those reports where I find I’m clenching my teeth as it becomes clear how much has gone wrong.

This incident begins the night before on the 23rd of May. The Airbus A319 registered G-EUOE was parked at Heathrow at Stand 513 for overnight maintenance.

Two British Airways technicians were working the short haul line maintenance shift at Terminal 5 from 18:45 to 06:45. The two technicians were assigned six aircraft. Each aircraft required a Daily Check and two of them required Weekly Checks: Airbus A319 G-EUOE and Airbus A321 G-EUXI. As a part of these checks, the technicians would determine what “defect rectifications” were required and carry them out.

Both were working overtime, it was not their regular shift. There was a shortage of staff at British Airways and so the staff had been putting in a lot of overtime. Although both technicians complied with the BA’s working time policy, the risk of fatigue was high.

They started their shift with a Daily Check on a Boeing 767 at Stand 546. The 767 and the two aircraft requiring Weekly Checks were prioritised as they were most likely to require additional maintenance following the checks.

They then continued to Stand 513 to wait for G-EUOE to arrive. G-EUOE was scheduled for both Daily and Weekly Checks, including a visual inspection of the IDG (Integrated Drive Generator) oil levels on both engines. The technicians split up so that each could check an engine. Fan cowl doors are heavy and cumbersome and it takes two technicians to lift and prop the doors open. However, the IDG oil levels can be checked by just holding the door up, so it is more efficient to split up so each technician can check one engine quickly if they don’t need to prop them open.

The IDG oil level was low on both engines. Both technicians lowered the fan cowl doors, not bothering to latch them as they were going to have to service them.

In order to top up the oil, they needed an IDG oil gun and oil from the maintenance stores. The north store room was being refurbished, which meant driving to the south store room which was about a five-minute drive away. They still had four aircraft to check, so rather than stop to drive to another part of the airport just for the IDG oil gun, they decided to continue on so that they would know how much work they needed to complete for the shift and what other equipment was required.

Technically, if the technicians leave the aircraft, the fan cowl doors should always be propped fully open using the stays or fully re-latched. Also, a warning notice should have been placed in the cockpit and then removed again when the work was complete and the fan cowl doors closed.

Realistically, it is common practice to skip these steps, not just at BA but generally. You can even top up the IDG oil leaving the fan cowl door resting on the hold-open device, so there’s no need to use the stays and they knew they would have the work completed before the end of their shifts.

The technicians made a note in the work allocation sheet to the effect that the IDG oil servicing task was incomplete. One updated the technical log on the aircraft’s flight deck and left an open entry in the technical log for incomplete Weekly Check.

They then moved on to the next aircraft requiring a Weekly Check, an Airbus 320 registration G-EUXI at stand 517. A few years ago, British Airways introduced a local procedure at Terminal 5 for a staff member to take the technical logs from all aircraft undergoing a Weekly Check to the maintenance office, where staff carried out administrative checks. The technical logs were usually removed before midnight and then returned an hour prior to the aircraft’s first flight of the day. By the time they completed the Weekly Check on G-EUXI, the technical logs had been removed for the night and the technical log entries were made later during the shift at the maintenance office.

After their rest break, one of the technicians popped by the south store room to pick up an IDG gun but there were none available. The technicians decided to pick up a second van, so that one technician could go to the Eastern Ancillary Area to see if he could pick up an IDG gun and oil there. They agreed to meet up again at Stand 509, where the final Daily Check of the night was located. From there, they could travel to G-EUOE to finish the Weekly Check. They had two vans so they drove separately southwards along Terminal 5A to complete the oil servicing.

The technician leading the way parked his van under the jet-bridge of the aircraft facing the left engine and the second technician parked alongside. To their surprise, they found that the fan cowl doors on both engines were closed and latched. This was strange but as they’d been away for three hours, they decided that someone else must have noticed the fan cowls left open and closed them. Both engines had cooled down and as they each checked an engine, they both found that the IDG level was now within the green band, which meant that the engines didn’t require servicing. They agreed that as the engines had cooled, residual oil had drained back into the IDG oil sump. They worked together to close and latch the fan cowl doors on both engines and verified each others work to ensure that the latches were correctly fastened. By now, they knew the tech log was at the office. One technician drove to the eastern store to return the IDG gun and oil before meeting his companion at the southern crew room. There, they completed the Weekly Check worksheet for G-EUOE and completed the aircraft technical log. It was just after 2am. They commented to colleagues about finding the fan cowls closed on the aircraft and that the oil levels had raised while the engines had cooled. No one present questioned this.

No one realised that they had serviced the wrong aircraft.

On their way back to complete the oil service, the two technicians drove straight past Stand 513, where G-EUOE was standing, and instead continued on to Stand 517 and G-EUXI, their other Weekly Check of the night. As both were driving, neither cross-checked which stand that they were supposed to go to. If the fan cowl doors had been propped fully open, one or the other might have noticed as they drove past but as it was, nothing seemed amiss.

The A321 is longer than the A319 and the registration is marked in various places on the aircraft, however both technicians were completely convinced that they knew where they were; neither checked the note on the work allocation sheet. The aircraft certainly looked familiar, as they’d worked on it earlier. The log book in the cockpit should have drawn their attention to the fact that it was the wrong plane. At that time of night, however, they had to collect the logbook from the office, so they simply requested the one that they needed and updated G-EUOE’s log to say the oil service wasn’t needed and the Weekly Check was complete.

The entire incident is a textbook example of confirmation bias, where we look for facts to support our assumptions rather than consider the evidence that we may be wrong.

The situation is not yet tragic, however, as the aircraft is still safely parked. All it needs is for someone to notice.

The next morning, the crew reported for duty at 05:45. The pilots and cabin crew met on the aircraft at Stand 513 45 minutes before the scheduled departure time. A British Airways staff member took photographs of the aircraft for use in a ground handling training video. The photograph below shows the latches clearly unlatched.

The two pilots reviewed the aircraft technical log together. Then the commander began preparing the flight deck while the first officer did the external walk-around check. The first officer did not notice anything wrong during the walk-around, despite the fact that all four latches were unlatched.

The pushback tug arrived 15 minutes before the aircraft was due to depart. The tug driver walked around the aircraft in an anti-clockwise direction as a pre-departure inspection. He did not notice anything wrong.

At 06:57, the aircraft was pushed back from the stand with both fan cowl doors unlatched.

At 07:16, G-EUOE was cleared for take-off on Heathrow Runway 27 left. The first officer was Pilot Flying with the commander as Pilot Monitoring. The left wing tanks held 3,687 kg of fuel and the right wing tanks held 3,701 kg.

During the take-off roll, passengers pressed cabin call buttons and shouted at the cabin crew. They saw in the inboard fan cowl doors flapping and one woman believed she saw smoke.

As G-EUOE took off, the fan cowl doors opened and then broke away completely.

From the accident report:

Aerodynamic loads imparted to the engine pylons by the departing fan cowl doors had caused a severe buckle in the primary structure of the right pylon and damage to the forward fairing of the left pylon. The detached fan cowl doors had struck and damaged the inboard leading edge slats, the fuselage skin close to the overwing emergency exits, the overwing fairings, the right inboard flap and the left belly fairing. The impact to the right overwing emergency exit had dislodged the cabin trim cover on its interior handle, causing a door r emer exit ECAM message to be generated. In addition, the right engine’s outboard fan cowl had struck the right wing leading edge at the outboard end of Slat 3, damaging this slat and the inboard end of Slat 4. The outboard flap track fairing on the right wing was punctured and the left horizontal stabiliser leading edge and lower skin were damaged.

The aircraft lining up on the runway after G-EUOE reported that there was significant debris on the runway.

Normally, cabin crew will not contact the flight deck from the point when engine power is applied for take-off until the aircraft is clearly airborne and the landing gear has been retracted. However, the senior cabin crew member was alarmed enough by the passengers reaction that she unstrapped herself from her seat to stand and call the cockpit.

The aircraft was still less than a thousand feet above the ground and the captain needed his full attention on monitoring the aircraft; he decided that he could call back once they were in the climb. The cabin crew member held on for 17 seconds and then gave up. She did not have any concrete information to pass on.

As the aircraft climbed through 1,500 feet, the autothrust disconnected as the engine control system determined a fault with the right engine. The captain checked the Electronic Centralised Aircraft Monitor and tried to make sense of what was happening as the first officer continued the climb at 250 knots to the first cleared altitude of 6,000 feet.

Within a few seconds, a yellow alert warned that the hydraulic system reservoir was low. Hydraulic fluid was leaking out. It was only 90 seconds after take-off when the fuel began to leak as well, coming out of the tanks at a rate of 61 kg per minute.

Just after take-off, a sensor detected that an interior panel had been removed in the right overwing exit, triggering a warning message. Now that they were in the climb, the Senior Cabin Crew Member went down the cabin to find out what was wrong with the right overwing emergency exit. The passengers near the exit were very agitated and told her that the right engine was damaged and leaking fuel. Here’s a photograph taken by a passenger:

She couldn’t see the engine very well from the aisle but the cowling damage was clear. Someone in the cabin crew (it is not clear who) reassured the passenger that the engine was not leaking fuel. The senior cabin crew member rushed back to speak to the flight crew.

As they levelled out at 6,000 feet, the first officer saw damage on the right wing’s leading edge and thought that something had struck the wing.

At 07:20:20, the commander declared a PAN (an urgent situation) to air traffic control, reporting that the aircraft had an engine problem and had lost a hydraulic system. He requested radar vectors for a return to Heathrow.

At 07:22:53, six minutes after her initial attempt to call the cockpit, the Senior Cabin Crew Member rang the flight deck again to tell them that the top of the right engine was “blown open”. The captain replied that they were returning to Heathrow and asked the cabin crew member to make an announcement to the passengers. He would make a further announcement once he had completed his checks.

If the captain had been made aware that fluid was visibly leaking from the engine, they may have identified the fuel leak more quickly. However, although passengers had reported it, the Senior Cabin Crew Member had not seen any fluid herself and so she did not pass this information on.

The first officer followed vectors from Air Traffic Control and reduced the speed to 220 knots. The commander reviewed the information from the Electronic Centralised Aircraft Monitor again.

A fuel imbalance displayed. It had been eleven minutes since the leak had started. “Right, we’ve got a fuel issue here,” said the first officer. “We might have a fuel leak, don’t you think?” They had lost 1,500 kilograms of fuel so far.

The next radio call is in my top three statements I never want to hear from ATC:

“You’ve left multiple engine parts and there was smoke as you left the runway at Heathrow.”

The captain made his announcement to the passengers. He told them that an engine cover had come off and struck one of the doors and he reassured them that both engines were operating normally. He then asked the Senior Cabin Crew Member to come to the flight deck for an emergency briefing.

He told her and another cabin crew member that the aircraft would be landing in about six minutes and to prepare the cabin for a normal landing.

As soon as the cabin crew left the flight deck, the first officer told the captain that he’d calculated that they had lost 100 kg of fuel in two minutes and twenty seconds.

They were 26 nautical miles from touchdown, with a total fuel remaining of 4,200 kg. There was only 1,000 kg left feeding the right engine.

They informed ATC that they may need to shut down the right hand engine and the commander reviewed the fuel imbalance checklist. The first officer re-calculated the fuel leak and realised they were at risk of losing the right-hand engine to fuel exhaustion before they landed. He told the commander that they should shut down the right-hand engine but the commander felt the risk was too great that the left engine might fail. This was a reasonable decision but the first officer remained concerned about the fuel leak.

If the leaking fuel had not ignited, the return to Heathrow could have been relatively straightforward although at the rate of fuel loss, they would have lost the right engine regardless.

The aircraft was cleared to land. When they were 9.5 nautical miles from the runway, the commander recalls that there was a loud bang and the aircraft seemed to swing unexpectedly. The commander said “Right…we’re gonna shut that engine down, shut that engine down, eh. I think its that engine that’s gone”. A moment later, the right engine fire warning activated. He shut down the right engine.

From the accident report:

The main fire initiated 23 minutes after aircraft rotation, as indicated by the engine fire warning at 07:39:27 hrs. However, passenger photographs taken prior to this show sooting already present in the region of the fan case adjacent to the outlet of the ventilation scoop on the nose cowl. It is likely that the hydraulic fluid which leaked in this area immediately after rotation created the correct hydraulic fluid vapour/air mixture to allow ignition by the arcing wiring early in the flight. At this stage of the flight the speed of the airflow over the engine, and the manner in which the leaking fuel was entrained in the airflow over the wing, may not have been favourable for ignition of the fuel. During the approach to land, the aircraft’s airspeed reduced and the leading edge slats were deployed.

These factors may have changed the fuel/air mixture in the region of the fuel leak sufficiently to create the correct ratio to allow ignition, resulting in the large fuel fire which triggered the fire warning. It is possible the source of ignition for the main fuel fire was the burning residue of the hydraulic fluid, or a very limited amount of fuel that had continued to burn within the confined area created behind the large section of fan cowl, but insufficient evidence was available to confirm this.

Inside the cabin, the lighting went orange from the flames and the temperature began to rise.

The greatest risk to the aircraft was now the fire. The commander discharged the first fire extinguisher bottle. The auto-pilot disconnected and the first officer flew the approach manually. The commander declared an emergency to air traffic control and released the second fire extinguisher bottle. At this stage, there was 333 kg of fuel left in the right wing; even without the fire, the fuel supply for the right engine would have run out before they landed.

The right engine warning light continued, which meant that the fire was still burning. The commander took control of the thrust levers, although he was not Pilot Flying, and told the first officer to concentrate on flying the approach. He then took control of the aircraft at just under 500 feet above ground level, 1.4 nautical miles from the touchdown.

The aircraft touched down at 07:43:51. The first officer deployed the reverse thruster and the brake pedals were depressed. One tyre burst as the flight crew got the aircraft under control. The commander then turned the aircraft to the right as it came to a halt, so that the right engine was on the downwind side of the fuselage.

There’s more in the report, to do with the details of the approach, the aircraft braking and the chain of command after the landing. I’m running out of room and the main thing here is that everyone was on the ground and safe.

The aircraft was evacuated without serious injury, although one passenger insisted on taking a large carry-on bag, only to drop it on entering the slide. The bag fell to the ground in the midst of the Airport Fire and Rescue Service personnel but no one was injured.

There’s a lot of human error in this incident but as always, there’s more to it than that.

Afterwards, the first officer described his walk-around and the standard method of checking the fan cowl door latches. He would lean over and look; however in order to get a clear view of the latches, he needed to be on his hands and knees. The Flight Crew Operating Manual procedure for the walk-around specified checking the latches but the walk-around was done from memory and no one had ever tested the first officers walk-around procedure.

The tug driver conducted a limited pre-flight inspection but there was no specific details and the investigators determined that he could not be realistically expected to identify the unlatched fan cowl doors.

In addition, the high visibility paint on the door latch handles was in poor condition; most of the paint was either missing or obscured by blue paint overspray.

Then there were the two technicians were working overtime that night and that this was a regular occurrence. British Airways made a number of changes at London Heathrow as a result of this accident, including recruitment of 26 additional staff to the maintenance team to deal with understaffing.

The end result:

Causal factors
The investigation identified the following causal factors:
1. The techicians responsible for servicing the aircraft’s IDGs did not comply with the applicable AMM procedures, with the result that the fan cowl doors were left in an unlatched and unsafe condition following overnight maintenance.
2. The pre-departure walk-around inspections by both the pushback tug driver and the co-pilot did not identify that the fan cowl doors on both engines were unlatched.

Contributory factors
The investigation identified the following contributory factors:
1. The design of the fan cowl door latching system, in which the latches are positioned at the bottom of the engine nacelle in close proximity to the ground, increased the probability that unfastened latches would not be seen during the pre-departure inspections.
2. The lack of the majority of the high-visibility paint finish on the latch handles reduced the conspicuity of the unfastened latches.
3. The decision by the technicians to engage the latch handle hooks prevented the latch handles from hanging down beneath the fan cowl doors as intended, further reducing the conspicuity of the unfastened latches

You can download the full report as a PDF here.

There are 38 instances of fan cowl doors becoming detached on the Airbus A320-family. In every instance, the doors were opened before the flight and then not correctly re-secured. In 69% of the cases, the cowl doors were opened to check or service the IDG with oil.

Human factors caused this accident but adherence to procedures and proper training would do a lot to keep it happening again.