Holdfast Training Services Trial Blog

For those who may have missed the news on Fri 26 May 2017, (note civvi date structure) there was a significant failure of the baggage system at Gatwick Airport which made national news. I will also share with you how BA managed to significantly divert media attention away from the baggage failure by their ability to seriously curtail the duration of their customer’s holidays.

Well, let me start by saying that the baggage system failure was not my fault despite the early conclusions drawn by my peers. The aim of this Blog is, to set the record straight, to regain control of my reputation as a polished E&M engineer, to give you an insight into how an airport baggage system functions and what went wrong on that fateful day; known as Bag Friday.

OVERVIEW OF THE BAGGAGE HANDLING SYSTEM (SOUTH TERMINAL)

To be able to describe the baggage system failure it is appropriate to give you the reader an overview of the mechanics of the Baggage System employed at Gatwick.

Meet Martha in Figure 1, she is a shy Check-In technician who takes control of your baggage at the Bag Drop. The bag is weighed and a bar-code is attached to the bag before it starts its very own journey through a myriad of conveyors, scanners, sorters and physical abuse sustained from the baggage handlers. The bar-code contains a list of descriptors which identifies you to the bag, flight details, destination, weight (of the bag), flight times and so on.

Figure 1 . Check in desk and Martha.

Your personal details are derived from your passport and tickets which are communicated to various agencies both internal and external to the airport.  Internally, an individual’s personal data is forwarded to the airline which will checks you onto their flight manifest and central database for use of tracking and marketing purposes. Externally, Customs and Excise are alerted to you checking out of the country; their database is linked to the Home Office and shared with annexing databases owned by UK security services (this is applicable to both manual and on-line check in). In short, if an individual is on a Home Office ‘Watch List’ then basically he/she will be interviewed at the airport as they pass through security.

Assuming you are not a terrorist, your baggage is permitted to enter the baggage system.  The bag is currently at Level 1 (blue insecure) and the Baggage System (governed by an Oracle database) is expecting the bag to arrive at the first scan point to perform a ‘handshake’ with the baggage.  The scan is a function performed by an Automated Tag Reader (ATR) – the four blue boxes elevated above the conveyor shown at Figure 2.  No matter where the tag is positioned on the bag and however the bag is oriented on the belt, the ATR can locate, read, and decode the bar-code achieving up to 99.8% read rates even for damaged or poorly printed codes.  If the tag is unreadable, an operator removes the bag from the conveyor and then manually codes the bag into the Oracle Database and reintroduces it to the conveyor.

Figure 2.  4No Automated Tag Readers elevated above the inbound conveyor.

The ATR also scans the orientation and size of the bag, from this point forward the bag will be physically rejected from the baggage system if it has moved over a tolerance of 20mm on the assumption it has been compromised.  For efficiency, conveyors default to idle when not in use and also the speed of conveyance is governed by Programmable Logic Controllers (PLCs).  Each of the conveyors have a dedicated PLC gate which notifies the next PLC and so on to guide the bag through the conveyor system.  The PLCs also feed data to the central Oracle database which tracks the bag’s progression through the baggage system.

The next step is to scan the baggage for illicit material.  The Level 1/2 scanners perform an automated check in under 3 seconds and if satisfied with the contents, the bag is permitted to progress onto the next stage. The level 1/2 scanner is shown below in Figure 3; note the Level 1 blue conveyor presents the bag to the scanner and emerges on to the red (scanned) conveyor on the far side.

Figure 3. Baggage Scanner detects illicit material within the baggage.

The bag is passed to a vertical sorter (a decision point) as shown in Figure 4. The bag is either permitted to progress onto the next stage or a PLC command will force a transfer to Level 3.   At Level 3, an operator has 15 seconds to check the bag on their screen and if the operator is unhappy with the contents, the passenger is summoned to open the bag for a Level 4 manual search.

Figure 4.  The vertical sorter allocates the bag to either Level 2 or 3 conveyors

Assuming the bag has successfully cleared level 2, the bag is allocated to a tip tray conveyor as shown in Figure 5.  PLC logic controls the speed of a delivery conveyor to ensure the bag’s approach is synchronised with the tip tray. The chutes also shown in Figure 5 are dedicated to particular aircraft, when the bag arrives at the appropriate chute it drops down to a Make Up Position where a baggage handler loads the dolly (bag truck).

Figure 5. To the left is an image of the Tip Trays and Chutes. To the right is an image of the Make Up Positions

SO WHAT WENT WRONG ON BAG FRIDAY?

To understand the error you need to understand how a baggage Oracle database works. The central Oracle database manages the automation of the baggage system which is fed data from a vast array of PLCs distributed across the baggage network.  The database stores the data into tables which dynamically increase/decrease in size in relation to the amount of data accrued by the system.  The data tables are called ‘Stacks’ and have a limited capacity before they start to overflow and discard data.

A bespoke programme queries the Oracle database and presents the baggage system performance characteristics to the Building Management System (BMS).  BMS is a Graphical User Interface which can manually or autonomously  manipulate mechanical systems around the airport, such as the baggage system and also the air conditioning.

The communication path between the Oracle database and PLCs contributed to Bag Friday.  The PLCs fed data to the Oracle Database, however due to latency issues in the communications network, the database was receiving data spikes causing the Stacks to increase in size beyond their maximum capacity. The Oracle Database became overwhelmed and gave a stop command to the PLCs which in turn closed down the baggage system.

As a consequence, within 7 mins of failure the Check-In desks were closed and puts the airport on alert status ‘Bronze’.  At 45 mins the Arrivals and Departures fills up with passengers which heightened the alert status to ‘Silver’, at 90 mins cars on the M23 and M25 were backed up and Gatwick hit the headlines. The baggage system was down for just under 3 hours resulting in a total of circa 3000 bags remaining in the terminal, later repatriated with their owners at a cost of approx. £150 per bag.

WHAT HAPPENED TO BRITISH AIRWAYS?

BA tweeted to the media that they have a data HUB near Heathrow that lost power; one would suggest they should have factored redundancy into their power supply if indeed this was the case.  According to a senior BA rep at Gatwick, the actual reason for the their system to collapse is because they sold their operating system to India to reduce costs! BA operate on a Virtual Local Area Network (VLAN) which allows their operating system to be placed anywhere in the world. Situating an entire Operating System on a VLAN is a sound concept and common place for many multinational companies, however in the case of the BA system failure, the link from OS in India to the satellite ‘dropped out’ to the Master/Slave servers.  The failure caused their entire management system to collapse, grounding all aircraft, ceasing all bookings, aircraft tracking and configuration of flight manifests. The entire fleet had to re-set to a start state two days out from the time the failure occurred.

In summary, the issue with BA overshadowed the Baggage failure at Gatwick, therefore we owe BA a great debt of gratitude.

Having received a detailed handover from Stu Douglas, I am now in the chair as an M&E Package Manager and currently involved in a number of interesting projects.  The Department for Transport has imposed a directive on all major UK airports which requires Gatwick to install new baggage scanners to ensure they conform to the Explosive Detection System 3 Standard by 1 Sept 2018. The machines are larger in all regards, including size, weight and heat rejection, therefore there are multiple projects running simultaneously in order to install these machines; I will introduce you to these projects over a number of blogs in due course.

The Gatwick Airport estate is vast and is distributed across two terminals, North and South. The South Terminal opened in 1958 and has a footprint of 160,000 m² and the North terminal opened in 1988 with a footprint of 98,000 m², with a supporting workforce of 24,000 employees spanning 252 companies.  GAL employs 2,800 staff directly which is sufficient to manage a throughput of 43.1m passengers in 2016. GAL is projecting that passenger numbers will increase to 52m by 2032 and the infrastructure must also expand to accommodate such a vast increase.

The airport boasts a colourful past, serving as an aerodrome in the 1930s and later requisitioned for military use for the RAF in 1940.  The estate was renovated in 1956 to become a commercial airport paving the way for Gatwick to become the UK’s second largest airport. Figure 1 shows Gatwick in general with the South Terminal at the bottom of the image.

Figure 1 – Ariel view of Gatwick Airport.

Gatwick is a heavily engineered and congested environment which makes the simplest construction project or electrical/mechanical installation very complex indeed.  That said, I have come across a small construction project which may be of interest to the Civils.  The principle contractor is MARCO (to be clear, this is not a person but the company name) whose brief was to Design and Build office space and a welfare facility for the Gatwick Handling Agents (GHA) to operate from in the South Terminal.

The project progressed well until the contractor had to excavate a 50m x 1m channel through a concrete base of 600mm to allow for soil pipes to transit waste from the toilets to a new foul drain.  Figure 2 is a capture of the design drawing which shows the location of the toilets, pipe runs (brown) to the foul drain and I Beam columns (in the red circles).

Figure 2 – A capture of the GHA design drawing showing the area of interest.

The foul pipe from B to C has a cross fall of 1:80 and both the grey and black water is gravity fed to the foul drain at C.

Unfortunately the channel is in a confined space with a low ceiling and restricted access, therefore the contractor is unable to use diesel powered excavators and dumpers to remove the rubble. Consequently they employed the use of a portable control BROKK hydraulic breaker and then handball the rubble from site. The BROKK and channel is shown in Figure 3.

Figure 3 – BROKK and Channel

The BROKK is also shown in Figure 4 below, it requires an 15kW/32A 3 phase supply from a generator which can be remoted away from the confined space. There are larger machines on the market that can be hired, this is at the bottom end of the scale.  The machine will produce a noise level well in excess of the HSE standard of 85 dB(A) therefore ear protection is required. The machine also produces a downward pressure on the tip at 25MPa.

Figure 4 – BROKK 100

THE PROBLEM

The vibration energy applied through the tip to the ground is transmitted through the concrete to the vertical I Beam columns circlied in red at Figure 2.  The vertical columns connect to horizontal beams supporting the Departures Lounge and a number of retail outlets.  The vibrations were having an impact on the outlets above and the Project Manager was summoned to Harrods to explain why their jewellery was dancing around the shop;  as a result the excavation was halted with immediate effect.  The noise level was measure by the on-site EHS representative at 76dB, to put this in perspective, employers must provide PPE to those exposed to noise above 85dB.  Another 9dB increase then the customers in Harrods would be issued with PPE!

THE SOLUTION

At first, the PM suggested that disruptive maintenance is conducted in silent hours which is between 2300hrs and 0400hrs, however  if this course of action was adopted, the contractor would fall behind considerably.  The contract is an NEC3 Option E (Cost Reimbursable) which translates into delay costs which are imposed onto the client (Gatwick).

The second option was to cut the concrete channel beyond 600mm depth using an orbital cutter down to the compacted earth beneath, doing so detached the channel from the rest of the concrete pad. This simple act prevented the majority of the vibration energy being transmitted through the pad to the vertical columns. The BROKK was re-introduced and the noise assessment was conducted again during silent hours and a notable reduction was observed from 76dB down to between 52-54dB.

Both contractor, client and Harrods are happy once more!