The failure of Delta Air Lines’ worldwide computer network this week spotlights the vulnerability of the information systems sustaining the biggest U.S. carriers, each of which has contended with major disruptions during the last year.
Complex networks cobbled together over the decades need major overhauls requiring significant new investments, said Bob Edwards, a former chief information officer for United Continental Holdings Inc. Recent flaws in computer systems quickly escalated into corporate black eyes that exacted costs in both money and reputation.
“I don’t believe the flight ops, maintenance, passenger service systems, crew and dispatch applications are engineered with the level of redundancy needed,” Edwards, who retired in 2014 under pressure after several service disruptions at United, said by telephone. More disruptions are a near certainty: “Mistakes will happen, devices will malfunction.”
The Delta debacle marks a wake-up call for an airline industry in which outdated information systems can strand thousands of passengers. The Atlanta-based airline, which had been leading major carriers in reliability, is far from alone in stumbling. Southwest Airlines said a computer failure July 20 would cost it “tens of millions” of dollars after more than 2,300 flights were canceled.
Delta scrapped about 800 flights on Tuesday, after about 1,000 cancellations Monday.
Delta spent “hundreds of millions of dollars” on technology upgrades and backup systems in the last three years to avoid such an outcome, Chief Executive Officer Ed Bastian said in a video message to customers Tuesday.
“I’m sorry that it happened,” he said. “This isn’t who we are.”
Chief Operating Officer Gil West said equipment controlling the flow of electricity at Delta's base in Atlanta malfunctioned early Monday morning, “causing a surge to the transformer and a loss of power.” Though electricity was restored quickly, “critical systems and network equipment didn't switch over to backups. Other systems did. And now we’re seeing instability in these systems.”
The cost of lost revenue, accommodating passengers on other flights and other issues may cut the airline's third-quarter earnings by as much as 10 percent, Dan McKenzie, an analyst at Buckingham Research Group, said in a note Tuesday. Unlike a factory hit by a disruption or strike, airlines already running near capacity have limited ways to make up lost revenue.
Delta isn’t the only airline struggling with outdated technology, said Mark Jaggers, an analyst at technology research firm Gartner Inc.
“A lot of airlines have been struggling with legacy systems that they are not able to shut down — decommission to move through their life cycle — because they have a 24/7 operation,” Jaggers said. “As they've grown in importance and stature with more flights and customers, taking time to do maintenance becomes a bigger issue.”
The starting point for preventing computer failures is to ensure reliable electricity sources, said Ron Peri, CEO of Radixx International, which provides passenger-service systems to airlines including FlyDubai and Air India Express. Radixx’s data center has backup power provided by three jet engines, Peri said.
The challenge of keeping systems upgraded is particularly steep because they operate around the clock and jetliners are in the air almost constantly these days.
“You've got these big systems and maintaining them is kind of like maintaining an aircraft in flight because they can’t go down,” Peri said. “The core design comes from an era when the presumption was the systems would go down every night.”
A connectivity flaw at American Airlines halted flights at its Chicago, Dallas and Miami hubs in September. A United Continental computer fault last summer lasted two hours and disrupted travel for thousands of fliers. It began with a router malfunctioning and prevented the carrier from ticketing passengers and dispatching crews.
The cost of having duplicate software and hardware at different locations is minor when compared to the expense of having a system down for several hours, not to mention the damage to the airline’s reputation, said Ahmed Abdelghany, a professor of operations management at Embry-Riddle Aeronautical University.
“We shouldn't talk about the cost of making your system reliable because you live on that system,” Abdelghany said. “It’s like an operating room at a hospital: You can’t say I don’t have power or I don’t have a backup for the system.”