New software intended to speed jetliners to their destinations is to blame for the debacle Saturday that tied East Coast air travel in knots, causing nearly 1,000 flight cancellations and delays, the Federal Aviation Administration said Tuesday.
The software caused a memory overload that shut down the en-route tracking system at the Washington-area air traffic control center, a facility that controls 160,000 square miles of airspace, federal officials said.
When that center failed, it caused mass confusion for airlines with flights in and out of the region’s three major airports and for north-south flights that were routed to pass through the Washington center airspace at cruising altitude.
The confusion multiplied throughout the day as the FAA tried to troubleshoot the problem and could not tell airlines whether it would be resolved. That uncertainty, passed on to mobs of anxious passengers at airports, amplified their frustration. Some Saturday travelers, who expected an uneventful trip on a day when the system normally operates smoothly, did not arrive at their destinations until Sunday or Monday.
If there was a silver lining for the beleaguered FAA, it was that Congress is away on August recess. On a typical Saturday, at least a few House and Senate members who missed Friday flights to their home states would pass through Reagan National Airport.
Congress this year is pondering a momentous overhaul of the FAA that would remove the entire air traffic control system from its hands and create a nonprofit, quasi-government agency to manage the system and its more than 14,000 air-traffic controllers.
If the split is approved, the FAA also would lose its leadership role in creating the new $40 billion air traffic control system known as NextGen. Getting NextGen on a faster track motivated Congress to propose the split.
The computer software that crashed Saturday is a centerpiece of the NextGen system, known as En Route Automation Modernization (ERAM). The problem developed after a recent software upgrade by contractor Lockheed Martin, the FAA said.
“I can understand why they would be very worried about ERAM as the source of the problem, given the tremendous scrutiny on them right now in Congress on delivering NextGen,” said Joshua Schank, president of the Eno Center for Transportation. “This probably could have happened to anybody. On the other hand, if they’re trying to win the PR war in saying, ‘Look, we’re making great progress on NextGen, see how well we’re doing,’ this puts a big wet blanket on that argument.”
The computer system at the control center in Leesburg, Va., began to break down about 11 a.m. Saturday. By the time it was restored five hours later, air travel along the East Coast was in turmoil, and the problem had rippled across the country, as flights to and from the Washington area were delayed or canceled.
The region’s three major airports — National, Baltimore-Washington International Marshall and Dulles International — all sharply curtailed flights but did not shut down entirely.
The FAA’s air control system is designed like a layer cake, with centers such as Leesburg taking over when planes reach cruising altitude of about 20,000 feet. The layer below that is handled by Terminal Radar Approach Control Facilities, or TRACON. The bottom layer — takeoffs and final landing approaches — is the business of airport control towers.
To mitigate Saturday’s mess, the FAA worked with airlines to allow planes to fly below 20,000 feet under the direction of TRACON controllers, who handed off planes to TRACON controllers in adjoining sectors rather than to controllers at the Washington center. The normal handoff as planes reached cruising altitude would have been to the Washington center.
The recent software upgrade allowed individual controllers to use a customized window on their computer screens for frequently referenced data. That information was supposed to be removed from the system when controllers deleted it. But when controllers adjusted their settings, some of the information remained in the system until the maximum storage limit was reached.
When that storage maximum was hit, it sapped processing power from the center’s overall system. After Saturday’s problems, the FAA suspended use of the customized window function and is working with Lockheed to determine why the glitch wasn’t identified in testing and to find a permanent solution, federal officials said.