At 4:15 p.m. on May 27, the lights flickered across Ontario.
Subway cars in Toronto rolled to a stop while safety signals were reset. Pizza oven doors flew open on the ground floor of the city's landmark CN Tower. Cement and steel plants paused while machinery was restarted. Tens of thousands of computers automatically shut down and rebooted.
Hydro One, the Toronto-based electric utility, quickly issued a press statement seeking to reassure the public that the utility's "equipment protection worked as designed to isolate the fault."
In fact, the situation was much more tenuous. The power blip involved an extremely rare, still unexplained failure of two protection systems, according to internal documents of the utility, reports to oversight agencies and eight engineers. The eight are part of a group that has been on strike since June 6. By their accounts, the failure brought the region's power grid to the verge of a blackout like the one that struck on Aug. 14, 2003, plunging 50 million people in the United States and Canada into darkness.
"It was very bizarre, very disconcerting. And we may have a major problem on our hands," said engineer Aaron Cooperberg.
He and other engineers said that because they are striking, the precise cause of the malfunction has not been identified, and they say it could happen again. "Fundamentally, we had a very close call, and we don't know why," said another engineer, one of three who asked not to be identified, saying they feared retaliation by the company.
The electrical grid handled by Hydro One, which has 17,000 miles of high-power electric transmission lines throughout Ontario, managed the blip "very well," said Peter Gregg, vice president for corporate communications at the utility. Much of its cause is still not understood and under investigation, he said, and "we continue to be disappointed with the striking staff who seem to want to scaremonger on this incident."
Striking engineers said they were not seeking to pressure the company as part of their labor dispute. They said they were speaking out because they were worried that a threat remains to a power system they care about. "We're not pointing fingers," said one. "This isn't about a strike. It's about reliability of the system."
The May 27 incident set power generators in New York, Massachusetts, New Brunswick and Ontario swinging in wild oscillation, fighting one another to try to cope, according to official reports filed with oversight authorities in Canada and the United States.
The competition among generators produced what experts call a "ring" on the grid. According to an internal company document submitted to regulators, "the ringing lasted about 12 seconds." Computer simulations showed that "the oscillations can grow with time and result in the breakup of the power system in about 30 seconds," the internal document noted.
The industry oversight agencies have not yet publicly drawn any conclusions from the event. Edward Schwerdt, head of the Northeast Power Coordinating Council, which reviews the system's grids in New York, New England and eastern Canada, said the May 27 event was an "extreme contingency" but that no additional precautions to prevent a recurrence are being taken until an inquiry is complete.
An official of the U.S.-Canada oversight agency, called the North American Electric Reliability Council, also said they are awaiting a report. "We could see a change in the system frequency on the continent. We knew that something big had happened," said Glenn Brown, who heads a disturbance analysis group for the council. He compared the irregularity to one that blacked out all of Jacksonville, Fla., in April 2002.
Two outside experts cited by the strikers confirmed their concerns. "This had the potential of cascading into a big blackout," said Puttaveeraiah Prabhakara, an electrical engineering expert in Toronto who has written standards for electrical grid protection that have been adopted worldwide.
The symptoms "were indications of a weak system," said Claudio Canizares, a power systems specialist at the University of Waterloo in Ontario. "This is a serious concern."
Sixty-six striking engineers sent a petition to Parliament expressing their concerns. Their affiliated union, the International Federation of Professional and Technical Engineers, said it would appeal to lawmakers in Washington on Tuesday to investigate dangers to the electrical supply.
The system was "a mere 20 seconds from reaching the proportion of the August, 2003 blackout," Gregory J. Junemann, the union president, said in a letter to members of Congress.
The problem began after routine maintenance of a huge circuit breaker at switching substation in Milton, Ontario, 30 miles west of Toronto. The company, engineers and regulators agree that when the work was completed, the breaker was inadvertently left switched to a grounded "test" position. When an operator at the Ontario Grid Control Center in Barrie threw the switch to resume flow of electricity, it was diverted into the ground, causing a massive short circuit in a line carrying 500,000 volts.
Immediately, the protection systems -- high-tech, computerized circuit breakers -- stopped power to the line that was erroneously grounded at Milton. That is supposed to happen within 50 milliseconds. Instead, it took about 150 milliseconds -- three times as long -- leaving massive strain on the system, according to David McGinn, 54, a senior protection and control specialist for the utility who has joined the strike.
"We don't design the system for that," said McGinn. "It did survive, but just by good luck."
It was a "significant shock to the system," said Schwerdt of the Northeast Power Coordinating Council.
Then, the protection systems for two other massive power-carrying lines, stretching eight miles to the Trafalgar substation and 20 miles to the Claireville substation, inexplicably tripped, causing both those 500,000-volt superhighways for power transmission to shut down, according to the engineers.
Within milliseconds, generators across New York, Massachusetts and eastern Canada reacted, first accelerating due to the loss of load, then automatically decelerating to compensate. There were four major swings before the system stabilized, according to reports filed with the Northeast Power Coordinating Council.
"We were lucky," said Emeka Okongwu, 61, an operating engineer. Had the weather been hot that day, said Okongwu, who is also on strike, the additional strain on the system would likely have produced a domino effect of failing lines.
This type of cascading from an initial local problem at a power plant in Ohio caused the huge 2003 blackout in just nine seconds, according to a U.S.-Canada Task Force report. That failure stranded millions of office workers and commuters from Pennsylvania to Boston and Toronto. It left Cleveland without water, shut down 22 nuclear plants, caused 60 fires, required 800 elevator rescues in New York City and cost businesses billions of dollars.
The May 27 glitch, Cooperberg said, has left him nervous. "If another line had failed, we would have been in the soup."
"Obviously, that did not happen," said Gregg, of Hydro One. "The system is robust and resilient and managed this fault very well. An investigation certainly will give us and all interested parties a more fulsome answer."