When hackers struck Collierville, Tenn. with a ransomware attack in 2019, the city’s IT staff worked around the clock to recover.

Vital services for the small city of 50,000 were back online within a few days. But, behind the scenes, the full recovery was far more complicated. It included rebuilding some digital systems from scratch and rigorously restoring others from backups. It took the city roughly a year and more than $100,000 to get all of its technology back where it was before the attack, the city’s IT project manager Don Petrowski told me.

“People were very patient, but it was an all-hands-on-deck situation,” Petrowski said. “We worked until we got it done.” 

Stories similar to Collierville’s have played out in more than 400 cities and counties across the United States in recent years.

As I reported this weekend, the scourge of ransomware attacks – in which hackers lock up computer systems and demand a payment to unlock them – has impeded emergency responders, stalled tax payments and forced government offices back to pen-and-paper operations for weeks on end.

There are plenty of local examples, as our colleague Karina Elwood recently reported.

“In April, D.C.’s police suffered an attack, with a group posting purported department data after making demands for money," she wrote. "In the fall, Baltimore County Public Schools and Fairfax County Public Schools faced similar attacks, causing online classes in Baltimore County to stop for a brief time. And the Hampton Roads Sanitation District and Bristol Police Department in Virginia became victims last fall and winter.”

The recovery costs have run to millions of dollars for many cities. 

Cities that are unable to recover on their own have been forced to pay hundreds of thousands of dollars to cybercriminals to unlock their computers. The FBI discourages such payments, but officials acknowledge they may be necessary in some cases. 

Public attention has focused mostly on ransomware attacks against critical infrastructure, such as an attack that hit Colonial Pipeline in May and hampered gas supplies to the southeastern United States. But attacks on cities have been among the most damaging and difficult to recover from. 

That’s partly because cities’ information technology has often been underfunded for years or decades, constantly losing out to seemingly more immediate priorities such as policing and social services. Cities also struggle to retain top-shelf IT staff who can attract far higher salaries in the private sector.

“The money just isn’t there and even if the money is there, the people aren’t,” Allan Liska, director of threat intelligence at the cybersecurity firm Recorded Future, told me.

The increase in ransomware attacks is driven by the rise of cryptocurrency. 

That has made ransoms far easier to pay and tougher to track.

The past few years have also seen a rise in ransomware-for-hire gangs based mostly in Russia that have made it far easier for other cybercriminals to conduct ransomware attacks with only minimal skills.

“That’s attracted a lot of cybercriminals that want to make money. Ransomware-as-a-service has been a force multiplier,” Liska said.

The coronavirus pandemic has also supersized the problem. 

The reliance on remote working has made it tougher for cities to protect against ransomware attacks. When attacks do hit, city IT staff are faced with the double problem of getting city services functioning again while also dealing with a workforce that’s often still mostly working remotely. 

When New Orleans was hit with a ransomware attack in December 2019, the IT staff worked seven days a week through February 2020 to ensure police communications and other city services were sufficiently restored to maintain public safety during Mardi Gras, Chief Information Officer Kim LaGrue told me. 

They had planned to slow the pace after that. But when the coronavirus struck in force days later, the seven-day weeks returned as IT staff struggled to manage a string of covid-related crises using technology that was still hobbled.

“We’d established a cadence with the cyberattack that allowed us to roll into the pandemic cadence so we could deliver what the city needed at the time,” LaGrue said.

It would take roughly one year and more than $5 million before New Orleans was fully recovered from the attack and confident the city wasn’t vulnerable to reinfection.

In other cases, IT staff must return to city buildings to manage the recovery from a ransomware attack, despite the pandemic.

That happened when a ransomware attack hit Tulsa in June. 

One piece of luck is that the attack struck in a narrow window when many city staff had already received coronavirus vaccines but the more-contagious delta variant hadn’t yet spread widely in the United States. 

“Everyone came from basically working from home and being isolated to all of a sudden being in a building and working together,” Chief Information Officer Michael Dellinger told me. “We tried to rotate people, make sure they weren’t working too many hours so they didn’t burn themselves out. You can push yourself too hard, mentally and physically, in an emergency like this.”

The keys

Apple delayed its controversial plan to scan devices for child pornography.

The delay comes after experts and activists warned that the technology could give hackers and government spies back doors into Apple devices, Reed Albergotti reports

Apple unveiled the system in August. It is designed to match image fingerprints known as “hashes” on people’s phones with known instances of child pornography. If enough such images are identified being uploaded to iCloud, Apple planned to alert the National Center for Missing and Exploited Children. 

A top E.U. official offered cautious praise for Apple

E.U. Home Commissioner Ylva Johansson applauded Apple for trying to strike a balance between combating the digital spread of child pornography while protecting privacy, in an interview with our colleague Ellen Nakashima. Johansson also told Ellen, however, that she welcomed a spirited public debate about whether Apple had struck the right balance. 

“I thinktoo often this is a debate that either we protect privacy or we protect children. And I can't accept that black-and-white picture because we need to protect both privacy and children,” Johansson said. “I think Apple [is] trying to make at least this balance. Whether they [found] the right balance or not, I can't answer that.” 

The interview took place before Apple announced the delay. Johansson discussed encryption with Attorney General Merrick Garland in Washington last week, according to her office.

Labor Day weekend didn’t see any economy-rocking cyberattacks, despite warnings from U.S. officials.

The FBI and CISA last week warned organizations to be on alert for major ransomware attacks. They noted that hackers have committed increasingly consequential cyberattacks on holiday weekends, but said they didn’t have specific intelligence about Labor Day weekend. This year alone, Colonial Pipeline, JBS and Kaseya suffered ransomware attacks that occurred just before holiday weekends.

Deputy national security adviser ​​Anne Neuberger also warned at a White House briefing last week about the potential hacks, specifically calling out companies that run critical infrastructure such as electric grids and pipelines.

The holiday weekend wasn’t bereft of cyber drama. U.S. Cyber Command urgently warned organizations to update corporate software offered by Australian company Atlassian, noting that “mass exploitation” of the vulnerable software “is ongoing and expected to accelerate.” 

Germany accused Russia of trying to hack its politicians and demanded a stop before elections. 

German officials told Russia to stop the “illegal cyber activities” immediately, Loveday Morris reports. The warning came after a Moscow-backed group stepped up phishing attacks on the national and local parliaments in the country, Foreign Ministry spokeswoman Andrea Sasse said.

There are “reliable findings” that hacking attempts against German politicians can be attributed to “cyber-actors of the Russian state,” Sasse said. Germany’s spy chief in July warned that the hackers had been launching “intensive attacks” since February.

Germany is preparing for a high-stakes parliamentary election set to take place Sept. 26. Berlin has long accused Moscow of targeting its ministries and parliament in cyberattacks.

Chat room

Here's a great Twitter query from former New York Times information security director Runa Sandvik: What's your favorite cybersecurity mystery?

Proofpoint’s Joshua Miller:

Amazon Web Services's David Oxley:

Hill happenings

National security watch

Global cyberspace

Encryption wars

Daybook

  • The Massachusetts legislature will hold a hearing on cyber policy on Wednesday at 1 p.m. 
  • National Cyber Director Chris Inglis discusses the United States’ cyber defenses at an event hosted by the Reagan Institute on Thursday at 10:30 a.m.
  • Rep. Lauren Underwood (D-Ill.), who previously chaired the House Homeland Security Committee’s cybersecurity subcommittee, discusses cybersecurity diversity and inclusion at an event hosted by the Aspen Institute on Thursday at noon.
  • Retired Adm. Michael S. Rogers, who led the National Security Agency and U.S. Cyber Command, speaks at a Heritage Foundation event on Thursday at noon.
  • Director of National Intelligence ​​Avril Haines; Gen. Paul Nakasone, who leads the NSA and U.S. Cyber Command; National Cyber Director Chris Inglis and others speak at the two-day Intelligence and National Security Summit, which begins Sept. 13.
  • Chris Krebs, the former Cybersecurity and Infrastructure Security Agency, keynotes the Insider Risk Summit on Sept. 14.

Secure log off