Russian Hackers Have Been Inside Austin City Network for Months

State-sponsored hackers believed to be from Russia have breached the city network of Austin, Texas, The Intercept has learned. The breach, which appears to date from at least mid-October, adds to the stunning array of intrusions attributed to Russia over the past few months.

The list of reported victims includes the departments of Commerce, Homeland Security, State, and the Treasury; the Pentagon; cybersecurity firm FireEye; IT software company SolarWinds; and assorted airports and local government networks across the United States, among others. The breach in Austin is another apparent victory for Russia’s hackers. By compromising the network of America’s 11th-most populous city, they could theoretically access sensitive information on policing, city governance, and elections, and, with additional effort, burrow inside water, energy, and airport networks. The hacking outfit believed to be behind the Austin breach, Berserk Bear, also appears to have used Austin’s network as infrastructure to stage additional attacks.

While the attacks on SolarWinds, FireEye, and U.S. government agencies have been linked to a second Russian group — APT29, also known as Cozy Bear — the Austin breach represents another battlefront in a high-stakes cyber standoff between the United States and Russia. Both Berserk Bear and Cozy Bear are known for quietly lurking in networks, often for months, while they spy on their targets. Berserk Bear — which is also known as Energetic Bear, Dragonfly, TEMP.Isotope, Crouching Yeti, and BROMINE, among other names — is believed to be responsible for a series of breaches of critical U.S. infrastructure over the past year.

The Austin breach, which has not been previously reported, was revealed in documents prepared by the Microsoft Threat Intelligence Center, or MSTIC, and obtained by The Intercept, as well as in publicly available malware activity compiled by the site VirusTotal. “While we are aware of this hacking group, we cannot provide information about ongoing law enforcement investigations into criminal activity,” a spokesperson for the city of Austin wrote in response to a list of emailed questions.

On Sunday, Reuters reported that a state-sponsored hacking group had breached the Treasury and Commerce departments, sparking an emergency weekend meeting of the National Security Council. The Washington Post later attributed the attacks to Cozy Bear, citing anonymous sources, and reported that the group breached the agencies by infecting a software update to Orion, a popular network management product made by SolarWinds, a firm based in Austin. “Fewer than 18,000” users downloaded the malicious software update, which has been available since March, SolarWinds said in a federal securities filing on Monday. The Intercept has seen no evidence that the Austin breach and the SolarWinds hack are related.

Russia’s dramatic intrusions into U.S. networks come at an awkward moment for Washington. In November, President Donald Trump fired Christopher Krebs, the director of the Cybersecurity and Infrastructure Security Agency, or CISA, for refusing to go along with attempts to overturn the presidential election results, and Trump has generally shied away from criticizing Russian hacking operations.

Berserk Bear is suspected to be a unit of Russia’s Federal Security Service, or FSB. Cozy Bear, the group behind the attacks on federal government agencies, is affiliated with the Russian Foreign Intelligence Service, or SVR. Both the SVR and the FSB are considered successors to the Soviet-era KGB.

CISA and the FBI singled out Berserk Bear in an October 22 advisory warning that the hacking group had targeted airports, energy companies, and state, local, and tribal governments around the country, and had “exfiltrated data from at least two victim servers.” The New York Times later reported that FSB hackers had “bored into local networks” in California and Indiana, without specifying which networks had been breached.

It is now clear that a group of highly sophisticated hackers, likely Berserk Bear, also hit Austin. An IP address belonging to the city appears on a list of indicators of compromise, or technical evidence that organizations can use to determine if they’ve been hacked by this threat actor, compiled by MSTIC. When employees of the city of Austin visit websites from their work computers, this is the IP address that those websites see them coming from.

VirusTotal, a service owned by Google that allows the public to submit files to be analyzed and scanned by dozens of anti-virus programs, has cataloged 97 malware samples that were observed communicating with the Austin IP address, and 88 of them were submitted to the site since January. When malware runs on a computer, often the first thing it does is receive instructions from the hackers who control it. In these cases, the instructions appear to be coming from Austin’s compromised network.

The list of indicators of compromise accompanied an alert about Berserk Bear that MSTIC shared with public sector Microsoft customers in mid-November. That document warned that BROMINE, Microsoft’s name for Berserk Bear, had also targeted the telecommunications, aerospace, and defense sectors, hitting entities in the United Kingdom and Turkey as well as the United States.

Austin’s internet address was the only government IP on the MSTIC list. The other IPs belonged to cloud hosting providers like Amazon, DigitalOcean, Microsoft Azure, and the German company Hetzner, as well as Turkish cellular carrier Turkcell. The next day, MSTIC distributed a copy of the same alert without Austin’s IP address included. It’s possible that Microsoft initially included Austin’s IP address as an indicator of compromise by mistake, but the malware activity from VirusTotal makes that scenario unlikely. And VirusTotal cataloged six malware samples that communicated with this IP and were submitted in November and December, suggesting that Austin’s network remains compromised.

“It’s not surprising that hackers, when they find an unsecured server that is in the country that they’re targeting, use that as a jumping off point for lots of other things,” said John Scott-Railton, a senior researcher at Citizen Lab at the University of Toronto’s Munk School of Global Affairs and Public Policy, in an interview. “It makes things a lot easier.”

The Austin City Council appears to have been aware of the breach since October. CISA and FBI published an initial advisory on October 9 warning of “advanced persistent threat actors,” or APTs, targeting state and local governments, before publishing a follow-up advisory on October 22 in which the agencies attributed the campaign to Berserk Bear. Four days after the initial advisory, on October 13, the City Council went into a closed meeting to discuss “confidential network security information,” according to the posted agenda. The council discussed the topic again two days later during an executive session of its regular meeting, according to that agenda. The portions of the meetings in which the council discussed network security were closed to the public, and the agendas cited an exemption in the state’s rules governing open meetings related to “the vulnerability of a network to criminal activity.” An assistant to Mayor Steve Adler declined to comment, as did three other council members. “Any info council would have received on this would have been in executive session, and as such, any council member would not be able to comment,” a staffer for a fourth council member wrote in an email. The remaining six members did not respond to The Intercept’s questions.

On December 8, according to a transcript of the City Council meeting, the city authorized a $2.4 million contract for cyber liability insurance — a product that typically covers losses from data breaches and hacks.

Berserk Bear’s campaign targeted hundreds of organizations across the United States in addition to the city of Austin. A heat map published by CISA, last updated on November 17, lists types of organizations that were compromised, scanned, or targeted with other reconnaissance activity from Berserk Bear’s hacking infrastructure. It includes 75 airports, four airlines, 13 cities, four counties, three states, and dozens of other targets in aviation, defense, information technology, health care, transportation, and other industries. CISA did not name any of the targets in the heat map.

Berserk Bear’s reputation for lurking fits a pattern that is common for espionage-related attacks, where “adversaries have already been sitting in the network for three months or so before someone realizes that they are there,” said Sami Ruohonen, a researcher with the Finnish cybersecurity firm F-Secure. “This is a technique that is specifically favored by these APT groups, just for the fact that the longer you go unnoticed, the longer you have a foothold in the network.” In a report published last year, F-Secure compared Berserk Bear and similar groups to the IT equivalent of a sleeper agent.

Yet cybersecurity experts warned that while the Berserk Bear hackers are not known for sabotage, they could rear up at any moment and wreak havoc in the United States, for example, by making cities go dark. “We should be cognizant of the level of information that they have,” said Vikram Thakur, a technical director at Symantec who has tracked the group for years. “Turning on valves or closing valves, things of that sort — they have the expertise to do it.” In 2010, Stuxnet, a digital weapon developed by the United States and Israel, temporarily took out as many as one-fifth of Iran’s centrifuges and infected systems around the world; in 2015, the Russian hacking group Sandworm triggered an extensive power blackout in Ukraine.

Like their counterparts in the National Security Agency, the Berserk Bear hackers are highly skilled and often narrowly focused on their targets. In 2013, Symantec found that Berserk Bear hit targets throughout the United States and Europe by using so-called watering hole attacks, which involves identifying websites frequented by members of an organization and infecting the sites with malware. The Russian hackers separately compromised software that is used by a small group of businesses in the energy sector. “This was very specific software for people who work in the industrial space,” says Thakur. “They knew who their targets were.”

Berserk Bear’s breach of Austin’s network may have been extensive. CISA’s October 22 alert said that the hackers exploited a critical vulnerability in Netlogon, Microsoft’s authentication protocol, allowing them access to valid usernames and passwords of all users of the network. In at least one instance, the alert said, they used those credentials to steal information, including documents, related to “sensitive network configurations and passwords” and “printing access badges.” And in at least one case, the hackers used valid credentials to compromise Microsoft Office 365 email accounts. While it is not clear from the CISA alert whether those two instances described Austin, full network access would have allowed the hackers to assume “the privileges of anyone in the network,” said Ruohonen, of F-Secure. “If you think about data that is only available to the CEO, or data that is only available to IT services, they would have all of this data.”

The breaches tracked by CISA and the FBI were so severe that the agencies recommended drastic measures. The only way an organization can be sure that it has removed the threat, they advised in the October 22 alert, is by systematically rebuilding the network from the ground up, noting that it’s “critical to perform a full password reset on all user and computer accounts,” an incredibly daunting and expensive task.

“The City follows the measures that the Cybersecurity and Infrastructure Security Agency (CISA) and the FBI recommend for local governments,” the Austin spokesperson said. But the city has apparently not reset all its accounts. One employee was asked to enable multifactor authentication on internal services, but they have not been asked to change any passwords or reset and reconfigure their computer.

Malware still appears to be communicating with Austin’s network. The most recent malware sample found on VirusTotal that was observed communicating with Austin’s IP address was submitted to the site for analysis on December 15, two days ago.