In 2010 a Chinese ISP momentarily hijacked the Internet. Due to a misconfiguration, some traffic that should have gone to Dell, CNN, Starbucks and Apple was sent to China instead. The incident lasted for only a few minutes and the responsible party claimed it was an accident. But it highlights a dangerous security weakness in one of the Internet's fundamental protocols.
Research released this week has revealed two more cases in which misconfigurations re-routed traffic far from their intended destination. For example, in one of the attacks, traffic traveling from Mexico to the United States took a circuitous and illogical route to Belarus.
And this time, the researchers believe that the peculiar routes, which re-routed up traffic from a number of networks all over the world, were not merely a misconfiguration. Rather, they think someone deliberately introduced errors into the Internet's routing system, perhaps with an eye to intercepting traffic as it flowed past a location controlled by the attacker.
Experts say that the Internet's fundamental routing protocol, called the Border Gateway Protocol (BGP), is surprisingly reliant on trust among the administrators of the many networks that comprise the Internet.
The Internet's glue
BGP is "essentially the glue that holds the disparate parts of the Internet together," says Jennifer Rexford, a computer science professor at Princeton University.
While we often think of the Internet as one huge network, it's really a collection of separate networks called autonomous systems that are connected via BGP. And the way those routes find each other is surprisingly simple.
"Each domain (AT&T, Princeton and so on) will tell it's neighbors it connects to directly which destinations it can reach and over what paths," Rexford says. "Those neighbors will then chose amongst those set of paths that are offered by their respective neighbors, add themselves to the front of the path, and tell their neighbors."
The process requires a lot of trust. "By default your neighbor just believes you, and doesn't have a really reliable way to tell if you are actually telling the truth," says Rexford. That reflects the fact that BGP was developed more than two decades ago, at a time when the Internet was a smaller and more trusting place.
"Like a lot of the technologies underlying the Internet, it was designed without security in mind under this sort of implicit assumption that all the guys on the network are good guys, and all the bad guys -- if they exist at all -- are outside the network," says Rexford. "People worried about the network being vulnerable to physical attacks, but the idea of cyberattacks wasn't really in people's thinking at the time."
As a result, BGP has been fraught with some security issues -- primarily in the form of BGP hijacking, which happens when a provider advertises itself as the best route to get to a destination but it isn't. In the most common incidents, that just results in the traffic being dropped because it has nowhere to go.
It's long been theorized that this sort of re-routing could be weaponized as a technique for intercepting traffic. In fact, Anton Kapela and Alex Pilosov demonstrated a technique for eavesdropping on traffic via BGP at DEFCON in 2008. But now Renesys, an Internet monitoring company, says it has seen a series of what they describe as "man-in-the-middle" attacks using BGP targeting "financial institutions, VoIP providers, and world governments" in the wild.
"Internet route hijacking has been around for years, it's really just the emergence of this specific man-in-the-middle variance that has taken off in 2013," Renesys Chief Technology Officer Jim Cowie told me last week. "We can see what appear to be multiple groups engaging in it from multiple countries around the world. It's a worrisome trend."
The two examples published Tuesday show traffic being routed through places that are significantly out of their way before continuing on to their destination. According to Renesys, the first incident began in February 2013. Traffic from "major financial institutions, governments, and network service providers" was re-routed from their typical paths through Belarus before sending it back through to the destinations.
The other example involves an Icelandic provider, Opin Kerfi. According to Renesys, on July 31, it began announcing origination routes for 597 IP networks, despite the fact that it normally only originates three IP networks and has no downstream autonomous system customers. The faulty routes appear to have exclusively come through one of Opin Kerfi's ISPs, Síminn.
That was just one of seventeen events Renesys says they have observed between July 31 and Aug. 19 following the same patterns: "False routes sent to Síminn’s peers in London, leaving ‘clean paths’ to North America to carry the redirected traffic back to its intended destination."
The issue is that when the traffic is rerouted like this, there's an opportunity for bad actors to snoop on traffic -- and that comes with some distinct risks. "You'd hope that a lot of people who are using the Internet these days are using encryption," Cowie said. "You hope that a lot of the big Web sites and content they are accessing would be SSL enabled and using HTTPS.
"But in fact, for various reasons there's still a lot of unencrypted flying around the Internet." he said. "There are probably good reasons to want to access somebody's Internet activity, particularly if you can do it from afar without leaving any fingerprints rather than through a routing table." Adversaries could potentially snoop in on that unencrypted traffic, or even modify it by sending it on to its final destination using the hijacking technique described.
Renesys is clear that its data can't explain the "exact mechanism, motivation, or actors" of these incidents. And Síminn told Renesys that its routing issues were the result of a bug in vendor software. But Síminn declined to give Renesys further supporting details "despite repeated requests" and has not responded to a Washington Post inquiry for similar details.
Renesys seems skeptical about Síminn's explanation, saying if it's a bug, "it’s a dangerous one, capable of simulating an extremely subtle traffic redirection/interception attack that plays out in multiple episodes, with varying targets, over a period of weeks." And Doug Madory at Renesys says that one thing they have observed is what appears to be a "perpetrator sort of honing their technique" In the traffic data, they say they can see the adversary try out one technique that doesn't quite work, but then tweak it repeatedly until they find a method that does re-route the intended traffic.
"It's very targeted, it's been perfected over time, and this is not an accident," Madory says. "It's for sure something malicious going on."
Andree Toonk, founder and lead developer at BGPmon.net, said his data supported the examples listed by Renesys, but he couldn't say if the disruptions were malicious without further information. "Data such as BGP and traceroutes show us what happened, it allows us to determine timestamps, geographical impact, sources," he says, but "it does not tell us if what we see is intended or by accident. This is one of the major flaws with routing on the Internet today."
Rexford says that "most of the outages caused by BGP are simpler" than what Renesys is describing. Most result from simple typos that result in traffic being dropped. But a traffic re-routing with the potential to be a man-in-the-middle attack is technically difficult because you need to tell part of the Internet that you are the correct route, but maintain an actual route to that destination at the same time. "It's extremely hard to imagine how it might happen by accident because you need to talk out of both sides of your mouth at the same time," Rexford says.
Limited shelf life
Other cybersecurity experts I spoke to were also curious about the context of these incidents. "The big question is what's the actual impact," Akamai CSIRT director Michael Smith said. "How quickly were these hijacks detected, how quickly were they mitigated, what volume of traffic actually arrived at the hijackers network versus how much of the routes?"
"As an attacker, you have to be really careful because if you hijack too much traffic you don't have the capacity to handle all of that traffic and it crashes your network," he said. "There are a lots of good movie plot scenarios you can do, but it all depends on the fact that you have capacity and you have the ability to actually handle the traffic you just hijacked."
Renesys believes this kind of attack is a serious threat to Internet security, but may have a very limited shelf life. "This is not a very subtle attack -- you can't carry it out without publishing your false routes all over the planet," said Cowie. "If everyone would take care to watch how their networks are being advertised around the world it would disappear overnight." So beyond the specifics on the incidents revealed by Renesys, one of the major takeaways from its research might be the need for increased scrutiny of the protocols that make the Internet tick.
"It definitely points out a weak link in the Internet where there is an ability for a local action to have global consequences," says Rexford, who has worked on some projects to create better security options for BGP. However, she has found it difficult get buy-in for transitioning global network infrastructure on the scale necessary to fix some of the current vulnerabilities. "That's just fundamentally a problem whenever you need to do things that relate to how the Internet is stitched together -- it can't just be one country or one company that fixes it."
Correction: This story originally stated that Renesys had asked Opin Kerfi about the anomalous routes they detected. In fact, Renesys's communications were with Opin Kerfi's ISP, Síminn. We have changed the story accordingly and we regret the error.