Edward Snowden’s revelations about the activities of the National Security Agency have taught us many things, among the most important of which are that the NSA has technological capabilities far beyond what most of us imagined, and that once technology enables the agency to engage in a particular form of spying, it will be almost impossible to resist. My reaction to many of these revelations has been, “This is unsettling, but what’s really scary is what they’ll be able to do when the technology advances a bit.” Well today we get a picture of just such an advancement.
As Barton Gellman and Ashkan Soltani report in today’s Post, the NSA now has the ability to record an entire country’s phone calls — all of them — and keep them for later examination, kind of like a DVR recording of everything everyone in that country said. It’s a rolling 30-day collection, in which the oldest recordings are dropped off to make room for the newest ones. They’ve deployed this system on one unidentified country and are planning to use it on others.
It’s natural when we hear about a system like this to think that maybe it isn’t so bad as long as they don’t deploy it against us here in the United States. And that’s what the Obama administration has assured us in the past, just as previous administrations have: We’re only going to use this kind of extremely invasive surveillance on foreigners, so don’t get too upset. And that foreign country is certainly one we don’t like very much, so if we knew which one it was, most of us would say, “Oh, them? Sure, go ahead and record their calls.”
But let’s look forward a bit. The NSA started by recording one country’s calls. It’s already planning to add a few more. What if a few years later, it decides that it might as well record the calls of every country in the world that is either actively or potentially hostile to us — let’s say everyone except the members of NATO. That could, without question, yield a lot of valuable intelligence. And then it might add our allies, too, because there could be some interesting things to be learned there as well; and after all, the NSA already monitored the communications of allies like German Chancellor Angela Merkel, and it would presumably still be doing it if ithadn’t gotten caught. It isn’t a leap to imagine that within a relatively short amount of time — years, not decades — the agency could be recording every non-American phone call on Earth.
Okay, that’s sounds like something out of a spy novel. How would the NSA store all that data, anyway? It seems impossible. Let’s go back to Gellman and Soltani’s article:
Telephone calls are often thought to be more ephemeral and less suited than text for processing, storage and search. Indeed, there are indications that the call-recording program has been hindered by the NSA’s limited capacity to store and transmit bulky voice files.
In the first year of its deployment, a program officer wrote that the project “has long since reached the point where it was collecting and sending home far more than the bandwidth could handle.”
Because of similar capacity limits across a range of collection programs, the NSA is leaping forward with cloud-based collection systems and a gargantuan new “mission data repository” in Utah. According to its overview briefing, the Utah facility is designed “to cope with the vast increases in digital data that have accompanied the rise of the global network.”
There are some technological problems that require creative leaps in order to solve. This is not one of them. Yes, we’re talking about an enormous amount of data. But the amount of data we can store increases all the time, just as the cost of a given amount of storage decreases. For some time, people have been predicting the eventual end of Moore’s law, which states that the number of processors that can fit on a chip doubles about every 18 months to two years. But it’s still going strong, and a similar curve of increases in memory capacity has followed right along. As our requirements for more and more storage space have increased, the tech industry has managed to keep up. As the NSA’s appetite grows, its ability to hold what it sucks up is likely to grow right along with it.
To see whether recording and storing the whole world’s phone calls is really possible, I tried to do some back-of-the-envelope calculations, which may be more than a little suspect given that it’s early in the morning and I’m getting estimates for the factors by bounding around the Internet (using figures like this and this). But with that caveat firmly in place, I think we’re talking about the necessary storage being something in the range of an exabyte or two to store low-quality recordings of a year’s worth of every phone conversation on Earth (don’t forget, most phone calls are pretty short). For reference, Randall Munroe guessed that all of Google’s servers hold around 10 exabytes (one exabyte is equal to a billion gigabytes), which suggests that while assembling that much storage would certainly be a huge undertaking for the NSA, it wouldn’t be so huge as to be impossible. Someone who knows more about these things than I do should do a more careful calculation of what it would really be, but if that’s the general neighborhood we’re in, before long it might not be out of the question.
I realize that this sounds a bit outlandish. But as I said, there isn’t much in the way of technical advancement that would be required to make it happen — just more storage, sufficient processing power to make use of the data, and the will to do it.
And if you aren’t unsettled enough, there’s another surveillance story that came out in the last couple of days that is in its way just as alarming. Facebook has announced that its engineers have for all intents and purposes solved the problem of facial recognition. They claim that their facial recognition software (which is still experimental and hasn’t yet been deployed) can now identify faces just as accurately as people can. Now consider the millions of security cameras, multiplying all the time, on street corners and highways and in stores and offices all over the country. Now imagine if you linked the images from all those cameras — oh, and throw in the license plate cameras that more and more law enforcement agencies are using — into a database that could tell someone like the NSA where nearly everyone in the United States has walked or driven on a given day. The technology is already here. All we need is more processing power and storage, and we’ll have that soon enough.
Orwell had no idea.
* REPUBLICANS PICK A POLITICAL NOVICE FOR GOVERNOR IN ILLINOIS: Democratic governor Pat Quinn is seen as extremely vulnerable, but Republicans may have done him a favor by handing the primary to Bruce Rauner, a businessman making his first run for office. Those types usually lose.
* RUSSIAN-BACKED FORCES TAKE OVER A UKRAINIAN NAVAL BASE IN CRIMEA: It doesn’t look like anything’s going to stop Russia’s annexation of the territory.
* CNN GETS A TEMPORARY BOOST FROM THE MISSING MALAYSIAN PLANE: What do you get when you have a mysterious story with almost no real information to be had and nearly unlimited room for baseless speculation? Boffo ratings, that’s what.
* FLORIDA, ARIZONA, MICHIGAN STATE AND LOUISVILLE RECEIVE A PRESIDENTIAL ENDORSEMENT: Apparently, there’s some sort of sporting contest going on in which leather spheres are hurled through iron rings by young men with overactive pituitary glands. It must be vaguely socialistic, because President Obama has anointed the four finalists and will later today reveal his full “bracket,” declaring which teams are approved to win each game.
* TWO LUCKY PEOPLE TO HAVE LIVES RUINED BY SUDDEN WEALTH The $400 million Mega Millions jackpot will be split by two ticket holders, one in Maryland and one in Florida. The winners will soon learn a depressing lesson in the depths of human depravity, as their family, friends and neighbors scheme to get their hands on the money, and it turns out that a Lamborghini is little solace when you’ve lost your capacity to trust another human being.