London’s Metropolitan Police Service, in collaboration with Accenture,  just tested a new predictive policing system to assess the likelihood of known gang members re-offending, and already people are comparing it to the pre-crime system in “Minority Report.” The pilot program combined four years of historic data of criminal activity across London with social media network analysis to predict which gang members were most likely to commit a violent crime in the following year. While the addition of social network analysis to existing big data capabilities represents the next big step in being able to predict wrongdoing and spot likely wrongdoers, we’re still a long way from having a pre-crime system that truly predicts, rather than just forecasts, potential crime.

The first problem, quite simply, is that any pre-crime system is only as good as the data that goes into the system. As Accenture notes in a 2013 report on how to improve police services in the future, data can be used to build a robust analytical model and provide real-time intelligence for police. London’s predictive policing model used historical criminal data and social network data, and models used by other police services have used weather data, traffic data and even school data to predict future criminal activity. At the end of the day, though, we’re asking a software program  to spit out a statistical guess of the most likely scenario based on all available data, some of which may be more qualitative than quantitative.

In “Minority Report,” the pre-cogs used a similar type of data-driven system to predict future crimes, which is why experiments like the one in London are usually referred to as “pre-crime.” The pre-cogs were theoretically able to reduce crime almost entirely by taking all these data points, mashing them together, and then arriving at a prediction about a future crime being committed. But remember — this prediction of a future crime was the “Majority Report” — the consensus of three different predictions. Usually, these three different predictions agreed, but there always existed the potential for the “Minority Report” – the one prediction that’s the outlier. In the real world of modern data analysis, this is tantamount to saying that you’re taking the consensus forecast of three different statistical models, one of which may be an outlier.

Secondly, while companies such as Accenture and IBM have investigated the promise of tapping social networks for clues as to possible perpetrators and victims of crime, this type of analysis only has value for certain types of crime. Social networks can be effective for analyzing relationships between “social” organizations, such as gangs or terrorist cells, that commit crimes or other wrongdoing as part of a coordinated activity. This is the reason why the London police and Accenture chose to focus on London gangs and analyze their social network chatter — these people were presumably talking to each other and discussing future criminal activity.

When it comes to crimes committed by individuals, however, this social network analysis breaks down. In the IBM report on predictive policing, it was noted that the best use of social network analysis for crimes committed by individual is to use it to starve a suspect of “social resources” – you go to the all the nodes of a suspect’s social network after a crime has been committed, and use those to narrow down where the suspect might be. In some limited cases, an individual’s social network activity may warn of future crimes (e.g the nut making threats on Facebook), but that is the exception, not the rule.

The third reason why we’re still a long way from a true pre-crime system is subtler – and that’s how hard it is to transform information into actionable intelligence. This is the point made by RAND in its 2013 report on predictive policing. Any predictive policing model is essentially the same predictive model that advertisers use to predict who might purchase their products or that pollsters use to predict elections. You’re trying to take a lot of variables and make sense of them all. It does help, but it’s not a magic bullet. Even if you can narrow down a specific geographic area where a crime is highly likely to occur, you still need to divert policing resources to the right place at the right time.

Consider that PredPol, a predictive policing model used by the Los Angeles police department that is similar to the one used by the police in London, is “based on models for predicting aftershocks from earthquakes.” That makes intuitive sense: small crime aftershocks are more likely in places where large crime earthquakes have already occurred. However, as Nate Silver points out in his bestseller “The Signal and the Noise,” earthquake prediction is a notoriously imprecise field. There is too much noise, not enough signal. It’s possible to forecast, but not predict an earthquake. And, as Silver emphasizes, even the USGS says that it’s impossible to predict a future earthquake.

Using the type of social network data analysis employed by Accenture and London’s Metropolitan Police – analyzing gang chatter as a way to predict possible wrongdoing — is a new interesting twist on predictive policing. It certainly goes far beyond what we were able to do before the big data and social media era – when police investigators merely plotted past crimes on a geographic map and used historical criminal data to forecast the future.

With police officers armed with real-time, digital tools and with access to more data than ever before, we’re getting closer to radically improving future policing strategies. By layering on ever more data, we can get a better forecasting model for when and where a crime might occur. But as anyone who has ever tried to predict the outcome of a football game or a political election knows, there is often just way too much noise and not enough signal. We are only forecasting, not predicting.