The country’s biggest seller of police body cameras on Thursday convened a corporate board devoted to the ethics and expansion of artificial intelligence, a major new step toward offering controversial facial-recognition technology to police forces nationwide.
The board’s first meeting will likely presage an imminent showdown over the rapidly developing technology. Shortly after the board was announced, a group of 42 civil rights, technology and privacy groups, including the American Civil Liberties Union and the NAACP, sent members a letter voicing “serious concerns with the current direction of Axon’s product development.”
The letter urged an outright ban on face recognition, which it called “categorically unethical to deploy” because of the technology’s privacy implications, technical imperfections and potentially life-threatening biases. Most facial-recognition systems, recent research found, perform far less accurately when assessing people with darker skin, opening the potential to an AI-enabled officer misidentifying an innocent person as a dangerous fugitive.
Axon’s founder and chief executive, Rick Smith, said the company is not currently building facial-recognition systems but said the technology is “under active consideration.” He acknowledged the potential for “bias and misuse” in face recognition but said the potential benefits are too promising to ignore.
“I don’t think it’s an optimal solution, the world we’re in today, that catching dangerous people should just be left up to random chance, or expecting police officers to remember who they’re looking for,” Smith said. “It would be both naive and counterproductive to say law enforcement shouldn’t have these new technologies. They’re going to, and I think they’re going to need them. We can’t have police in the 2020s policing with technologies from the 1990s.”
Axon held the board’s first meeting Thursday morning at its Arizona headquarters with eight company-selected experts in AI, civil liberties and criminal justice. The board, whose members are paid volunteers and have no official veto power, will be asked to advise the company on “future capabilities Axon's AI Research team is working on to help increase police efficiency and efficacy,” the company said in a statement.
Face recognition has long had major appeal for law enforcement and government surveillance, and recent advances in AI development and declining camera and hardware costs have spurred developers to suggest it could be applied for broader use. Roughly 117 million American adults, or about half the country, can be found in the vast facial-recognition databases used by local, state and federal law enforcement, Georgetown Law School researchers estimated in 2016.
Faces are regarded as a quick, reliable way to identify someone from video or afar — and, in some cases, seen as easier to acquire than other “biometric identifiers,” such as fingerprints, that demand close proximity and physical contact. The Department of Homeland Security scans the faces of international travelers at many of the country’s biggest airports, and plans to expand to every traveler flying overseas.
But critics say facial-recognition systems are still unproven in their ability to uniquely identify someone. Faces age over time and change because of circumstance. Identical twins have been shown to be able to fool the facial-recognition systems used to unlock Apple’s iPhone X.
“Real-time face recognition would chill the constitutional freedoms of speech and association, especially at political protests,” the letter from the dissenting groups states. It “could also prime officers to perceive individuals as more dangerous than they really are and to use more force than the situation requires. No policy or safeguard can mitigate these risks sufficiently well for real-time face recognition ever to be marketable.”
Axon has moved aggressively to corner the market on police technologies, offering free one-year trials for its body cameras and online storage to police departments nationwide. The company said in February that more than half of the major city law-enforcement agencies in the United States have bought Axon body cameras or software, including Los Angeles, Chicago and Washington.
The company, which changed its name last year from Taser International, also advertises itself as “the largest custodian of public safety data in the U.S.,” saying more than 20 petabytes — or 20 million gigabytes — of police photos, body-camera video and other criminal-investigation documents have been uploaded to its cloud-storage service, Evidence.com.
Police video is seen as a major growth market for AI-development firms, both for real-time surveillance and after-crime review: One company, BriefCam, allows city officials and police investigators to narrow hours of video down into seconds by filtering only the footage of, for instance, red trucks or men with suitcases.
Axon’s long-established contracts with nationwide police forces could push the technology’s real-world deployment rapidly forward. Instead of signing new deals with tech firms, police departments with Axon body cameras could push facial-recognition features to its officers in potentially the same way they apply a software update.
Face recognition is one of the most competitive and hotly debated subsets of AI in today’s consumer tech, with Apple, Facebook and Google all devoting teams to expanding its use in security, photo tagging and search.
Most facial-recognition systems today depend on “deep-learning” algorithms that analyze facial photos and scan for similarities across a huge data set of similar images. Supporters of body cameras say the upgraded systems could help alert officers to a passing criminal suspect or spot a missing child in a crowd.
But the technology does not always deliver perfect results and instead suggests the probability of a possible match, with an accuracy rate that can vary wildly based on the photo’s quality, the person’s skin color or other factors. Privacy advocates worry that the systems could instill a false confidence and lead to police misidentifying innocent people as suspects or wanted criminals, with potentially fatal results.
“There’s always going to be a possibility of error and, in a real-time scenario where a police officer is likely armed, the risks associated with potential misidentification are always going to exceed any possible benefits,” said Laura Moy, the deputy director of Georgetown Law's Center on Privacy & Technology. “There's a real concern that it could exacerbate the risk of police use of force.”
Today’s facial-recognition systems also show troubling implicit biases, often due to the lack of diversity in images its systems have been trained on. Researchers from the Massachusetts Institute of Technology’s Media Lab said earlier this year that the three leading facial-recognition systems — from IBM, Face++ and Microsoft — performed consistently better at identifying the gender of people with lighter skin, averaging 99 percent accuracy for lighter-skinned men and 70 percent accuracy for darker-skinned women.
Body cameras, which gained popularity in recent years as tools for checking police misconduct, have been criticized for contributing to pervasive surveillance and potentially worsening the problems in heavily policed neighborhoods. Police also largely decide the rules of use. Sacramento police officers last month muted their body cameras after fatally shooting Stephon Clark, an unarmed black man, in his grandmother's back yard.
Critics have questioned how effective the volunteer ethics board, meeting twice a year, will be in steering the decisions of a private company. But Smith said he saw some parallels between face recognition and Tasers, which saw initial resistance but have rapidly proliferated into one of law enforcement's most commonly used weapons.
“We’ll probably see some missteps along the way. As I look back on the Taser journey, when you introduce things with this much of a change, it’s rarely a smooth process,” he said. But “getting this wrong is not just a bad thing for society. Companies that get these things wrong pay a big price. ... We don’t want to create an Orwellian state just to make a buck.”