To add insult to injury — or injury to injury — Mr. Parks was barred from release on bail when another algorithm interfered. This tool evaluates a defendant’s risk to determine whether he may be released before trial. Because Mr. Parks was incarcerated a decade ago for selling drugs, he was awarded a public safety assessment score that kept him behind bars until his family hired an attorney to help. This history also almost meant he didn’t dispute the charge despite his innocence: “I was afraid to go to trial. I knew I would get 10 years if I lost.” The ability to challenge flawed facial recognition software isn’t necessarily enough.
The tale illustrates the insidious implications of an imperfect technology that is known to perform even less accurately on people of color. Black Americans are already disproportionately policed and punished. Scanning mugshot databases for similar faces to a suspect’s reinforces this trend — especially because many systems return results no matter whether there’s a high-confidence match. There are myriad tales of grosser misuse, many involving attempts to compensate for poor-quality pictures of suspects. Celebrity lookalikes to beer thieves have been run through tools to try to turn up leads; the New York Police Department once Googled the phrase “Black Male Model,” cut out the mouth on the image they found and pasted it on their target.
There are surely positive applications of facial recognition technology used correctly, and used lawfully. Yet today, too many authorities have proved that as long as there aren’t rules imposed on them, they won’t impose any on themselves. Congress needs time to develop those rules in a manner that protects public safety and civil liberties at the same time. Meanwhile, it needs to impose a moratorium so that the third person wrongfully arrested will also be the last.