HOW MANY more Black men will be wrongfully arrested before this country puts a stop to the unregulated use of facial recognition software?

So far, the count is three, and that’s only those we know of. The latest is Nijeer Parks, who according to the New York Times was accused of stealing snacks from a gift shop and trying to hit a police officer with his car at a Hampton Inn in Woodbridge, N.J., despite having been 30 miles away when the crime occurred. The story is familiar, because similar things happened to Robert Williams and Michael Oliver in Michigan, and because advocates have been warning of precisely these pitfalls for years. A detective sent a fake driver’s license, supplied by the actual crook before he escaped, to state agencies equipped with a facial recognition tool. The state returned Mr. Parks as a match; the detective and a witness agreed, in what seems a classic case of confirmation bias. It is unclear what software was used.

To add insult to injury — or injury to injury — Mr. Parks was barred from release on bail when another algorithm interfered. This tool evaluates a defendant’s risk to determine whether he may be released before trial. Because Mr. Parks was incarcerated a decade ago for selling drugs, he was awarded a public safety assessment score that kept him behind bars until his family hired an attorney to help. This history also almost meant he didn’t dispute the charge despite his innocence: “I was afraid to go to trial. I knew I would get 10 years if I lost.” The ability to challenge flawed facial recognition software isn’t necessarily enough.

The tale illustrates the insidious implications of an imperfect technology that is known to perform even less accurately on people of color. Black Americans are already disproportionately policed and punished. Scanning mugshot databases for similar faces to a suspect’s reinforces this trend — especially because many systems return results no matter whether there’s a high-confidence match. There are myriad tales of grosser misuse, many involving attempts to compensate for poor-quality pictures of suspects. Celebrity lookalikes to beer thieves have been run through tools to try to turn up leads; the New York Police Department once Googled the phrase “Black Male Model,” cut out the mouth on the image they found and pasted it on their target.

There are surely positive applications of facial recognition technology used correctly, and used lawfully. Yet today, too many authorities have proved that as long as there aren’t rules imposed on them, they won’t impose any on themselves. Congress needs time to develop those rules in a manner that protects public safety and civil liberties at the same time. Meanwhile, it needs to impose a moratorium so that the third person wrongfully arrested will also be the last.

Read more: