The system said she was a shoplifter. The alert did not say how sure it was.

From 2012 to 2020, Rite Aid, a US pharmacy chain, used facial recognition technology in hundreds of its stores [1]. Two outside companies ran the system on Rite Aid's behalf, mostly in and around New York City, Los Angeles, San Francisco, Philadelphia, Baltimore, Detroit, Atlantic City, Seattle, Portland, Wilmington and Sacramento [2]. Its stated purpose was to "drive and keep persons of interest out of [Rite Aid's] stores" [2]. Rite Aid did not tell customers the technology was in use, and told employees not to reveal it to customers or the media [1][2].

The watchlist held at least tens of thousands of people. They were added from CCTV footage, photos taken on employees' phones, driver's licences and, sometimes, news reports — many of the images low-quality, generally kept forever, with store security staff trained to "push for as many enrollments as possible" [1][2]. Cameras captured everyone who walked in and compared each face against the list. For each comparison the system produced a confidence score: a number for how sure it was that two images showed the same person. An alert fired when that score passed a line Rite Aid set with its suppliers [2]. The alerts sent to store staff generally did not include the score. So the employee deciding what to do did not know how confident the match had been [2]. Most entries on the list carried the instruction "Approach and Identify": approach the person, ask them to leave, and if they refuse, call the police [2].

The system produced thousands of false matches. Rite Aid largely did not record whether its alerts were right, yet employees still logged thousands of false matches between December 2019 and July 2020 [2]. In the same period it produced over 5,000 alerts in stores more than 100 miles from the store that had added the person to the list [2]. In one five-day stretch, a single entry set off over 900 alerts across more than 130 different stores — most of the locations then running the technology. Hundreds came in New York and Los Angeles, over 100 in Philadelphia, and more in Baltimore, Detroit, Sacramento, Delaware, Seattle, Manchester (New Hampshire) and Norfolk (Virginia). Employees acted on some of them, including by asking customers to leave [2].

The FTC — the US consumer watchdog — alleged the burden fell unevenly. Around 80 per cent of Rite Aid's stores are in areas where White residents are the largest group, but about 60 per cent of the stores using facial recognition were in areas where they are not. The company had prioritised what it called "urban" locations and stores on public transport routes, without asking who that would fall on [2]. Alerts in areas where Black or Asian residents are the largest group were significantly more likely to carry low confidence scores. So were alerts against entries with typically feminine names. Low-score alerts were more likely to be wrong, and Rite Aid did not change its policies in response [2]. It never tested, before or after switching the system on, whether its accuracy varied by race or gender, and never asked either supplier what accuracy testing had been done [1][2]. The failures, the FTC alleged, made harm especially likely for Black, Asian, Latino and women customers [2].

What that meant in a shop: employees followed people around, searched them, ordered them out, stopped them buying the medicine they had come for, and accused them of shoplifting in public — sometimes in front of coworkers, employers or children. Some called the police to confront or remove them [1][2]. In one incident Rite Aid looked into itself, staff called the police and asked a Black woman to leave after an alert matched her to an entry whose photo employees themselves described as "a white lady with blonde hair" [2][3]. In another, employees stopped and searched an 11-year-old girl on a false match; her mother told Rite Aid she had missed work because her daughter was so upset [2][3]. An internal presentation arguing to expand the programme had named exactly one risk: "[m]edia attention and customer acceptance" [2][3].

On 19 December 2023 the FTC voted 3-0 to file a complaint and a proposed settlement order in a federal court in Pennsylvania [1]. The order's first provision bans Rite Aid for five years from using any facial recognition or face analysis system in any store, pharmacy or online shop. It requires the company to delete, within 45 days, every photo and video collected through the system, along with any data, models or algorithms built from them [4]. If it adopts such a system later, it must tell people when they are added and when it acts against them. It must test for statistically significant bias — differences too large to be chance — and shut the system down if it cannot fix the risks it finds [1][3]. Rite Aid was in bankruptcy at the time, so the order needed approval from both the bankruptcy court and the district court; the FTC's docket records the agreed final order (a stipulated order for permanent injunction) filed on 8 March 2024 [1][5]. Rite Aid said it was pleased to reach an agreement but disagreed with the allegations. It described them as relating to "a facial recognition technology pilot program the Company deployed in a limited number of stores" that it had stopped using more than three years earlier, before the FTC's investigation began [6].

The honest case for this technology is narrower than the sales pitch, and NIST — the US standards body whose testing the FTC cites — draws the line clearly. One-to-one checking asks a different question. Two photos, one of them offered by the person themselves, are compared to answer "is this the same person or not?" That is what unlocks a phone or clears a passport gate. When it fails, it usually fails by refusing you — an annoyance you fix by trying again [7]. The one-to-many search Rite Aid was running turns that around: one face against a whole list. NIST found differences between demographic groups in most of the systems it tested, with false-alarm rates often varying ten- or a hundred-fold between groups. The differences were widespread, though not in every system — which is exactly why asking a supplier for its test results is not a formality [7]. NIST also describes the setup in which one-to-many search remains defensible: as "part of a hybrid machine-human system", where the software returns candidates for a person to judge, and staff are actually given the time to judge. It is candid that the evidence on human reviewers is itself uneven [7]. Rite Aid had the search. It did not have the judging.

Follow one false match to the end. An alert lands on a store employee's company phone showing two photos and an instruction. For most entries that instruction is "Approach and Identify" — approach, ask them to leave, call the police if they refuse [2]. The confidence score the system worked out is generally not in the alert [2]. So the employee is not weighing a probability against what they can see in front of them. They are carrying out an instruction attached to something that reads as a finding. Commissioner Bedoya named the mechanism: "a computer is telling a person that the customer is suspicious. And people trust computers" — automation bias, "a human tendency to believe that what a machine tells us must be true" [3]. The harm was not that the system produced a wrong match; every such system does. It was that nothing standing between the wrong match and the confrontation was designed to doubt it.