The system said she was a shoplifter. The alert did not say how sure it was.

From 2012 to 2020, Rite Aid deployed facial recognition technology in hundreds of its retail pharmacies [1]. Two third-party vendors operated the system on Rite Aid's behalf, mostly in and around New York City, Los Angeles, San Francisco, Philadelphia, Baltimore, Detroit, Atlantic City, Seattle, Portland, Wilmington and Sacramento [2]. Its stated purpose was to "drive and keep persons of interest out of [Rite Aid's] stores" [2]. Rite Aid did not tell customers the technology was in use, and instructed employees not to reveal it to customers or the media [1][2].

The watchlist held at least tens of thousands of people, enrolled from CCTV footage, photographs taken on employees' mobile phones, driver's licences and, occasionally, news reports — many of them low-quality images, generally retained indefinitely, with store security staff trained to "push for as many enrollments as possible" [1][2]. Cameras captured everyone entering, compared each face against the database, and produced a confidence score: a number expressing how sure the system was that two images showed the same person. An alert fired when that score cleared a threshold Rite Aid set with its vendors [2]. The alerts sent to store staff generally did not include the score, so the employee deciding what to do did not know how confident the match had been [2]. A majority of enrolments carried the instruction "Approach and Identify" — approach the person, ask them to leave, and if they refuse, call the police [2].

The system produced thousands of false positives. Rite Aid largely did not record the accuracy or outcome of its alerts, yet employees still logged thousands of false matches between December 2019 and July 2020 [2]. In that same period it generated over 5,000 alerts in stores more than 100 miles from the store that had created the enrolment [2]. In one five-day stretch, a single enrolment triggered over 900 match alerts across more than 130 different stores — a majority of every location then running the technology — including hundreds each in New York and Los Angeles, over 100 in Philadelphia, and more in Baltimore, Detroit, Sacramento, Delaware, Seattle, Manchester, New Hampshire, and Norfolk, Virginia. Employees acted on some of them, including by asking customers to leave [2].

The FTC alleged the burden fell unevenly. Around 80 per cent of Rite Aid's stores sit in plurality-White areas, but about 60 per cent of the stores using facial recognition were in plurality non-White areas — the company had prioritised what it called "urban" locations and stores on public transport routes without assessing who that would fall on [2]. Alerts in plurality-Black and plurality-Asian areas were significantly more likely to carry low confidence scores, as were alerts against enrolments with typically feminine names; low-score alerts were more likely to be false; and Rite Aid did not change its policies in response [2]. It never tested, before deployment or after, whether its technology's accuracy varied by race or gender, and never asked either vendor what accuracy testing had been done [1][2]. The failures, the FTC alleged, made harm especially likely for Black, Asian, Latino and women consumers [2].

What that meant in a shop: employees followed people around the store, searched them, ordered them out, stopped them buying medication they had come for, publicly accused them of shoplifting — sometimes in front of their coworkers, employers or children — and called the police to confront or remove them [1][2]. In one incident Rite Aid investigated internally, staff called the police and asked a Black woman to leave after an alert matched her to an enrolment image employees themselves described as "a white lady with blonde hair" [2][3]. In another, employees stopped and searched an 11-year-old girl on a false match; her mother told Rite Aid she had missed work because her daughter was so distraught [2][3]. An internal presentation arguing to expand the programme had identified exactly one risk: "[m]edia attention and customer acceptance" [2][3].

On 19 December 2023 the FTC voted 3-0 to file a complaint and a proposed stipulated order in the Eastern District of Pennsylvania [1]. The order's first provision bars Rite Aid for five years from deploying or using any facial recognition or analysis system in any retail store, retail pharmacy or online retail platform, and requires it to delete, within 45 days, every photo and video collected through the system along with any data, models or algorithms derived from them [4]. Should it adopt such a system later, it must tell people when they are enrolled and when it acts against them, test for statistically significant bias, and shut the system down if it cannot address the risks it finds [1][3]. Rite Aid was in Chapter 11 at the time, so the order needed bankruptcy and district court approval; the FTC's docket records the stipulated order for permanent injunction filed on 8 March 2024 [1][5]. Rite Aid said it was pleased to reach an agreement but disagreed with the allegations, describing them as relating to "a facial recognition technology pilot program the Company deployed in a limited number of stores" that it had stopped using more than three years earlier, before the FTC's investigation began [6].

The honest case for this technology is narrower than the marketing, and NIST — whose evaluation the FTC cites — draws the line clearly. One-to-one verification asks a different question: two photographs, one of them offered by the person themselves, compared to answer "is this the same person or not?" That is what unlocks a phone or clears a checkpoint, and when it fails it usually fails by refusing you, an inconvenience you fix by trying again [7]. The one-to-many search Rite Aid was running inverts that. NIST found demographic differentials in the majority of algorithms it evaluated, with false positive rates often varying by one or two orders of magnitude across groups — present broadly, though not in every algorithm, which is precisely why asking a vendor for its test results is not a formality [7]. NIST also describes the configuration in which one-to-many search remains defensible: as "part of a hybrid machine-human system", where the algorithm returns candidates for human adjudication and staff are actually resourced to adjudicate — and it is candid that the evidence on human reviewers is itself uneven [7]. Rite Aid had the search. It did not have the adjudication.

Follow one false positive to the end. An alert lands on a store employee's company phone showing two photographs and an instruction; for most enrolments that instruction is "Approach and Identify" — approach, ask them to leave, call the police if they refuse [2]. The confidence score the system computed is generally not in the alert [2]. So the employee is not weighing a probability against what they can see in front of them; they are executing an instruction attached to what reads as a finding. Commissioner Bedoya named the mechanism: "a computer is telling a person that the customer is suspicious. And people trust computers" — automation bias, "a human tendency to believe that what a machine tells us must be true" [3]. The harm was not that the model produced a wrong match; every such model does. It was that nothing standing between the wrong match and the confrontation was designed to doubt it.