90% accurate in the lab. Then it met the lighting in a real clinic.

Diabetic retinopathy causes blindness and is treatable if caught early, which makes screening a high-value problem: there are far more patients needing eye checks than ophthalmologists to check them. Google built a deep-learning system to read retinal photographs and identify the disease, and in the lab it performed at more than 90% accuracy — what the team called "human specialist level" [2].

Researchers then did something unusual: they studied it in the field, across eleven clinics in Thailand, observing and interviewing the nurses who actually operated it. The study was published by Emma Beede and colleagues at CHI 2020 [1][2].

In those clinics the system rejected more than a fifth of the images nurses captured. It had been trained on high-quality scans and refused anything below its quality threshold — and the dominant cause was lighting, because real examination rooms are lit however they happen to be lit [2].

The rest of the environment intruded too. Slow connections made uploads to the cloud take a long time, frustrating staff and patients waiting on results. Nurses who could see the eye looked healthy found their photograph rejected anyway, and patients were sent to another clinic on another day [2].

The published paper frames the finding not as a model defect but as a mismatch: tensions between the model's thresholds for data quality and the quality of data that arises in an imperfect, resource-constrained environment [1].

When conditions allowed it to work, it worked — an instant result at the point of care instead of a wait for a specialist, in a screening programme that cannot possibly staff enough ophthalmologists to see everyone. And the study itself is the helped story: a team publishing, in detail, the ways their own celebrated system underperformed in the field. That is how a deployment gap gets closed rather than repeated, and it is rarer than it should be [1][2].

Nothing here was a modelling error. The accuracy figure was honest, the training was sound, and the system did exactly what it was built to do — refuse to guess on a poor image, which is the responsible behaviour. It simply had never been asked to work in a room with the wrong lamp. The gap was not between the model and the truth; it was between the data it was validated on and the data it would actually be given, and no amount of additional lab accuracy would have revealed it [1][2].