A yes/no field for Dutch citizenship fed the fraud score, and tens of thousands of parents were told to repay everything at once.

The Dutch childcare benefits scheme began in 2005, administered by the tax authorities under social-security enforcement policies that had been progressively hardened. In 2013 they adopted an algorithmic system for fraud detection — the "risk classification model" — which included self-learning elements and built risk profiles of applicants thought more likely to file inaccurate claims. Applicants it selected had their benefits suspended and were investigated [2].

One of the model's parameters was "Dutch citizenship: yes/no". Applicants of non-Dutch nationality consequently received higher risk scores [2].

What triggered an accusation could be very small. A missing signature on a childcare contract, or a late or incomplete payment of the mandatory personal contribution, was enough. Parents were required to repay large sums immediately and in a single lump sum, and were recorded as fraudsters; the consequences reported ran from debt and unemployment to forced evictions, and to mental-health harm and broken relationships [2].

Parents who tried to establish what was wrong with their claim were, in Amnesty's account, met with silence: the tax authorities consistently refused to explain their decisions. Nor could anyone else see the machinery — it was not possible for parents, journalists, politicians, oversight bodies or civil society to obtain meaningful information about the existence or the workings of the risk classification model [2].

The number of people affected is not precisely known. As of 2021, 42,000 parents and caregivers had come forward as victims, of whom 19,000 had received compensation of €30,000 or more, and around 70,000 children had been compensated [2]. The scandal became public in 2018 and brought down the Dutch cabinet in 2021 [2].

A parliamentary inquiry committee reported on 17 December 2020 under the title "Ongekend onrecht" — "Unprecedented injustice". It found that basic principles of the rule of law had been breached in the administration of childcare allowance, and attributed that failure not only to the administration but to the legislator and the judiciary [1].

Its account of why is specific. Legislation was adopted that left little or no scope for individual circumstances such as an administrative error made with no ill intent, and contained no hardship clause. The allowance was administered as a mass process, and the committee identified the "all or nothing" approach, group-based action and the application of malice and gross negligence as a serious breach of the principle that a person's circumstances should be considered. It also found that for many years the administrative justice system helped perpetuate the ruthless application of the law and neglected its function of safeguarding individual legal rights. Its summary of the cumulative effect was that "for years, parents never had a chance" [1].

The inquiry did not, however, examine the algorithm. It records that risk profiles including nationality, and the use of blacklists, fell outside the committee's remit, and that the use of nationality in risk selection was investigated instead by the data protection authority [1].

That investigation ended in a fine. On 13 December 2021 the Dutch data protection authority fined the tax administration €2.75 million over its handling of nationality data in the childcare benefits system, including the use of nationality as a risk indicator, which legislation did not permit for assessing these applications. The investigation also found nationality data on 1.4 million people still held in 2018 that should have been deleted in 2014 [3].

There is one thing a scoring model did here that a room full of officials would not have done, and it is the only reason this case has a documented ending. The assumption was written down. "Dutch citizenship: yes/no" existed as a parameter in a system [2], so a regulator with statutory powers could establish it as a fact and fine the department for it [3]. The same suspicion distributed across a thousand individual judgements would have left nothing to find and nothing to fine — no field, no file, no number. This is not a defence of the model, and the transparency was not voluntary: the workings were closed to parents, press, parliament and oversight bodies alike [2], and it took an investigation by a separate authority to open them [1]. But encoding a bias does make it discoverable, and discoverability is what turned a decade of denied suspicions into a finding of unlawfulness.

Everything else. The model was one input into a process that had no way back: entitlement could be lost entirely over a clerical slip, repayment fell due at once, the label was fraudster, and the legislation carried no hardship clause and no room for an error made in good faith [1]. A parent could not learn what was wrong, could not see what had flagged them, and for years could not get relief from the courts either [1][2]. Note where the algorithm sits in that sentence: it decided who got looked at, and every part that ruined people was the machinery around it. A fairer model would have produced a fairer selection of families to subject to the same unappealable process [1].