Last reviewed:

What is algorithmic bias? Definition and business implications

An algorithmic bias is a systematic deviation in an AI model, inherited from training data, design choices, or deployment context, that produces unfair or erroneous decisions to the detriment of a population group or a type of case.

Algorithmic bias can emerge at three stages of the model's life cycle. First, in the training data: if the historical corpus reflects biased human decisions (past hiring, credit granting, case law), the model learns and reproduces those biases. Second, in the design choices: the chosen optimisation criterion (overall accuracy, for instance) can produce good average performance but errors concentrated on certain subgroups. Third, in the deployment context: a model trained on French 2018 data used on Belgian 2026 files will produce unexpected gaps. Algorithmic bias is not an exotic feature of AI models, it is their default condition. The question is not whether it exists, but which biases exist, at what scale, and with what operational consequences. Bias auditing is now a de facto obligation for any AI deployment with decisional impact on people.

Concrete example

The Dutch childcare benefits scandal, known as the toeslagenaffaire, is one of the most documented European case studies. Between 2013 and 2019, the Dutch tax administration used an algorithmic system to detect childcare allowance fraud. The system incorporated dual nationality as a risk signal. Result: about 26,000 families, mostly from immigrant backgrounds, were wrongfully accused of fraud, forced to repay between 20,000 and 60,000 euros, some driven into over-indebtedness, with children placed into foster care. The case led to the resignation of the Rutte III government in January 2021. Source: Amnesty International Xenophobic Machines report, October 2021.

See also

Further reading

Xenophobic Machines, Amnesty International, October 2021 (external resource)

Sources

  1. Xenophobic Machines: Discrimination through unregulated use of algorithms in the Dutch childcare benefits scandal, Amnesty International, October 2021. https://www.amnesty.org/en/documents/eur35/4686/2021/en/ (accessed 2026-05-24)
  2. AI Incident Database, Incident 101: Dutch Families Wrongfully Accused of Tax Fraud Due to Discriminatory Algorithm. https://incidentdatabase.ai/cite/101/ (accessed 2026-05-24)

← Back to glossary

Address copied