# What is algorithmic bias? Definition and business implications

> An algorithmic bias is a systematic deviation in an AI model, inherited from training data, design choices, or deployment context, that produces unfair or erroneous decisions to the detriment of a population group or a type of case.

## Definition
Algorithmic bias can emerge at three stages of the model's life cycle. First, in the training data: if the historical corpus reflects biased human decisions (past hiring, credit granting, case law), the model learns and reproduces those biases. Second, in the design choices: the chosen optimisation criterion (overall accuracy, for instance) can produce good average performance but errors concentrated on certain subgroups. Third, in the deployment context: a model trained on French 2018 data used on Belgian 2026 files will produce unexpected gaps. Algorithmic bias is not an exotic feature of AI models, it is their default condition. The question is not whether it exists, but which biases exist, at what scale, and with what operational consequences. Bias auditing is an expected good practice for any AI deployment with decisional impact on people.

## Example
The Dutch childcare benefits scandal, known as the *toeslagenaffaire*, is one of the most documented European case studies. From 2013, the Dutch tax administration used an algorithmic system to detect childcare allowance fraud. The system used nationality as a risk factor: people of non-Dutch nationality received a higher score. Result: tens of thousands of parents, disproportionately affected when they had an immigrant background or a low income, were wrongfully accused of fraud and forced to repay, sometimes tens of thousands of euros. The case led to the resignation of the Rutte III government in January 2021. Source: Amnesty International *Xenophobic Machines* report, October 2021.

## For a leader
The classic pitfall is to treat algorithmic bias as a technical problem to be solved by data scientists alone, isolated from business functions and governance. That is a framing error. Three mechanisms make it a senior-management issue. First, an undetected algorithmic bias that manifests at scale (hiring, credit scoring, insurance pricing) generates direct legal and reputational risk, which no technical audit alone can anticipate without knowledge of the business context. Second, the absence of a formalised bias audit will become hard to defend: for high-risk systems, the European AI Act will require an examination of possible biases in the data (article 10) and documentation of the measures taken. These obligations will apply from 2 December 2027, a date set by the 2026 Digital Omnibus. It is better to prepare now. Third, algorithmic bias often resides in the way the model is used, not in the model itself. An audit must therefore include downstream decision processes, not just the code.

## FAQ

**What is algorithmic bias?**
A systematic deviation in an AI model, inherited from training data, design choices, or deployment context, that produces unfair or erroneous decisions against a population group or a type of case. It is not an exotic defect but the default condition of models. The real question is not whether it exists, but which biases exist, at what scale, and with what operational consequences.

**At which stages can bias appear?**
At three stages of the model's life cycle. In the training data, when the historical corpus reflects biased human decisions (past hiring, credit granting, case law). In the design choices, when the chosen optimisation criterion performs well on average but concentrates errors on certain subgroups. In the deployment context, when a model trained on one country's and era's data is used elsewhere.

**Why is it a senior-management issue, not just a technical one?**
Because an undetected bias that manifests at scale (hiring, credit scoring, insurance pricing) creates direct legal and reputational risk that no purely technical audit can anticipate without knowledge of the business context. Bias often resides in how the model is used, not in the code itself.

**What does the AI Act require on bias?**
For high-risk systems, the European AI Act will require an examination of possible biases in the data (article 10) and documentation of the measures taken. These obligations will apply from 2 December 2027, a date set by the 2026 Digital Omnibus. The absence of a formalised bias audit will become hard to defend.

**Is auditing the model enough?**
No. Bias often resides in how the model is used, not only in the model. An audit must therefore include downstream decision processes, not just the code. Bias auditing is an expected good practice for any deployment with decisional impact on people.

## See also

- [Alignment](https://wyp.agency/en/glossary/alignment/)
- [Guardrails](https://wyp.agency/en/glossary/guardrails/)
- [AI Act](https://wyp.agency/en/glossary/ai-act/)
- [GDPR and AI](https://wyp.agency/en/glossary/gdpr-ai/)
- [Training](https://wyp.agency/en/glossary/training/)
- [AI sycophancy](https://wyp.agency/en/glossary/ai-sycophancy/)
- [ISO/IEC 42001](https://wyp.agency/en/glossary/iso-42001/)
- [Computer vision](https://wyp.agency/en/glossary/computer-vision/)
- [Explainable AI (XAI)](https://wyp.agency/en/glossary/explainable-ai/)
- [Machine learning](https://wyp.agency/en/glossary/machine-learning/)

## Sources

- Xenophobic Machines: Discrimination through unregulated use of algorithms in the Dutch childcare benefits scandal, Amnesty International, October 2021 (https://www.amnesty.org/en/documents/eur35/4686/2021/en/)
- AI Incident Database, Incident 101: Dutch Families Wrongfully Accused of Tax Fraud Due to Discriminatory Algorithm (https://incidentdatabase.ai/cite/101/)
- Regulation (EU) 2026/1744 (Digital Omnibus on AI), EUR-Lex (https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX%3A32026R1744)

---
Entry from the WYP.agency AI glossary · https://wyp.agency/en/glossary/algorithmic-bias/ · Last updated : 2026-10-14