Bias: AI Glossary Definition

In AI, bias has two distinct meanings. Statistical bias is the systematic error between a model’s average prediction and the true value, forming one half of the bias-variance tradeoff. Algorithmic bias refers to systematic unfairness in outcomes for specific groups, usually traced back to patterns in the training data or design choices made during model development.

Two different meanings of “bias”

When you read about bias in AI, you are likely encountering one of two concepts that share the same word but describe different phenomena. Confusing them leads to misdiagnosed model failures.

Statistical bias is a mathematical property. It measures how far off a model’s predictions are on average from the ground truth. A model with high statistical bias is too simple. It misses real patterns in the data and fails to capture the underlying relationships.

Algorithmic bias, often called social bias, is an ethical or operational property. It measures whether a model treats different groups of people differently or unfairly. This bias does not necessarily mean the model is mathematically inaccurate. It means the inaccuracy or the decision distribution is skewed against a specific demographic.

Statistical bias

Statistical bias is central to the bias-variance tradeoff, a fundamental concept in machine learning. Every model makes errors. These errors come from two sources: bias and variance.

A model with high bias makes strong simplifying assumptions. It assumes the data follows a simple pattern, like a straight line, when the reality is more complex. Because of this rigidity, the model underfits. It performs poorly on the training data and continues to perform poorly on new, unseen data.

High bias usually results from using a model that is too simple for the problem. For example, using linear regression to predict a highly non-linear relationship will introduce significant statistical bias. You reduce this bias by using more flexible models or adding relevant features. However, doing so often increases variance.

Variance is the other half of the tradeoff. A high-variance model fits the training data too closely, including the noise and random fluctuations. This leads to overfitting. The model performs exceptionally well on training data but fails on new data because it memorized the noise instead of learning the signal.

Balancing bias and variance is a core practical concern. You must choose a model complexity that minimizes total error without sacrificing generalization.

Algorithmic bias occurs when an AI system produces systematically skewed outcomes for particular groups of people. This bias is usually traced back to the data the system was trained on or to choices made during the building process. It is rarely a single deliberate decision to discriminate.

This type of bias often stems from historical data reflecting societal imbalances. If past hiring decisions favored men, a model trained on that history will likely learn that pattern. It is also influenced by how a problem is framed and which features are selected.

How bias gets into a model

Bias enters AI systems through several distinct pathways. Understanding these sources helps engineers identify where fairness breaks down.

Historical bias occurs when the real-world data used for training reflects past inequities. For example, if a dataset of loan approvals reflects decades of discriminatory lending practices, the model will learn to replicate those patterns.

Sampling or representation bias happens when certain groups or situations are underrepresented in the training data relative to how the system will actually be used. If a medical imaging dataset contains mostly images from one demographic, the model may not generalize well to other groups.

Measurement bias arises when the labels or features used as proxies for what the model is trying to predict are themselves imperfect. If the proxy variable is unevenly reliable across different groups, the model inherits that inaccuracy.

Real examples

Several well-documented cases illustrate how bias manifests in production systems.

In October 2018, Reuters reported that Amazon built and later scrapped an internal AI recruiting tool. The system had taught itself to penalize resumes that included the word “women’s,” such as “women’s chess club captain.” It also downgraded graduates of two all-women’s colleges. The bias emerged because the tool was trained on ten years of past hiring data, which was dominated by resumes from men.

Another widely discussed example involves the COMPAS criminal risk-assessment tool used in parts of the US justice system. A 2016 analysis by the investigative newsroom ProPublica found that the tool’s risk scores were substantially more likely to falsely flag Black defendants as high-risk for future crime than white defendants. This sparked an ongoing debate over how such tools should be measured and used.

Facial recognition systems have also shown measurable disparities. Independent testing has demonstrated higher error rates for some demographic groups compared to others. This is why accuracy claims for facial recognition are usually reported broken down by group rather than as one overall number. You can read more about these specific challenges in our facial recognition entry.

How teams try to reduce it

There is no single technique that fully removes bias. Most practical approaches involve measuring specific fairness metrics for a specific use case and trading them off against other goals like overall accuracy.

Common mitigation strategies include auditing training data and model outputs for disparities across groups before deployment. Teams often deliberately rebalance or augment underrepresented data. Sometimes, synthetic data is used to fill gaps in representation.

Using explainability tools helps engineers inspect which features are driving a model’s decisions. Explainable AI techniques allow teams to see if a model is relying on a protected attribute, either directly or indirectly, to make predictions.

Reducing bias is an iterative process. It requires continuous monitoring and a willingness to accept that a model might be slightly less accurate overall if it is significantly fairer for underrepresented groups. This balance is a key component of AI ethics in practice.

FAQ

What is the difference between statistical bias and algorithmic bias?

Statistical bias measures the average error between a model’s predictions and the true values. Algorithmic bias measures systematic unfairness in outcomes for specific demographic groups.

Can a model be unbiased statistically but biased socially?

Yes. A model can be mathematically accurate on average but still produce unfair outcomes for specific groups if the underlying data reflects historical inequities.

How do you measure bias in AI?

Teams use fairness metrics such as demographic parity, equalized odds, or disparate impact to compare outcomes across different groups. These metrics are often calculated alongside standard accuracy scores.