Confusion Matrix: Definition, Example & Metrics

A confusion matrix is a table that compares a classification model’s predicted labels with the actual labels, counting correct and incorrect predictions for each class. It shows not only how often the model is wrong but which kinds of mistakes it makes.

Consider a spam filter tested on 1,000 emails. The model flags 90 emails as spam, but only 80 of those really are spam. Without a matrix, you might look at overall accuracy and feel confident. But the matrix reveals that 10 legitimate emails were wrongly blocked and 20 spam emails slipped through. This distinction matters when you decide whether false positives or false negatives are more costly for your users.

What is a Confusion Matrix in Machine Learning?

At its core, a confusion matrix is a supervised learning evaluation tool used for classification problems. For binary classification, it takes the form of a 2x2 table. In standard libraries like scikit-learn, the rows represent the actual classes, and the columns represent the predicted classes. Some textbooks and tools flip these axes, so always read the axis labels before interpreting the data.

The table is built from four specific outcomes:

True Positive (TP): The instance is actually positive, and the model predicts positive.
False Negative (FN): The instance is actually positive, but the model predicts negative. This is a type II error.
False Positive (FP): The instance is actually negative, but the model predicts positive. This is a type I error.
True Negative (TN): The instance is actually negative, and the model predicts negative.

Understanding these cells is the first step to moving beyond simple accuracy. A false positive might annoy a user by blocking a real email, while a false negative might let a malicious message through.

Confusion Matrix Example: A Spam Filter

Let’s look at the specific numbers from our spam filter scenario. The dataset contains 1,000 emails: 100 are actually spam, and 900 are not. The model’s performance breaks down as follows:

	Predicted spam	Predicted not spam
Actual spam	TP = 80	FN = 20
Actual not spam	FP = 10	TN = 890

This table tells the full story. The model correctly identified 80 spam emails (TP). However, it missed 20 spam emails (FN) and incorrectly flagged 10 legitimate emails as spam (FP). It correctly identified 890 non-spam emails (TN).

Metrics You Can Calculate From It

Once you have the four cells, you can derive several key metrics that give a clearer picture of model performance than accuracy alone.

Accuracy: The proportion of correct predictions.
Formula: (TP + TN) / total
Calculation: 970 / 1,000 = 97%
Precision: Of the emails flagged as spam, how many were actually spam?
Formula: TP / (TP + FP)
Calculation: 80 / 90 ≈ 88.9%
Recall (Sensitivity/True Positive Rate): Of all the actual spam, how many did the model catch?
Formula: TP / (TP + FN)
Calculation: 80 / 100 = 80%
Specificity (True Negative Rate): Of all the non-spam emails, how many did the model correctly identify?
Formula: TN / (TN + FP)
Calculation: 890 / 900 ≈ 98.9%
False Positive Rate: The rate at which non-spam emails are incorrectly flagged.
Formula: FP / (FP + TN)
Calculation: 10 / 900 ≈ 1.1%
F1 Score: The harmonic mean of precision and recall, useful when you need a balance between the two.
Formula: 2 × precision × recall / (precision + recall)
Calculation: ≈ 0.842

These metrics help you weigh the cost of errors. If missing a spam email is worse than blocking a real one, you prioritize recall. If annoying users is the bigger risk, you prioritize precision. See our guide on precision and recall for deeper dives into these trade-offs.

Why Accuracy Alone Misleads

Accuracy is a deceptive metric when your data is imbalanced. In our spam example, accuracy is 97%, which sounds excellent. But imagine a simpler model that just labels every single email as “not spam.”

Total emails: 1,000
Actual spam: 100
Actual not spam: 900

A “lazy” model that predicts “not spam” for everything would have:

TN = 900
TP = 0
FP = 0
FN = 100

Its accuracy would be (0 + 900) / 1,000 = 90%. This is still high, but the model caught zero spam. Its recall is 0%. The confusion matrix exposes this failure instantly, whereas accuracy hides it. This is why evaluating on test data is crucial to ensure these metrics reflect real-world performance.

Threshold Dependence

Most classifiers don’t output a label directly; they output a score or probability. You then apply a threshold (e.g., 0.5) to decide if the prediction is positive or negative.

Changing this threshold shifts the values in all four cells. Lowering the threshold catches more spam (TP rises, FN falls) but also flags more legitimate mail (FP rises, TN falls). ROC curves and precision-recall curves help visualize performance across all possible thresholds. For a technical deep dive, check out area under the curve, AUC.

Multiclass Confusion Matrix

When you have more than two classes, the matrix expands to an N x N table. The diagonal cells represent correct predictions for each class. The off-diagonal cells reveal which classes the model confuses with each other. For example, a digit classifier might frequently confuse the number 3 with the number 8.

You can calculate per-class precision and recall from each row and column. Normalizing each row by its total gives you the recall for each individual class, helping you identify which specific classes are problematic.

Confusion Matrix in Python

Here is how you generate and visualize a confusion matrix using scikit-learn. The code assumes y_true contains the actual labels and y_pred contains the predicted labels.

from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay

cm = confusion_matrix(y_true, y_pred)  # rows: actual, columns: predicted
ConfusionMatrixDisplay(cm).plot()

# binary case with labels 0 and 1
tn, fp, fn, tp = confusion_matrix(y_true, y_pred).ravel()

This code uses the standard convention where rows are actual classes and columns are predicted classes. Always verify this alignment with your specific library version or documentation.

FAQ

What is a confusion matrix used for?

It is used to evaluate the performance of a classification model by showing the counts of true positives, false positives, true negatives, and false negatives. It helps identify specific types of errors that accuracy might hide.

How do you calculate precision from a confusion matrix?

Precision is calculated by dividing the number of true positives (TP) by the sum of true positives and false positives (TP + FP). It measures the proportion of positive identifications that were actually correct.

What does a false negative mean in a confusion matrix?

A false negative occurs when the actual class is positive, but the model predicts it as negative. This is also known as a type II error and is critical in scenarios like medical diagnosis where missing a disease is costly.

Why is the confusion matrix better than accuracy?

Accuracy treats all errors equally and can be misleading in imbalanced datasets. The confusion matrix breaks down errors by type, allowing you to weigh the cost of false positives against false negatives based on your specific business needs.