Precision and Recall: Definitions, Formulas, and Trade-offs

Precision measures the proportion of positive identifications that were actually correct, while recall measures the proportion of actual positives that were correctly identified. Together, they provide a nuanced view of model performance, especially when class distributions are uneven.

While accuracy gives a broad overview, it often hides critical flaws in imbalanced datasets. For instance, a model predicting “no spam” for every email might achieve 90% accuracy but completely fail to catch any spam. This is where understanding the distinction between false positives and false negatives becomes essential for selecting the right metric.

Precision and recall formulas

To calculate these metrics, you need the counts of true positives (TP), false positives (FP), and false negatives (FN) from a confusion matrix.

Precision is calculated as:

precision = TP / (TP + FP)

This answers the question: “When the model says yes, how often is it right?” High precision means that when the model predicts a positive class, it is usually correct. It minimizes the risk of false positives, also known as Type I errors.

The formula for recall (also called sensitivity or true positive rate) is:

recall = TP / (TP + FN)

This answers: “Of everything it should have found, how much did it find?” Recall measures the model’s ability to capture all actual positive instances. It relates directly to false negatives and Type II errors.

Worked example

Consider a spam filter tested on 1,000 emails, where 100 are actually spam. The model flags 90 emails as spam, and 80 of those are truly spam.

From this scenario, we derive the following counts:

TP = 80
FP = 10 (flagged as spam but were actually not spam)
FN = 20 (actual spam that was missed)
TN = 890 (correctly identified as not spam)

Plugging these counts into the formulas:

Precision = 80 / 90 ≈ 88.9%
Recall = 80 / 100 = 80%

This example illustrates precision vs recall in practice. The model is quite precise but misses one-fifth of the actual spam.

The precision-recall trade-off

Most classifiers output a probability score rather than a binary label. A threshold determines whether that score becomes a “yes” or “no.” Adjusting this threshold creates a direct trade-off between precision and recall.

Raising the threshold makes the model more conservative. It predicts “positive” only when very confident, which typically increases precision but lowers recall because more actual positives are missed. Lowering the threshold makes the model more aggressive, catching more positives but also increasing false alarms.

Using the same spam data, if we lower the threshold, the model might flag 150 emails. Of these, it catches 95 of the 100 spam emails.

TP = 95
FP = 55
FN = 5

In this scenario:

Precision drops to 95 / 150 ≈ 63.3%
Recall rises to 95 / 100 = 95%

This demonstrates the precision-recall trade-off: you gain recall but sacrifice precision.

When to prioritize precision or recall

The choice depends on the cost of errors in your specific use case.

Prioritize recall when missing a positive instance is costly. Examples include:

Cancer screening: Missing a tumor (false negative) is worse than an unnecessary biopsy (false positive).
Fraud detection: Catching every fraudulent transaction is critical, even if it means investigating some legitimate ones.
Safety-critical defect detection: Missing a defective part can lead to product failures.

Prioritize precision when a false alarm is costly or annoying. Examples include:

Spam filters: Users lose trust if legitimate emails are buried in the spam folder.
Recommendations: Showing irrelevant items reduces user engagement.
Alerts: Frequent false alarms lead to “alert fatigue,” where users start ignoring warnings.

F1 score and F-beta

When you need a single metric to compare models, the F1 score is commonly used. It is the harmonic mean of precision and recall:

F1 = 2 × precision × recall / (precision + recall)

The F1 score is high only when both precision and recall are high. It balances the two metrics equally.

For cases where you want to weight one metric more than the other, you can use the F-beta score:

F_beta = (1 + beta²) × precision × recall / (beta² × precision + recall)

If beta > 1 (e.g., F2), recall is weighted more heavily.
If beta < 1 (e.g., F0.5), precision is weighted more heavily.

Precision-recall curve

A precision-recall curve plots precision against recall for every possible threshold. This visualization helps you choose the optimal operating point for your classifier.

The area under this curve is often summarized as Average Precision (AP). On heavily imbalanced datasets, the precision-recall curve is usually more informative than the ROC curve because it focuses exclusively on the positive class. For a deeper look at related metrics, see area under the curve (AUC).

In information retrieval and ranking systems, these metrics are often adapted as precision@k and recall@k, measuring quality within the top k results. This is critical in information retrieval and learning to rank applications.

Calculating precision and recall in Python

You can easily compute these metrics using scikit-learn. The following code calculates standard scores and generates the precision-recall curve data.

from sklearn.metrics import precision_score, recall_score, f1_score, precision_recall_curve

precision = precision_score(y_true, y_pred)
recall = recall_score(y_true, y_pred)
f1 = f1_score(y_true, y_pred)

# y_scores: predicted probabilities or scores for the positive class
prec, rec, thresholds = precision_recall_curve(y_true, y_scores)

FAQ

What is the difference between precision and recall?

Precision measures the accuracy of positive predictions, while recall measures the coverage of actual positives. Precision answers “how many selected items are relevant?” and recall answers “how many relevant items are selected?”

Why is accuracy not enough for imbalanced datasets?

Accuracy can be misleading when classes are imbalanced. A model that predicts the majority class for all samples can achieve high accuracy but zero recall for the minority class.

What is the F1 score used for?

The F1 score provides a single metric that balances precision and recall. It is useful when you need to compare models without favoring one metric over the other.