Evaluating AI Models Beyond Accuracy

A correct answer is only the beginning

When a machine-learning model produces predictions, it is tempting to summarize performance with one number: accuracy. But accuracy is the proportion of all examples it classified correctly. That number can be misleading when important cases are rare or the costs of different mistakes are unequal.

Consider a fictional alert system that almost always says “nothing to report.” If only a tiny fraction of situations need an alert, the system might appear accurate while failing to detect the cases it was built to identify. The relevant question is not merely how often a prediction is right. It is which mistakes occur, for whom and under what conditions.

Understanding a confusion matrix

For a binary classifier, four outcomes are possible. A true positive is a correctly detected positive case. A false positive is an incorrect positive alert. A true negative is a correctly rejected negative case. A false negative is a positive case that the system missed.

Precision asks: of all the cases flagged positive, how many really were positive? It is calculated as true positives divided by true positives plus false positives. Recall asks: of all actual positives, how many were found? It is calculated as true positives divided by true positives plus false negatives.

When the denominator is zero, we cannot simply invent a percentage. A careful interface should tell the user when a metric is undefined for the selected threshold.

What happens when the threshold moves?

Many classifiers assign a score and label an example positive only if the score meets a threshold. Raising the threshold commonly reduces the number of positive predictions. On a fixed dataset this can reduce false positives, but it may also miss more actual positives. Lowering the threshold may capture more positive cases while flagging extra negatives.

Try the Classifier Trade-offs Lab to see the arithmetic. It uses a small fictional set of scores with known labels. Move the threshold, watch the confusion counts change, and explain why a different application could reasonably prefer a different operating point.

Generalization is a separate problem

A model can fit its training examples well while performing poorly on new cases. This is the problem of overfitting. Good evaluation separates training, validation and testing roles and checks whether the data resemble the situations in which predictions will be used. If populations, instruments or circumstances shift, historical scores may not describe future performance.

Model evaluation therefore connects engineering to ethics. A metric matters only within the system's real purpose, assumptions, possible harms and decision process. NIST's AI guidance emphasizes trustworthy behavior beyond a single predictive score, including transparency, fairness, security and accountability.

Use the model, question the measurement

Before trusting a performance claim, ask: What dataset was used? Which errors count? How were labels defined? Does the reported measure reflect the decisions people need to make? And what happens when the inputs change?

This page teaches foundational classification metrics. It does not endorse a particular commercial system, assess individual eligibility, or guarantee that any model is safe to deploy.