AI evaluations · Lesson 4

Measure the parts that matter

Estimated time: about 12 minutes

In plain English. One overall score can make a system look reliable while one important group of users still gets bad results. Break the score into the behaviors and situations that matter to your task.

A metric is a number that summarizes some part of performance. Accuracy is one familiar metric: the number of correct results divided by the number of results checked. It is useful only when “correct” is defined and the cases match the task.

Use a tiny example to expose a blind spot

An assistant answers 20 questions. It gets 18 ordinary questions right, then invents answers for the 2 questions that the source cannot answer. The overall score is 18/20, or 90%. But the two unsupported cases both failed.

Case groupCorrectCasesRate
Answerable questions1818100%
Unsupported questions020%
All questions182090%

Read a score table by asking: “What was counted, what is the denominator, and which cases are grouped together?” Always show the number of cases beside percentages, especially when a group is small.

Measure each stage of a system

For a retrieval-augmented generation (RAG) system, the assistant first retrieves source passages and then writes an answer using them. A correct final answer needs both steps to work, so measure them separately.

StageQuestion to measureExample check
RetrievalDid the needed source appear in the retrieved results?Recall at k: the share of expected sources found in the first k results
AnswerIs the answer correct and supported by the retrieved text?Task-specific rubric or reviewed answer criteria
End-to-end behaviorDid the user get an appropriate response?Correct answer, useful citation, or correct refusal when evidence is missing

If the source is missing, changing the writing prompt may not fix the real problem. If the right source is present but the answer is wrong, investigate how the system uses evidence. Stage-level measures help you choose the next place to look.

Compare carefully

Run the same cases before and after a change, and report quality alongside practical costs such as latency and provider errors. If the model can vary between runs, repeat the run before concluding that a small score difference is meaningful. For a small eval set, one changed answer can move the percentage sharply.

Metrics describe the checks you chose. They do not prove that the dataset covers every user, that a reference label is correct, or that future answers will behave the same way.

Practice: interpret the breakdown

Scenario. Retrieval finds the expected document in 19 of 20 questions. The final answers are correct in 14 of 20. What can you infer, and what should you inspect next?

Retrieval is likely not the only issue. Inspect the six answer failures and whether their sources were actually present. If the source was present, examine how generation used it; if not, inspect retrieval on those specific cases.

Further reading

OpenAI’s evaluation best practices recommends task-specific checks and combining numerical scores with human judgment. For an overview of relevant measures, use the eval design reference.

Practice

Answer before you reveal the explanation.

Quick recall

When you see an evaluation percentage, what should you check alongside the score?

Say your answer first, then reveal the explanation.

Quick recall

In a RAG system, why measure retrieval and answer quality separately?

Say your answer first, then reveal the explanation.

Quick recall

Why can a small difference in score be misleading on a very small eval set?

Say your answer first, then reveal the explanation.