In plain English. An eval is most useful when it changes what you do next. Treat failures as clues: inspect the examples, find the likely cause, fix that cause, and rerun the same checks.
A failed case is not just a lower score. It is a concrete example of where the system missed an expectation. Review the input, the system’s intermediate steps when available, the output, and the grader’s reason for marking it wrong.
Find the part that failed
Suppose a document assistant gives a wrong answer and cites a source. The failure could come from a missing document, a poor retrieval result, an incorrect reading of the source, or a grader that misunderstood the answer. Each cause calls for a different change.
What you observe
What it may mean
What to check next
Expected source is absent
Retrieval or indexing problem
Query, indexed text, and retrieved passages
Source is present but the answer contradicts it
Generation or instruction-following problem
Prompt, context, and the exact unsupported claim
Answer seems acceptable but grader says fail
Rubric or grader problem
Compare with human judgment and clarify the criterion
Provider request failed or timed out
Service reliability issue
Report it separately from answer quality
Keep a simple improvement loop
Run the eval set and save the version of the system and settings.
Inspect failures and group them by likely cause.
Make one focused change based on the evidence.
Rerun the full set, not only the case that failed.
Check that the target behavior improved and important other behaviors did not regress.
Add a useful, reviewed failure case to the regression set.
A regression set is the collection of past cases you rerun after changes to catch previously fixed problems returning. Keep some new or held-back examples for a less biased comparison when making bigger decisions. If you tune repeatedly against every case, the score can become a measure of how well you fit those cases rather than how well the system handles new requests.
Keep checking as the system changes
AI results can vary, and real user requests change over time. Run the checks after meaningful changes to prompts, models, tools, or data. When new failure patterns appear, add good examples. Watch live quality signals and human feedback too; a fixed eval set cannot predict every new situation.
OpenAI’s best-practices guide describes evaluation as an ongoing process and recommends using human feedback to calibrate automatic scoring. The same idea applies whether your checks run by hand, in a command line, or in continuous integration.
Practice: choose a next step
Scenario. After a prompt change, the assistant stops making unsupported claims but now refuses many answerable questions. What is the next useful action?
Inspect the new refusal failures, compare them with the system’s prior behavior, and adjust the prompt based on the cases. Then rerun the full eval set to check both unsupported-question behavior and answerable-question accuracy.
Further reading
See the sections on iteration and ongoing evaluation in OpenAI’s evaluation best practices. The platform-specific Evals API is being deprecated, so use the guidance here as a process, not a requirement to use that API.