Quick reference · AI evaluations

A useful eval case, one field at a time

Copy this structure when adding an example to an evaluation set. Keep the expected result specific enough that another person can judge it consistently.

{
  "id": "stable-name-for-this-case",
  "input": "The request and any needed context",
  "expected_behavior": "What a good system must do",
  "must_include": ["required fact or action"],
  "must_not": ["harmful or unsupported behavior"],
  "case_type": "ordinary | edge | unsupported | adversarial",
  "source": "Where the expected answer came from"
}

Before you add a case

A case can test one or several requirements, but make the requirements explicit. An ambiguous case with no defensible expected behavior can make your score less trustworthy.

Return to Lesson 2 · Eval design checklist.