Practice: Evaluating LLM and agent systems
Generated by Claude through GigAI 0.1.11.10 on 2026-10-11.
Main tool: OpenAI Evals · OpenAI Evals puts datasets, code and model graders, human annotation, eval runs and trace grading for agents in one place, so every layer of this lesson can be practised against the same test set. · Alternatives: promptfoo (local, config-file driven, runs from the command line with no hosted account), Inspect AI (Python framework with strong agent and sandbox support), LangSmith and Braintrust (hosted tracing plus datasets and experiment comparison), the Anthropic Console evaluation tool (prompt-centric test cases in a web console), and pytest (plain code assertions with no built-in model graders or run history).
Every step links into the tool's own documentation; links were checked on 2026-10-11 · version 1
Setup (one time)
1. Install the OpenAI CLI on your laptop using the package installer named in the installation section.
The CLI lets you drive requests and collect outputs from a terminal, which you will reuse when you automate eval runs at the end of the path.
Explained in: OpenAI CLI (developers.openai.com), checked 2026-10-11
Done when: Asking the CLI for its version or help text prints a result in your terminal.
2. Create a dedicated project, a service account and an API key for your eval practice; note that this is a hosted API with usage billing, so keep the test set small.
A separate project keeps eval runs, datasets and spend apart from other work, and a service account key is what an automated job would use later.
Explained in: OpenAI CLI (developers.openai.com), checked 2026-10-11
Done when: A new project exists with a service account and you hold one API key scoped to it.
3. Configure CLI authentication with the new key through an environment variable, not a file in your repository.
Every later step calls the API, and keeping the key in the environment is the habit that carries over to CI.
Explained in: OpenAI CLI (developers.openai.com), checked 2026-10-11
Done when: The CLI reports that it is authenticated without you pasting the key into a command.
4. Send one first request from the CLI to confirm the whole stack works end to end.
A single successful call rules out key, project and network problems before you start building evals on top.
Explained in: OpenAI CLI (developers.openai.com), checked 2026-10-11
Done when: A model response to your test prompt is printed in the terminal.
Steps
1. Create a dataset by uploading a CSV of 15 to 20 realistic inputs with expected outputs for one task you know well, such as ticket classification.
A test set drawn from real cases is the foundation of every later measurement. Starting small keeps cost low and makes you read every row.
Explained in: Getting started with datasets | OpenAI API (developers.openai.com), checked 2026-10-11
Done when: The dataset shows your rows with an input column and an expected output column.
2. Write the success criteria for the task as three to five checkable statements, using the testing and verification guidance to decide what the model must prove about its own output.
Criteria written before any grader exists stop you from tuning the eval to whatever the model already does. Each statement becomes one grader later.
Explained in: Using GPT-6 | OpenAI API (developers.openai.com), checked 2026-10-11
Done when: You have a short written list where each criterion can be answered pass or fail for a single output.
3. Create an eval for the task by defining the data source schema and one code-style testing criterion that compares the output with the expected label.
The eval object, its item schema and its testing criteria are the core model of the tool. A deterministic check is the fastest and cheapest grader layer.
Explained in: Working with evals | OpenAI API (developers.openai.com), checked 2026-10-11
Done when: The API returns an eval identifier and the eval lists your schema and one testing criterion.
4. Load your test set into the platform as a JSONL file whose fields match the eval schema.
Persisting test data as a versioned file means every run is graded against exactly the same items, which is what makes runs comparable.
Explained in: Working with evals | OpenAI API (developers.openai.com), checked 2026-10-11
Done when: The upload returns a file identifier and the row count matches your local file.
5. Create an eval run that applies your prompt to the uploaded test data and grades each output.
A run ties one prompt and one model to one score, turning a vague sense of quality into a stored number.
Explained in: Working with evals | OpenAI API (developers.openai.com), checked 2026-10-11
Done when: The run reaches a completed status and reports pass and fail counts.
6. Add a model grader to the dataset that scores one criterion a string check cannot judge, such as whether the explanation is faithful to the input.
Model-graded checks cover quality dimensions that code assertions miss. You need to see where they agree and disagree with your own judgment.
Explained in: Getting started with datasets | OpenAI API (developers.openai.com), checked 2026-10-11
Done when: Each row shows a grader score next to the generated output.
7. Annotate every generated output yourself with a pass or fail rating and a short note, then compare your ratings with the model grader's scores.
Human labels are the reference that tells you whether a model grader can be trusted. The notes are the raw material for error analysis.
Explained in: Getting started with datasets | OpenAI API (developers.openai.com), checked 2026-10-11
Done when: You can name the rows where your rating and the grader's score disagree and say why.
8. Compare two eval runs of the same eval, one per prompt variant, and group the failing rows into named failure categories.
Error analysis on failures tells you what to change next. Comparing runs shows whether the change fixed that category without breaking another.
Explained in: Working with evals | OpenAI API (developers.openai.com), checked 2026-10-11
Done when: You have a short table of failure categories with a count per run and a decision on which prompt to keep.
9. Configure every run with an explicitly named model version instead of a default, and record it beside the prompt version.
A score only means something when you know exactly which model produced it. Pinning the model is the first requirement for a reproducible eval.
Explained in: Models and providers | OpenAI API (developers.openai.com), checked 2026-10-11
Done when: Rerunning the eval with the same pinned model and prompt gives a score close to the first run.
10. Run trace grading on a small agent that calls at least one function tool, with a grader that checks the tool calls and their order as well as the final answer.
For agents the unit under test is the whole trajectory. A correct final message can hide a wrong tool call or a skipped step.
Explained in: Evaluate agent workflows | OpenAI API (developers.openai.com), checked 2026-10-11
Done when: At least one trace is marked as failed because of a step inside the trajectory, not because of the final message.
11. Create a dataset from the agent cases you just debugged and run it as a repeatable eval run.
Traces are for understanding behavior, while datasets and runs are for proving that it stays fixed. This move is how a one-off finding becomes a regression test.
Explained in: Evaluate agent workflows | OpenAI API (developers.openai.com), checked 2026-10-11
Done when: The agent cases run as an eval run with a score you can compare across agent versions.
12. Write a small script that starts the eval run, retries transient failures, and exits with an error when the pass rate drops below your threshold; run it the way a CI job would, using only the service account key.
An eval in the delivery loop must run unattended and must separate a flaky API call from a real quality regression. This is the closest laptop equivalent of a gated pipeline.
Explained in: Errors and recovery | OpenAI API (developers.openai.com), checked 2026-10-11
Done when: The script exits cleanly on your good prompt and fails on a deliberately worsened prompt.