Eval suite
A set of graded test cases that score an agent's quality and gate releases against regressions.
category ▸ Operations
Eval suite, in plain language.
An eval suite is to an AI agent what a test suite is to ordinary software: a curated set of cases with known-good expectations, run automatically to measure quality and catch regressions before they ship. For agents, evals typically grade dimensions like faithfulness to source, citation coverage, and safety, not just exact-match correctness.
Evals are how AI quality stops being a matter of opinion. A green demo proves nothing about the next thousand runs; a passing eval suite, run as a release gate, gives you a defensible, repeatable basis for saying an agent is ready.
How Cortex implements it.
This term isn't abstract here — it maps to a real capability in the runtime. Here is exactly how Cortex enforces or relates to it.
Cortex runs gold eval suites as release gates and scores quality on every run — faithfulness, citation coverage, and safety — with live runs returning a quality { overall } reading that traces back to the run that earned it. A simulation lab lets you A/B prompts and models on real traffic before promoting a change.
Eval pass rate feeds the reliability score, so failing evals don't just show on a chart — they pull the score down and can hold a release at the publish gate.
Keep building the vocabulary.
These terms sit next to this one in the governed-AI model — follow the thread to see how the controls connect.
Reliability score
A 0–100 quality score per agent that gates which versions are allowed to publish.
Agent Studio
Where governed agents are built, chat-tested, and published — behind a reliability gate.
Policy-as-Code
Governance rules written as testable, versioned code that the runtime enforces directly.
Control Tower
The live operations console for the whole agent fleet — watch, gate, pause, or kill.
See Eval suite enforced, not just defined.
Book a walkthrough and watch the controls in this glossary return real verdicts, seal real evidence, and trace every fact back to its source.