Eval suite
A set of graded test cases that score an agent's quality and gate releases against regressions.
category ▸ Operations
Eval suite, in plain language.
An eval suite is to an AI agent what a test suite is to ordinary software: a curated set of cases with known-good expectations, run automatically to measure quality and catch regressions before they ship. For agents, evals typically grade dimensions like faithfulness to source, citation coverage, and safety, not just exact-match correctness.
Evals are how AI quality stops being a matter of opinion. A green demo proves nothing about the next thousand runs; a passing eval suite, run as a release gate, gives you a defensible, repeatable basis for saying an agent is ready.
How Cortex implements it.
This term isn't abstract here — it maps to a real capability in the runtime. Here is exactly how Cortex enforces or relates to it.
Cortex runs gold eval suites as release gates and scores quality on every run — faithfulness, citation coverage, and safety — with live runs returning a quality { overall } reading that traces back to the run that earned it. A simulation lab lets you A/B prompts and models on real traffic before promoting a change.
Eval pass rate feeds the reliability score, so failing evals don't just show on a chart — they pull the score down and can hold a release at the publish gate.
Keep building the vocabulary.
These terms sit next to this one in the governed-AI model — follow the thread to see how the controls connect.
Reliability score
A 0–100 quality score per agent that gates which versions are allowed to publish.
Agent Studio
Where governed agents are built, chat-tested, and published — behind a reliability gate.
Policy-as-Code
Governance rules written as testable, versioned code that the runtime enforces directly.
Control Tower
The live operations console for the whole agent fleet — watch, gate, pause, or kill.
See Eval suite enforced, not just defined.
Join the waitlist and watch the controls in this glossary return real verdicts, seal real evidence, and trace every fact back to its source.