Reliability score
A 0–100 quality score per agent that gates which versions are allowed to publish.
category ▸ Operations
Reliability score, in plain language.
A reliability score condenses an agent's measured quality into a single comparable number, so you can tell at a glance whether an agent is performing well and whether it is good enough to ship. It is computed from signals the runtime already collects rather than from a one-off benchmark, which makes it a live reading instead of a stale label.
The point of the score is to make it actionable: instead of a chart someone glances at, it becomes a gate. An agent below the bar — with enough evidence to judge it fairly — is refused at publish, so a regression cannot slip into production silently.
How Cortex implements it.
This term isn't abstract here — it maps to a real capability in the runtime. Here is exactly how Cortex enforces or relates to it.
Cortex blends success rate (0.5), eval pass rate (0.3), and incidents (0.2) over a 30-day window into a 0–100 score with bands: excellent ≥85, good ≥70, fair ≥50, poor <50. Trend snapshots record how the score moves over time.
The registry refuses to publish an agent that is sufficient (enough run signal) and scores below RELIABILITY_MIN_PUBLISH — returning 409 RELIABILITY_TOO_LOW. New or low-data agents publish freely, and a scoring outage fails open so availability never blocks ops.
Keep building the vocabulary.
These terms sit next to this one in the governed-AI model — follow the thread to see how the controls connect.
Eval suite
A set of graded test cases that score an agent's quality and gate releases against regressions.
Agent Studio
Where governed agents are built, chat-tested, and published — behind a reliability gate.
Control Tower
The live operations console for the whole agent fleet — watch, gate, pause, or kill.
Risk tier
A classification of how consequential an agent or action is, used to set the controls it needs.
See Reliability score enforced, not just defined.
Book a walkthrough and watch the controls in this glossary return real verdicts, seal real evidence, and trace every fact back to its source.