Reliability score
A 0–100 quality score per agent that gates which versions are allowed to publish.
category ▸ Operations
Reliability score, in plain language.
A reliability score condenses an agent's measured quality into a single comparable number, so you can tell at a glance whether an agent is performing well and whether it is good enough to ship. It is computed from signals the runtime already collects rather than from a one-off benchmark, which makes it a live reading instead of a stale label.
The point of the score is to make it actionable: instead of a chart someone glances at, it becomes a gate. An agent below the bar — with enough evidence to judge it fairly — is refused at publish, so a regression cannot slip into production silently.
How Cortex implements it.
This term isn't abstract here — it maps to a real capability in the runtime. Here is exactly how Cortex enforces or relates to it.
Cortex blends success rate (0.5), eval pass rate (0.3), and incidents (0.2) over a 30-day window into a 0–100 score with bands: excellent ≥85, good ≥70, fair ≥50, poor <50. Trend snapshots record how the score moves over time.
The registry refuses to publish an agent that is sufficient (enough run signal) and scores below RELIABILITY_MIN_PUBLISH — returning 409 RELIABILITY_TOO_LOW. New or low-data agents publish freely, and a scoring outage fails open so availability never blocks ops.
Keep building the vocabulary.
These terms sit next to this one in the governed-AI model — follow the thread to see how the controls connect.
Eval suite
A set of graded test cases that score an agent's quality and gate releases against regressions.
Agent Studio
Where governed agents are built, chat-tested, and published — behind a reliability gate.
Control Tower
The live operations console for the whole agent fleet — watch, gate, pause, or kill.
Risk tier
A classification of how consequential an agent or action is, used to set the controls it needs.
See Reliability score enforced, not just defined.
Join the waitlist and watch the controls in this glossary return real verdicts, seal real evidence, and trace every fact back to its source.