Evaluator definitions
Inspect current tracked definitions, approved prompt templates, and the configuration available for recorded verdicts.
Watch the workflow: Inspect the current read-only Quality definition · Authored dummy data
Open the current bundled Quality definition in Tensile, read its rubric and system instructions, then return to the configuration catalog. The template does not establish the governing prompt of a past verdict or the deployed settings. This clip uses an authored dummy workspace and runs no evaluation.
Read the walkthrough transcript
- 00:00 — Open the Quality definition in Tensile. It does not prove past execution.
- 00:05 — Return to the catalog. Configured definitions do not establish the deployed settings.
Where Executions is available, tracked intent definitions are also read-only records under Executions → Check library → Legacy definitions; use View read-only intent definition on an intent's entry here to open that record. Where it is not, this page shows each tracked intent's recorded-label count and the labels without a tracked definition. See Intents.
An evaluator definition is one prompt or rule used to judge a result. An evaluator pack is a versioned decision definition with evidence requirements and answer criteria, reviewed against source examples. The definitions page is read-only; it shows current workspace definitions and bundled runtime templates, not a verified inventory of deployed settings.
Configured intents are listed here even when they have no message labels. A tracked intent's recorded-label count describes labelled events within the selected time, customer, environment and classification filters. It is separate from evaluator verdicts. Labels carry only a name: renaming a definition does not rename recorded labels. Duplicate names or rubric keys have an explicit unknown or ambiguous relationship instead of a guessed attribution.
Read the current configuration
The catalog contains workspace intent and rule definitions alongside bundled runtime definitions for Quality, Safety, and Task success. Bundled source identifies code packaged with the viewer. Configured means the workspace source definition exists; it does not prove that a deployed worker executed it. The workspace’s continuous-evaluation setting is shown separately. From Evals, use Browse evaluator definitions in the top-right navigation, alongside Evaluator Packs and User Stories, to open this current catalog.
Select Quality in the catalog to inspect the approved generic whole-input system instructions and request template. A recorded verdict can link to the matching Quality prompt when its saved version verifies. The conversation is represented by a placeholder; private conversations are not included. Safety, SOP and Task success verbatim prompts are withheld pending a product/security decision. Definitions are read-only.
Source, status, creation date where recorded, and one-shot rubric version where available explain the current entry. Missing owner, definition version, last update date, and deployed model remain Unknown. A creation date is not an update date, and a bundled version alone does not prove a historical match.
Inspect a recorded verdict
Evals shows a judge verdict alongside its rubric and keeps the recorded model separate. It matches the complete persisted verdict receipt against organization-scoped input history; a logical verdict ID alone is insufficient. When a unique recorded one-shot Quality version matches the bundled template, View matching Quality prompt (read-only) opens that template. Deployment status remains unknown. If the bundle later changes, the link shows the recorded version as unavailable and offers current configuration separately.
The details panel explains whether version history is unavailable, ambiguous, not recorded, or points to a definition that cannot be inspected. A history lookup failure does not remove the saved pass/fail result. For other cases, the link Browse today’s definitions (read-only) opens current configuration; it does not show or change the configuration used for the historical verdict. Name-joined message labels do not retain the historical definition version.
Safety and Task success show explicit summaries while their verbatim rubrics remain withheld. Scoring and eligible input differ by kind. Task-success proxies appear as neutral Proxy positive/negative signals and have a separate list filter; they are not judge Pass/Fail. The compact verdict explanation appears on hover or keyboard focus. Held, unset and failed-to-run evaluations are not rubric Fail; a judge Pass alone does not prove customer success.
A configuration source failure is shown separately from the saved verdict. Reload to retry. An unavailable source is not treated as an empty configuration, and an unavailable selected definition is not replaced by a similarly named record. Click anywhere on a verdict row to open its inspector, or use its details button with the keyboard. The judge result sits beside the inspector title; View session is directly beneath it when a source timestamp is available. The raw session ID is under Technical details. On small screens, use Back to list; keyboard users can open the details button and explanation with Enter and close the inspector with Escape.
Quality evidence
When Quality evidence inspection is enabled, Evals shows how many sessions were Judged, have Missing evidence, are Processing, are Held, or have an Error. Quality pass rate uses judged sessions as its denominator. A session without enough recorded agent input or output receives no Quality score.
The filters, including the time period, are together above the cards. Select the Review sessions without a Quality score card to open a searchable session table in the right pane. Read each status and reason, then select a session to open its recorded work. Closing the pane returns to the same filtered overview. Source-identified evaluator spans and their children are excluded from the agent evidence. Background agent work can still qualify; a chat message is not required when the source records the task input and output.
Quality (agent evidence) identifies the matching bundled prompt for these judgments. Historical cleanup retains original verdicts and records corrected judgments or evidence states. An unavailable saved verdict shows Error for that session; publication lag shows Processing. A source/store outage shows a read error. Older scores never become the current result during these states. When versions differ, Quality pass-rate and average-score cards are paired under a scoring-version heading, with separate rates, and original historical scores stay separate until cleanup. Cleanup does not establish human calibration or task completion across sessions.