Evals
Read recorded judge verdicts by check type, inspect one verdict and its rationale, and see what a proxy signal does not show.
Watch the workflow: Follow an authored proxy signal to its recorded trace · Authored dummy data
Filter one authored Task Success proxy in Tensile, read its recorded explanation and inspect its session trace. No evaluator or provider was run for this fixture. The exact historical input and rubric are unavailable, and the signal is not a judge result or customer outcome. No feedback is saved.
Read the walkthrough transcript
- 00:00 — Filter an authored Task Success proxy in Tensile. This signal is not a judge result.
- 00:05 — Open the recorded proxy. Its exact historical input and rubric were not recorded.
- 00:11 — Read the authored explanation. No evaluator or provider was run for this example.
- 00:18 — Follow the recorded session trace. Two events do not establish customer success.
Open Evals to read the verdicts that evaluators recorded for your sessions. Use it to see pass rates over time and to open a single verdict with its rationale.
Before you start, configure evaluators that produce recorded verdicts. For live evaluations, check Live evals in Settings → General and the evaluator prerequisites. The switch alone does not configure an evaluator. See Settings.
Set the scope
Use the time control, the customer filter and the page filters at the top. Use the save-view control to keep a filter for later. Select the information button, named About eval metrics, to read how the rates work.
Rates show recorded positive outcomes, including task-success proxies. A proxy signal is not a judge Pass or Fail. A held, unset or failed-to-run evaluation is not a rubric Fail.
Read the cards and charts
- Quality judge pass rate and Avg judge quality score summarize Quality verdicts.
- Safety judge pass rate summarizes Safety verdicts.
- Quality judge-human agreement compares judge verdicts with human labels. It shows a dash when no label exists.
The charts show the pass rate and the average score for each day. Each check type has its own line: Quality, Safety, Intent, SOP and Task Success. Legacy Custom verdicts do not appear in these charts.
Inspect a verdict
- Select a row in the verdicts table. A details pane opens.
- Read the check type, the result, the session and the version history of the evaluator.
- Read the rationale. The page loads it for the selected verdict only. Select Retry if it fails to load.
A judge pass means the rubric criteria were met. It does not confirm customer success. For the definition behind a verdict, open Evaluator definitions. For task verdicts, open Task verdicts and evidence assessments.
Empty and failed states
- A card shows a dash and a message such as No judged Quality results when no verdict exists.
- If page filters hide every verdict but the scope has some, the page says so and lets you remove each filter.
- If a query fails, an error state appears.
- A verdict from an older record can lack an identity. The pane then says that the rationale is unavailable.
Environment tests
Open Environments to inspect registered Slack or Gmail scenarios. Use Scenarios to find a test case or Run history to open a saved run. Search or filter the collection, then open the full record. A queued request is not a completed test. Missing or stale evidence stays Unknown.
A signed-in workspace owner or admin can select Run test for an active, registered scenario. The runner must be available. If no scenarios are registered, the collection shows an empty state. If a source read fails, select Retry after the source is available.
Open Settings → Environments → Slack or Gmail for setup requirements. A workspace owner must register a versioned scenario and approved intent. The operator must configure its isolated runner. These tests do not require a customer provider login or connect to a live mailbox or Slack workspace.
Gmail run records show expected versus observed results, recorded actions, the first deviation and evidence gaps. They keep task, policy, environment, reset and cleanup results separate. Open technical details for the exact versions, or download the selected run evidence.
Compare saved Gmail runs only when their starting conditions and expected results match. A mismatch explains why comparison is unavailable. Scripted fixture results do not establish live model behavior or provider acceptance. Outlook adapters and scheduled environment tests are not available.
Next steps
Open Evaluator packs to review the checks that produce these verdicts.