AssuranceProof that every agent works, before it ships and after.

Observability Agent DashboardAll agents 7d1m3m

Each agent, day by day Last 14 days

Prior auth intake		Healthy
Remittance matching		Healthy
Member outreach		Needs action
Vendor invoice coding		Watch
Claims appeal draft		Healthy

Ran fine Failed runs No runs

Why it failed

Opt-out reply misread58%
Portal timed out27%
Document unreadable15%

Needs attention 3

Got worseMember outreachSuccess 96% → 81% since v9 went live
Stopped runningVendor invoice codingNo runs in 11 days
Waiting on a personPrior auth intake3 requests past their deadline

Before it ships

A new version goes live only after it has been tested on your own cases and an admin approves it.

Experiments on your casesRun the new version and the live one on the same tasks, and see what it fixed and what it broke, check by check.
Release reviewA submitted version reaches its approver with its benchmark scores and a comparison to the version before it.
Blocking risksApproval is blocked while a risk stands: no benchmark run, no value to measure it by, or an unattended agent with no human-approval step.

See Benchmark

Approvals Member outreach · v10Needs approval

Compare to the prior version

	v9 live	v10
Benchmark score	0.88	0.94
Checks passed	11 of 13	13 of 13
Cost per run	2.4 OU	2.3 OU

Fixed: Opt-out honored passed 3 of 3 runs, the baseline 1 of 3. Broke nothing.

Before it can be approved

Benchmark run on the submitted versionMember outreach cases · 40 tasks, 3 runs each
Value driver mappedMembers reached
Runs unattended, no approval gateAdd a human-approval step before it sends messages

ApproveRequest changesCannot approve: a blocking risk stands.

In production

The Agent Dashboard shows every agent's runs, success, failures, cost and value, and flags the ones that need you.

Observability Agent DashboardOct 5 – Nov 4

Runs: 12,480; +8% on the prior month
Success rate: 97.6%; of all runs in the range
Failed: 296; 41 more than last month
Cost: 1,940 OU; 0.16 OU a billed run
Live agents: 5 of 8; 3 not live

Agent	Version	Runs	Success	Failed	Cost	Health
Remittance matching	v7	4,912	99.1%	44	610 OU	Healthy
Prior auth intake	v13	3,284	98.2%	59	702 OU	Healthy
Member outreach	v9	2,106	81.4%	152	318 OU	Needs action
Claims appeal draft	v4	1,830	97.9%	38	254 OU	Healthy
Vendor invoice coding	v3	348	99.4%	3	56 OU	Watch

HealthEach agent day by day, with the ones failing, getting worse, stopped or waiting on a person flagged first.
Costs and ROICost in Orchestration Units per agent, and the value each one delivers against your targets.
AlertsAn admin is alerted when a gate did not stop a run, a reviewer was never reached, or a check blocked an answer.

Every run, traced

Open any run and see every model call and tool call, in order, with its timing and what came back.

Run traceA step-by-step record of what the agent did and what each tool returned.
Ask ACTAVAAsk about a run in plain words, right from its trace.
Replay and exportPrepare a replay package, or download the run as JSON.

API Runs Prior auth intakeCompleted · 48s

Started: Today, 9:41 AM
Model calls: 3
Tool calls: 3
Cost: 2.1 OU

model: plan4.1s
read_referral7.6s
check_eligibility3.2s
model: match criteria10.4s
match_policy5.8s
model: draft12.3s

check_eligibility returned

Member active · plan rules loaded · prior authorization required for this service

Ask ACTAVA Prepare replay package Download JSON

When a release slips

Every release is measured against the version it replaced, and the last good version is one switch away.

Release logEvery version that went live, was rolled back or taken offline: who switched it, and why.
Better, steady or worseThe success rate before and after each switch, so a release that slipped shows on the day.
Roll backPut an earlier version back live. The log records the rollback and the reason given.

Agent Dashboard ReleasesLast 30 days

Went live: 3
Rolled back: 1
Taken offline: 1
Got worse: 1

Live versions by agent

Release log

TodayMember outreach v9 → v8Reason: Opt-out replies misreadRolled backBetter since: 81% → 96%
Nov 1Member outreach v8 → v9Went liveWorse since: 96% → 81%
Oct 28Remittance matching v6 → v7Went liveBetter since: 93% → 99%
Oct 24Benefits questions v2 → offlineTaken offline
Oct 21Prior auth intake v12 → v13Went liveSteady: 98% → 98%

One brake for everything

The emergency brake holds or stops every run in the organization at once, until an admin releases it.

HoldPause running work and keep pending approvals. Decisions made while held resume on release.
StopCancel running work and clear pending approvals. Nothing resumes on release.
Every run, on recordCovers ACTAVA, scheduled agents, evaluations and benchmarks. Only org admins engage or release it, and each brake is kept with its reason.

Security & Audit Emergency brakeRuns are allowed

Engage the emergency brake

Every run in this organization, including ACTAVA, scheduled agents, evaluations and benchmarks.

HoldPauses running work and keeps pending approvals. Decisions made while held resume when the brake is released.
StopCancels running work and clears pending approvals. Runs do not resume when the brake is released.

Choose a reason

Misconfiguration

Type Northwind Health exactly to confirm.

Northwind Health

Engage brakeCancel

History (2 brakes)

Oct 12Hold · External incidentReleased after 18 min

Ship agents you can stand behind

We'll walk you through a release review, the Agent Dashboard and the brake on a workflow like yours.