BenchmarksScore every agent version on real tasks before it ships.
Create with ACTAVA
Test the intake agent on retro-authorizations and out-of-network imaging, using last month's cases.
DraftDrafted 4 tasks from 312 cases · each with a checker
retro_auth_after_admission_01Request two days after admission; check timely-filing rulesoon_imaging_nearby_in_network_01Out-of-network MRI with an in-network center nearbyretro_auth_missing_order_01Retro request with no physician's order attachedoon_imaging_continuity_01Out-of-network imaging for an ongoing course of care
- Turn last month's appeals into tests
- Add the cases our agent failed
- Import a benchmark ZIP
Benchmark Library
Test agents on the work they actually do, and on healthcare benchmarks the field already uses.
- Built-in benchmarksThousands of healthcare tasks, from long prior auth workflows to clinical questions, ready to run.
- Create with ACTAVADescribe what to test or attach your cases, and ACTAVA drafts the tasks and asks about what is missing.
- Import your ownBring an existing benchmark as a ZIP. It keeps its own checks and metrics.
- Prior auth casesyours1206
- CHI-Benchbuilt-in752
- HealthBench Hardbuilt-in1,0000
- HealthBench Professionalbuilt-in5250
- MedXpertQA Textbuilt-in2,4500
Task Map
See what your agents are actually asked to do, then turn it into a benchmark.
- Workflows and clustersProduction runs grouped into workflows and the kinds of case inside them.
- Outliers worth a testRare or failing cases are flagged, so they can be decided on, not lost.
- Benchmark proposalEvery cluster gets a test, with its input and what a good result looks like.
Task Map
Prior auth intake · last 30 days · 5,840 runs
- Referral missing the physician's order38%97% completed
- Out-of-network imaging request23%94% completed
- Retro-authorization after admission17%88% completed
- Duplicate submission9%99% completed
Outliers
- Request for a drug the plan doesn't coverInclude
- Same patient, two requests in one hourInclude
- Test request from the payer's sandboxLeave out
Every version, scored
Run an agent version against a benchmark, and see it beside every version before it, stage by stage, on the same tasks.
Prior auth intake on Prior auth cases · 75 tasks · reward 0 to 1
| Version | What changed | Intake | Triage | Nurse review | MD review | P2P | Reward | Cost |
|---|---|---|---|---|---|---|---|---|
| v9 | First plan | 0.74 | 0.70 | 0.58 | 0.55 | 0.48 | 0.61 | 188 OU |
| v10 | + Policy lookup tool | 0.78 | 0.74 | 0.66 | 0.60 | 0.52 | 0.66 | 196 OU |
| v11 | Model: Opus 5 · medium | 0.80 | 0.79 | 0.70 | 0.66 | 0.55 | 0.70 | 214 OU |
| v12 | Skill: peer-to-peer scheduling | 0.81 | 0.80 | 0.77 | 0.75 | 0.77 | 0.78 | 209 OU |
| v13 | Prompt: cite the criteria | 0.86 | 0.85 | 0.84 | 0.83 | 0.82 | 0.84 | 212 OU |
v13 against v12: +0.06 reward on the same 75 tasks, 74 evaluated, one candidate error to diagnose.
- Change one thingCompare agent versions, models, or a skill or prompt change, on a fixed sample or every task.
- Every case explainedOpen any case for its reward, runtime, cost and its checker's verdict, or diagnose a failure.
- Spend you can seeCost per experiment, a cap on what a run may spend, and completion over time.
Benchmark your agents with us
Bring one workflow. We'll map what your agents do, draft the benchmark, and run your first experiment.