What Is AgentOps? The Continuous Evaluation Model for Agentic AI
You hired the agent. Now who does its performance review? AgentOps is the operational discipline for running AI agents in production: monitoring performance, detecting drift, managing thresholds, and evaluating outcomes continuously rather than once at go-live.
Most healthcare AI programs treat evaluation as a gate. The agent gets tested, the agent passes, the agent ships. Everyone moves on to the next use case.
That model works for software that does the same thing every time. It does not work for an agent that reasons, calls tools, and produces a slightly different trajectory on every run. An agent that scored 0.91 on task completion in March is not guaranteed to score 0.91 in September. Nothing about the model changed. The world it operates in did.
AgentOps is the answer to that problem. It borrows the shape of DevOps and MLOps, then adds the thing neither one had to solve: agents make decisions, and decisions need supervision that never stops.
Non-human resources
Here is a framing that lands with most operations leaders faster than any architecture diagram.
Every agent you deploy is a worker. It has a job description, a scope of authority, a set of tools it is allowed to touch, and a performance standard. You would never hire 300 people and then decline to manage any of them. Yet plenty of health systems are on track to have more deployed agents than employees inside three years, with no equivalent of an HR function to govern them.
Call it non-human resources, if you will accept the turn of phrase. Someone has to own the hiring, the onboarding, the evaluation, and the offboarding of every agent in the estate.
AgentOps is that function. And continuous evaluation is the performance review that runs every single turn instead of once a year.
The four moving parts
Strip AgentOps down and it comes to four jobs that have to run at the same time, forever.
Monitor
Per-agent cost, latency, tool errors, and volume, traced at the level of the individual turn.
Evaluate
Score live production output against defined metrics, not just pre-launch test cases.
Detect drift
Catch the moment measured behavior separates from the behavior you validated.
Govern
Enforce policy, write the audit trail, and route the exception to a named human.
Miss any one of the four and the other three quietly stop being useful. Monitoring without evaluation tells you the agent is fast, not that it is right. Evaluation without governance produces a score nobody is accountable for.
How continuous evaluation works in ACTAVA
Continuous evaluation is not a feature bolted onto one product. It runs through the whole ACTAVA suite, and it maps onto the four pillars the platform was built around: Build, Test, Learn, Guide.
The agent lifecycle starts here
Every agent built in ACTAVA KORA gets a lifecycle the way a new hire gets a file. Job description, scope, handbook, measurement model. Deployed agents land in a governed registry where each one is attested, so nobody is guessing which agents exist or who approved them.
A runtime safety layer sits underneath: multi-approver human-in-the-loop governance, admin-configurable write gates, and HIPAA-aware safeguards on what an agent may read and change.
Adversarial testing before anything touches a patient
Agents get red-teamed continuously in ACTAVA KORA, with hallucination and bias detection and regulatory validation calibrated for healthcare scenarios rather than generic chat.
Evaluation runs against a defined metric collection: Role Adherence, Task Completion, Tool Correctness, Faithfulness, Answer Relevancy, and Safety. Each metric carries a threshold. Testing is a named lifecycle stage, not a phase that ends.
The agent gets better between releases
Once an agent runs, in test or in production, the KORA learning layer turns what happened into improvement: trajectory-driven optimization, dataset synthesis, and versioned upgrades.
This is the part that separates an agent from a script. A script is as good on day 400 as it was on day 1. An agent under a learning layer improves from release to release, and the version history shows exactly what changed and why.
Governance that runs while the agent runs
ACTAVA Compliance is the agentic governance and compliance suite: continuous policy enforcement, audit trails, and AI standards applied to the live estate.
The govern dashboard surfaces the operational signals an executive actually needs, including uptime, drift, and request volume across the last 24 hours. Drift as a first-class number on the dashboard is the AgentOps difference in one glance.
Thresholds are two different things
This trips up most teams, so it is worth separating cleanly. In ACTAVA, threshold management means two distinct mechanisms, and they answer different questions.
| Threshold type | Question it answers | Who owns it |
|---|---|---|
| Evaluation threshold Set per metric, at metric creation and at the collection level |
Did this turn pass or fail? Is Faithfulness above the bar we set for this agent? | Agent builders, clinical informatics, the AI governance committee |
| Attainment threshold Baseline, target, and direction on each ROI metric |
Is this agent on track, regressing, or in need of attention against the business case? | Finance, operations, the executive sponsor |
An agent can pass every evaluation threshold and still miss its attainment threshold. That agent is behaving correctly and delivering nothing. Without both numbers on the same page, you find out at renewal.
Drift, precisely
Drift is the most abused word in this category, so here is how we use it. Two distinct things get called drift, and conflating them wastes real engineering time.
Infrastructure drift is the environment moving away from the configuration you declared. The deployed state and the declared state disagree, and the deployed state wins. It is detected by comparing the two directly. Unglamorous, and the cause of a surprising share of production incidents.
Behavioral drift is the agent moving away from the behavior you validated. It is caught by the evaluation loop: score production turns against their thresholds continuously, then flag the regression through a status engine that says on-track, regressing, or needs-attention.
Note what that second definition does not claim. It is not a statistical model-output drift detector in the classic machine learning sense. It is measured behavior against a defined standard, which for a healthcare agent is the more useful signal anyway. Nobody in a compliance review wants a distribution distance. They want to know whether the agent still does the job the way it was approved to do it.
Evaluation on production traffic, not just test cases
The word doing the work in continuous evaluation is continuous. Pre-launch benchmarking is table stakes. What matters is what happens on turn 40,000.
In ACTAVA, evaluation is wired into the agent runtime. After a turn completes, the system looks up that agent's metric collection, filters to the active metrics, builds a test case from the actual trajectory, and scores it in context, with session, agent, version, query, and response all carried through.
Two design choices matter here.
- The same rubric works for single-turn and multi-turn. A metric written once evaluates a one-shot answer and a 20-turn conversation, scoped to either the final turn or the full conversation. You are not maintaining two evaluation systems.
- Sampling is configurable per metric. Evaluate every turn where the stakes justify it, or a fraction of traffic where they do not. Continuous evaluation should not cost more than the agent it evaluates.
Three kinds of metric cover most of what a health system needs to check.
| Metric type | How it judges | Good for |
|---|---|---|
| Rubric | LLM-as-judge against weighted criteria, at the metric or item level, including negative points for known failure patterns | Quality calls that need reasoning: tone, completeness, grounding in the retrieved record |
| Python | Custom code with full access to conversation context | Deterministic checks: was the required field populated, was the correct tool called, did the total reconcile |
| Plan compliance | LLM-as-judge against the system prompt or plan, returning a structured checklist of which items were completed | Multi-step workflows where finishing the job matters more than any single answer |
When a new agent is created with a plan or task list defined, a plan compliance metric attaches automatically. The default is that the agent gets measured, not that someone remembers to set it up.
Outcomes, not just behavior
Behavioral scores tell you the agent is doing the job correctly. They say nothing about whether the job was worth doing.
That is why outcome capture runs alongside evaluation. Business outcomes are captured automatically from agent runs and rolled up across windows from hourly to monthly, with each metric reporting how much of its value came from runtime measurement, emitted events, or manual entry.
Cost displacement is the clearest example. Rather than asserting a number in a slide, the calculation is explicit: events completed, multiplied by minutes saved per unit, converted to hours, multiplied by the loaded hourly rate, multiplied by an attribution coefficient that says how much of the saving the agent is honestly responsible for.
The attribution coefficient is the honest part. If a number falls outside a valid range, the metric is treated as misconfigured and skipped rather than reported. An ROI figure you cannot defend line by line is worse than no figure at all.
What to ask your vendor, and yourself
If you are evaluating agentic AI right now, these five questions separate a demo from a production system.
- What percentage of production turns get evaluated, and can you show me the metric definitions?
- Where is the threshold set for each metric, and who has authority to change it?
- How would I learn that this agent regressed, and how long would that take?
- Which named human reviews the exceptions, and what is the audit trail they leave?
- What business outcome is this agent credited with, and how much of that credit is attributed to the agent versus the process change around it?
A vendor who can answer all five is running AgentOps. A vendor who answers with model benchmarks is selling you a pilot.
The part worth taking away
Agentic AI does not fail in healthcare because the models are not good enough. It fails because organizations deploy something that makes decisions and then supervise it the way they supervise a report.
Continuous evaluation closes that gap. Score the work as it happens. Set thresholds someone owns. Watch for drift in both senses. Tie behavior to outcomes you can defend. Keep a human on the exceptions.
That is the whole discipline. The agents are already in the building. The question is whether anyone is managing them.
See continuous evaluation running
ACTAVA is the AI factory for healthcare. Master your agentic future. If you want to see how evaluation, thresholds, drift, and governance work on a live agent estate, we will walk you through it on your own workflow.
