Sign in

Engineering

Continual learning from production runs

Production trajectories become simulation environments and a task map of real work, supplying executable tasks for benchmarking and model training.

By Weiran Yao

5 min readNovember 2, 2026

A production run can contain the next task an agent needs to learn. In an illustrative authorization case, a second request arrives while the existing episode is waiting for a physician’s order. A reviewer corrects the draft so the agent continues that episode, preserves the pending status, and prepares the follow-up.

ACTAVA uses trajectories like this to reconstruct simulation environments and build Task Map coverage. The environment recreates the records and tools involved in the decision. The map organizes the workflow conditions that deserve their own tasks. Together, they turn production experience into executable material for benchmarking and model training.

Continual learningProduction experience to executable tasks
Production trajectory
1
Incoming requestSame episode as R-418
2
Tool observationsEarlier order is pending
3
Reviewer correctionContinue the existing case
Case recordsTool definitions
Simulation environment
Initial state
RequestEpisodeState
R-418E-042Order pending
R-419To linkIncoming
read_caselink_requestsave_followup
Restore state for each attempt
Task Map
Prior authorizationMissing orderDuplicateClosed case
Duplicate requestEarlier episode is waiting for an orderSelected
Executable task familiesDuplicate requests under different case conditions
EnvironmentInstructionChecker
Benchmark tasksCompare agent versions on reserved evaluation tasks
Run experiment
Training tasksPractice selected cases and learn from outcome reward
Model Factory
Illustrative production-learning pipeline: run evidence supplies a simulation environment, Task Map organizes the case families, and executable tasks support benchmarks and model training.

Reconstruct the decision the agent faced

The trajectory contains the request, tool calls and responses, artifacts, and review events. Authorized source material adds the records and tool definitions needed for reconstruction. These inputs describe the information available to the agent and the actions that could change the case.

For the duplicate-request episode, the simulation holds the earlier request, the incoming duplicate, and the outstanding document requirement. Reconstructed tools let an agent look up the episode, link the request, and save a follow-up. Their responses reflect the environment’s state as each action runs.

A reusable starting state makes that decision available for repeated attempts. One candidate may open a new case; another may find the existing episode. The resulting records expose which continuation each candidate chose and whether the document status was preserved.

Task Map reveals the variations worth practicing

Production runs arrive as individual episodes. Task Map groups them into workflows and case clusters, giving the team a view of coverage. An authorization workflow can include duplicate submissions, missing-document follow-ups, and requests requiring another review. Outliers identify less common conditions to represent.

Within the duplicate-request family, the earlier episode might be pending an order, ready for submission, or already closed. These conditions change the correct action. The team uses those distinctions to choose starting states and expected outcomes for the task collection.

A task then combines the environment with an instruction and an executable checker. For the pending-order case, the instruction asks the agent to handle the incoming request. The checker inspects the episode link, preserved document status, and follow-up artifact after the attempt. Other tasks exercise the same tools under the different conditions represented in the map.

The reconstructed workflow serves evaluation and training

Benchmark tasks make a proposed agent change inspectable. An experiment runs the revised model, instructions, or tool configuration through the represented workflow situations. Task results identify which case families changed, while the complete attempts connect those results to actions and resulting records.

Training tasks give the customer’s model practice in the same kind of working environment. A reviewer correction or teacher attempt can supply a reference continuation during task development. Replaying that continuation connects the intended behavior to the state the checker will evaluate.

Model FactorySimulation environment / Training tasks
Training task
Continue the existing episode
Simulation environmentRestore the two requests and pending order
link_requestExecuted
episode: "existing-request"
status: "order_pending"
Follow-up artifact saved
prior-auth-v3Versioned training dataset
Environment and tasksVerified continuationsOutcome checker
Ready for training
Model Factory
TrainPlaygroundDeploy
Task validation
Replay the referenceVerify the intended case outcomeTask ready
Reinforcement learning
1Attempt2Attempt
Learn from outcomesEpisode link, pending status, follow-up
Deploy
v1v2
Stable model nameprior-auth-specialistServing
New production cases extend the simulation environment and Task Map
Illustrative training flow: a reference continuation validates the reconstructed task, then Model Factory trains the customer's model through new attempts and outcome feedback.

Model Factory uses new attempts in the environment for training. The model starts from the task’s initial state, chooses its actions, and receives reward from the resulting outcome. Linking the duplicate while retaining the pending order becomes a behavior it can practice. Each training record connects the attempt to its task, environment version, and feedback.

Related episodes stay together when case families are assigned to training or evaluation. Reserved cases let the team inspect how the update handles fresh instances of the workflow. The useful development record includes the family the task represents and the particular decision each attempt made.

A new production condition extends the loop

After deployment, another duplicate may arrive after an order was cancelled. That condition calls for a different continuation from the pending-order case. Its trajectory supplies evidence for extending the simulation and adding coverage to Task Map.

The next benchmark measures behavior on the expanded workflow. The next training run practices the newly represented decisions. Production experience accumulates as runnable environments and tasks, giving the team a way to investigate an episode, develop the model’s behavior, and observe it again in production.

Read the training walkthrough in Model Factory, or explore Benchmarks and Model Factory.


Weiran Yao

Written by

Weiran Yao

CTO & Co-Founder

Weiran Yao is Co-Founder and CTO at ACTAVA, working on model training and enterprise agent systems.

More stories

All articles →
Share this