Research
Learning a cost-aware model policy with xRouter
A reinforcement-learning router chooses model calls using a reward that combines task success with inference cost.
Choosing a language model can happen within the execution of a task. xRouter treats that choice as a learned policy: a router can produce an answer itself or invoke one or more external models through tool calls.
Model access becomes an action space
External language models appear behind a common tool-calling interface. The router receives a task, selects an action, and uses the resulting response as part of its work. This formulation gives model orchestration the same action-and-feedback structure used in reinforcement learning.
The policy can learn different behavior for different requests. A direct response and an external call each have an observable cost and contribute to the task outcome. The training system records those quantities for evaluation.
Reward joins completion with cost
xRouter uses an explicit reward that incorporates success and the total cost of model invocations. Successful trajectories receive a reward adjusted by their inference cost. A tunable cost coefficient controls the trade-off used during training.
The reward multiplies a success indicator by a fixed success bonus adjusted for total invocation cost. For multi-turn interactions, episode-level accounting connects a decision with the later calls it causes. This gives the learning objective a view of the complete interaction.
A policy that can be examined
The research covers training, cost accounting, deployment, and evaluation. Experiments compare cost and task performance across workloads and investigate which training choices help the router learn effective orchestration.
This framing gives developers a useful way to describe a routing decision. They can identify the available models, the outcome being rewarded, and the spending associated with the selected path. Those definitions make a cost-performance comparison easier to reproduce and discuss.
Cheng Qian and collaborators conducted xRouter at Salesforce AI Research and the University of Illinois Urbana-Champaign. Haolin Chen and Weiran Yao are co-authors. Read the technical report.