Training agents to ask useful questions

A user asks an agent to prepare and send a referral packet. The agent can find documents, assemble a packet, and choose a destination, yet the quality of its work depends on details the user has left implicit. A useful question reveals information that changes an action: which clinic should receive the packet, which document is needed, or how the user wants to resolve a trade-off.

Training this behavior begins by treating a question as an action with a downstream effect. The agent chooses what to ask, observes the reply, updates its understanding, and uses that information in a tool call or recommendation. The full sequence supplies the learning signal.

Find the missing detail that changes the action

Consider a worked example in a simulated referral application. A user asks, “Prepare the packet for the specialist.” The record identifies the patient and the referral, and the available tools list two eligible recipient clinics. The user’s intended destination is stored privately in the task and becomes available when the agent asks.

The agent has a decision to make before sending. Asking, “Which clinic should receive the packet?” reveals the missing requirement. Once the user selects a clinic, the agent can retrieve its recipient identifier, assemble the required documents, and send the packet to that destination. The final record should carry the selected recipient.

A training task can encode that chain explicitly: the initial request, the hidden requirement, the reply available when asked, the grounded options, and the conditions for completion. Changing the intended recipient while keeping the initial request fixed creates paired examples where a different answer leads to a different action.

Verify the outcome of the conversation

Build the task around structured options. Each recipient should have attributes the evaluator can inspect, such as service, clinic, identifier, and document requirements. The simulator can then judge the final action against the user’s stored intent and the options available in the environment.

Keep a record of how requirements enter the conversation. Mark which facts appear in the initial request, which facts the user reveals in response to a relevant question, and which preferences change after the user sees an option. This creates useful variation in the interaction and gives reviewers a way to trace each choice to the information available at that turn.

Include examples where the agent already has enough information to act. Those cases teach it to move from understanding to execution. In longer conversations, let a tool result expose a meaningful trade-off, such as a preferred recipient requiring another document, so the next question arises from the work itself.

Train questions against the final task outcome

A question’s value often becomes clear several turns later. In the referral example, the destination question helps the agent select the correct recipient. A training objective can connect that final outcome to the earlier information-gathering action through trajectory-level reward or reward-to-go.

Use an outcome component for satisfying the user’s requirements and an efficiency component for the interaction cost. Record both during evaluation. This gives an experiment a concrete way to study whether the agent reaches a suitable result and how much conversation it needs.

to: "riverside-cardiology"
packet: "referral-042"

Inspect the decisions between the question and the outcome. Did the agent use the selected clinic in its lookup? Did it check the returned recipient record? Did the final packet preserve that destination? These checks turn a broad success signal into feedback that can guide the next training examples.

Vary how users express the same need

Supervised examples can provide an initial pattern for asking, listening, and acting. Interactive training can then exercise that pattern across users who answer briefly, volunteer extra detail, change a preference, or describe the same requirement in different words.

Evaluate with several user simulators and held-out requirement combinations. Review conversations where extra questions changed the selected action, alongside cases where the agent moved directly to a good choice. The next data batch can target a specific gap, such as asking about a relevant constraint or carrying the answer into the final action.