Research
LaTRO learns a distribution over reasoning paths
A variational training objective connects sampled reasoning with the model’s own answer-likelihood estimates.
A language model can produce several reasoning paths for the same question. LaTent Reasoning Optimization, or LaTRO, treats those paths as a latent distribution that can be optimized during training.
Give reasoning a place in the objective
The method starts with question-and-answer pairs and introduces a sampled rationale between the question and the answer. A variational formulation connects the distribution over rationales with the likelihood of the reference answer.
This turns the reasoning path into a trainable component of the learning problem. The objective can favor paths that help the model assign higher probability to the answer while keeping the rationale distribution connected to its prior.
Use the model's probability estimates as feedback
LaTRO uses a self-rewarding mechanism based on the model's own probability estimates. The training procedure jointly develops its reasoning behavior and its ability to evaluate the usefulness of sampled reasoning.
The paper derives the optimization framework and implements sampling-based training. That connection between derivation and procedure gives researchers a way to inspect how a rationale receives learning signal.
Study reasoning across models and tasks
The experiments evaluate multiple model families on GSM8K and ARC-Challenge. They compare the resulting models with base-model and supervised-fine-tuning configurations and examine zero-shot reasoning behavior.
The research contribution lies in linking a latent-variable account of reasoning to a concrete training algorithm. For researchers investigating model improvement, this makes three choices explicit: how candidate rationales are sampled, how they influence answer likelihood, and how that signal updates the model. The released code supports further experiments with those choices.
Haolin Chen and collaborators developed LaTRO at Salesforce AI Research, with Weiran Yao among the authors. The paper was first posted in 2024. Read the paper and inspect the training implementation.