Using diffusion models to refine a system prompt

A system prompt sits at the beginning of an interaction, while evidence about its behavior arrives afterward: the user’s request, the model’s response, and feedback on the result. Updating the prompt means connecting that later evidence to instructions that will guide the next interaction.

Prompt Optimization Via Diffusion Language Models studies a direct editing procedure for this task. A diffusion language model regenerates selected parts of a system prompt while conditioning on the surrounding interaction trace.

Make the editable region explicit

The method represents the interaction as a system prompt, user query, model output, and optional feedback. It masks the prompt spans selected for optimization, or inserts mask positions for additional instructions. The diffusion model then fills these positions through iterative denoising.

This layout gives the optimizer both an editing target and the context that explains its behavior. User requests, observed outputs, and evaluator feedback can all inform the regenerated instructions. The target model’s parameters stay fixed throughout the prompt-optimization experiment.

Evaluate the instructions on completed tasks

The experiments use Dream-7B to refine prompts for GPT-4o-mini. Evaluation covers simulated tool-use interactions in the airline and retail domains of τ-bench, sentiment classification, paraphrase recognition, and natural-language inference.

In the reported table, airline task success rises from 43% with the original prompt to 50% with the diffusion-optimized prompt. Retail rises from 42% to 46%. On SST-5, the score increases from 55% to 67%. The paper also studies the number of diffusion steps, treating the refinement schedule as an experimental choice.

These measurements connect an editable text artifact to downstream behavior. They give prompt-development teams a way to inspect a proposed change, run the resulting instructions, and compare the outcome under a fixed target model.

Keep a record of each revision

The procedure suggests a practical workflow for experimentation: save the original prompt, the evidence used for editing, the regenerated spans, and the evaluation results. A versioned record makes it easier to trace a successful instruction back to the cases that motivated it.

It also makes the optimization process concrete. Researchers can vary feedback, editing scope, and denoising steps while tracking the same downstream tasks. That gives prompt refinement a measurable set of inputs, artifacts, and outcomes.

Research by Shiyu Wang and Haolin Chen, who contributed equally, with Liangwei Yang, Jielin Qiu, and collaborators.

Source: Prompt Optimization Via Diffusion Language Models.