Revision Prompting
The Problem that Revision Prompting solves
We prompt LLMs in two ways:
Ad-hoc prompting
Prompts LLMs manually, with a custom instruction per call.
Examples
Asking a coding agent to implement a new feature.
Asking a chatbot to draft an email.
Industrial prompting
Prompts LLMs as part of an automated process, with the same instruction across calls.
Examples
Extracting structured information from invoices as part of an accounting pipeline.
Translating documentation pages into other languages as part of a release process.
Industrial prompting typically processes some Input data with an Instruction to produce some Output.<br>Whenever the Input gets updated, industrial prompting naively re-runs the Instruction on the UpdatedInput to produce the UpdatedOutput.<br>This approach has two downsides:
Lack of consistency<br>LLMs are non-deterministic, so the UpdatedOutput differs from the original Output beyond what the UpdatedInput necessitates.<br>Full processing time and token costs<br>Although only parts of the input have changed, we produce the UpdatedOutput in full. This is as expensive as the production of the original Output.
Revision prompting resolves both downsides of naive re-runs by operating on the input and output revisions instead of the full input and output.
How Revision Prompting works
Assume you have processed some Input with an Instruction to produce some Output by prompting an LLM with
Instruction: Input
Now, Input has been updated, and you also want to process the UpdatedInput.<br>Revision Prompting processes the UpdatedInput by constructing the RevisionPrompt as
Instruction: Input produced Output.
The input got updated as follows: diff(Input, UpdatedInput).
Please produce a patch to update the output.
The LLM responds to the RevisionPrompt with the OutputPatch that we apply to the Output to obtain the UpdatedOutput.
Example<br>You translate the product page of an e-bike to German with the prompt
InstructionTranslate to German:<br>InputThe Vela 3 e-bike has a range of 80 km.<br>Its battery recharges in three hours.<br>The frame is made from recycled aluminium.<br>Every Vela 3 includes a two-year warranty.
and the LLM produces
OutputDas E-Bike Vela 3 hat eine Reichweite von 80 km.<br>Sein Akku lädt in drei Stunden auf.<br>Der Rahmen besteht aus recyceltem Aluminium.<br>Jedes Vela 3 hat zwei Jahre Garantie.
Later, a battery upgrade increases the range from 80 km to 100 km. Instead of re-translating the whole page, you prompt
InstructionTranslate to German:<br>InputThe Vela 3 e-bike has a range of 80 km.<br>Its battery recharges in three hours.<br>The frame is made from recycled aluminium.<br>Every Vela 3 includes a two-year warranty.<br>produced<br>OutputDas E-Bike Vela 3 hat eine Reichweite von 80 km.<br>Sein Akku lädt in drei Stunden auf.<br>Der Rahmen besteht aus recyceltem Aluminium.<br>Jedes Vela 3 hat zwei Jahre Garantie.<br>The input got updated as follows:<br>diff(Input, UpdatedInput)- The Vela 3 e-bike has a range of 80 km.+ The Vela 3 e-bike has a range of 100 km.<br>Please produce a patch to update the output.
The LLM responds with
OutputPatch- Das E-Bike Vela 3 hat eine Reichweite von 80 km.+ Das E-Bike Vela 3 hat eine Reichweite von 100 km.
Applying the OutputPatch to the original Output produces the updated translation.
The OutputPatch contains only two lines of text instead of a full re-translation. Unchanged content stays consistent with the original translation.
Why Revision Prompting works
Consistency<br>By supplying the LLM with the input revision diff(Input, UpdatedInput), we ensure that the OutputPatch is limited to what the input changes necessitate. Everything not touched by the OutputPatch remains identical to the original Output. Therefore, the UpdatedOutput is consistent with the original Output.<br>Time & cost savings
Revision Prompting feeds the original Output back in as part of the prompt, so the LLM only generates the short OutputPatch. Most tokens thereby move from the output to the input. Since processing time scales roughly with the length of the output, this eliminates most of the processing time. It also converts most of the output token cost into much cheaper input token cost. If the re-run happens within a couple of minutes of the original run, prompt caching reduces part of the input token cost as well.
Revision Prompting in practice
Revision Formats<br>The ideal formats for encoding diff(Input, UpdatedInput) and the OutputPatch depend on the Instruction. The POSIX diff utility is a useful generic format. For JSON outputs, the JSON Patch format works well.<br>Expected Savings<br>The time & cost reduction scales with the size of the input changes and the sensitivity of the output to changes in the input. In our own industrial prompts, Revision Prompting reduces time by ~80%, and costs by ~65%.