Say It Four Times - by Nitin Khola
SubscribeSign in
Say It Four Times<br>Repeating an instruction in your system prompt does work. It quits working around the fourth time, and the average hides who it worked for.
Nitin Khola<br>Aug 19, 2026
Share
The short version: repeating an instruction in your system prompt genuinely helps. It stops helping at around four repetitions. Everything after that is superstition, and it costs you tokens.<br>That’s the whole finding. It cost about a dollar to get, and I think it’s a nice little thing to know on a Tuesday.<br>Thanks for reading Khola.Blog: Post-Human Engineering! Subscribe for free to receive new posts and support my work.
Subscribe
This is the first of what I want to make a weekly habit here. Pick one claim that floats around about AI coding agents, test it in a weekend, publish the numbers whether or not they’re flattering. Not research. Just somebody actually checking.<br>Why I bothered
You’ve seen the advice. Repeat the important instruction. Put it at the top and the bottom. Say it twice so the model takes it seriously. Everybody does some version of this, myself included, and I’ve never seen a number attached to any of it.<br>Then a paper came through my daily brief with an actual shape for it. Han-yu Wang’s When More Becomes Less: Position-Dependent Repetition Effects in Language Models (paper: arXiv 2608.04021, briefing: 6 August) tests what happens as you add more copies of a target, and finds the answer depends on where the copies sit. Copies stacked next to each other climb and then flatten out. Copies spread away from where the model reads out produce a hump, rising to an early peak and then falling.<br>That’s a specific, checkable claim about something I do every week, so I checked the half that matches how I actually write prompts.<br>I wrote my guess down first, which is a rule I’m keeping. My guess was that I’d see the hump, including the fall. I was wrong, and being wrong sent me back to read the paper properly, which is its own small lesson.<br>What I actually did
The setup is deliberately boring.<br>I picked one rule a model can either follow or not: use single quotes, never double quotes. Then I asked for six ordinary Python functions, the kind of thing you’d write on any given afternoon. Merge some intervals. Flatten a dictionary. Parse a version string.<br>The only thing that changed between runs was how many times that quote rule appeared in the system prompt: zero times, once, twice, four, eight, or sixteen. Same rule, just repeated more.<br>Thirty tries of each combination. 1,080 runs total, on Gemini 2.5 Flash, all of it on Vertex.<br>Checking the answers needed no judgment calls. I ran Python’s own tokenizer over the generated code and counted strings that opened with a double quote. Zero of them means it followed the rule. That’s it. No model grading another model, no me squinting at diffs deciding what counts.<br>The zero-repetition runs are the important control. That’s where I never mention quotes at all, which tells me what the model does when left alone.<br>One detour worth mentioning: my first three candidate rules were all duds. I tried “no comments,” “no docstring,” and “no type hints,” and the model obeyed all three about 99% of the time on the first ask. You can’t measure whether repetition helps when there’s no room left to improve. So I went looking for a rule the model actually resists, and quote style turned out to be one.<br>What came back
The control row is my favorite number in the table. Left to itself, the model used double quotes every single time. Not most of the time. All 171 of them. So its habit here is about as strong as a habit gets, which makes the rest of the table mean something.<br>Say the rule once and you’re at 74%. Say it four times and you’re at 97%. Those two are far enough apart that I’m comfortable calling it real.<br>Past four, the line goes flat. Eight and sixteen land inside the same range as four. My predicted decline never showed up, and to be straight with you, at this sample size I could miss a small one. What I can say is that nobody is getting paid back for repetitions five through sixteen.<br>Here’s the part I only understood afterward. I stacked all my copies of the rule right next to each other, which is the adjacent case in Wang’s paper, and adjacent is exactly the case that’s supposed to climb and then flatten. The hump I went looking for belongs to the other case, where the copies are spread out away from where the model is reading. So this didn’t contradict the paper. It landed on the paper’s prediction from a completely different direction, with a natural-language rule handed to a coding model instead of tokens in a probe. That’s a better outcome than the one I predicted, and I’d have missed it entirely if I hadn’t gone back to the source.<br>Three things I didn’t expect
The average is hiding almost everything. Two of my six tasks hit 100% on the very first mention and never wavered. Another one, merging intervals, sat at 20%...