LLM text watermarking, by hand | Srce Cde Skip to content LLM text watermarking, by hand<br>Aug 20, 2026

Source: https://arxiv.org/abs/2301.10226<br>Paper: A Watermark for Large Language Models - Kirchenbauer et al., 2023 (arXiv 2301.10226)

The paper lays out a technique to bias an LLM’s word choice toward a reproducible list of tokens, re-drawn at every position, so the output carries a statistical signature which a detector can use for verification without model access. This note covers concepts first, then a hand-worked example on a tiny vocabulary, from generation all the way to detection.

Assuming an understanding of

tokenization (text split into tokens with a fixed ID in a fixed vocabulary)

the model emits a raw score (logit) per vocabulary token

softmax turns those scores into probabilities

Concepts you will need

Pseudo Random Number Generator: Pseudo Random means reproducible. Pseudo Random Number Generator is used to shuffle the vocabulary followed by a split at the gamma fraction. The idea is to shuffle the vocabulary using the ID of the previous token as seed, so the shuffle is always deterministic.

Gamma: Gamma is a hyperparameter [0,1]. On the generation side it decides how to split the shuffled vocabulary among green and red sections. Basically, its a green list fraction.<br>On the detection side there is a gamma-percent chance that human written text (tokens) fall into a green list. Value of Gamma depends on the use case. Smaller gamma means smaller green list to pick from and quality may degrade - you might ask, can’t it pick from red list. Definitely it can but there is a delta (defined next) parameter that gives boost to green list weights which in turn shrinks share of red list so it is very possible that before the boost the plausible token appear in red list and 2nd plausible option is in green list and after the boost the green overtakes. The boost can reorder tokens but cannot make implausible options plausible. On the detection side if the gamma is small which means the noise floor is low - you can think of it as green share going beyond 10% of total tokens, then that excess above 10% is the watermark signal. The shuffle of the list is not based on meaning, so any writer who is not aware of the partition would land on green list exactly at gamma rate.

Delta: Delta is a hyperparameter to boost the green list logits. The paper’s default is set to 2 and we will use that throughout. So before transforming the logits into probabilities, 2 is added to the green token’s logit and it becomes e^2 multiplication to green weights before applying softmax.

Entropy: Entropy measures the spread of probability across the available options. Suppose the input prompt is “Albert” and one token that would probably hold most of the probability mass will be “Einstein” - so this is an obvious option which means the entropy (surprise element) is low and the delta boost of 2 will not change that because other tokens will not be in the range of ~7.4x to become a leader. But if the delta is large then it is possible at the cost of quality so its a trade-off. Whereas if the input prompt is “The weather is” then the plausible options (like awesome, sunny, rainy, mindblowing) are many with similar probabilities. So here the entropy is high and the boost can promote one of the already plausible green options like rainy past the leader without quality degradation as all the candidates are good. Basically, the watermark can be written at high-entropy position where multiple plausible options are available. However, if we look at the reverse (low-entropy positions) where facts, code, names tokens can still land in green or red share but at the chance rate of gamma which is same as any human. So the count accumulates but the signal doesn’t.

z-score: z-score is the watermark detection test. In simple terms, it tells us how far the observed count is from expected value in standard deviation units. Paper flags anything above z=4 as watermarked. For example there is a text with T scorable tokens and anyone without the partition knowledge will produce about gamma x T green tokens by chance. The z-score answers “Is the observed token count close enough to be produced by luck or too far above it?“. So anything below z=4 the observed count cannot be distinguished from luck (cannot reject the null hypothesis) and anything above z=4 means that the count that high can occur only about 3 times in 100000 (coming from standard normal tail) by luck. Its rare enough to reject the null hypothesis and flag the text as watermarked.

Worked example end-to-end

A short sentence generated, detected, scaled, attacked and diluted.

Setup

vocabulary (8 tokens, with ids)<br>0 = and 1 = cat 2 = door 3 = on<br>4 = mat 5 = sat 6 = the 7 = slept

gamma = 0.5 -> 4 green per partition<br>delta = 2.0 -> boost alpha = e^2 = 7.389<br>prompt -> "the cat"

gamma - decides how many tokens are green.

delta - decides how hard green gets a boost.

The...