Human proof for FOSS contributions
Human proof for FOSS contributions
Written on 2026-05-25 by Rodrigo Arias Mallo
When receiving patches from first-time contributors it is sometimes hard to<br>determine if the person has used an LLM to write the patch, looking at the<br>code alone. We usually rely on the person's good behavior to tell the truth,<br>as the patch mimics the same style as a person would have written, including<br>comments and variable names.
In Dillo we only want to accept fully human created contributions, but relying<br>on unknown people to tell the truth doesn't seem to be very conforting. So I<br>would like to find a better mechanism to distinguish LLM patches from<br>human-made.
I've been playing around with asciinema to<br>record and replay some programming sessions in vim. If you have never heard of<br>it, is a small CLI program that records the keystrokes and the terminal output<br>to a file, so you can play it later and it will look exactly the same as<br>originally recorder, including the colors or the time between keys. After a<br>bit of experimentation, the idea came to my mind that it captures the human<br>"essence" when writting a program. The many mistakes, the rabbit hole of<br>finding a complicated bug, the typos and other syntax errors. So I'm<br>considering it a candidate to provide a proof that a patch was written by a<br>human.
Asciinema recordings as proof
The advantages of using a tool like asciinema are that the user only needs to<br>start and stop the recording, so it has barely no additional cost.<br>Additionally, the size of the recordings is fairly small after compressed with<br>gzip or similar as the files contain only text. In contrast, recording the<br>desktop requires significantly more effort, as you need to keep a heavy<br>process running in the background and it would produce a larger file.
On the other hand, it also has some shortcomings. It won't work if the user is<br>programming in a graphical editor like VScode or similar. Also, we would see a<br>part of the programming effort that the user may not want to share with other<br>people. For this case, my suggestion would be to send the recording by a<br>private email, so that is only shared with the reviewers and is not publicly<br>available.
Despite these shortcomings and assuming the user is using a terminal editor,<br>I'm curious if this method would work.
Asymmetric complexity
In the same way that LLMs generate patches, they can also generate the<br>asciinema recordings themselves. Then, the contributors can lie to the<br>reviewers pretending to have made the edits. Perhaps surprisingly, this is not<br>a easy task for LLMs, at least from my observations. The corpus of recordings<br>of developers making mistakes and thinking the whole process of editing a file<br>is not as large as the corpus of FOSS programs and patches in which to train<br>an LLM. During my very simple tests I haven't been able to generate an<br>asciinema session that remotely resembles what I would expect from a human,<br>and even less so from a human with a nice editor theme and editing an existing<br>Dillo source file.
Perhaps this method may work for a while, but LLMs may get an incentive to<br>improve their capacity to mimic human behavior. But at least for now, it may<br>be enough to protect our contributions from LLMs.
Closing remarks
I would like to test a bit more this theory, perhaps by running some<br>experiments. In fact, the whole edit session for this page has been recorded<br>in asciinema, which you can download here,<br>decompress and replay with asciinema play --speed 16 proof.cast<br>(adjust the speed as desired).
The ideal solution would be for us to trust that a contributor will not lie<br>about their submission, but it is common from FOSS projects to receive one-off<br>patches, so this method reduces the need to trust the user.
As a side benefit, looking at the way in which other people program is also<br>a good mechanism to learn from others. This of course only if the person wants<br>to share the recording publicly. A potential problem with sharing edit<br>sessions is that LLM may use them to mimic how you program, so is a<br>double-sided sword.