LLMs can't program  ❤︎  samir : coffee → nonsense

LLMs can't program

by Samir Talwar

Monday, 13 July 2026 at 09:00 CEST

The word “generative” in “generative AI” refers to what it does, not what it is useful for.

But that’s the conclusion. Let’s start from the beginning.

Years of ignoring the world

I have wondered, for a long time now, why I haven’t written on the topic of generative AI for programming. After all, everyone else is.

I recently realised that that’s why. Everyone else is. I have nothing to add, except that I find it personally distateful. (And if you were wondering what my bias is, now you know.) I have long been an advocate of machine-assisted work, and in the past, I even claimed programming will be fully automated, though I no longer believe this… and yet, I never felt the need to even bother trying. A large language model was obviously so detached from an understanding of the code that I didn’t believe it could ever be useful. And, of course, I was seeing the evidence reinforcing my beliefs all over social media.

So I mostly ignored it, even when my colleagues were experimenting with outsourcing their own jobs to a chatbot. I installed Cursor last year when I was asked to, and tried it out. I was told to “treat it like an intern”. I asked it to help me debug a failing CI pipeline. It misquoted the documentation at me and then doubled down when I argued with it. So I fired uninstalled it and carried on with my work as usual. For a year, no one asked me to try it again.

In which I learn to be one with the agent… or not

Recently, I was asked to attend a two-day course in “spec-driven development with agentic AI”. If you’ve not heard of it, spec-driven development is the idea that waterfall software development is good, actually, as long as you do it fast enough.

The principle is fairly simple: write down every decision in text files, never rely on the agent’s context (so that instances of agents become fungible), and work in discrete (though very large) steps. Start with a roadmap, broken down into high-level goals, and then generate a spec for the first goal, implement, repeat. Classic top-down Theory X management, with the programmer acting as the manager. (To be fair, if Theory X was ever useful for something, it’d be for dictating to a machine.)

So each of us, in the workshop, built a little habit-tracking app with either Claude Code or OpenAI Codex. It went quick. I had Codex generate a 5000-word specification, prompt me for clarification, and update it until it (the agent, not me) decided it had a high confidence of being worthwhile.

(What it means for an agent to have “high confidence”, I have no idea. As far as I can tell, the bullshit machine works better if you ask it to bullshit itself too.)

I had a UI and some basic functionality within, I don’t know, 30 minutes? It was an atrocious UI, which is not necessarily a problem: it allowed me to reject a design and come up with something new. But while staring at the awful design, I was a bit confused.

You see, I’d asked the chatbot to also create screenshot-based approval tests so I could see the changes as diffs. And it did. But they looked nothing like the actual UI. So I scrolled through the approval test code, and discovered that it was not actually running the application and taking screenshots, but instead generating some random nice-looking screenshots with code, rendering them as PNGs, and saving them to disk.

Strike one.

I proceeded anyway, ignoring the tests. The bot was not bad at redesigning, though I found myself really wanting a UX designer by my side, not something that could only say “yes”. It turns out tracking habits is not a solved problem; the general problem is infinitely complex, and constraining it in such a way that it is still useful is fascinating in its own right, even setting aside the computer. But time was marching on, and so I accepted a design that was, well, fine, I guess, and continued with step 2 on my roadmap: authentication.

I asked the agent to generate a spec, and it complied. It even asked me if I wanted to use passkeys, which I had not considered, and for the first time, I was intrigued. I didn’t know how to implement authentication with passkeys. This was a learning opportunity for me! Of course I agreed, and once the spec was refined, I asked the bot to implement it.

It did. At least, it said it did. I launched the webapp, signed in as a user, and found two issues. First of all, it claimed that it would transfer habits recorded as an anonymous user to the known user upon sign-in… and it did not. But the more worrying issue was that there was no passkey prompt. Nothing. I typed a username, I was signed in.

It turns out that chatbots are good at chat. It claimed to have implemented authentication. Actually, it had basically written:

function authenticate(user: string): bool {<br>return true;

I am fairly sure this is not compliant with the WebAuthN/Passkeys specification,...