It Still Can't Do My Job<br>聾">

It Still Can't Do My Job

Four years of moving goalposts, with receipts

I started keeping notes in December 2022, mostly to document why the panic was overblown. The notes turned into this. The quotes in orange boxes are real. You can look them up. The gray comments are paraphrased from a few thousand comment sections. You know the ones. You may have written some. I did.

November 2022

The party trick

ChatGPT launches on a Wednesday. By the weekend it has a million users and my whole feed is screenshots of it apologizing for code that doesn't compile. It invents functions. It hallucinates whole APIs. I asked it for Snake, the game you write in an afternoon as a teenager. It gave me a snake that ate itself on move one. Five days in, Stack Overflow bans it:

"Because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site."

Stack Overflow temporary policy, December 5, 2022

The verdict was easy, and it was also mine: a stochastic parrot that learned to sound like a senior dev without ever meeting a compiler.

The goalpostCall me when it stops making things up. It can't even do Snake.

March 2023

The exam season

GPT-4 ships. One prompt now gets you a working Snake. The same game it face-planted on four months earlier. The comment sections adjust instantly and never slow down:

It's just a simple game bro. There are ten thousand Snake tutorials on GitHub, it's literally copy-pasting. Wake me up when it does something that's NOT in the training data.

the comment section, spring 2023, paraphrased

Meanwhile the party trick starts passing exams. OpenAI claims the bar exam at the 90th percentile. Microsoft researchers publish a paper called "Sparks of Artificial General Intelligence". A real paper, with that real title. To be fair, the skeptics landed punches here. A later re-evaluation put the bar exam closer to the 60th percentile, and around the 48th among people who actually passed. Both sides were flinging numbers. Only one side was flinging them at a thing that kept improving.

The goalpostToy scripts and exams aren't engineering. Call me when it builds something real. A proper game, say. In 3D.

March 2024

The staged demo

A startup called Cognition announces Devin, "the first AI software engineer". The demo video is everywhere for a week. A month later a veteran developer named Carl Brown (YouTube channel: Internet of Bugs) goes through it almost frame by frame. The impressive parts were curated. Devin didn't do the Upwork task from the demo. It generated its own errors, then heroically fixed them. The skeptics take a well-earned victory lap. I watched the takedown twice. It felt great.

That same spring, the CEO of Nvidia stands on a stage in Dubai:

"It is our job to create computing technologies that nobody has to program, and that the programming language is human. Everybody in the world is now a programmer."

Jensen Huang, World Governments Summit, February 2024

Nobody I know quit programming that year. But everybody I know quietly installed Copilot.

The goalpostDemos are staged. Call me when real developers use this for real work, daily.

October 2024

The earnings call

"More than a quarter of all new code at Google is generated by AI, then reviewed and accepted by engineers."

Sundar Pichai, Alphabet earnings call, October 2024

The comment sections don't blink. That's just autocomplete acceptance metrics. Boilerplate doesn't count. Half of it is import statements. And fine, some of it probably is. But "a quarter of Google" is a strange thing to keep calling a party trick.

The goalpostGenerating lines isn't the job. Call me when it takes a ticket and ships the feature.

February 2025

The vibes

"There's a new kind of coding I call 'vibe coding', where you fully give in to the vibes, embrace exponentials, and forget that the code even exists."

Andrej Karpathy, February 2, 2025

Three weeks later Pieter Levels prompts a multiplayer 3D flight simulator into existence. It takes him about three hours. He has zero gamedev experience. He puts it online at fly.pieter.com. Remember the 2023 goalpost? A proper game, in 3D? Here it is. It sells $29.99 fighter jets and blimp ads to real customers, and he claims a $1M annual run rate within seventeen days. The comment sections know exactly what to do:

It has no vibe bro. It doesn't even feel like a fun game to play. Floaty physics, asset-flip graphics, zero game design. This is a tech demo with a Stripe account.

the comment section, March 2025, paraphrased

Same season: Zuckerberg tells Joe Rogan that Meta expects AI that codes like a "midlevel engineer" within the year. Dario Amodei says AI may be writing 90 percent of code within six months. And vibe coding grows its own disaster genre. Leaked API keys. Wide-open databases. "My app got hacked and I don't know where to look" postmortems. The seniors are unimpressed, and they...