What if useful AI is a fantasy? What if useful AI is a fantasy?

Blog

Marginalia

What if useful AI is a fantasy?

Posted July 28, 2026 5 min read I tried to use AI in my work, but was it actually useful?<br>I’m a programmer, and I have been using AI for development work since around summer of 2023. At that point I tried Claude, and it was good enough at generating code that I did manage to make use of it for some of my projects. Jumping ahead to the beginning of this year, I subscribed to Claude Pro and had the opportunity to try “agentic”, tool-using AI for the first time (via Claude Code).At first I was enthralled, and used the AI to generate wholesale entire hobby projects, just like many others did. I was extremely impressed and the feeling of accelerated productivity felt like a drug.But once the code had been generated and the software was ‘working’, reality started to set in. I didn’t have a mental model for the thing that was in front of me. If there is a bug, or if a new feature needed to be added my mind was precisely where it was before I started prompting, and I couldn’t even begin to make changes until I had built a thorough understanding of the code.I like to write code, and its how I build my mental model when I’m programming. I fully admit to not having the discipline or perhaps even the capacity to design large systems in my head before my hands reach for the keyboard. My mental model is to a significant degree built one statement, declaration, and expression at a time.I see a clear trend in my use of AI for programming, and that trend ultimately is to use it less. First I gave up on “set and forget” vibe coding, and transitioned to an active, guided approach. Still feeling frustrated and unsatisfied, I’ve recently transitioned to AI assisted ideation: attempting to build those large mental models just through conversation with the AI, and then writing the actual code on my own, just as I always have.Has it worked? The ridiculous answer is that I don’t know for sure, and its for the same reasons vibe coding seems so dubious. The AI is fundamentally fickle, and independent of anything I do there’s always a risk that the output will be useless. An example would a software component I designed with the AI that I never end up using, because I realized later that it was massively more complex than it needed to be, or that I didn’t even need it at all. Of course the opposite can happen as well. I have gained genuine insights while rubber-ducking a problem with Claude, and used those insights to make real progress on a project. But at the end I’m always stuck with the same thought: that I probably could have just finished this without using AI at all.My experience is not unique, and its not new. It’s plainly obvious to see anywhere that anyone is discussing their use of AI. Its possible that this technology is just too new, and that we have yet to discover the true ‘killer app’ for LLM’s that is obviously and unequivocally useful. I don’t believe that has happened yet, or that its at all obvious that ever will.How long is it reasonable to wait for this technology to reveal its true utility? It may be an unfair comparison, but the smartphone was unambiguously a transformative technology, and it did not take long for that to become obvious.I’ve recently asked myself an interesting rhetorical question: What if in a decade from now there is no large-scale use of LLM’s? What if developers, engineers, writers conclude (like I did) that this technology is not useful enough to bother with? What if large organizations that ultimately spend the most on tokens decide that they can’t get a reasonable return on investment?At first this seems far-fetched, but consider what happened to Facebook’s Metaverse. For a brief window of time it actually seemed reasonable to believe that we would all be spending most of our waking hours with high-tech ski goggles strapped to our heads. That we would work, relax, and socialize with these bulky headsets tricking our brains into thinking they were in a different world.The technology was and still is amazing, I’ve owned Quest headsets at various times and really enjoyed gaming on them. But the original proposition is insane. That’s clear now just as it was a decade before anyone had heard of the Metaverse. So what if LLM’s are the same? What if we’re all participating in a ridiculous fantasy that intellectual effort, thinking itself can be automated? What if in a few short years we look back with astonishment that anyone would have ever believed in any of this? If you are enjoying my content, I'd love to hear from you. Send me a message at mail@lzon.ca or one of my social accounts.

I'd also appreciate a small donation if you are willing to offer one. Paypal is fine, as well as the other digital tip jars I have listed on the homepage.

Settings<br>Theme

Primary<br>Secondary<br>Tertiary

Secrets