Cannibalism | The “B” Ark

Brett Kosinski

If you’re not in tech you might not realize just how much of a panic has set in among the C-suite and investor class in the industry. Everyone is, of course, putting on a brave face, but as fears of a SaaS-pocalypse persist and a fictional financial analysis on Substack caused a real sell-off in markets, it’s hard not to see the sweat on everyone’s brows.

Meanwhile, CEOs and CTOs are falling over themselves trying to position themselves as visionaries, with the likes of Jack Dorsey, Sebastian Siemiatkowski, and Matt Biilmann each trying to out-AI-pill one another. Of course, one has to wonder how much of this is AI-washing and how much of it is intentionally eye-catching hyperbole, but these are just the most extreme and visible tip of a very large iceberg.

It’s hard not to enjoy the bitter irony, here. The tech industry once revelled in disruption. Whether it was Uber wiping out the taxi industry, Amazon destroying traditional bookstores, or Spotify wiping out artist revenues, time and time again we’ve seen the Silicon Valley enrich themselves by disrupting an established market and then transforming into rent seekers.

AI now threatens to do the same to countless professions. Everyone from writers to graphic designers to financial analysts can feel the wolf at the door.

But this time things are a little different.

This time it’s coming for tech first.

Anyone who’s working with large language models knows that, despite being incredibly powerful, they are also deeply (and in some ways fundamentally) flawed.

Take, for example, an incredible common use case: summarizing meeting notes.

I’ve used the latest, greatest, and most sophisticated models to perform this incredibly mundane task, and I’d put them at a solid 90% accuracy.

But that 10% definitely matters.

It’s entirely normal for the AI to incorrectly attribute one person’s words with another; to make claims or draw conclusions that aren’t evident from the meeting transcript; to inflate or exaggerate statements or claims in the meeting. The list goes on.

The challenge with AI in these contexts is that there’s no objective source of truth for determining what “correct” is. In the industry, the term for this would be an “oracle”, something that you can use to interrogate your conclusions to assess validity.

The same is true across a wide variety of domains where you might apply AI. Whether it’s writing prose, or generating an image or video clip, or offering a customer a resolution an automated support call, “correct” is often a matter of taste, opinion, or interpretation. As a consequence, application of AI is incredibly uneven across industries.

Moreover, the very nature of large language models and how they’re trained can increase the odds they will be unsuccessful in a variety of domains. LLMs are, after all, ultimately trained on large quantities of publicly available data, and particularly for deep niches, much of what makes and expert an expert isn’t available for easy scraping online.

And then there’s tech.

First, assuming a decent spec or set of requirements (yes, that’s a big if!), software is fundamentally verifiable: either the code does what it’s supposed to do or it doesn’t. It’s this very property that makes things like ralph loops possible, as you can give the LLM a set of success criteria and then free it to iterate toward a solution.

Second, and this is where open source plays a deeply ironic role in this business: enormous quantities of source code is widely and publicly available for use in LLM training. There is truly no other industry that has a training corpus as rich and varied as software development.

Third, the AI industry is itself made up of software professionals, and as a result they are uniquely positioned to create incredibly effective tools for generating code with AI. In fact all major AI labs end up dogfooding their own tools to create those tools, which only speeds up the rate of iteration and advancement.

The end result: the tech industry, from the C-suite to the coding grunts on the ground, is shocked and terrified about what’s to come as, for the first time, the call is coming from inside the house.

I’m honestly not sure how I feel about this whole situation.

My first instinct is to laugh and shake my head.

One need not look very far to find indignant software developers absolutely certain that their jobs cannot possibly be automated away by the very tools their industry contemporaries are creating to replace them. I suspect you’d also not have to look far into their posting histories to find those same people comparing cabbies to buggy whip makers.

Meanwhile, on the other end of the spectrum, you have CEOs and CTOs pushing folks to token maxx while pounding the table that we all have to AI harder and faster lest we be tossed into the dustbin of history.

The result is a bizarre combination of denial and boosterism that’s hard to square.

Ultimately, if...