On Accountability · Addison Crump

Published: 2026-07-13

In 2024, I attended ICST in Toronto.<br>I went to attend the first "Artificial Intelligence in Software Testing" workshop (AIST), where I gave a tutorial on a potential competition and participated in a panel (which, turned into a roundtable discussion) about the field.<br>Our concerns at the time were principally about LLMs with two prominent questions: "how can we research this reliably?" and "how much can we reasonably expect to come from this?"

Readers who have visited before will know that I'm not the greatest fan of LLMs, to say the least.<br>The many arguments against it are weary of needing to be repeated so often, and I am weary of repeating them.<br>Back in 2024, the scale of these harms was not yet well-understood, and so the panel itself was mostly concerned with the fact that we could not realistically do research on LLMs themselves, but only the viability of their applications.<br>No empirical result about efficacy would be generalisable, and anything could be denied (in favour of or against LLMs) by simply claiming that the research was studying the wrong targets or applying them in the wrong way.

Ironically, though I had attended ICST specifically to attend AIST, it was not the most impactful part of the conference for me.<br>That title belongs to the second keynote from the main conference track, "We build the world we measure" by Mike Hoye.<br>The keynote, ironically, was mostly not about measurement, but its consequences.<br>The speaker derided the industry for its utter disregard for the well-being of its customers, and the lack of accountability faced by companies that develop software.<br>Ultimately, he called for accountability in software engineering, akin to the measures put into place by other engineering fields, all justified by a simple fact: software is simply too integral to our daily lives, and too consequential.

I barely remember the workshop I had joined for anymore, but I remember where I sat during the keynote, specific slides, the shape of the room, the rapt attention and silence of the audience, and the brief conversation I had with the speaker afterward.<br>I also remember the two Google engineers (one senior, one junior) who joined the conversation -- rather forcefully -- to assert that accountability is toxic, and counter to the freedom of software development.<br>I remember the smugness with which he said this, and the certainty with which he claimed this, and his inability to justify that claim beyond repeating the claim itself or excuses already accounted for in the keynote.

At the time, I just found this infuriating.<br>A quintessential self-report on avoiding responsibility for the software one develops, knowing that people depend on it.<br>Now, I remember it in the context of LLMs.

Our industry is so poisoned by the idea that consent is optional; that responsibility is something to be dismissed in a limitation of warranty; that negligence is excusable in the name of progress; that features are more valuable than stability; that usability is secondary to commercialibility; that somehow, there is no monetary gain too small to justify an indignity to users.<br>The idea that companies are somehow not responsible for the software they create and distribute is pervasive and normalised, while people's lives revolve around programs working as anticipated often without any alternatives.

I'm coming back to the ideas of the keynote more often these days.<br>LLMs have sponsored a radical shift in how programs are developed for many -- mostly for worse.<br>Now, the very same companies which have, for decades now, shirked responsibility for the consequences of their programs, are using these tools to increase the output of code at the cost of maintainability, understanding, and accountability in the software they distribute.

Effectively any programmer can produce code with an LLM that accomplishes a task that can be commercialised.<br>I see our field shifting from one that already has problems with accountability to one where there simply is none.<br>I see a world where individual programmers, who maybe historically had enough stake in their work to see it done well, become disconnected from the code they are responsible for, assigned enormous workloads which can only be accomplished by simply caring less and relying on the code generator more.<br>Where those same programmers are "held accountable" by being replaced when something fails; after all, since nearly any programmer can operate the tools that actually produce the code, we become more expendable, more exchangeable.<br>To me, and certainly to those who truly benefit from the commercialisation of software, this developer sounds like perfect scapegoat to be held accountable for the harms without consequences for the system or the people that actually cause those harms.

When something causes harm in the traditional engineering domains, the company faces severe penalties, and a proper investigation is done...