Circumstantial Complexity, LLMs and Large Scale Architecture | datagubbe.se

{ datagubbe }

datagubbe.se "<br>circumstantial complexity, llms and large scale architecture

Circumstantial Complexity, LLMs and Large Scale Architecture

On software architecture and the paradox of machine-assisted programming.

February 2026

Sic transit gloria mundi

Late 1995, or is it early 1996? Scrawny legs straining against the wind, heart beating with exertion and excitement as I pedal my bike across town and pass the truly affluent neighbourhoods. The harsh winter sea is roaring to my left and the enormous villas overlooking it tower to my right, calm and dignified in the dusk.

Fresh, empty 3.5" floppy disks in my backpack, fresh batteries in my Walkman FX16. Cheap headphones are blasting Chemical Brothers' Leave Home at maximum volume as I cross the tracks of a derelict railway, once connecting Limhamn and its now dwindling fishing industry and recently closed limestone quarry with the city of Malmö proper. I keep going, past Kockum's no longer bustling shipyard, past the decommissioned SAAB factory, past the still busy grain pier. A perfect image of the 1990s: Decades of accumulated decline, of circumstantial industrial death, yet an inevitable need for sustenance.

Then I see it, like I have almost every night for the past few months. A beacon of promise, of future economic opportunities yet unexplored: Kajplats 305, Berth 305, the youth center with a dedicated 256 kbit/s Internet connection.

The world wide web. There seems to be some money in that. But presently, my mind is far from the budding dotcom boom and untapped business opportunities. It's time to download some new Amiga demos, check in on IRC, and update my homepage.

It's a bit over three decades since I wrote my first ever lines of HTML on one of the 486 PCs at Kajplats 305. Since then, the kind of web coding I really love has disappeared from the professional world. Not because of LLM-powered coding agents, but because of increasing complexity.

Even though I'm a senior full stack web developer, things are not as they used to be. The days when a single developer installed Linux, compiled and configured Apache, designed a MySQL database and wrote all of the front- and backend code in an unholy mix of PHP, SQL, HTML and (maybe) a tiny bit of JavaScript, are long gone. How fitting, then, that Kajplats 305, like the heavy industries and dockside warehouses preceding it, is now being demolished. It will likely be replaced by modern offices, where the complex software of tomorrow can be designed.

Because complexity is - up to a point - both beneficial and profitable. On the web, most systems and processes are now so complex and involved that even full stack, while indeed dealing with both sides of an HTTP request, is a specialized, compartmentalized role. Builds and deployment? Please, we've got experts dealing with that. I've got colleagues who, in their professional role, never write anything but SQL or React. Expectations and demands on software increase steadily. Few can keep the entire development process, or the entire code base of a modern enterprise web app, in their heads. As complexity increases, so does the need for specialization.

A recent blog post by Paul W. Homer, Systems Thinking, divides software architecture into two overarching philosophies. They're called Evolution and Engineering, and the article argues in favour of the latter. Evolution - letting a system grow organically and fixing stuff as you go along - is described as leading to a haphazard patchwork of poorly integrated systems, unnecessary complexity, and terrible fragility. Engineering, on the other hand, means solving the entire problem upfront, in a large unified system.

This is where I think the article contradicts itself. The evolutionary example it mentions involves an ecosystem that's been running and growing for a period of fifty years. Show me the most intelligent and talented team of hardcore systems thinkers ever to exist, and I will show you a team that will never, ever be able to anticipate all the real-world twists and turns that can affect a software system across the span of five years, let alone the span of five decades.

Web development has become what it is today not because someone planned it that way 30-odd years ago, but because of a myriad decisions, big and small, intended to solve technical issues encountered along the way. Since the publication of Fred Brooks' 1986 essay "No Silver Bullet", this is usually called accidental complexity - a technical problem that must be solved before an essential, real-world problem can be solved. For example: A solution to a specific problem involves a certain amount of irreducible, essential complexity, perhaps the act of adding numbers together. This might lead to accidental complexity, such as the need for arbitrary-precision arithmetic.

Hence, a software system that deals with a complex process must, at some level,...