Building the systems that build the software - nilenso blog

Atharva Raykar

Read more by Atharva<br>here

I gave this talk at The Fifth Elephant 2026 Pune Edition in February. This was a result of me grappling with the question of: “What do software engineers do when AI is writing all the code? What are the new skills required of them?”

This was an early glimpse into what the zeitgeist is now calling “loop engineering”. I draw from Donella Meadows’ work on systems dynamics to ground what this skill looks like concretely, along with examples of how to apply “the bitter lesson” to AI product building and also make a case of empirical rigour being an underrated skill to navigate the turbulent waters of AI hype cycles.

00:00-00:54.

Before I begin, I wanted to get a quick poll, just to get a sense of what tools people are using. How many of you are Claude Code users? A couple. What about Copilot? Okay, a lot more Copilot. Codex? Okay, one Codex. Anything else I did not mention? That seems to cover just about everything. I think all of you have used agentic tools, and I am going to talk about building the systems that build the software.

00:54-01:09.

In light of all of these coding tools writing most of our code, this is going to be an exploration of what we must do now. What are the new skills we require for this transformed SWE role?

01:09-01:33.

It is very easy to confuse the essence of what you are doing with the tools you are using. As Abelson has argued, software engineering is not really about code. It is not even really about computers. Our job is to formalize intuition about process, or about how to do things.

01:33-01:51.

There is some implicit knowledge that we have to make explicit by encoding various representations of the relevant aspects of the world and handling edge cases. That gives us code that does things.

01:51-02:06.

This is the part that LLMs are now doing really well. I would argue that many parts of this are being done better than the median programmer.

02:06-02:24.

So what is our job here? We still formalize intuitions about a process. That has not changed. But the process is different: it is now the process of managing a sociotechnical system and orchestrating code production so that it creates value. That could be business value, impact on the world, and so on.

02:24-02:33.

Just like before the whole change with AI, our concern is the same: complexity.

02:33-03:18.

This time we have to contend with new forms of complexity. There is stochasticity: randomness, unpredictability, the chance-driven nature that comes from LLMs. There is non-determinism, where there are multiple possible outputs for the same input. There is jaggedness, which Claude Code users will have noticed: sometimes the model is good, and sometimes it is absolutely bad, and it is not clear in advance which one you are going to get. And obviously, you have to manage risk. Sometimes things blow up. What do you do when that happens?

03:18-03:33.

I like to characterize all of these forms of complexity as “the fuzz”: this big, hairy cloud of unknowing, of not being sure what you are going to get. How do we deal with it? For this, I propose three skills.

03:33-03:54.

Our first skill is systems thinking. Systems thinking serves us well because it helps us understand how structure generates behavior. What is important in a world where you are dealing with the fuzz is that behavioral patterns are more important than any specific event or precise parameter.

03:54-04:09.

Systems thinking gives us structural intuition. When you know the structure and the patterns of behavior it generates, you are able to work better with the fuzz.

04:09-04:48.

What is a system? Let me be more specific. A system is an interconnected set of elements, coherently organized in a way that achieves something. In this case, our system will consist of stocks, which you can think of as buffers or quantities that can increase or decrease; inflows and outflows, which are valves that increase or decrease the quantity in a buffer; and loops, which we will see now.

04:48-06:09.

Let us take an example of a system we are very familiar with: the code review backlog. Our stock is the white box in the center: the number of backlog items, or the number of PRs that need to be reviewed. We have an inflow and an outflow. The number of PRs raised increases the quantity in the code review backlog, and resolving PRs decreases that buffer. There are two loops here, B1 and B2. These are balancing loops that happen in response to information. If the backlog is high, a team lead might ask the team to stop some feature development because code review is the bottleneck. You prioritize code reviews, increase the outflow, and bring the backlog down. Similarly, once the backlog is small and we do not need so many people prioritizing review, we can deprioritize it.

06:09-06:33.

That brings the backlog back toward an equilibrium. If the review backlog is...