Did AI speed us up? - by Sean Madigan - AI Builder Series

SubscribeSign in

Essays<br>Did AI speed us up?<br>A trend analysis of 2,410 completed tickets at Kerno spanning January 2025 to May 2026, with focus on pre and post AI tooling adoption, September 2025.<br>Jun 09, 2026

Share

Background

As the conversation about the return of investment (ROI) associated with AI intensifies I decide to do some analysis across our Linear database to see if AI has had an impact on engineering at Kerno. Notionally, being part of a small team you can ‘feel’ the productivity gains, but I wanted to anchor this in data.<br>Preface: During my ROI hunting expedition, I did not link ROI back to customer driven $. This expedition was focused purely on productivity. In next editions I will traverse back to features and $-value creation.<br>Context

To help you digest this piece, there is context that had a material impact on the numbers:<br>Like most companies in 2025 we deemed that AI coding agents weren’t good enough to start writing production acceptable code until around August/September last year.

The majority of our team were skeptical of LLMs in the application of code generation. This meant that there was an initial education ramp through August and September, where engineers were provided with space to experiment and explore.

We had 40% attrition during the past 12 months as we changed product direction and some roles were no longer needed.

We released a major refactor of our product in March which attributed to a spike across all graphs [tickets, tokens, bugs]. While I didn’t adjust for this in the dataset, I will add notes and draw my final conclusion based on this.

All our engineers have 10+ years of experience.

While not all code is generated by AI, the rough estimated is that 70% is.

I have not accounted for the PoC scrapyard or technical debt in this analysis.

Beyond the obvious

If I asked each of my engineers the question… is AI helping you, the answer is an easy yes, but I wanted to look beyond the usage metrics. I wanted to understand the nuances and see if there were correlations across certain dimensions:<br>Adoption: While every team have AI power users, AI code generation can only have a true org wide uplift if the majority of your engineers have bought in to it and are using effectively*. I wanted to check what adoption looked like across the team.<br>*effectively means something different for everyone, so let’s just say they are moving beyond basic prompting and investing time to make this tool better for them.<br>Bug Velocity: Shipping fast means nothing if what you ship is slop, it undermines progress. While software bugs are part of doing business (and always will be), I was interested in seeing if the bug velocity was up or down post introduction of AI.<br>Tokens: Another trend I was interested in, is how token consumption is trending. My assumption was we would have a sharp rise in tokens as we started using AI initially, but then it would fall somewhat and stabilise as we started using more skills, better scaffolding, memory layers and better prompting.

📊 Full hosted report here. [⚠️ does not contain conclusion or background context]<br>p.s if you want the exact skill to build the report just respond to this email

Summary

The analysis covers 17 months of completed Linear tickets, January 2025 to May 2026, split at the September 2025 adoption point: 2,410 tickets in total, 376 of them bug-labelled.<br>At a glance:<br>+180% uplift in output per FTE

Ratio of bugs:output is no different between pre and post AI (adjusting for major refactor in March).

Token consumption starting to stabilise and decline.

Adoption: did the whole team use it?

Total throughput rose by 41% (~117 → ~164 tickets/month), which undersells the attrition story. Accounting for this, per engineer output went from ~9.2 to ~25.7 tickets/month - roughly +180%

The gain shows up across both teams, not in one or two power users: Backend rose +158% (~7.4 → ~19 per engineer) and Frontend +258% (~16 → ~59), though the Frontend figure is volatile on a small team and is best read as directional. The signal is a broad, sustained uplift.<br>Share AI Builder Series

Bug velocity: quality or slop?

No defect explosion, and no miracle either. Bugs were already being closed at ~17–31/month before AI; afterward they rose a modest +52%, in line with simply doing more work. Bug share held steady in a 12–27% band the whole period, and bug cycle time actually fell (0.54 → 0.19 days). This is a nice win!

Adjusting for March major refactor (non-normal), the average ratio would restore to somewhere between 15% - 17%, which is lower than pre-AI era.<br>Tokens: did spend stabilise?

Based on raw data — No. The expectation was a spike followed by a plateau as scaffolding improved. Instead, once all tooling sources are counted (the first pass undercounted by ~44%), tokens per ticket rose from ~10M (Nov–Jan) to ~16M (Feb–May), and total monthly spend climbed to a ~4.2B peak in March....