The Hub of Heliopolis - Busting performance issues, AI edition

Coding agents are fast becoming the main generators of new source code, but they are also being used for code review, finding and fixing bugs. How can they be used to the best extent to discover and fix performance issues?

Table of contents:<br>Enter AI

bytecode: Our Case Study<br>A Bit of Setup Required

Establishing the Baseline

Communicating Our Intent to the Agent

On a Mission

A Guided Tour of Profiles<br>A Bonus: Learning to Read Profiles

Is Profiling Data Really Necessary?

The Takeaway

In a previous post we looked at how to find and resolve performance issues<br>lurking in any codebase by simply running some of its code through a profiler.<br>The simplest thing we can do, especially when we come to a new codebase that is<br>totally unknown to us is to run its test suite (or a subset of it) and see what<br>code paths are being exercised. When we look at them through a profiler, the<br>typical approach is to start investigating its plateaux, that is functions at<br>the top (or bottom, depending on the orientation of the flame graph!) that have<br>the largest own time. This is generally where the largest opportunity of<br>improvements are. Of course, there could be cases where a function is simply<br>repeating unnecessary work, and this won't show up through a mere visual plateau<br>scan.

Enter AI

Can we automate this process, and perhaps put it on steroids, with the prowess<br>of the most powerful, state-of-the-art coding agents?

Modern coding agents have reached a level of sophistication where they can<br>effectively understand and modify codebases with good practical results.<br>Benchmarks such as SWE-bench (which tests multi-file bug fixes in real GitHub<br>repositories) and LiveCodeBench (which uses fresh competitive programming<br>problems to avoid data contamination) show that frontier models now achieve high<br>pass rates. However, there still is quite the margin for improvement: studies on<br>real-world class-level code generation show a significant gap between synthetic<br>benchmark performance (84-89%) and actual project performance (25-34%)<br>(arxiv-2510.26130), highlighting both the progress made and the room for<br>improvement in practical coding scenarios.

In this post we will explore how well a coding agent can come up with<br>performance improvements when it is provided with focused profiling data. The<br>focus, however, is on the method and the tooling involved, rather than<br>benchmarking a set of models to determine which one is better at finding<br>performance issues.

bytecode: Our Case Study

To make the exposition concrete we will focus on an actual project: we will try<br>to improve the performance of the bytecode library.

bytecode is a Python library for generating and modifying Python bytecode. It<br>provides an abstraction over Python's low-level bytecode instructions, allowing<br>you to programmatically create bytecode objects, convert them to actual code<br>objects, and even inspect and modify existing code's bytecode. It is useful for<br>metaprogramming, dynamic code generation, and bytecode analysis.

These are the tools that we will use for our little experiment, all free:

Austin, for Python profiling, easily installed with "pip install<br>austin-dist"

The Austin VS Code extension, easily installed from the VS Code<br>marketplaces. The latest version comes with an MCP server that can feed<br>profiling data and flame graph navigation commands to a coding agent.

Opencode with the MiniMax M2.5 free model as our coding agent setup.

The Opencode VS Code extension to never leave VS Code throughout our<br>experiment

A Bit of Setup Required

After having cloned the bytecode repository that we will be working on, and<br>opened it in VS Code, there is a minimum of setup required. We need to allow the<br>coding agent to discover the Austin MCP. If you're using GitHub Copilot instead<br>of Opencode, there is no extra setup required for this, since the Austin MCP<br>server integrates natively with that. Otherwise we need to allow other coding<br>agents to discover the server. This is as simple as running the Austin:<br>Generate .mcp.json command: press<br>Ctrl+Shift+P, type austinmcp and press<br>Enter. That's it. Well, not quite, because we decided to use Opencode<br>for our experiment, and as at the moment of writing, it still does not support a<br>.mcp.json file. So instead we open an Opencode session with<br>Ctrl+Escape and politely ask the agent to convert the<br>.mcp.json configuration to something that opencode understands

The Austin MCP server uses an ephemeral port to run, so every time you re-open<br>VS Code there will be a new port assigned to it. If a .mcp.json was<br>previously created, the Austin extension will update the port automatically so<br>that you don't have to regenerate it every time. However, Opencode won't do<br>this unless you instruct it to.

After Opencode configured itself, you might have to start a new session for it<br>to pick up the connection to the server.

We have asked Opencode to configure itself based on the contents...