Python 3.13 gets a JIT

Happy New Year everyone! In late December 2023 (Christmas Day to be precise), CPython core developer Brandt Bucher submitted a little pull-request to the Python 3.13 branch adding a JIT compiler.

This change, once accepted would be one of the biggest changes to the CPython Interpreter since the Specializing Adaptive Interpreter added in Python 3.11 (which was also from Brandt along with Mark Shannon).

In this blog post, we’re going to have a look at this JIT, what it is, how it works and what the benefits are.

What is a JIT?¶

JIT, or “Just in Time” is a compilation design that implies that compilation happens on demand when the code is run the first time. It’s a very broad term that could mean many things. I guess, technically the Python compiler is already a JIT because it compiles from Python code into Bytecode.

What people tend to mean when they say a JIT compiler, is a compiler that emits machine code . This is in contrast to an AOT (Ahead of Time) compiler, like the GNU C compiler, GCC or the Rust compiler rustc which generates the machine code once and distributes as a binary executable.

When you run Python code, it is first compiled into bytecodes. There are plenty of talks and videos about this process online so I don’t want to rehash this too much, but what is important to note about Python bytecodes is:

They mean nothing to the CPU and require a special bytecode interpreter loop to execute

They are high level and can equate to 1000’s of machine instructions

They are type agnostic

They are cross-platform

For a very simple Python function f() that defines a variable a and assigns the value 1:

def func():<br>a = 1<br>return a

It compiles to 5 bytecode instructions, which you can see by running dis.dis:

>>> import dis<br>>>> dis.dis(func)<br>34 0 RESUME 0

35 2 LOAD_CONST 1 (1)<br>4 STORE_FAST 0 (a)

36 6 LOAD_FAST 0 (a)<br>8 RETURN_VALUE

I have a more interactive disassembler called dissy as well if you want to try something more complicated.

For this function, Python 3.11 compiled into the instructions LOAD_CONST, STORE_FAST, LOAD_CONST, and RETURN_VALUE. These instructions are interpreted when the function is run by a massive loop written in C.

If you were to write a very crude Python evaluation loop in Python equivalent to the one in C, it would look something like this:

import dis

def interpret(func):<br>stack = []<br>variables = {}<br>for instruction in dis.get_instructions(func):<br>if instruction.opname == "LOAD_CONST":<br>stack.append(instruction.argval)<br>elif instruction.opname == "LOAD_FAST":<br>stack.append(variables[instruction.argval])<br>elif instruction.opname == "STORE_FAST":<br>variables[instruction.argval] = stack.pop()<br>elif instruction.opname == "RETURN_VALUE":<br>return stack.pop()

def func():<br>a = 1<br>return a

If you gave this interpreter our test function, it would execute them and print the results:

print(interpret(func))

This loop with a big switch/if-else statement is an equivalent, albeit simplified version of how CPython&rsquo;s interpreter loop works. CPython is written in C and compiled by a C compiler. For the sake of simplicity I&rsquo;ll build out this example in Python.

For our interpreter, everytime you want to run the function, func it has to loop through each instruction and compare the bytecode name (called the opcode) with each if-statement. Both this comparison and the loop itself add an overhead to the execution. That overhead seems redundant if you run the function 10,000 times and the bytecodes never change (because they are immutable). It would be more efficient to instead generate the code in a sequence instead of a evaluating this loop every time you call the function.

This is what a JIT does. There are many types of JIT compiler. Numba is a JIT. PyPy has a JIT. Java has lots of JITs. Pyston and Pyjion are JITs.

The JIT that is proposed for Python 3.13 is a copy-and-patch JIT.

What is a copy-and-patch JIT?&para;

Never heard of a copy-and-patch JIT? Don&rsquo;t worry, nor had I and nor have most people. It&rsquo;s an idea only proposed recently in 2021 and designed as a fast algorithm for dynamic language runtimes.

I&rsquo;ll try and explain what a copy-and-patch JIT is by expanding our interpreter loop and rewriting it as a JIT. Before, the interpreter loop did two things, first it interpreted (looked at the bytecode) then it executed (ran the instruction). What we can do instead is to separate those tasks and have the interpreter output the instructions and not execute them.

A copy-and-patch JIT is the idea that you copy the instructions for each command and fill-in-the-blanks for that bytecode arguments (or patch ). Here&rsquo;s a rewritten example, I keep the loop very similar but each time I append a code string with the Python code to execute:

def copy_and_patch_interpret(func):<br>code = 'def f():\n'<br>code += ' stack = []\n'<br>code += ' variables = {}\n'<br>for instruction in dis.get_instructions(func):<br>if...