Running Multiple Interpreters in Python Code — Incredible Speed | alexeev-dev notes

Running Multiple Interpreters in Python Code — Incredible Speed

19 May, 2026

On June 5, 2025, PEP-0734 was accepted. Judging by the information on the official website, it is a continuation of PEP-0554. This PEP proposes adding a new interpreters module to support checking, creating, and running code in multiple interpreters within the current process. And going further, it is a continuation of PEP-0684, which proposes one GIL per interpreter.

Several full-fledged interpreters working side by side. What are the advantages?

One process;

One thread, but you can manually create more;

Data between interpreters is always transferred through serialization similar to pickle, including primitive types;

One GIL per interpreter, you can still get the benefits of true multitasking over the network;

Works with asyncio.

GIL (Global Interpreter Lock) in Python is a global interpreter lock. This is a mechanism built into the standard Python implementation (CPython) that prevents simultaneous execution of Python bytecode by multiple threads.

Among the downsides — this PEP significantly changed the C code, and therefore the stability of C extensions is not always guaranteed. By the way, I talked about how to create them in my previous article.

There are several important non-technical aspects about the process of creating this feature:

PEP-734 and Free-Threading do very similar things — they allow implementing true multitasking, but in different ways;

Initially, subinterpreters appeared in 3.10 only as a C-API;

There is a separate PyPI package (https://pypi.org/project/interpreters-pep-734/) with this code;

The Python part in the form of PEP-734 was added to 3.14 after the feature freeze;

It was originally planned to add it as the interpreters module, but at the last moment it became concurrent.interpreters, there's a lengthy discussion about this.

The PEP adds the interpreters module (concurrent.interpreters). This includes Interpreter objects representing the underlying interpreters. The module also provides a basic Queue class for communication between interpreters.

For users, there will be a simple API:

interp = interpreters.create()<br>try:<br>interp.exec('print("Hello from PEP-554")')<br>finally:<br>interp.close()

Right now, if you use Python 3.14, you can import the concurrent.interpreters package:

import concurrent.interpreters as interpreters

interp = interpreters.create()

a = 15<br>print(f"A in main: {a}")

try:<br>interp.exec('print("Hello from PEP-554")\na = 10\nprint(f"A in subinterp: {a}")')<br>finally:<br>interp.close()

Output when run:

A in main: 15<br>Hello from PEP-554<br>A in subinterp: 10

❯ Why is this PEP important?<br>The interpreters module will provide a high-level interface for the multiple interpreter functionality. The goal is to make the existing multiple-interpreters feature of CPython more accessible to Python code. This is especially relevant now that CPython has a per-interpreter GIL (PEP 684), and people are more interested in using multiple interpreters.

Without a stdlib module, users are limited to the C API, which restricts their ability to experiment with and take advantage of multiple interpreters.

The module will include a basic mechanism for communication between interpreters. Without it, multiple interpreters would be a much less useful feature.

❯ Architecture<br>Essentially, an "interpreter" is a collection of all the runtime state that Python threads need to share together.

Processes in Python can have one or more OS threads executing Python code (or interacting with the C API). Each of these threads works with the CPython runtime.

Interpreters are created through the C API using Py_NewInterpreterFromConfig() (or Py_NewInterpreter(), which is a lightweight wrapper around Py_NewInterpreterFromConfig()). This function does the following:

Creates a new state;

Creates a new thread state;

Sets the thread state as current (current state is needed for interpreter initialization);

Initializes the interpreter state using the thread state;

Returns the thread state (still current).

When a Python process starts, it creates one interpreter state (the "main" interpreter) with one thread state for the current OS thread. Then the Python runtime is initialized using them.

After initialization, the script or module or REPL is executed using them. This execution happens in the interpreter's __main__ module.

When the process finishes executing the requested Python code or REPL in the main OS thread, the Python runtime is finalized in that thread using the main interpreter.

❯ C API<br>Inside, you can find many different C modules. Let's break them down in more detail.

Python/crossinterp.c

This file contains the API for managing actions between isolated interpreters. The foundation, in general.

We'll skip some functions if they are minor (like _Py_GetMainfile), you can view them yourself.

Main...