protobuf-py: Protobuf for Python, without compromises · Buf<br>Skip to main content 11.2kLoginSign UpContact us

Blog July 7, 2026<br>Team Buf

protobuf-py: Protobuf for Python, without compromises

Today we’re announcing protobuf-py, a Protocol Buffers library for Python written completely from scratch. It passes every binary and JSON case in the Protobuf conformance suite across proto2, proto3, and editions, and it supports extensions, custom options, unknown fields, dynamic messages, and well-known types. It generates readable, typed Python, has no runtime dependencies, and runs on pure Python 3.10+. With its Rust accelerator installed, it’s just as fast for production workloads as upb, the C engine Google’s Python package runs on.

For Python developers, the choice has historically been between a complete Protobuf implementation and a library that feels like Python. Google’s package is complete, but it carries an API shaped by C++ and Java. betterproto is pleasant, but it gives up too much of the spec. protobuf-py gives Python developers both.

Why build another Protobuf runtime?

Python is too important for Protobuf to feel like an afterthought. It sits in data pipelines, ML systems, AI agents, infrastructure scripts, RPC services, and developer tooling. Still, the experience of using protobuf in Python does not match what developers expect from such a popular language. Google’s package is complete and battle-tested, but the API and generated code still feel like a binding around someone else’s runtime. betterproto proved that Python developers wanted something nicer, but it never implemented the whole spec. grpcio brought the same problem into RPC. It’s powerful and widely used, but painful to build around.

We originally set out to fix the RPC layer with connect-py, a ConnectRPC implementation that speaks both Connect and gRPC. But the transport layer was not enough. A good RPC stack still depends on the messages underneath it, and Python did not have the Protobuf runtime we wanted as the foundation. We wanted something complete enough for real schemas, readable enough for everyday Python, and fast enough that nobody has to apologize for choosing it.

protobuf-py is the result of asking whether those constraints could all be true at once. A complete implementation built for Python, it provides the spec coverage, generated code, and performance profile we wanted connect-py to stand on, without wrapping Google’s runtime or trimming the spec.

Why Google’s package feels the way it does

Install protobuf from PyPI and the engine you normally get is upb, written in C. Your message lives in a C arena, while the Python object is a handle into that arena. Reading a field crosses into C, finds the value, and materializes a Python object on the way back.

It is a good way to share an engine among multiple languages, but it leaves fingerprints everywhere:

Generated _pb2.py files are hard to read because there is not much Python to read. Classes are assembled at import time to configure the C engine, so go-to-definition lands on a wall of serialized descriptor bytes.

SerializeToString, HasField, WhichOneof, and CopyFrom make up a Python API designed for C++ ergonomics. HasField raises if you call it on a proto3 scalar. WhichOneof returns a string you hand back to getattr to retrieve a value it already located.

Generated imports are absolute and break the moment you nest them in a package. There is a separate tool called fix-protobuf-imports that exists on PyPI only to rewrite Google’s output.

Types register in one process-wide pool, so importing two builds of the same .proto raises at runtime.

None of these are random defects in the binding. They are what happens when a Python API is designed for consistency in a primarily C++/Java codebase, rather than as an idiomatic Python package. The clumsy API and the speed shipped together, and for years you took both or neither.

What we built instead

protobuf-py keeps your message in Python. It is a plain object with __slots__, and its fields are ordinary Python values: ints, strings, lists, submessages. A Rust accelerator speeds up the operations that need it, mostly parsing and serializing, and writes the results straight into the object. Once parsing finishes, reading a field is just accessing a Python attribute.

Because the data is Python, the generated code is real code. protoc-gen-py emits a class you can read:

class User(Message[_UserFields]):<br>__slots__ = ("first_name", "last_name", "active", "manager", "locations", "projects", "contact")

if TYPE_CHECKING:<br>def __init__(<br>self,<br>*,<br>first_name: str = "",<br>last_name: str = "",<br>active: bool = False,<br>manager: User | None = None,<br>locations: list[str] | None = None,<br>projects: dict[str, str] | None = None,<br>contact: Oneof[Literal["email"], str] | Oneof[Literal["phone"], str] | None = None,<br>) -> None: ...

first_name: str<br>last_name: str<br>active: bool<br>manager: User | None<br>locations: list[str]<br>projects: dict[str,...