Profiling | Internals for Interns📚<br>Understanding the Go Runtime (11 of 11)<br>▼1.<br>The Bootstrap<br>2.<br>The Memory Allocator<br>3.<br>The Scheduler<br>4.<br>The Garbage Collector<br>5.<br>The System Monitor<br>6.<br>The Network Poller<br>7.<br>Slices, Maps, and Channels<br>8.<br>The select Statement<br>9.<br>Stacktraces<br>10.<br>The Reflect Package<br>11.<br>Profiling<br>You are here
In the previous article<br>we took apart the reflect package and found that its magic is mostly the compiler leaving very good notes — type descriptors frozen into read-only data at build time, and a package that knows how to walk them. The whole article was about reading metadata that was already sitting in memory before main even started.<br>Today we shift the perspective. Profiling is the runtime catching your program in motion — sampling what it’s doing and where it’s spending its time, then accumulating that into something you can open with go tool pprof. The heart of every sample is a call stack , produced with the same unwinder we saw in the Stacktraces article<br>. So profiling is really live moment-catching sitting on top of build-time stack-reading.<br>But Go doesn’t have just one type of profile — it provides five : CPU, heap, block, mutex, and goroutine. At first they look like five unrelated subsystems, but they all share the same skeleton, and once you see it, the whole thing collapses into one idea repeated five ways. (A sixth, a goroutine-leak profile, is on its way in Go 1.27 — Alex Rios has a great series of posts<br>digging into it — but we’ll stick to the five that ship today.) Let’s start there.<br>The shape they all share<br>The deepest similarity between all five profiles is the thing you actually walk away with: the file . No matter which profile you collected, what lands on disk is the same format — and it’s surprisingly simple: the pprof profile, a gzip-compressed protocol buffer<br>. It’s not even Go-specific; it’s the same format Google’s C++ profiler (gperftools) emits, and one go tool pprof shares with the wider pprof ecosystem. So before we look at how each profile is collected, let’s look at what they’re all collected into.<br>At the top level it’s one Profile message, and the part that matters is just a handful of repeated fields:<br>message Profile {<br>repeated ValueType sample_type = 1; // what each number in a sample means<br>repeated Sample sample = 2; // the actual data<br>repeated Mapping mapping = 3; // the loaded binary / shared libraries<br>repeated Location location = 4; // a PC, resolved to a place in the code<br>repeated Function function = 5; // name, file, start line<br>repeated string string_table = 6; // every string, deduplicated
The clever part is how little is stored inline. A Sample is almost nothing — a list of values and a list of location IDs, leaf first:<br>message Sample {<br>repeated uint64 location_id = 1; // the call stack, as references<br>repeated int64 value = 2; // e.g. [sample count, cpu nanoseconds]<br>repeated Label label = 3; // extra key/value tags attached to the sample
Each of those pieces lives in its own table inside the Profile, and they all reference each other by id. Put together, the whole thing looks like this (a simplified visual of it):
What the diagram makes visible is the indirection. A Sample never holds a function name or even a stack frame — it holds a list of location_ids, one per frame in the stack trace, leaf first. Follow one of those into the location table and you reach a Location, which points (via a Line) at a Function and also records which Mapping it came from — the loaded binary or shared library that address lives in. The Function finally holds the human-readable bits — name, file, start line — except those aren’t strings either; they’re integer indices into the one shared string_table at the bottom. So it’s a chain of lookups: sample → location → function → string. And nothing is duplicated: the same Location, Function, and string are referenced by every sample that touches that spot in your code, so a stack that shows up in ten thousand samples stores its function names just once.<br>This is also where the abstract “value” of each profile gets its meaning: the sample_type declares what the numbers in each sample actually measure. That’s the one part of the shape that genuinely differs between profiles — same container, different labels on the columns — so we’ll fill it in as we get to each profile type in the rest of the article.<br>So much for what a profile is. Now let’s see how the runtime actually fills one in.<br>How the data gets there<br>Whichever profile you’re collecting, one thing is constant: somewhere along the way the runtime captures a call stack with the unwinder, and at the very end it all comes out as the pprof file we just saw. How a stack gets from the one to the other, though, is not the same across the five — and that difference is the real story.<br>The five sort into three collection models :<br>CPU records...