Detecting goroutine leaks with synctest/pprof
Anton Zhiyanov<br>projects<br>books<br>blog<br>about
Deadlocks, race conditions, and goroutine leaks are probably the three most common problems in concurrent Go programming. Deadlocks usually cause panics, so they're easier to spot. The race detector can help find data races (although it doesn't catch everything and doesn't help with other types of race conditions). As for goroutine leaks, Go's tooling did not address them for a long time.<br>A leak occurs when one or more goroutines are indefinitely blocked on synchronization primitives like channels, while other goroutines continue running and the program as a whole keeps functioning. We'll look at some examples shortly.
Things started to change in Go 1.24 with the introduction of the synctest package. There will be even bigger changes in Go 1.26, which adds a new experimental goroutineleak profile that reports leaked goroutines. Let's take a look!<br>A simple leak •<br>Detection: goleak •<br>Detection: synctest •<br>Detection: pprof •<br>Algorithm •<br>Range over channel •<br>Double send •<br>Early return •<br>Take first •<br>Cancel/timeout •<br>Orphans •<br>Final thoughts<br>A simple leak<br>Let's say there's a function that runs the given functions concurrently and sends their results to an output channel:<br>// Gather runs the given functions concurrently<br>// and collects the results.<br>func Gather(funcs ...func() int) chan int {<br>out := make(chan int)<br>for _, f := range funcs {<br>go func() {<br>out f()<br>}()<br>return out
And a simple test:<br>func Test(t *testing.T) {<br>out := Gather(<br>func() int { return 11 },<br>func() int { return 22 },<br>func() int { return 33 },
total := 0<br>for range 3 {<br>total += out
if total != 66 {<br>t.Errorf("got %v, want 66", total)
PASS
Send three functions to be executed and collect the results from the output channel. The test passed, so the function works correctly. But does it really?<br>Let's pass three functions to Gather without collecting the results, and count the goroutines:<br>func main() {<br>Gather(<br>func() int { return 11 },<br>func() int { return 22 },<br>func() int { return 33 },
time.Sleep(50 * time.Millisecond)<br>nGoro := runtime.NumGoroutine() - 1 // minus the main goroutine<br>fmt.Println("nGoro =", nGoro)
nGoro = 3
After 50 ms — when all the functions should definitely have finished — there are still three running goroutines (runtime.NumGoroutine). In other words, all the goroutines are stuck.<br>The reason is that the out channel is unbuffered. If the client doesn't read from it, or doesn't read all the results, the goroutines inside Gather get blocked on sending the f() result to out.<br>Let's modify the test to catch the leak.<br>Detecting the leak: goleak<br>Obviously, we don't want to rely on runtime.NumGoroutine in tests — such check is too fragile. Let's use a third-party goleak package instead:<br>// Gather runs the given functions concurrently<br>// and collects the results.<br>func Gather(funcs ...func() int) chan int {<br>out := make(chan int)<br>for _, f := range funcs {<br>go func() {<br>out f()<br>}()<br>return out
func Test(t *testing.T) {<br>defer goleak.VerifyNone(t)
Gather(<br>func() int { return 11 },<br>func() int { return 22 },<br>func() int { return 33 },
playground ▶<br>--- FAIL: Test (0.44s)<br>goleak_test.go:28: found unexpected goroutines:
Goroutine 8 in state chan send, with play.Gather.func1 on top of the stack:<br>play.Gather.func1()<br>/tmp/sandbox4216740326/prog_test.go:16 +0x37<br>created by play.Gather in goroutine 7<br>/tmp/sandbox4216740326/prog_test.go:15 +0x45
Goroutine 9 in state chan send, with play.Gather.func1 on top of the stack:<br>play.Gather.func1()<br>/tmp/sandbox4216740326/prog_test.go:16 +0x37<br>created by play.Gather in goroutine 7<br>/tmp/sandbox4216740326/prog_test.go:15 +0x45
Goroutine 10 in state chan send, with play.Gather.func1 on top of the stack:<br>play.Gather.func1()<br>/tmp/sandbox4216740326/prog_test.go:16 +0x37<br>created by play.Gather in goroutine 7<br>/tmp/sandbox4216740326/prog_test.go:15 +0x45
The test output clearly shows where the leak occurs.<br>Goleak uses time.Sleep internally, but it does so quite efficiently. It inspects the stack for unexpected goroutines up to 20 times, with the wait time between checks increasing exponentially, starting at 1 microsecond and going up to 100 milliseconds. This way, the test runs almost instantly.<br>Still, I'd prefer not to use third-party packages and time.Sleep.<br>Detecting the leak: synctest<br>Let's check for leaks without any third-party packages by using the synctest package (experimental in Go 1.24, production-ready in Go 1.25+):<br>// Gather runs the given functions concurrently<br>// and collects the results.<br>func Gather(funcs ...func() int) chan int {<br>out := make(chan int)<br>for _, f := range funcs {<br>go func() {<br>out f()<br>}()<br>return out
func Test(t *testing.T) {<br>synctest.Test(t, func(t *testing.T) {<br>Gather(<br>func() int { return 11 },<br>func() int { return 22 },<br>func() int { return 33 },<br>synctest.Wait()<br>})
--- FAIL: Test (0.00s)<br>panic: deadlock: main bubble goroutine has exited but blocked goroutines remain [recovered, repanicked]
goroutine...