eBPF in Go: Observability for AI-Generated Services | by Cheikh seck | Jun, 2026 | MediumSitemapOpen in appSign up<br>Sign in

Medium Logo

Get app<br>Write

Search

Sign up<br>Sign in

eBPF in Go: Observability for AI-Generated Services

From blind debugging to kernel-level visibility in 10 minutes

Cheikh seck

8 min read·<br>3 days ago

Listen

Share

I shipped a service in March that looked fine in staging. P95 latency was 40ms, errors were under 0.1%, everything green. Two weeks later, on a Tuesday afternoon during our weekly demo, the service started timing out. Not gradually. Suddenly. P95 jumped to 4 seconds in under a minute.<br>I had no idea why.<br>The Go service was clean. Standard net/http, a goroutine pool, Redis for caching. I'd even added Prometheus metrics. But the metrics showed nothing useful. CPU was normal. Memory was normal. GC pauses were normal.<br>The problem was in the kernel. Specifically, in how my service interacted with the network stack, the file system, and other kernel subsystems. Prometheus couldn’t see any of it because those interactions happen below the Go runtime.<br>That’s when I started learning eBPF. And that’s why I’m writing this.<br>If you use AI to generate Go services, you have a visibility problem. The Go runtime can tell you what your code is doing. eBPF can tell you what your code is doing to the kernel. You need both.<br>This tutorial shows you how to add eBPF-based observability to a Go service using Cilium’s eBPF library. I’ll explain what eBPF is, why it matters for AI-generated code, and walk through a working example you can run in under 10 minutes.<br>Press enter or click to view image in full size

What is eBPF?<br>eBPF (extended Berkeley Packet Filter) is a technology that lets you run sandboxed programs inside the Linux kernel without modifying kernel source code or loading kernel modules. The kernel was first extended to support eBPF in Linux 3.18 (2014), and the technology has matured significantly since then.<br>According to the eBPF Foundation, eBPF programs can be attached to various kernel hooks: network events, system calls, function entry/exit points, and more. They execute safely in the kernel and can pass data to userspace programs through shared maps.<br>In practice, this means you can observe and modify kernel behavior from userspace, without recompiling the kernel or loading risky kernel modules.<br>For Go services, eBPF lets you see:<br>Which syscalls your service makes and how long they take<br>TCP connections, retransmissions, and drops<br>File system operations<br>Security events (privilege escalation attempts, unusual syscalls)

Why AI-Generated Code Makes eBPF More Important<br>When I write Go code by hand, I have context. I know which libraries I chose and why. I know which syscalls they’ll trigger. I can reason about the kernel interactions.<br>When an LLM writes Go code, I don’t have that context.<br>This is where eBPF comes in. It gives you X-ray vision into what your AI-generated service is actually doing at the kernel level. You can see the syscall patterns, the network traffic, the I/O behavior, even if you didn’t write the code yourself.

Prerequisites<br>You need:<br>Linux kernel 4.19+ (for stable eBPF support)<br>Go 1.21+<br>clang and llvm (for compiling eBPF programs)<br>bpftool (for loading eBPF programs)<br>On Ubuntu:<br>sudo apt install clang llvm libbpf-dev linux-headers-$(uname -r) bpftoolVerify your kernel supports eBPF:<br>uname -r # Should be 4.19 or higherStep 1: Install the eBPF Go Library<br>Cilium’s eBPF is the most mature Go library for eBPF. It’s used in production by Cilium, Falco, and many other projects.<br>go get github.com/cilium/ebpf<br>go get github.com/cilium/ebpf/link<br>go get github.com/cilium/ebpf/perfStep 2: Write an eBPF Program<br>eBPF programs are written in C and compiled to bytecode. Create a file called trace.c:<br>#include "vmlinux.h"<br>#include

struct event_t {<br>u32 pid;<br>u64 timestamp;<br>char comm[16];<br>};

struct {<br>__uint(type, BPF_MAP_TYPE_PERF_EVENT_ARRAY);<br>__uint(key_size, sizeof(u32));<br>__uint(value_size, sizeof(u32));<br>} events SEC(".maps");

SEC("tracepoint/syscalls/sys_enter_openat")<br>int trace_openat(struct trace_event_raw_sys_openat *ctx) {<br>struct event_t event = {};<br>u64 pid_tgid = bpf_get_current_pid_tgid();<br>event.pid = pid_tgid >> 32;<br>event.timestamp = bpf_ktime_get_ns();<br>bpf_get_current_comm(&event.comm, sizeof(event.comm));

bpf_perf_event_output(ctx, &events, BPF_F_CURRENT_CPU,<br>&event, sizeof(event));<br>return 0;

char _license[] SEC("license") = "GPL";This program traces the openat syscall (used for file opens) and sends events to userspace.<br>You can always ask your LLM to update the C program to track the system actions you want.<br>(See end of post for link to Github with full source code)<br>Step 3: Load and Attach the eBPF Program<br>Before Go can load the eBPF program, it needs generated bindings for the C code. The easiest way to do that is with bpf2go.<br>Because go generate only scans .go files, place the following directive at the top of your main.go:<br>//go:generate go run...