Reimplementing pf as an eBPF/XDP Dataplane on Linux — nfSensei Blog

Abstract

Stateful firewalling on Linux conventionally lives in netfilter, several layers into the kernel's receive path. XDP [1] moves programmable packet processing to the earliest point in the stack — the driver, before an sk_buff is allocated — but offers a restrictive execution model. We describe PFL, an attempt to express the full pf configuration language [2] and its stateful semantics within that model: a userspace compiler (pfld) that lowers pf.conf to a set of BPF maps, and a Rust/eBPF dataplane that evaluates them as a tail-called pipeline of XDP programs. We discuss how the verifier's 512-byte stack and bounded-execution requirements shape the design, report which pf features map cleanly and which are architecturally precluded, and give measured latency on a controlled (virtualized) benchmark alongside an explicit account of why those numbers are not yet a wire-speed claim.

1. Motivation

I started pfSense in 2004 as a fork of m0n0wall. It is built on FreeBSD, and its firewall is pf — OpenBSD's packet filter, whose configuration language is, in my biased opinion, still the clearest expression of firewall intent anyone has shipped. Two decades of operating that codebase at scale also taught me where its architecture, a product of its era, shows its age: a control plane and a web UI that can drift out of sync, and a dataplane bound to the host kernel's netfilter path.

PFL is not pfSense, and it is not a drop-in replacement for it. It is a narrower experiment with a specific question: can pf's language and stateful semantics be expressed efficiently on Linux's programmable datapath — XDP — rather than on netfilter? The appeal is concrete. XDP runs in the NIC driver (or in hardware-offload mode on capable NICs), before the kernel allocates a socket buffer, and prior work has shown it sustaining packet rates an order of magnitude beyond the conventional path [1][9]. The cost of admission is a far stricter programming model, described in §4.

The contribution here is not a new packet-processing primitive — XDP, LPM tries, and BPF conntrack patterns are established. It is the integration: taking a complete, real-world firewall language with state, NAT, normalization, anchors and policy routing, and showing how much of it survives translation into a verifier-accepted XDP program, where it breaks, and what it costs.

2. Background

2.1 The pf configuration language

pf rules read close to intent: block/pass, direction, quick short-circuiting, keep state, address , macros, anchors, scrub normalization, nat-to/rdr translation, and policy routing via route-to. The language is also a contract with operator muscle memory; a partial imitation that silently drops a clause is worse than none. PFL therefore treats parity — including the awkward corners — as the primary requirement, not a marketing checklist (§5).

2.2 XDP and the eBPF execution model

An XDP program runs on each received frame at the driver level and returns a verdict (PASS, DROP, TX, REDIRECT). It executes only after the in-kernel verifier proves termination and memory safety: bounded loops, a 512-byte stack per program, every pointer access provably in-bounds, and a total complexity ceiling [11]. State persists across invocations only in BPF maps. These constraints are the central engineering problem of this project, not an implementation detail (§4).

2.3 Why not the existing options

netfilter/nftables is mature and full-featured but lives past the driver and the sk_buff allocation. bpfilter aimed to JIT iptables rules to BPF but did not land as a general solution. XDP is widely used for stateless DDoS scrubbing and L4 load balancing — Cloudflare's L4Drop, Meta's Katran [9], and Cilium [10] — but those are purpose-built datapaths, not general stateful firewalls driven by a pf-grade policy language. VPP [12] achieves very high rates in userspace at the cost of taking the NIC away from the kernel entirely. PFL occupies a different point: a general pf firewall that stays inside the kernel's datapath and leaves the interface usable by the host.

3. Architecture

PFL has two halves. pfld (~3,700 lines of Rust) parses pf.conf, lowers it to an intermediate representation, and writes the result into BPF maps; it owns no packet path. The dataplane (~7,000 lines) is written in Rust and compiled to eBPF with aya [8] — a deliberate choice; the verifier is equally strict regardless of source language, and Rust's type system removes a class of pointer-arithmetic mistakes that are easy to make in C at this layer.

3.1 The pipeline as a tail-call graph

A single XDP program cannot hold a full firewall and stay under the complexity ceiling. The datapath is therefore split into stages that tail-call one another through BPF program arrays (PROG_ARRAY_*), with the matching set duplicated on the TC egress hook for outbound rules:

Per-frame pipeline — XDP ingress (TC clsact mirror on...