There Is a Mountain Lion in the Datacenter - HFT University

LIVE SUBMISSIONS — 3 DAYS

There Is a Mountain Lion in the Datacenter

Published: May 29, 2026

There Is a Mountain Lion in the Datacenter

P-22 in the backyard, EINTR in the syscall return, and the oldest guild fight in computing.

On the twelfth of December, 2022, a Los Feliz homeowner walked out the back door of a house in the hills above Hollywood and found a mountain lion sleeping under the deck. The homeowner called the California Department of Fish and Wildlife. The lion, tagged P-22, had been the unofficial mascot of Los Angeles for a decade. He had crossed the 405 and the 101 to reach Griffith Park, lived in the middle of a city of four million people, been blamed for the death of a koala at the LA Zoo, been photographed in front of the Hollywood sign, and ended up under a Los Feliz deck because he had been hit by a car and was dying. He was euthanized five days later. The city wrote love letters in the LA Times.

The homeowner who found him was not in any moral sense being invaded. The lion had the older claim. Griffith Park existed before the suburb did. The hills had been a lion habitat for as long as there had been hills. The Los Feliz house was the new arrival, built on the assumption that the design of the house and the city around it would keep the wilderness at a polite distance. The design turned out to be insufficient on the morning of December 12th. The lion had an opinion about decks.

That is the relationship between the systems programmer and the application programmer in modern infrastructure. The application programmer built the suburb. The systems programmer is the lion. The wilderness, in this telling, is the hardware: the cache, the page table, the syscall return value, the signal-delivery race, the CCX boundary, the four kinds of memory the engineer now has to keep straight. The application programmer was sold a model of the machine in which the wilderness had been kept at a polite distance. The wilderness has a different view of the matter, and increasingly the view shows up under the deck at three in the morning.

A couple of weeks ago I wrote about a YouTube video that compressed forty years of C++ history into one geographic slogan and treated the existence of POSIX EINTR as evidence that the operating system was "freaking out." The piece focused on the rhetorical compression. What it did not name, because the piece was already long enough, is the prejudice underneath the rhetorical move. The complaint that read() returns EINTR to userspace, when stripped of the dramatic terminal demo, is the complaint of a Los Feliz homeowner finding a mountain lion under their deck. The complaint is that the wilderness was supposed to stay in the wilderness. The reply is that it never did, and the suburb was built on the assumption that it would.

It is the oldest guild fight in computing. It is older than the video, older than C++, older than Unix in any form that the modern reader would recognize. It is the fight between the people who believe the operating system exists to launder hardware reality away from the application programmer, and the people who believe the application programmer has to know enough hardware reality to ship a system that runs.

Application Programmer vs Systems Programmer

The cleanest way to name the divide is not by salary, not by geography, not by ethnicity, not by the colour of anyone's badge. It is by the layer of the stack where the engineer signs their commits.

The application programmer writes code above the syscall boundary. Their world is composed of types, control flow, and abstractions the language gives them. Their mental model of the machine is the model the language presents. If the language hides a thing, that thing does not exist in their workspace. When the abstraction leaks, it is a bug in the abstraction, not a feature of the world.

The systems programmer writes code that touches the boundary. Their world includes the cache hierarchy, the page table, the syscall table, the NIC's descriptor ring, the scheduler's runqueue, and the signal-delivery race the kernel can produce between any two instructions. Their mental model of the machine is the machine. When the abstraction leaks, it is the abstraction confessing what was always there.

The two populations do not necessarily dislike each other. They have, however, been talking past each other for fifty years, and a recurring feature of the conversation is the application programmer treating the systems programmer's concerns as evidence of the systems programmer's bad taste. The systems programmer's reply is usually a measurement.

The Classical Statement of the Elitism

Edsger Dijkstra wrote On the cruelty of really teaching computing science in December 1988 (EWD 1036). The essay is the cleanest single statement in the literature of the position that computing should be taught as a branch of mathematics, that the engineering side...