Row, Table, Join: No Longer Primitives — datumwise<br>skip to content Positions

Row, Table, Join: No Longer Primitives

A position from datumwise · 2026-07-30

For twenty years, one movement has been quietly taking the database apart — layer by layer, from the bottom up. It never had a manifesto. It has a pattern, and the pattern has now reached the last layer.

The pattern, four times over

Storage fell first. The row-oriented file — data laid out the way transactions arrive — turned out to be a convention, not a necessity. Columnar formats decomposed it: values of one variable, stored together, typed together. Parquet made it the lake’s native tongue; Arrow made it the in-memory standard. And the commercial verdict is in — this is no longer an insurgency. Snowflake is columnar under the hood. Databricks is columnar under the hood. Every serious analytical engine of the last fifteen years made the same choice. The war at the storage layer is simply over.

Execution fell next. The monolithic engine — parser to optimizer to executor, one vendor, one binary — decomposed into composable parts. DuckDB put a full analytical engine in a process. DataFusion and Velox made execution a library you embed. The engine stopped being a place you send data and became a component you assemble.

Plans are falling now. The query plan — always an internal, engine-private data structure — is decomposing into a portable artifact. Substrait standardizes the plan itself: a typed, serialized, engine-neutral description of a computation, produced by one system and executed by another. DataFusion speaks it in both directions; DuckDB consumes and produces it; Ibis compiles to it; and the earliest travelers are already trying to carry semantics on it as cargo. The plan stopped being implementation detail and became interchange.

Transport fell alongside. ADBC did for result-movement what the rest did for their layers: a standard, columnar, engine-neutral wire.

Four layers, one identical move each time: something the industry treated as a primitive turned out to be an inheritance — and decomposed into typed, lawful, portable parts native to data itself. The row-oriented file was not fundamental. The monolithic engine was not fundamental. The private plan was not fundamental. Each decomposition released a wave of capability the old bundle had been suppressing.

So here is the question the movement has been walking toward for twenty years without asking it out loud: what about the concepts themselves? The row. The table. The join. Are those bedrock — or are they the last inheritance left standing?

Where row, table, and join actually come from

Be fair to them first, because they were never arbitrary. The row is an event — something happened, several facts arrived together. The table is an entity ledger — a collection of things of one kind. The join is a connection — this relates to that. Row, table, and join are the physical world’s ontology, projected onto data: honest, intuitive, and productive for fifty years precisely because the projection is so natural. We even borrowed the semantics along with the shapes — governing data operations by reasoning about the events, entities, and connections the data depicts, as if the laws of the referents were the laws of the records.

That is the actual finding, and it is deeper than “these are conventions”: data has been living under a borrowed ontology. And a borrowed ontology can describe data while being unable to govern it — the physical world’s concepts carry no aggregation laws, no grain discipline, no account of what absence means. The recurring silent failures of analytics are not accidents of implementation. They are the interest paid on the loan.

The last layer just moved

We are publishing the answer, and it is the movement’s own move, one level deeper: at the theoretical foundation, row, table, and join are no longer primitives — they are projections of another world’s primitives. Data has its own ontology, autonomous and more natural to it, with meaning defined by rules over abstract structural elements — and it has now been written down.

The row is a co-recording: several observations that happened to be captured together. Useful for bookkeeping and write-paths — and carrying no semantic authority. Nothing about a variable’s meaning, its lawful aggregations, or its population lives at the row.

The table is a recording convention. The industry’s own oldest term gives the game away: we have called tables “structured data” for fifty years, but a table’s arrangement is tabulation — bookkeeping order. The structure that supports identity and law begins one level down, at the column: many observations of one variable, organized by one typed binding. The structure was never in the table.

The join — and this one will sting — is not one operation at all. It is the physical world’s “connection” projected onto data, and under the projection it bundles seven distinct decisions into one...