Abstraction is Expensive | Speculative Branches

Abstraction is Expensive

calendar

Dec 7, 2022

· 10 min read · Software Engineering

Share on:

twitter

facebook

linkedin

copy

As you build a computer system, little things start to show up: maybe that database query is awkward<br>for the feature you are building, or you find your server getting bogged down transferring gigabytes<br>of data in hexadecimal ASCII, or your app translates itself to Japanese on the fly for hundreds of<br>thousands of separate users. These are places where your abstractions are misaligned - your app<br>would be quantitatively better if it had a better DB schema, a way to transfer binary data, or<br>native internationalization for your Japanese users. Each of these misalignments carries a cost.

For many computer systems, abstraction misalignment is where we spend the majority of our resources:<br>both in terms of engineering costs and compute time. Building apps at a high level pays dividends<br>in terms of getting them set up, but eventually the bill comes due, either in the form of tech debt,<br>slow performance, or both. Conversely, systems where abstractions are well-aligned throughout the<br>tech stack, like high-frequency trading systems and (ironically) chat apps, are capable of amazing<br>feats of engineering.

Most small-scale web services that do normal things don't pay much for<br>abstraction misalignment, but large-scale systems and systems that do odd things pay huge costs.<br>Misalignments can also show up as systems age and things change - migrations are difficult, and you<br>would rather add a feature than do a refactor (okay, maybe not you, but your manager definitely does).

When I left Google, I was working on a new storage system that leveraged several cool technologies<br>to go fast. The real power we had, however, was that we could design the stack so that every<br>abstraction was well-aligned with the layers above and below it, eliminating the shims that<br>sucked up lines of code and compute cycles. I saw this kind of system in high-frequency trading,<br>too. I didn't know how good I had it...

Assumptions, Values, and Requirements

Every project is built with a set of assumptions, requirements, and values. These will come to<br>define the constraints under which the system is built, and ultimately the characteristics<br>of the system. Requirements are simple: they are what you absolutely need for your product. For<br>example, if you are building a NoSQL database, you require a key-value interface of some sort and<br>a storage medium of some sort. A project's values define what you want to aim for. Perhaps you<br>are building a performance-focused NoSQL database: In that case, you require a NoSQL database<br>interface, and you value performance. Bryan Cantrill has done a good talk on<br>values. Assumptions are the final pillar: these<br>tell you about the invisible pseudo-requirements you build your system under. For example, most of<br>us assume certain things about programming languages and computing environments: at the very least,<br>most systems are built on the assumption that computers will run them. Breaking that assumption<br>can result in great outcomes<br>(and a lot of work).

Assumptions, values, and requirements tend to determine the final characteristics of a system. A<br>typical startup CRUD app might have the following characteristics:

Require that we do our startup's CRUD

Value time to market

Value the ability to run cheaply until you get product-market fit

Assume that we run in a hosted provider or a cloud

A high frequency trading system is built with different constraints:

Require that we execute a trading strategy

Value trading profitability

Value speed

Value safety/compliance with regulations

Assume that you control all of your hardware inside a co-located datacenter

The former set of constraints gives you millisecond response times, features, and comparative<br>affordability. The latter set of constraints gives you 10-100 nanosecond response times, exotic<br>hardware, comparatively high visibility, and MASSIVE startup costs.

Comparing Two Databases

Let's look at two easily comparable examples. A product like ScyllaDB might have the following<br>characteristics (disclaimer: I don't work with ScyllaDB, so these are not their words):

Required: Distributed NoSQL database

Value speed

Value scalability

Value compatibility with existing NoSQL DB ecosystem (Cassandra)

Assume NVMe flash and modern network cards on Linux machines

These two products drive most of their design decisions from their values, within the space of their<br>requirements and assumptions. However, products with different assumptions and requirements turn out<br>very different. Contrast ScyllaDB to a project with these assumptions, values, and requirements:

Required: Distributed SQL database

Value scalability

Value global consistency and durability

Value compatibility with existing SQL DB ecosystem (PostgreSQL)

Assume that you run on Linux machines

These are very similar requirements, values,...