The Wheels We Keep Reinventing - Blain Smith

The industry has a maddening relationship with solved problems. A thing gets figured out, packaged, documented, and shipped, and a decade later half the field is building it again from scratch. The old solution was rarely wrong. Somewhere along the way the proven thing got traded for something that looked cheaper, and it turned out more expensive the moment anyone counted the hours. I watch this happen on repeat and it wears on me.

Heroku did rolling deploys, blue/green releases, and health-checked rollbacks over a decade ago. You pushed to a remote and it handled the rest. Now every team running Kubernetes rebuilds that same machinery: readiness probes, deployment strategies, surge and unavailability tuning, rollback logic, and the observability to know when any of it went wrong. Kubernetes promised to hand you this. What it actually hands you is the primitives and a config surface large enough to get every piece of it subtly wrong. You traded a convenience you already had for a customization you now babysit every quarter, and somehow that counts as progress.

The Usual Suspects

Auth is the clearest case, and the one that irritates me most. There is no shortage of auth providers. Engineers write the same ad hoc routes anyway, the same users and sessions tables, the same password reset flow, the same JSON payloads, the same token refresh logic. Everyone knows the provider exists. They build it themselves regardless, ship it with at least one hole in it, and then maintain that hole forever.

Background jobs are the same story. You need to run arbitrary work outside the request cycle, so you wire together a queue, a worker pool, retry handling, dead letter behavior, and a way to see what failed. Sidekiq, Celery, and a dozen managed queues have existed for years. It gets rebuilt regardless, per stack, per team, per service.

Once you start looking, the list is long. A few others:

Retries, timeouts, backoff, and circuit breaking. Service meshes were supposed to own this so your application code never had to. Most retry logic still lives in the app, hand-written, usually without jitter and often applied to operations that were never safe to run twice.

Rate limiting. Token bucket and leaky bucket are old, well-described algorithms. Gateways offer rate limiting as a checkbox. It still gets reimplemented as middleware, often with a race condition around the counter.

Idempotency keys. Stripe wrote down the canonical pattern years ago and it is not complicated. Teams rebuild it per endpoint anyway, and get the retry-safety edge cases wrong in the process.

Feature flags. A managed service exists. Teams build a flags table and an admin page instead, then discover they have reinvented targeting rules and gradual rollout.

Caching and invalidation. Redis already exists. What gets hand-rolled is the part that was always hard: when to consider a cached value stale, and how to avoid serving the wrong thing.

Config and secrets. Parameter stores, Vault, Consul, and etcd all solve this. Every service still hand-writes its own loading, precedence, and validation, usually discovering the precedence bug in production.

Search. Postgres full text search and Elasticsearch both exist. The common path is a LIKE query, then a slow reinvention of relevance ranking once LIKE stops being enough.

State machines and workflows. Temporal and Step Functions exist to model long-running stateful work. The default is a status enum column and a growing pile of conditionals that no one can reason about after six months.

Multi-tenancy isolation. The tenant_id filter on every query is a wheel that has been built thousands of times, and the version with the security hole has been built almost as many.

The Pattern

The pattern underneath all of these is the same, and it is not subtle. A mature solution exists. The ecosystem swears a platform or provider already covers it. The engineer rebuilds it anyway, because the abstraction leaked, or did not fit the exact shape of the problem, or solved an adjacent problem and left a gap sitting right where the real work was. So you get half a solution you did not want and a second half you had to write yourself, which is the worst of both.

Keeping It Simple and Extensible

None of this is an argument for overengineering the first version. Keep it simple is still the right default, and I tell people that constantly. The point is narrower: do not engineer yourself into a box while you keep it simple. A small system and a system with every future decision already foreclosed can look identical on the first day.

Building the core product on standards and known protocols is what keeps the door open. If your auth speaks OIDC, swapping in a provider later is a configuration change. If your messaging speaks a documented protocol, adding an external system or a plugin is an integration rather than a rewrite. The trap is the opposite: hardcoded vendor...