Don’t Build a General Purpose API (4 Years Later) - Max Chernyak

Don’t Build a General Purpose API (4 Years Later)

written by Max Chernyak on<br>11 Dec, 25

In 2021 I wrote an article encouraging people not to build general purpose APIs for their own front-ends. (You should probably read it before reading this one.) It got featured on Hacker News twice, albeit with a worse reception (and more heated discussion) the second time around. My guess is, more front-enders showed up. 😛

Having observed this approach for 6 years, I’ve only grown more confident in its success. Part of that confidence comes from seeing it play out. Our team vastly simplified maintenance, reduced bugs, and boosted performance by making the jump. (The jump was made possible by Long Term Refactors btw.) Another reason for my confidence is the many comments I’ve received over the years confirming that it works for other teams too. Finally, I’ve received a number of challenges and condemnations, although they were either entirely theoretical, or based on a misunderstanding of the article. Since I consider it somewhat my fault for not being clear enough, I want to address these misunderstandings. So here they are, organized into topics, each one addressed in turn.

1. You reinvented HTML!

Multiple readers have informed me that I can apparently serve HTML directly from the server, instead of doing all that JSON payload nonsense. Having started building websites in 2003, I get it. However, the reality is that today we work with front-end teams, they work with React, and React comes with certain practices. 15 years of them in fact, in addition to the baggage of pre-built components. That’s worth some respect. If a JSON payload is preferred, and I have no trouble delivering it, why would I mind? Personally, I much prefer the good old Ruby on Rails-esque stack with server-side HTML rendering, but we work in teams, and should probably play to each other’s strengths.

That said, the bigger issue with the “you rediscovered HTML” crowd is that they’ve misunderstood the advice. HTML involves content, structure, and style (esp. in the case of Tailwind) — which is basically everything you can serve, aside from assets. In our JSON world we only serve content, a faint hint of structure, and no style at all. That’s way higher level than HTML. It’s important to understand that I’m not asking the back-end to serve JSON.dump(html) to the front-end, only the gaps the page requires to be filled. The rest of the page’s static content should just be hardcoded on the front-end.

I do, however, need to correct one mistake. My old advice went: “content and structure come from the back-end”, but I didn’t realize that folks would interpret it as literally as “serialize the entire HTML into JSON”. That’s not at all what I meant. You only need the bare minimum JSON structure to help the front-end engineer understand which values should go into which parts of the page. For example, if your HTML is something like this:

div><br>article><br>h1>Titleh1><br>p>Bodyp><br>article><br>div><br>You don’t need this: { "div": { "article": { "h1": "Title", "p": "Body" } } }. You simply supply the keyword arguments to this otherwise hardcoded ArticlePage’s constructor:

"title": "Title",<br>"body": "Body"<br>If there are multiple articles, you make an array of these. As simple as that. Cater to the needs of the page.

2. Pages will load slower without async!

There exists a concern that with my approach you can’t load parts of pages asynchronously, and that this would result in bad performance. If you are a front-end engineer who has only ever worked with generic API endpoints, I get why you have this impression. You needed 10 endpoints to render a single page, you parallelized the requests, you saw that they can be flaky and slow at random, you prioritized some over others. Right? I’m sorry, but that pain was self-inflicted. If you asked a back-end engineer to give you all of that data in one bundle, the server would most likely produce it in under 30 milliseconds in a single streamlined 200ms roundtrip rather than juggling 10x200ms roundtrips competing with one another.

The whole idea of asynchronously loading pages is a truism of the front-end world that only feels correct in theory. In practice, it’s like hiring 10 trucks to deliver 10 USB drives in parallel, realizing how slow it is to manage 10 trucks, and concluding “maybe I need some more trucks”. Async loading only makes sense when you are actually dealing with slow, heavy, or streaming data sources. If it’s your own back-end, attached to your own database, spitting out kilobytes of content, you are making things hundreds if not thousands of times slower by running parallel requests. On top of that, you’re making it harder for the back-end engineering team to bundle, optimize, and cache data, because it must be sent piecemeal.

Then there’s the reverse concern as exemplified by this whole thread, and this comment. It argues that you should be...