Write in Markdown, serve it dynamic | Andros Fenollosa

Skip to content

We have three kinds of visitors: one is a primate and the other two are robots. Three very different readers, with very different needs, and almost everyone serves all of them exactly the same thing. That's why, on my blog, I dynamically serve the same content in three different formats, adapted to each way of consuming it.

Before you keep reading, I want you to know that this could be seen as an extended explanation of one of the points in my other article The Dark Side of Static Sites.

One article, three readers

Think about who reaches this page:

Humans : They want a rich experience. Pleasant designs with good contrast, features like infinite pagination, comments, smooth navigation without reloads. In other words: design, content, and experience. Everything that makes the web a pleasant place to be.

Indexers : The bots from Google, Bing, or DuckDuckGo. They want clean, stable HTML, without depending on JavaScript running. They only read your content and classify it by the tag structure. They don't care about fireworks, dropdown menus, or sliders. They only want the content, and they want it fast.

AIs and agents : Claude, ChatGPT, Perplexity, and company. Like the indexers, they don't want your layout, your menu, or your footer. But not the structure, hierarchy, or content tags either (, , , ). They want your raw text, and cheap, spending the fewest tokens possible.

But all three reach the same URL.

The same content, negotiated

The solution is not to have three websites, but a single source of content served in the format each visitor asks for. And it turns out HTTP already had the answer decades ago: the Accept header.

Each request to the server states which format it prefers. A browser asks for text/html. An agent can ask for text/markdown. The server looks at that header and responds accordingly. It's called content negotiation, and it has been in the HTTP standard for almost thirty years. The same mechanism that serves HTML to a browser, JSON to an app, or RSS to a feed reader. The strange thing is that, until recently, almost no one used it to serve Markdown to AIs.

In my case, anyone can check it. Ask for this same article in Markdown:

curl -H "Accept: text/markdown" https://en.andros.dev/blog/...<br>And if you want HTML, ask:

curl -H "Accept: text/html" https://en.andros.dev/blog/...<br>The same URL. The same content. A different format.

flowchart TD<br>A[Request to the same URL] --> B{What does the Accept header say?}<br>B -->|text/markdown| C[AI / Agent: pure Markdown, minimal tokens]<br>B -->|text/html, indexer bot| D[Indexer: static HTML, no JS execution]<br>B -->|text/html, browser| E[Human: HTML over WebSockets, dynamic SPA]<br>C --> F[Same content, a single source]<br>D --> F<br>E --> F<br>Maybe you have heard of llms.txt, the file that lists your content in Markdown for AIs. It's complementary, not the same thing: llms.txt is for being discovered, content negotiation is for serving each page once they have found you. I'll go with the second one, because it doesn't force me to maintain a separate index to keep in sync.

Why you give Markdown to AIs

It's not an aesthetic whim. It's economics.

A language model pays per token. Every HTML tag, every CSS class, every nested are tokens it consumes to reach your text. Put a number on it: my article The Dark Side of Static Sites weighs around 36 KB in HTML and around 8 KB in Markdown. More than four times smaller, a 76% reduction, and that's with my page already being fairly light. On heavier sites the difference reaches 80% to 99%. Fewer tokens means less cost, less noise, and a higher chance the model understands what you say.

Think about it from the other side. If your content is easy for an AI to read, it's more likely to cite you correctly when someone asks. In a world where more and more people ask an assistant instead of a search engine, that is the new optimization. The SEO of the next decade might just be called serving good Markdown.

Isn't this cheating?

It's a fair question. If you serve something different to bots than to humans, isn't that cloaking, the technique Google penalizes?

No, and the difference matters. Cloaking means serving different content at the same URL to deceive: one thing to the search engine to rank, a different one to the human. Here the content is the same. The only thing that changes is the presentation format, and it changes because the client itself asks for it with its Accept header. There's no deception. There's courtesy. The article says exactly the same thing in all three versions.

The rule is simple: same facts, same structure, same links and dates. You only remove the wrapper that isn't useful to that visitor.

Humans, without giving up anything

For people I do exactly the opposite of what I do for AIs: I give them the richest experience possible. The site behaves like a SPA with HTML over WebSockets: you navigate without reloads, the search...