Why I moved agentique's LLM layer to BAML · agentique
2026-07-21 · prompt-engineering
Why I moved agentique's LLM layer to BAML
When I started building https://agentique.ch, I thought the only hard part would be<br>ranking AI articles. It turned out there was another problem - keeping the<br>LLM layer maintainable while constantly experimenting with different models<br>and providers.
A bit of context
Agentique is an AI news aggregator for developers. The goal isn't to cover<br>all AI news - funding rounds, acquisitions, or how large AI labs cope with<br>government affairs is out of scope. I'm much more interested in individual<br>contributors or small teams sharing insights: blog posts about agent<br>orchestration systems, GitHub repos solving a problem I ran into the week<br>before, dev workflows that made an assistant twice as useful.
To surface that, agentique collects articles from a curated list of<br>technical sources and runs them through a pipeline: fetch, deduplicate, score<br>relevance, improve titles, generate summaries, publish. Most stages involve<br>one or more LLM calls, which means every stage has prompts, schemas, model<br>selection, retries, and tests. After a while the LLM layer became its own<br>subsystem. That's when I started looking for better tooling.
Looking at the alternatives
Before settling on BAML, I compared the<br>existing ecosystem. Vercel AI SDK is excellent for building AI directly into<br>a web app - streaming, React integration, provider abstraction. Mastra looks<br>great for long-running agents, workflows, memory, RAG. LangChain/LangGraph is<br>still the most complete orchestration framework for complex agent systems.
My problem was different - I wasn't building autonomous agents or<br>conversational UIs. I needed a reliable, well-organized way of expressing<br>dozens of independent LLM tasks: classify article, improve title, generate<br>summary, extract metadata, score relevance. Each one should look like a<br>normal function in the codebase - typed inputs, typed outputs, easy testing,<br>easy model switching, minimal boilerplate. That's the niche BAML fills: a DSL<br>for turning prompts into typed functions with reliable structured output.
What I liked most
Prompt tests. Every prompt lives together with its tests, right in the<br>same .baml file, and the BAML VSCode extension runs them like unit tests<br>from the editor. When I switch from Gemini to GPT-OSS, or tweak a prompt, I<br>immediately know whether I've broken something. That removes a surprising<br>amount of uncertainty.
Prompt tests
Every prompt lives together with its tests.
Here's a simplified example from my title rewriting pipeline.
function ImproveTitles(articles: ArticleInput[]) -> TitleFix[] {<br>client GeneralClient<br>prompt #"<br>{{ _.role("system") }}<br>You are a technical editor. You write clear, informative article titles.
Output rules (strict):<br>- For each input article, return one TitleFix whose `url` exactly matches that article's URL.<br>- The `title` field contains plain title text ONLY. No markdown - strip ** or * or _ or backticks.<br>- Write in English. Never emit any other language or script.
{{ _.role("user") }}<br>Decide for each article whether the existing title is already good. A good title is specific, informative ...<br>- If the existing title is already good, return it verbatim.
{% for a in articles %}<br>--- Article {{ loop.index }} ---<br>url: {{ a.url }}<br>current_title: {{ a.title }}<br>snippet: {{ a.snippet }}<br>{% endfor %}
{{ ctx.output_format }}<br>"#
And here's the test:
test improve_titles_rewrite_short {<br>functions [ImproveTitles]<br>args {<br>articles [<br>url "https://example.com/claude"<br>title "Claude 4"<br>snippet "Anthropic today released Claude 4, a new flagship model with improved reasoning and a 1M-token context window."
test improve_titles_keep_unchanged {<br>functions [ImproveTitles]
args {<br>articles [<br>url "https://example.com/post"<br>title "Anthropic releases Claude 4 with 1M-token context window"
@@assert(no_markdown, {{ "**" not in this[0].title }})<br>@@assert(no_cjk, {{ "伊" not in this[0].title }})<br>@@assert(max_12_words, {{ this[0].title|split(" ")|list|length<br>Structured output without the headache
Every prompt returns a typed object. No JSON parsing.
No "please output valid JSON."
No markdown cleanup.
No repairing malformed responses.
Provider fallbacks that live in the DSL, not my code.
Routing between providers is declared once, not scattered across application logic:
client GeneralClient {<br>provider fallback<br>options {<br>strategy [GeminiFlash, NvidiaGptOss, NvidiaMinistral]
If Gemini is unavailable, the next provider in the strategy is tried<br>automatically. The application code still just calls ImproveTitles(...) -<br>it stays unaware of which provider ultimately serves the request.
The funny part
One of the goals of Agentique was to help me discover technical content I<br>probably wouldn't have found otherwise. BAML ended up being one of those<br>discoveries - I found the project on GitHub,<br>added it to the feed because it looked interesting, gave it a try a few...