-->

Contra DSPy and GEPA

Contra DSPy and GEPA

Workflows are out. Agents are in.

Posted Dec 17, 2025 by Benjamin Anderson

I have something shameful to confess—I have been holding hate in my heart. It pains me to say this, because the folks behind it are brilliant, kind, and thoughtful people, and deserve nothing but love and appreciation. Nevertheless, I must share my authentic truth: I hate DSPy and GEPA to the core of my being. I tried to like them. I really did. I went through the 45-minute Colab notebook. I even made an effort to add a flavor of GEPA to lm-deluge, my open-source LLM SDK, before giving up in a fit of rage (sorry, Claude). My conclusion? Trying to treat LLM workflows as modular programs is (often) a mistake. It's backwards, rigid, and the wrong fit for the most interesting tasks.

In this post (polemic? rant?) I try to unpack the "why" behind the hate, and consider whether it's possible to salvage the good parts of GEPA.

What Is DSPy?

If you're new around here, you might have no idea what I'm talking about. That's OK. DSPy is a toolkit created in 2023 by Omar Khattab (@lateinteraction on Twitter), who also known for inventing ColBERT and popularizing the general concept of "late interaction" in information retrieval. (See, like I said, smart guy.) The idea is to make AI programs more reliable by (a) breaking them down into modules, and (b) decoupling the function of those modules from their specific implementation .

For instance, suppose you want to to use an LLM to summarize a long story into a single paragraph. Probably the first thing you'd do is make a prompt like "Summarize this long story into a single paragraph" and send that to an AI model like Claude. You might then spend a bunch of time improving that prompt: reminding Claude that a paragraph should have no less than 4 but no more than 8 sentences, noting the aspects of a story that should generally be captured in a summary, and telling it for the love of God to not use emojis.

Then, the next week, your budget gets slashed by 90% and you can no longer afford Claude. Or a better model comes out, and you decide you'd rather use that one. Suddenly, all that work you did was basically useless. Other models don't fail in the same ways that Claude did. Maybe Gemini works better with few-shot examples than detailed instructions. Maybe Grok doesn't have the emoji problem. Maybe you want to fine-tune now instead of prompting. You have to start all over.

DSPy says: This seems bad. How about you define what you want your AI "program" to do, and we'll take care of the optimization for you. You write a PyTorch-like syntax with different "modules", and DSPy can test out a bunch of different few-shot examples, do prompt optimization, and even fine-tuning on a small dataset of examples, so that you don't have to worry about how to get your AI model to do what you want. For example (from DSPy docs):

def search_wikipedia(query: str) -> list[str]:<br>results = dspy.ColBERTv2(url="http://20.102.90.50:2017/wiki17_abstracts")(query, k=3)<br>return [x["text"] for x in results]

rag = dspy.ChainOfThought("context, question -> response")

question = "What's the name of the castle that David Gregory inherited?"

rag(context=search_wikipedia(question), question=question)<br>This little program does what you want (RAG) with any model out of the box, but then you can apply a DSPy "optimizer" to it, which uses one of a few techniques to make it do the task better. Sounds great! To really get mileage out of it, you can compose these little modules into a full workflow, e.g. search → filter → summarize → extract, and DSPy can help optimize all the pieces of the pipeline so it works great end-to-end. This sounds great, what's not to love? I'll get to that.

What is GEPA?

GEPA is short for "Genetic Pareto", and is one of the optimizers you can use to improve your "LLM program" (this is what a pipeline composed of little modules is called in DSPy-land). The required ingredients are simple: the modular LLM program (consisting of components and their associated prompts), plus a training dataset of (input, expected-output) pairs. The optimizer works by using LLMs to reflectively improve the policy (i.e. the prompts), based on analysis of trajectories on the training dataset.

More specifically, it implements a genetic prompt-evolution algorithm, which maintains the titular Pareto frontier of policies. To track the frontier, GEPA stores a "grid" of scores, the performance of each policy on each sample. To be kept around, a policy must be the best policy on at least 1 input, and can't be strictly dominated by any other policy. This makes sure that the pool of candidates stays diverse, versus just taking the top-scoring N candidates across the whole dataset, which might all be very similar.

That Pareto piece determines how candidates are kept or discarded. The other piece is how candidates are "evolved" (the policy-improvement step). Instead of the sparse signal...