Dumb Core, Smart Edge: Agentic Design

Sign in

Subscribe

Many agentic systems I've watched fail in production had the same shape: intelligence concentrated at the center, where it was hard to test, replace, or reason about. The orchestrator was doing too much — routing requests, holding domain knowledge, managing memory, selecting tools, and shaping outputs, all at once. When something broke, you couldn't isolate it. When requirements changed, you couldn't surgically update it. The whole thing had to move.<br>This is not a model quality problem. The teams building these systems weren't using inferior LLMs. They were running into an architectural error recurring enough to be useful to name, with a clear counter-principle.

The principle is this: Dumb Core, Smart Edge. The orchestrator — the<br>central node that sequences work — should be nearly stateless, encoding only control flow.<br>The intelligence belongs at the periphery, in specialist nodes that own their domain<br>boundary and can be reasoned about, tested, and replaced in isolation. This is not<br>just a stylistic preference. It is a useful topology when the system has multiple<br>domains, tools, or responsibilities that need to evolve independently.

TL;DR — Key Takeaways:

A "smart core" orchestrator that handles routing, domain logic, memory, and tool selection in one prompt is a common architecture failure in production agentic systems.

The Dumb Core, Smart Edge principle: orchestrators encode state transitions; specialists own domain intelligence within explicit boundaries.

The economic advantage comes from narrower context: specialists receive the slice needed for their task instead of the full system prompt and every tool schema.

The replaceability test: you should be able to swap any specialist implementation without modifying the orchestrator or adjacent nodes.

Every specialist edge should emit a typed Pydantic contract — the orchestrator consumes the type, never raw text.

LangGraph conditional edges map directly to this pattern: typed state carries routing metadata, nodes carry intelligence.

Latency is a real tradeoff. Measure it against the gains in testability, replaceability, and parallel execution.

Why Intelligence at the Core Fails<br>The instinct to centralize intelligence is understandable. You have one powerful model. You write one system prompt. You give it all the tools and let it figure things out. In a demo, this works. The model is capable enough to juggle concerns simultaneously, and the happy path looks clean.<br>Production breaks this in four distinct ways.<br>First: entanglement. When the orchestrator encodes both routing logic and domain knowledge, changing one requires touching the other. A new tool means rewriting the prompt. A new domain rule means re-testing the routing. The system develops coupling that has no architectural boundary — just a long, fragile string of natural language instructions.<br>Second: untestability and opacity. You cannot write a unit test for an LLM prompt that does five things at once. You can only run end-to-end evaluations and observe whether the emergent behavior is stable. When an agent misbehaves, you need to know whether the failure was in routing, domain reasoning, tool selection, or output formatting. A smart core conflates all of these — regressions are invisible until production, and then you're debugging a black box with a flashlight.<br>Third: cost amplification. A smart-core agent sends every token of context — full history, all tool schemas, all domain rules — through the same path, for every step. In a multi-step workflow, this repeatedly pays for context the current step may not need. A dumb core routes to specialists. The specialists receive only what they need. At model-routing economics scale — millions of calls — the economics compound fast.

Dimension<br>Smart Core (Monolithic)<br>Dumb Core, Smart Edge

Testability<br>End-to-end only; regressions invisible until production<br>Each specialist unit-testable with typed inputs/outputs

Cost per request<br>Full context and all tool schemas sent through one path<br>Each specialist gets the context slice relevant to its task

Blast radius<br>Any prompt edit risks cascading regression across all behaviors<br>Changes isolated to one specialist; orchestrator untouched

Debuggability<br>Single opaque prompt; failure source ambiguous<br>Typed contracts at each edge; failure pinpointed to specific node

Model flexibility<br>Locked to one model for all tasks<br>Each specialist can use the model or deterministic function that fits its task

Latency<br>Single call path, but one large prompt<br>Extra routing step; specialists can run in parallel when the graph allows it

Figure 1: Two topologies — intelligence concentrated at the core (fragile) versus distributed to the edge (controllable)The Principle Defined

The Principle of Structural Periphery: In agent systems with<br>multiple domains or tool boundaries,<br>the capacity for autonomous judgment should be maximally distributed toward the<br>edges of the...