Building scalable AI agents with modular prompt transpilation
- Google Developers Blog
Search
Building scalable AI agents with modular prompt transpilation
JULY 16, 2026
Simerus Mahesh
Site Reliability Engineer
Share
When you’re first building an AI agent, a single, monolithic system prompt is usually fine. You have a few instructions, maybe a tool definition or two, and everything lives in one readable file.<br>But as you start using them for production purposes, that format simply just breaks down. Teams start layering on safety policies, domain-specific rules, formatting requirements, and escalation behaviors. Suddenly, you have your entire agent’s control plane within a single instruction file which is exactly where the trouble starts.<br>This is a classic software engineering scaling problem. When you push every concern into a single file, you lose the ability to reason about the system. Collaboration becomes a nightmare, testing gets finicky, and a small change meant to improve one workflow can quietly break another.<br>At production scale, prompt maintainability becomes agent reliability.<br>Why monolithic prompts break down<br>We typically see three main failure modes when prompts grow beyond a certain size:
Obscured blast radius: In standard software engineering, it’s easy for reviewers to reason about the scope of a change through module boundaries, call sites, and tests. System prompt diffs are harder though. Adding a sentence could have unintended side effects across the entire agent, which is often hard to predict or test.<br>Copy-paste drift: As organizations scale, many teams end up duplicating shared logic for various applications such as internal service usage instructions, PII handling, safety policies, or escalation protocols. This leads to copy-pasting or multiple versions of the same functionality leading to inconsistencies.<br>Deferred runtime errors: To manage the sprawl, teams often resort to ad-hoc string formatting or simple templates. While this helps with authoring, it pushes error detection to runtime. You might deploy a prompt that only fails when a specific, rarely-used workflow is triggered because of a missing variable or an invalid import path.
Templates are a good start, but they aren't enough. Production systems require deterministic builds, static validation, and CI/CD integration.<br>Treat prompts like software artifacts<br>The solution here is to treat prompts like build artifacts as opposed to just static text.<br>Instead of maintaining one monolithic prompt file, you can author modular skill files. This allows you to reduce the scope of each file and encapsulate a specific behavior, which allows teams to separate concerns and iterate on components individually.<br>A top-level agent prompt template might look something like this:
# agents/sre_agent.prompt.md (prompt template file)
{% include "shared/safety.prompt.md" %}<br>{% include "shared/tool_usage.prompt.md" %}
You are an SRE triage agent operating in the {{ environment }} environment.
{% if allow_remediation %}<br>You may recommend remediation steps, but destructive actions require human approval.<br>{% else %}<br>You may inspect, summarize, and explain the issue, but do not recommend remediation actions.<br>{% endif %}
{% macro bullet_section(title, items) %}<br>## {{ title.rstrip() }}<br>{% for item in items %}<br>- {{ item.rstrip() }}<br>{% endfor %}<br>{% endmacro %}
{{ bullet_section("Required investigation steps", [<br>"Inspect recent deployment events",<br>"Check service metrics for latency or error-rate changes",<br>"Review logs for repeated failure patterns"<br>]) }}
Plain text
Copied
This gives you the best of both worlds. The templating layer lets you compose shared instructions, inject environment-specific values, and make use of macros. But for the build system, every include is a dependency, and every variable is a requirement. The result is a deterministic, fully rendered artifact that you can test, audit, and diff before it ever reaches the model. We can then use a transpiler to resolve the template imports to generate a file that is ready to be ingested by an agent.<br>For example, if environment = production and allow_remediation = true, the transpiled artifact would look like this:
You are an SRE triage agent operating in the production environment.
You may recommend remediation steps, but destructive actions require human approval.
## Required investigation steps
- Inspect recent deployment events<br>- Check service metrics for latency or error-rate changes<br>- Review logs for repeated failure patterns
Plain text
Copied
A high-level transpilation pipeline would look something like this:
Build-time validation is mandatory<br>A production-grade transpiler should catch errors before runtime.<br>We should be running validation checks for missing imports, undefined variables, and circular dependencies during the build process. Dependency graphs are invaluable here, reinforcing the need for a solid template engine. If you...