The anatomy of an AI agent on Google Cloud: a complete guide | Awesome Agents on Google Cloud

Awesome Agents on Google Cloud

The anatomy of an AI agent on Google Cloud: a complete guide

You built an agent. It works on your laptop: a model, a loop, a few tools. Now you want it in production on Google Cloud, and suddenly you’re staring at a wall of product names: ADK, Agent Runtime, Cloud Run, GKE, Gemini Enterprise Agent Platform. This post is a map of that landscape and a way to think about it: every layer of the stack, what Google offers at each one, and how to decide. Product names are current as of this writing, with former names in parentheses where the Next ‘26 consolidation renamed things.

The anatomy of an agent

Strip away the frameworks and every agent is the same five parts: a model that reasons, tools it can call, memory so it isn’t goldfish-brained, a runtime loop that ties them together until the task is done, and increasingly connections to other agents . Everything Google Cloud offers fills one of these slots, plus the cross-cutting production concerns (observability, security, evaluation) that apply to all of them.

The stack, layer by layer

The model: Gemini, Claude, or open weights

A typical brain is Gemini 3.x via the API: Gemini 3.1 Pro for hard reasoning, the Flash tier as the cost-effective workhorse many agents actually need. And it isn’t Gemini-only: Model Garden offers 200+ models inside your Google Cloud project, including Anthropic’s Claude and open-weights leaders like GLM, Kimi, and DeepSeek, many as pay-per-token services. To compare candidate brains on agentic work, Agent Arena ranks models on tool orchestration and task completion across a million-plus real agent sessions.

Self-hosting an open-weights model is also practical now. Gemma 4 ships with function calling and structured output, and Cloud Run GPUs serve it with scale-to-zero economics: both NVIDIA L4 GPUs (24 GB, fits small and mid-size models) and 96 GB NVIDIA RTX PRO 6000 Blackwell GPUs for 70B-class models are generally available, with no quota request needed for the L4s (Run Gemma on Cloud Run walks through the pattern with the vLLM inference engine). But for low-traffic agents, pay-per-token Flash is often cheaper than keeping a GPU warm (run the numbers for your own traffic: Gemini pricing vs Cloud Run pricing); self-hosting wins on data control, fine-tuning freedom, and high sustained utilization.

The framework: ADK first, but not ADK only

Agent Development Kit (ADK) is Google’s open-source, code-first framework (Python, Go, Java, TypeScript, Kotlin); ADK 2.0 moved it to a graph-based execution engine with parallel workflows, automatic retries, and human-in-the-loop pauses. It’s the framework the rest of the stack is tuned for: one command deploys to Agent Runtime or Cloud Run, with evals and tracing included. It’s not the only option, though: Agent Runtime also supports LangGraph, LangChain, LlamaIndex, and AG2, anything containerized runs on Cloud Run, and ADK itself drives non-Google models via LiteLLM.

The runtime: where the agent lives

Agent Runtime is the “managed everything” option: hand it framework-native code and get sessions, memory, sandboxed code execution, and observability without building any of it. Cloud Run runs your agent as a standard container and is a popular community choice, covering chat backends, worker pools for queue-driven agents, jobs for batch runs up to 7 days, and GPU serving. GKE carries the most operational complexity and makes sense when you’re running a fleet of agents or self-hosting models.

And these compose: Sessions and Memory Bank are callable a la carte from an agent on Cloud Run or GKE, so pick where the agent runs and borrow managed services as needed.

Agent Runtime (formerly Vertex AI Agent Engine)<br>Cloud Run<br>GKE

What it is<br>Managed agent runtime<br>Serverless containers<br>Managed Kubernetes

You bring<br>Python agent code<br>Any container<br>Containers + Kubernetes config

Manages for you<br>Sessions, Memory Bank, code sandbox, tracing, evals<br>Autoscaling (to zero), TLS, versioned deploys<br>The core of the cluster (all of it in Autopilot mode)

Best for<br>Fastest path to managed production state<br>Most agents, MCP/A2A servers, GPU serving<br>Multi-agent platforms, self-hosted LLMs

Tools and knowledge: MCP, A2A, and the RAG spectrum

Agent tools are converging on MCP : Google runs managed MCP servers for its databases and services, the open-source MCP Toolbox covers 40+ data sources, and Cloud Run hosts custom MCP servers. Agent-to-agent coordination has its own protocol, A2A: MCP connects an agent to its tools, A2A connects agents to each other.

For knowledge, think in a spectrum of abstraction: Agent Search (formerly Vertex AI Search) is turnkey retrieval, RAG Engine is a managed pipeline you can tune, Vector Search is raw infrastructure. Apply the data-location test first: if your data already lives in AlloyDB, Cloud SQL, BigQuery, or Firestore, use that database’s native...