How We Built a Customer Support Agent That Actually Learns, with Hindsight and Google’s ADK

A customer opened a support ticket on Hindsight Cloud at 2:41 PM: “I want to change my user email.”

Our AI support agent answered first, as it always does. It knew that billing contact emails exist and can be changed, so it pointed her at the billing settings — and then at an /account section that doesn’t exist. When she asked where to find it, the agent doubled down with directions to “the Settings link, typically located in the left-hand navigation bar.” There is no left-hand navigation bar.

Customer:  … there is no left-hand nav bar, are you just guessing?

Yes. It was.

The agent apologized and escalated. Three minutes of AI confidence, then a human (me) picked up the ticket an hour later and gave the real answer: you can’t change your email address . The email is the primary key in our database. What you can do is invite the new address via the Team Members page, promote it to org Owner, and delete the old user — or start a fresh org on the new address. The customer thanked me and moved on.

Two days later, a different customer asked the same question: “how to change user email address?” Here’s what the agent said, with no human in the loop:

To update your primary account email address, you’ll need to use the team-based migration process, as there isn’t a direct “edit email” setting.

Invite your new email:  Go to the Team Members page in your Hindsight Cloud dashboard and send an invitation to your new email address.

Assign permissions:  Once the new email account has accepted the invite, promote it to the Owner role.

Remove the old account:  After verifying your new account has full access, you can remove your old user account from the organization.

Please note that the email preferences page in the dashboard is only for managing notification settings and cannot be used to change your login email.

No deploy. No prompt change. No fine-tune. Nobody edited a knowledge base. The agent learned the answer from watching a human give it once — and it didn’t just parrot my reply, it restructured a freeform two-message ticket reply into numbered steps and proactively headed off the misconception that started the whole mess.

This post is the full build: a customer support agent on Google’s Agent Development Kit (ADK) , deployed to the managed Agent Engine  runtime on Google Cloud, with Hindsight  as the memory layer that makes the learning loop work.

TL;DR

The agent is built with google-adk 2.x , runs gemini-3.1-flash-lite , and deploys to Vertex AI Agent Engine  (Agent Runtime) as a reasoningEngine — we had a GKE cluster available and chose the managed runtime anyway.

Memory is two Hindsight banks: hindsight-cloud-product-kb  (shared product knowledge, read-only from conversations) and hindsight-cloud-interactions  (every user’s conversation memory in one bank, isolated by tags).

The knowledge bank is seeded from our docs sites by walking their sitemaps — each page retained as a document with a stable ID, so re-runs upsert instead of duplicating.

The learning loop is one function: every human support reply is automatically retained as a (question, answer) pair into the product bank. That’s how the email-change answer got in. There’s also a “Teach the agent” box in our admin UI and a feedback tool the agent calls on its own.

The agent has eleven account tools (billing, usage, members, API keys) that execute against the customer’s own org via a short-lived delegated token — the agent never holds a cross-tenant credential.

Why the Wrong Answer Happened

The first answer wasn’t a wild hallucination. It was worse: plausibly wrong. The agent knew billing emails were a real, changeable thing. It misread the intent — the customer wanted to change her login identity, not her invoice recipient — and then filled the gaps in its UI knowledge with statistically likely furniture: a /account section, a left-hand nav.

Customers catch this immediately. “Are you just guessing?” is the moment trust dies, and it’s the default failure mode of every support bot that answers from model weights plus a static RAG index. The email-migration procedure lives in no documentation page. It’s tribal knowledge: the kind of answer that exists only in a support engineer’s head until the day someone writes it down.

Our requirement was never “deflect more tickets.” It was: when a human answers a question once, the agent should know that answer forever.  That’s a memory problem, and it’s the specific problem Hindsight is built for.

The Stack

Google ADK  (google-adk ≥ 2.0) — the agent framework. We started from the public adk-samples customer-service agent and adapted it.

gemini-3.1-flash-lite  — the model. Flash-lite is enough because the hard knowledge lives in memory, not in the model. The model’s job is intent, tool selection, and composition.

Vertex AI Agent Engine  on Google Cloud — the managed runtime. The agent deploys as...