Why AI Orchestration Belongs in the Browser

Software Engineering Blog

Why AI Orchestration Belongs in the Browser

By:<br>Dhrumil Shah

Most AI assistants today run on the server. The user interacts with a web application, the request gets sent to a backend, and the backend handles the orchestration, agents, tools, application logic, and interaction with the LLM.

This article looks at the architecture behind the AI Components released as part of version 5.0 of the ArcGIS Maps SDK for JavaScript, and why we moved orchestration closer to where much of the application context already exists: the browser.

Traditional AI Architecture

In this context, orchestration is the coordination layer that decides which agents to call, what context to provide them, and how to turn their results into a useful response.

Why Web Mapping Applications Are Different

But web mapping applications are different.

Applications built with the ArcGIS Maps SDK for JavaScript are already highly client-rich. The browser already contains a significant amount of runtime context:

the current map extent

visible and hidden layers

renderers and legends

selected features and pop-ups

user interactions

application state and business logic

Beyond UI state and map context, the ArcGIS Maps SDK for JavaScript also provides substantial client-side APIs and workflows, including querying, geometry processing, and GPU-accelerated WebGL rendering. This enables fast, highly interactive experiences and allows more application logic to run directly in the browser.

For examples of these client-side workflows, see FeatureLayerView query by distance and FeatureLayerView query by geometry.

The browser already has much of the live runtime context needed to understand and interact with the map.

As we started building AI assistants for Web GIS applications, we asked a simple question:

"If the map, the state, and the context already live in the browser, why move everything to the server to run AI?"

Browser-Native AI Architecture

Instead of pushing all map context and application state to a backend orchestration layer, we explored a different approach: running AI orchestration and agents directly in the browser, alongside the map itself.

The architecture is built around:

browser-native orchestration

browser-executed agents and tools

direct interaction with live map state

hybrid browser/server AI workflows

This allows AI assistants to integrate with web mapping applications while reducing the need for additional backend orchestration services.

Example Workflow

To understand how browser-native orchestration works in practice, consider the following request:

"Show nursing homes in Texas "

Instead of sending the request to a backend service for orchestration, it is handled by the browser-side orchestrator.

Based on the available agents and their capabilities, the orchestrator selects the data exploration agent to handle the task.

Before sending context to the LLM, the application performs vector search in the browser to identify the most relevant layers and fields for the request. This helps narrow the amount of context sent to the model.

The selected agent then generates the appropriate query, updates the map, highlights the matching features, and summarizes the result back to the user.

Because orchestration runs near the runtime context in the browser, agents can interact directly with live map state without a heavy backend layer.

Why Vector Search Matters

Vector search is one of the key pieces that makes browser-native orchestration practical.

Embeddings are vector representations that allow semantic similarity comparisons.

An ArcGIS web map can contain many layers, and each layer can contain many fields. Sending all of that metadata to the LLM for every request is inefficient and can make the model more likely to use the wrong context.

Instead, we use embeddings and vector search in the browser to identify the most relevant layers and fields for the user’s request. This acts as a context engineering step before the LLM is called.

Embeddings are generated by calling an embedding model, but once the layer and field embeddings are available, the vector search itself runs in the browser.

For example, in the “Show nursing homes in Texas” workflow, vector search helps identify the nursing homes layer and the relevant state field before the agent generates the query.

This keeps the prompt smaller, more focused, and more grounded in the actual map.

Hybrid AI Architecture

Browser-native orchestration does not mean everything runs in the browser. The browser is where the application context, orchestration, agents, and map interactions live, but the architecture can still use server-hosted models, enterprise APIs, and long-running AI tasks.

Browser-native orchestration is not a fit for every scenario. Large-scale batch processing, long-running workflows, and sensitive orchestration logic such as proprietary prompts may still be better...