Choosing the Right Database for AI Agents: LLM Generated SQL | Learn - Predictable Blog
AuthorsNameJaiTwitter@jkntji<br>AI agents are rapidly evolving from simple chatbots into configurable systems that can adapt to different domains and data sources. One particularly interesting use case is an agent that users can configure with their own structured data.<br>An agent might work with store inventory data for one user and customer records, product catalogs, or entirely different datasets for another. In these systems, the structure of the data is not defined by the developer in advance; it is defined by the end user at runtime.<br>This creates a unique set of challenges for the underlying database. The system needs to support flexible schemas, strong multi-tenancy so that users' data remains isolated, easy ingestion and querying of files such as CSVs, and, increasingly, reliable LLM-generated SQL.<br>In this article, we'll explore these requirements and look at the database architectures that make the most sense for configurable AI agents.<br>The Core Requirements of Configurable Agents<br>Unlike traditional applications, where developers define the database schema, configurable agents need to work with structured data provided by the end user. This creates several important requirements:<br>Schema flexibility: Different instances of the same agent should be able to work with completely different data structures without requiring application code changes or migrations.<br>Queryability: The data must remain fully searchable and filterable so the agent can answer questions or take actions based on it.<br>Multi-tenancy: Data belonging to different users or organizations must remain strictly isolated.<br>Tabular data ingestion: Users may upload CSVs containing hundreds or thousands of rows, and those rows need to become queryable records.<br>Compatibility with LLM-generated SQL: Many agentic systems rely on an LLM to generate the SQL needed to retrieve data. The simpler and more standard the SQL is, the more reliably the LLM can generate it.<br>These requirements create an interesting trade-off between flexibility, isolation, and query reliability.<br>Why Traditional Relational Schemas Can Be Difficult<br>A traditional relational database with predefined columns works extremely well when the developer controls the schema. It becomes more difficult when the end user defines the structure.<br>You cannot define every possible column a user might need in advance. One dataset might contain price, category, and stock, while another contains customer_name, email, and subscription_status.<br>This is why document-oriented and semi-structured approaches such as JSONB initially appear attractive.<br>Postgres + JSONB: Flexibility with Caveats<br>PostgreSQL's JSONB data type provides an appealing middle ground. You can store arbitrary structured data in a JSONB column while still benefiting from Postgres's mature ecosystem, transactions, indexing, and security features.<br>JSONB is fully queryable using operators such as ->, ->>, and @>, and it supports powerful GIN indexes.<br>A typical design might look like this:<br>CREATE TABLE dataset_rows (<br>id UUID PRIMARY KEY,<br>tenant_id UUID NOT NULL,<br>dataset_id UUID NOT NULL,<br>row_data JSONB NOT NULL<br>);
Each row of a user-uploaded CSV becomes a row in this table, with the CSV columns stored inside the JSONB document.<br>Multi-tenancy can be handled using a tenant_id column together with Row Level Security (RLS). When configured correctly, the database itself can enforce tenant isolation rather than relying entirely on application-level filtering.<br>This architecture works well from a storage perspective. However, it introduces additional complexity when the primary way of querying the data is through LLM-generated SQL.<br>LLMs are generally more reliable when generating ordinary column-based SQL than when they need to produce JSONB path expressions, type casts, and containment operators. Every additional layer of syntax gives the model another opportunity to generate an invalid or incorrect query.<br>Multi-Tenancy Is Solvable; Dynamic Queries Are Inevitable<br>The possibility of data from different users mixing in the same table is a legitimate concern, but it is a solvable one.<br>A tenant_id or agent_id column combined with Row Level Security is a mature approach for multi-tenant applications. Stronger forms of isolation, such as schema-per-tenant or database-per-tenant, are also possible depending on the application's requirements.<br>Dynamic queries, however, are unavoidable.<br>Because the fields are defined by the end user, neither the application nor the LLM can rely on a fixed set of column names known at development time. The system needs to understand the schema of each dataset and construct queries against it at runtime.<br>This challenge exists regardless of whether the underlying data is stored in Postgres JSONB, relational tables, or a document database.<br>Handling User-Uploaded CSVs<br>Consider a user uploading a CSV containing 1,000 product records.<br>The...