Treat the Context Window as a Data Assembly Problem - Pydantic-resolve

Skip to content

Advanced

API Reference

Reference

Articles

Redefinition: LLM context = response tree

Mechanism mapping

Walkthrough: a customer support agent context

Comparison with other approaches

One Entity graph, four consumer types

When to use it, when not to

Conclusion

Redefinition: LLM context = response tree

Mechanism mapping

Walkthrough: a customer support agent context

Comparison with other approaches

One Entity graph, four consumer types

When to use it, when not to

Conclusion

Treat the Context Window as a Data Assembly Problem: Where pydantic-resolve Fits in AI Workflows

A typical piece of AI code

Open any service in your project that calls an LLM. You will most likely see a function that looks something like this:

async def build_support_context(ticket_id: int) -> str:<br>ticket = await db.get(Ticket, ticket_id)<br>customer = await db.get(Customer, ticket.customer_id)

recent_tickets = await db.query(Ticket).filter(<br>Ticket.customer_id == customer.id<br>).order_by(Ticket.created_at.desc()).limit(5).all()

# Retrieve similar past tickets<br>embedding = await embed(ticket.description)<br>similar = await vector_store.search(embedding, top_k=3)

# Each similar ticket needs its resolution pulled in<br>similar_with_resolution = []<br>for s in similar:<br>resolution = await db.query(Resolution).filter(<br>Resolution.ticket_id == s.id<br>).first()<br>similar_with_resolution.append({<br>"title": s.title,<br>"resolution": resolution.text if resolution else "",<br>})

# Collect tags<br>all_tags = []<br>for t in recent_tickets:<br>all_tags.extend(t.tags)

# Finally, ask the LLM to summarize<br>summary = await llm.summarize(<br>customer=customer,<br>recent_tickets=recent_tickets,<br>similar=similar_with_resolution,

return f"""<br>Customer: {customer.name} (id={customer.id})<br>Recent tickets: {len(recent_tickets)}<br>Tags: {', '.join(set(all_tags))}<br>Similar past cases:<br>{format_similar(similar_with_resolution)}<br>Summary: {summary}<br>"""

The function is not long, but the problem is already visible: this is a build_context() function that is fundamentally doing data assembly, but its shape is entirely procedural .

It is isomorphic to the FastAPI code that Clean Architecture for Python criticizes — only "assembling an API response" has been swapped for "assembling a prompt context". The problems are unchanged:

Data-fetching logic is scattered through the function body with no structure.

Dependencies of derived fields (all_tags, summary) are held together by comments and line ordering.

Vector retrieval, database queries, and LLM calls live in one function. Every new piece of context means editing this function.

Concurrency optimization (fetching similar tickets in parallel) requires a rewrite.

Reuse — say, exposing recent_tickets to the frontend too — is impossible.

This code is not badly written. It has no home .

"The context window" is a data assembly problem

When people discuss LLM applications, attention usually lands first on prompt templates, model choice, and temperature. Those matter — but as applications grow, the real bottleneck shifts from prompt engineering to context assembly .

The reason: prompt templates are stable, model choice is stable, but "what data to feed the LLM" differs on every call . A support agent handling ticket A and ticket B can share the same prompt template, yet the underlying data-assembly path may diverge completely — A is a VIP customer requiring SLA context and similar-case retrieval; B is a regular customer needing only the basics.

This "same template, different data-assembly path" requirement is exactly what API response assembly does . Your FastAPI project already solves it — different endpoints assemble different response trees. An LLM context is just another endpoint, only the consumer is an LLM rather than an HTTP client.

Once that perspective lands, the problem becomes concrete. The things pydantic-resolve solves well on the API side hold equally well on the LLM side:

API response assembly<br>LLM context assembly

Multi-level nesting (Sprint → Task → Owner)<br>Multi-level nesting (Customer → Ticket → Similar Ticket)

Batch-load related data<br>Batch-recall related context

Derived fields (task_count, contributors)<br>Derived context (summary, aggregated_tags)

N+1 database queries<br>N+1 vector retrievals + N+1 LLM calls

Cross-subtree aggregation (deduplicate all owners)<br>Cross-subtree aggregation (merge evidence across similar tickets)

Every item in the right column already has a solution on the left. We only need to bring the same machinery over.

Three classic assembly pain points

Breaking the build_support_context snippet apart reveals three symptom classes. They are not specific to support scenarios — they recur in nearly every LLM application.

Pain point 1: N+1 LLM calls

for s in similar:<br>resolution = await db.query(Resolution).filter(...).first()

This is a classic N+1 on the ORM side. In the LLM world it gets worse — you might be calling...