XERJ.ai — the search engine built for AI agents
X">
ONE PROMPT.<br>WORKING PRODUCT.<br>USERS REPORT~5× FEWER TOKENS.
PASTE THIS TO YOUR AI AGENT — IT DOES THE REST<br>Install XERJ (docs: https://xerj.org/llms.txt), index this project's sources, and set up reference coding: clone and index the open-source repos closest to what we're building, and search how they solved a problem before writing code.<br>COPY THE PROMPT
One paste — your agent installs XERJ, indexes your project and<br>the open-source repos worth learning from, and looks implementations up<br>instead of re-deriving them.
01·REFERENCE CODING · THE CODING USE CASE
GREP RE-READS.<br>XERJ RETRIEVES.
A coding agent hitting an API it hasn't memorised invents method names,<br>fails to compile, and loops — spending output tokens, the<br>expensive kind, on every lap. Grep doesn't save it: grep tells the agent<br>where to look, and the recovery is still reading source into context —<br>up to 1.06M input tokens on one corpus in our<br>measurements. Reference coding flips the loop: clone the open-source<br>repos closest to what you're building,<br>xerj autoindex them once, and the agent<br>retrieves the exact definition — with its contract — before writing.<br>Measured (8 tasks across 4 languages, 16 runs per arm, real<br>claude -p token counts): 2.7× fewer<br>output tokens than grep-driven Claude Code at the same 16/16<br>solve rate, 26× fewer than working from memory<br>(260,916 → 9,982), 2.1× cheaper ($1.58 vs $3.27). In a<br>companion run on a Rust library the model had never seen:<br>9/9 with retrieval vs 0/9 from memory . The three helper<br>scripts below ship with the case study — the hero prompt sets them up.
$ xc-corpus.sh xerj-storage https://github.com/spacejam/sled … # clone the peers, once<br>$ xc-index.sh xerj-storage # xerj autoindex, once per corpus<br>$ xc.py xerj-storage "how does sled fsync its WAL segment on rotation?"<br>→ the exact function, file:line, contract included — a passage to read, not a tree to grep
THE FULL CASE STUDY — EVERY NUMBER, EVERY TASK, THE EXACT PROMPT →·WHERE THE USERS-REPORT ~5× FIGURE COMES FROM →
02·POINT XERJ AT A FOLDER · ZERO CONFIG
ONE COMMAND.<br>ANY FOLDER.
The fastest way to make data useful to an AI agent is to not write a<br>pipeline at all. xerj autoindex<br>— a subcommand of the same binary — walks the tree, sniffs every file's<br>format by content (extensions are never trusted) across<br>13 format families — JSONL, JSON, dialect-sniffed CSV, logs, SQL dumps,<br>SQLite, PDF, DOCX, HTML, XML, YAML, plain text, gzip — infers field<br>types and date encodings from the data itself, writes explicit mappings,<br>and streams everything in with idempotent IDs. Junk files are recorded,<br>never fatal. It ends by writing a catalog index — the data map — so the<br>agent's first question, "what is even in here?", is answered by<br>the engine. Real captured run:
$ xerj autoindex ./sample<br>autoindex: 4 files (0 MB) under /tmp/xerj-pubpass/sample<br>phase A: sniffing + sampling 4 files…<br>phase A: 3 datasets inferred, 1 junk/skipped files<br>phase B: indexing 3 files with 8 workers → http://localhost:9280
done in 0.2s — 3 datasets, 5801 records live, 0 junk records, 1 junk/skipped files<br>ax-logs 5000 docs<br>ax-exports 800 docs<br>ax-docs 1 docs
next: `xerj autoindex map --url http://localhost:9280` for the data map; search via GET /ax-*/_search
GROUND-TRUTH EXAM · 1,995 FILES / 518 MB
80/81 CHECKS
secret-manifest corpus, 25 formats · the one miss: a Shift-JIS file indexed as mojibake
518 MB → LIVE, TYPED INDICES
38.1 s
31 datasets · 2,018,398 records · ~38–51 s across runs · 33.7k rec/s end-to-end on 923 MB
RESUMABLE · IDEMPOTENT
KILL -9 SAFE
journal resume converges to identical final counts · client memory flat ~250 MB at 5× input growth
Honesty first: in a controlled 10-question exam on that 518 MB corpus,<br>a XERJ-backed agent scored 9 correct + 1 partial vs a fair grep/python<br>baseline's 10/10 — a tie on accuracy, not a rout . What<br>XERJ wins is structural: a full corpus inventory in 4 API calls,<br>sub-second aggregations over millions of rows, and uniform access to<br>SQLite, DOCX, gzip, and decimal-comma CSV through one API — the<br>advantages that matter at scale, over remote/API-only access, and under<br>repeated querying. The pipeline is streaming and resumable, verified on<br>multi-GB corpora.
THE FULL RECIPE, EVERY NUMBER TRACED TO A RUN →
03·BUILT FOR AGENTS · MEMORY · RECIPES
AGENTS ARE<br>THE CUSTOMER.
Every operation an agent needs is a plain HTTP call on<br>:9200 — no SDK, no signup, no external<br>embedding key. Store what an agent learns, recall it later by meaning<br>(the built-in embedder is lexical hash-based — honest hybrid<br>lexical+vector retrieval, not neural understanding). Each namespace is<br>physically isolated, so agents never read each other's memories. Real<br>run against an empty XERJ:
$ curl -sXPOST localhost:9200/_memory/agent-demo \<br>-H 'content-type: application/json' \<br>-d '{"text":"The user prefers metric units and a dark UI...