LLM Wiki Maintenance: Drift, Contradictions and Review - Rost Glukhov | Personal site and technical blog
Page content
An LLM Wiki fails when old facts remain plausible, contradictions become polished, and generated summaries drift from their sources.
Maintenance is the real product of any compiled knowledge system. Creating wiki pages is straightforward compared with keeping them trustworthy across months of ingest, edits, rewrites, and new sources.
This article covers the operational side of LLM Wiki systems: drift detection, contradiction checks, citation discipline, linting, Git review, and maintenance workflows. It assumes you already understand the basic pattern described in LLM Wiki - Compiled Knowledge That RAG Cannot Replace: raw sources are compiled into durable Markdown pages that humans and agents can query later. The calm but opinionated view is that an LLM Wiki without maintenance is just a nicer-looking knowledge graveyard.
What Makes LLM Wiki Maintenance Different
Traditional wikis rot because people stop updating them. RAG systems drift because the corpus changes, chunks get stale, metadata is weak, and retrieval keeps finding plausible but outdated fragments. The deeper reason is that retrieval and representation solve different problems: retrieval can be re-run against fresh data, but a representation you already compiled has to be actively kept honest.
An LLM Wiki has a different failure mode. It can look clean even when it is wrong.
The pages may be well formatted. The links may work. The summaries may sound balanced. But underneath that neat surface, the system may have dropped critical facts, merged incompatible concepts, cited summaries instead of sources, or preserved an old decision as if it still applied.
That is why LLM Wiki maintenance must check both structure and meaning.
The Maintenance Goal
The goal is not to make every page perfect. That is too expensive, and it usually leads to abandoned systems.
The goal is to keep the wiki useful, inspectable, and recoverable.
A maintained LLM Wiki should make it easy to answer these questions:
What sources support this claim?
When was this page last reviewed?
Has this concept changed?
Are there conflicting pages?
Is this summary still current?
Did the agent rewrite more than it should?
Can we roll back a bad update?
Can a human understand why the page says what it says?
If the answer is no, the problem is not only content quality. It is system design.
A Simple Maintenance Loop
A useful LLM Wiki needs a repeatable loop. The loop should be simple enough to run often and strict enough to catch drift before it becomes invisible.
flowchart TD<br>A[Add or update source] --> B[Compile into wiki pages]<br>B --> C[Update links and indexes]<br>C --> D[Run structural lint checks]<br>D --> E[Run semantic review checks]<br>E --> F[Human review of risky changes]<br>F --> G[Commit approved changes]<br>G --> H[Schedule stale-page review]<br>H --> A
This loop is not glamorous. That is the point.
A knowledge system becomes durable through boring maintenance: source preservation, explicit review, predictable structure, and small safe updates. The same instinct drives evergreen notes: a note or a wiki page only compounds in value if someone keeps refining it instead of leaving it to rot after the first draft.
Types of Drift in an LLM Wiki
Knowledge drift is not one thing. Different drift types require different checks. A good maintenance system should name them clearly.
Source Drift
Source drift happens when the underlying source material changes.
For example, a tool releases a new version, a policy is updated, an API changes, or a vendor deprecates an old feature. The old wiki page may still be accurate for the previous version, but wrong for current use.
Source drift is dangerous because the old claim may still be true in historical context. The problem is not that the claim is fake. The problem is that it no longer answers the current question.
Maintenance response:
Record source dates
Record last reviewed dates
Mark version-specific pages clearly
Link old pages to superseding pages
Avoid mixing old and new versions without labels
Concept Drift
Concept drift happens when the meaning of a term changes over time.
This is common in AI and software architecture. Terms like “agent”, “memory”, “RAG”, “workflow”, “tool use”, “structured output”, and “context engineering” can shift meaning quickly.
A wiki can accidentally preserve several meanings of the same term without explaining the difference. That creates confusing pages that sound coherent but combine incompatible ideas.
Maintenance response:
Maintain glossary pages
Add “meaning in this wiki” sections
Separate overloaded concepts into distinct pages
Link related meanings explicitly
Avoid letting the model silently merge terms
Terminology Drift
Terminology drift is smaller than...