TL;DR: Let LLM maintain a persistent wiki by reading → extracting → integrating.
Why we need LLM Wiki?#
RAG is good, but we still need to retrives relevant chunks at every query time, that is not accumulation.
LLM wiki incrementally build and maintain a persistent wiki, that is a structured interlinked collection of markdown files.
How to use it?#
Three layer architecture#
- Raw sources: immutable source of articles, papers, images, data files, etc.
- Wiki: directory of LLM-generated markdown files. LLM owns this layer only.
- Schema: the document that tells how LLM to build the Wiki. User and LLM can both change this document.
Three operations#
- Ingest: Drop a new source into the raw collection, and tell LLM to process it.
- Query: Ask the question against the wiki, LLM search for relevant pages, read them and synthesizes a anser with citations.
Good ansers can be filed back into the wiki as new pages.
- Lint: Periodically ask the LLM to health-check the wiki.
Two special files#
index.md: contet-oriented, each page in wiki will be listed with a link, a one-line summary, and optionally meta data.log.md: chronological, append-only record of what happened and when.
Retrieval strategies#
| Strategy | How | Best for |
|---|---|---|
| Agentic Navigation | LLM picks entry pages from index.md, follows [[wikilinks]] | < 1k pages |
| Grep / Ripgrep | Full-text keyword search | < 5k pages |
| Hybrid | BM25 + embedding + LLM rerank | Any size |
| Graph Traversal | Pull 1–2 hop neighbors via wikilinks | Densely interlinked |
[Optional] CLI tools#
Some time we want the LLM can find markdown pages in wiki more efficiently, we can use qmd. It’s a lical search engine for markdown files.