Short answer: LLM Wiki does not replace RAG and does not guarantee 100% accuracy. But for a small core of critically important knowledge—policies, contracts, technical documentation, and complex research—a connected semantic map can be more reliable than naive RAG with mechanical text chunking. For large and constantly updated datasets, RAG or a hybrid architecture is more practical.
The problem is not RAG itself, but its simplified implementation: a document is cut into fragments without regard for structure, converted into vectors, and the system hopes that search will find exactly the piece needed for an answer. When the cost of error is high, mathematical text similarity alone may not be enough.
What RAG is and why chunks lose context
RAG, or Retrieval-Augmented Generation, is an architecture in which a language model receives information from an external source before answering. The process usually looks like this:
- Documents are split into fragments.
- Vector representations are created for the fragments.
- The user query is also converted into a vector.
- Search selects the most similar fragments.
- The LLM generates an answer based on the retrieved context.
Chunk size has no universal standard. It can be measured in characters, words, or tokens and should be selected based on the documents and query type. Therefore, claiming that RAG always uses fragments of 400–600 characters is incorrect.
Problems arise when mechanical chunking separates a rule from an exception, a definition from a note, or a contract clause from its related appendix. Search may find text similar to the query, but not the set of provisions needed for a correct conclusion.
Such errors are especially dangerous in legal documents, internal policies, medical instructions, financial rules, and technical procedures. Here, an AI system must not merely find familiar words, but reconstruct links between multiple sources.
What LLM Wiki and a semantic map are
LLM Wiki is a knowledge base pattern proposed by Andrej Karpathy, in which the model not only searches original documents but also gradually creates a maintained structure of linked Markdown pages.
The basic architecture uses three layers:
- Raw sources — immutable original documents.
- Wiki — model-created pages about concepts, objects, sources, contradictions, and relationships.
- Schema — instructions defining the database structure, source processing rules, and answer requirements.
The file index.md serves as a catalog: it contains page titles, short descriptions, and links. Upon receiving a question, the AI agent first examines the index, then opens relevant pages and, if necessary, follows cross-links.
This is not a ready-made universal technology or a recognized standard with proven accuracy. Rather, it is a practical knowledge management pattern designed primarily for moderate volumes of information. Karpathy himself describes working with around 100 sources and hundreds of Wiki pages, not millions of documents.
RAG vs. LLM Wiki: key differences
| Criterion | Classic RAG | LLM Wiki |
|---|---|---|
| Data preparation | Document chunking and search index creation | Reading sources and creating linked topic pages |
| Search | Semantic, full-text, or hybrid search | Navigation through the index, pages, and cross-links |
| Working with relationships | Depends on chunking, metadata, and retrieval architecture | Relationships are recorded in advance in the Wiki structure |
| Scaling | Suitable for large and dynamic datasets | More practical for a moderate volume of curated knowledge |
| Preparation cost | Relatively low with simple indexing | Higher due to source analysis, summarization, and linking |
| Updates | Changed documents can be automatically reindexed | Pages, relationships, conclusions, and the index need updating |
| Verifiability | The retrieved original fragment can be shown | An explicit link between the Wiki page and the original source is needed |
Why comparing it with “naive RAG” can be misleading
Modern RAG is no longer limited to random equal-sized chunks and nearest-vector search. Production systems use:
- Splitting by headings, sections, and semantic boundaries.
- Parent and child fragments.
- Hybrid search using embeddings and keywords.
- Metadata filtering.
- Query rewriting.
- Reranking retrieved fragments.
- Chunk contextualization.
- Knowledge graphs and multi-stage search.
- Answer verification against primary sources.
For example, Anthropic showed that adding context to chunks and then reranking them reduces the number of failed searches in its tests. Microsoft GraphRAG builds a graph of entities and relationships and uses hierarchical summaries for questions requiring analysis of the entire dataset.
In Microsoft tests, GraphRAG outperformed naive RAG in answer completeness and diversity in around 70–80% of comparisons. But this does not mean 70–80% overall accuracy and does not prove the superiority of the graph approach for every query. The study evaluated a specific type of global questions on two datasets using an LLM judge.
Can a semantic map eliminate hallucinations
No. A semantic map reduces some risks of context loss, but creates new ones:
- The model may incorrectly summarize the original document.
- An important detail may not make it into a Wiki page.
- A relationship between concepts may be constructed incorrectly.
- After a source update, some pages may remain outdated.
- Conclusions from different sources may be combined without sufficient grounds.
- The agent may choose the wrong route through the index.
Therefore, claims of “80% RAG accuracy,” “10 out of 10 for LLM Wiki,” or “100% accuracy” cannot be used when choosing an architecture. Without a test set of questions, correct answers, and a measurable methodology, these figures remain marketing claims.
For processes where errors are costly, source citations, action logging, and human confirmation of critical decisions are necessary. These measures are discussed in more detail in the article on AI agent security and data access restrictions.
When to choose RAG
RAG is usually better suited if:
- There are many documents, and they are updated regularly.
- Fast search across a catalog, correspondence, or support ticket database is needed.
- Most questions relate to individual facts or documents.
- Automatic synchronization with a CRM, cloud storage, or website is important.
- A large number of requests must be handled.
- You can create a test set and tune retrieval for real questions.
This architecture is suitable for customer support, knowledge base search, document analysis, and other AI business automation scenarios.
When an LLM Wiki is useful
A connected Wiki may be more useful if:
- There are relatively few sources, but each one is important.
- The answer requires comparing several documents.
- Conclusions need to be accumulated rather than reconstructed for every request.
- Contradictions, dependencies, versions, and change history matter.
- An expert should review the knowledge base regularly.
- The result needs to be stored in a human-readable format.
Examples include a project's regulatory framework, technical documentation for a complex product, due diligence materials, a research database, or internal company policies.
Hybrid architecture: a practical option for business
In most corporate projects, there is no need to choose only one approach. It is more reliable to divide knowledge into two levels:
- RAG layer indexes a large volume of documents, emails, tickets, and updated data.
- Wiki layer stores verified definitions, rules, relationships, contradictions, and conclusions on critical topics.
- Router determines which source to use for a specific question.
- Model generates an answer with references to the source documents.
- Control layer checks for evidence and refers disputed cases to a specialist.
Such a workflow can be included in a broader system: request source → classification → search in RAG and Wiki → evidence check → answer → recording the result in CRM or handing it over to a manager. This is one way to implement an AI assistant or corporate chatbot.
Which MVP to start with
Do not index all corporate information at once. To validate the architecture, it is enough to:
- Choose one process and 20–50 real documents.
- Collect 30–100 questions that employees or customers actually ask.
- Document the correct answers and mandatory sources.
- Create a basic RAG with meaningful chunking and reranking.
- Create a separate Wiki for the most interconnected and critical rules.
- Compare answer completeness, citation accuracy, latency, and cost.
- Test updating and removal of outdated data.
- Keep human approval for critical actions.
Such a pilot follows the “process analysis → plan → MVP → testing → launch” approach described in the CenterAI development process.
Key takeaway
A semantic map can indeed be more reliable than naive document chunking when the answer depends on relationships, exceptions, and multiple sources. But an LLM Wiki is not a universal replacement for RAG and does not eliminate model errors.
The practical choice is made not by the trendy name of an architecture, but by testing results on real questions. Large, dynamic datasets usually require a mature RAG. A maintained Wiki is useful for the critical core of knowledge. In complex projects, their combination often becomes the optimal solution.
To determine the right architecture and pilot scope, you can discuss the process and assess an MVP with CenterAI.
FAQ
Does a RAG system need a vector database?
Not always. Retrieval can use full-text, vector, or hybrid search. A local index is enough for a small project, while large-scale systems may use specialized databases or search platforms.
Can an LLM Wiki be built without Claude Code?
Yes. It is an architectural pattern, not a feature of a single product. It can be implemented using Codex, Claude Code, a custom AI agent, or another tool that can read and modify files according to established rules.
Which is cheaper: RAG or LLM Wiki?
The answer depends on the number of documents, update frequency, number of requests, and models used. A Wiki requires upfront analysis and structure maintenance. RAG requires spending on indexing, search, reranking, and context processing. Costs should be measured on a specific MVP.
Is an LLM Wiki suitable for a legal database?
It can be used as a navigation and analysis layer, but it should not replace original regulatory documents. Every conclusion must be linked to the primary source, its effective date, and a verifiable citation.
Sources and verification
Last reviewed: 20.08.2026
- https://arxiv.org/abs/2005.11401 — the original research paper on Retrieval-Augmented Generation.
- https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f — Andrey Karpathy's description of the LLM Wiki pattern.
- https://github.com/microsoft/llmwiki — Microsoft's open implementation of LLM Wiki.
- https://www.anthropic.com/engineering/contextual-retrieval — Anthropic's research on chunk contextualization and reranking.
- https://www.microsoft.com/en-us/research/project/graphrag/ — the Microsoft GraphRAG project.
- https://www.microsoft.com/en-us/research/blog/graphrag-new-tool-for-complex-data-discovery-now-on-github/ — an evaluation of GraphRAG compared with naive RAG.
