One of the biggest challenges in AI today is not building smarter models. It’s building systems that can remember. I have seen this over and over again throughout my years of experience as an AI engineer.
Over the past year, I’ve spent a significant amount of time exploring memory architectures for AI agents. Like many developers working in the agentic AI space, I started by looking at semantic memory systems powered by vector databases like ChromaDB and Qdrant, which are systems that allow agents to retrieve information based on meaning rather than exact matches. I also experimented with structured memory stored in PostgreSQL, where agents can maintain state, store facts, and track events over time.
Both approaches work well, but they introduce a common challenge. Every time an agent needs context, it must retrieve information from multiple sources, interpret the results, determine what is relevant, and then build an understanding of the situation before it can respond. As systems become more sophisticated, this process can become increasingly expensive, complex, and time-consuming. This is what initially drew me to the concept of an LLM Wiki.
Rather than treating memory as a collection of disconnected records, an LLM Wiki allows an AI system to continuously build and maintain a structured knowledge layer. Instead of repeatedly retrieving and interpreting the same information, the agent works from a curated repository of knowledge that evolves over time.
The concept was popularized by AI researcher Andrej Karpathy, but it has quickly become one of the most interesting ideas in AI knowledge management. More recently, the emergence of the Open Knowledge Format (OKF) has taken the concept a step further by introducing a standardized way to structure and share AI-maintained knowledge bases.
In this guide, we’ll explore what an LLM Wiki is, how it works, how it relates to the Open Knowledge Format, and why it may become a critical component in the future of AI agent memory.
What Is an LLM Wiki?
An LLM Wiki is a knowledge management pattern where a large language model continuously organizes information into interconnected wiki-style pages.
Instead of repeatedly searching through raw documents whenever a question is asked, the AI maintains a curated knowledge layer that can be updated, expanded, and refined over time. Think of it as a Wikipedia that is maintained by AI.
When new information becomes available, the AI can:
- Read source material
- Extract key insights
- Create new pages
- Update existing pages
- Link related concepts
- Generate summaries
- Remove duplicate information
Over time, the wiki becomes increasingly valuable because knowledge accumulates rather than being recreated from scratch during every interaction. This approach allows information to move from raw data into structured understanding.
Who Introduced the LLM Wiki?
The concept gained widespread attention in 2026 when AI researcher and former OpenAI scientist Andrej Karpathy published a document called LLM Wiki. Rather than launching a software product, Karpathy described a pattern for how AI systems could maintain persistent knowledge using markdown files and AI agents.
The proposal addressed a growing problem in AI systems. While language models had become extremely capable at reasoning, they still lacked efficient long-term memory mechanisms. The LLM Wiki pattern offered a practical solution by allowing knowledge to be continuously organized and refined outside of the model itself.
The idea quickly gained traction among AI engineers because it offered a lightweight alternative to increasingly complex retrieval systems.
How Open Knowledge Format (OKF) Is Related to LLM Wiki
As interest in LLM Wikis grew, the need for a standard became increasingly apparent.
This led to the introduction of the Open Knowledge Format (OKF), an open specification designed to formalize the LLM Wiki pattern. While an LLM Wiki describes the concept, Open Knowledge Format defines how that knowledge should be structured and exchanged.
You can think of the relationship like this:
| LLM Wiki | Open Knowledge Format |
| A concept | A standard |
| AI-maintained knowledge base | Knowledge exchange format |
| Flexible implementation | Structured specification |
| Focuses on memory and knowledge | Focuses on interoperability |
The goal of OKF is to make knowledge repositories portable between different AI systems and tools. Just as OpenAPI standardized how APIs are described, Open Knowledge Format may eventually standardize how AI knowledge bases are created and shared.
For businesses building AI infrastructure, this could become increasingly important as multiple agents, tools, and platforms need access to the same organizational knowledge.
LLM Wiki as a Memory Layer for AI Agents
This is where the concept becomes particularly interesting. Most AI agents today rely on some combination of:
- Context windows
- Vector databases
- Retrieval systems
- SQL databases
- Knowledge graphs
Each of these approaches solves part of the memory problem. Vector databases are excellent for semantic retrieval. Relational databases are ideal for structured facts and state management. Knowledge graphs excel at representing relationships between entities. However, these systems often store information in its raw form.
An LLM Wiki introduces an additional layer: synthesized knowledge. Instead of forcing an agent to repeatedly interpret source material, the wiki becomes a continuously maintained representation of what the agent has learned.
In many ways, an LLM Wiki begins to resemble a knowledge graph. Both systems focus on relationships between concepts, people, projects, and events. The difference is that traditional knowledge graphs often require specialized databases and schemas, while an LLM Wiki can represent those relationships through interconnected markdown documents that remain understandable to both humans and AI systems.
For example, a business intelligence agent might maintain pages for:
- Customers
- Products
- Competitors
- Industry trends
- Market research
- Internal projects
Instead of searching across multiple systems every time a question is asked, the agent can consult a continuously evolving repository of organizational knowledge. This creates a form of long-term memory that becomes more useful over time.
How an LLM Wiki Works
Most implementations follow a three-layer architecture.
Layer 1: Source Data
This contains the original information.
Examples include:
- PDFs
- Documentation
- Meeting notes
- CRM records
- Research papers
- Knowledge base articles
- Internal databases
These sources remain the source of truth.
Layer 2: The Wiki
The AI processes information and creates structured pages.
For example:
/wiki
customers.md
products.md
ai-agents.md
market-research.mdEach page contains curated knowledge rather than raw information.
Layer 3: Continuous Maintenance
The AI acts as an editor.
It continuously:
- Updates pages
- Adds summaries
- Creates cross-links
- Merges duplicate information
- Improves organization
The result is a living knowledge repository rather than a static collection of documents.
LLM Wiki vs Traditional RAG
Retrieval-Augmented Generation (RAG) and LLM Wikis are often compared, but they solve different problems.
| Feature | LLM Wiki | RAG |
| Storage | Curated wiki pages | Raw document chunks |
| Memory | Persistent | Temporary |
| Knowledge Growth | Compounds over time | Retrieval only |
| Summaries | Maintained permanently | Generated per query |
| Cross-Linking | Native | Limited |
| Human Readability | High | Low |
With RAG, the agent retrieves information every time it needs context. With an LLM Wiki, much of that interpretation work has already been performed. The agent works from a knowledge layer rather than directly from source material.
In practice, many future architectures will likely combine both approaches.
Why Businesses Should Care
Businesses building AI agents often face a challenge that has nothing to do with model quality. Their knowledge is fragmented.
Critical information is spread across:
- CRM platforms
- Internal documentation
- Email conversations
- Shared drives
- Databases
- Meeting notes
- Project management systems
Traditional retrieval systems require agents to constantly pull information from these sources and interpret the results. I have seen this can take anywhere between 3-10 minutes depending on how many sub agents the system spins up for a query or how complex the user’s query is.
While effective, this process adds latency and complexity. An LLM Wiki offers a different approach. Instead of repeatedly querying every system, the agent maintains an internal knowledge layer that has already been organized and refined. This creates several advantages.
Faster Responses
The agent works from pre-processed knowledge instead of repeatedly analyzing source documents.
Better Consistency
Knowledge can be standardized and maintained in a single location.
Lower Context Costs
Smaller wiki pages are often far more efficient than loading large collections of documents into an LLM context window.
Improved Knowledge Retention
Important insights remain available long after the original interaction has ended.
Stronger Organizational Memory
The agent develops a persistent understanding of the business over time. For organizations deploying AI agents at scale, this could significantly improve both performance and reliability.
How to Create an LLM Wiki
Now that we have covered what an LLM Wiki is, and how businesses can use them in their internal operations, let’s go over how to actually create one. The process is relatively straightforward.
Step 1: Gather Your Source Material
Collect the information you want the AI to learn from.
This might include:
- Documentation
- Research papers
- Internal knowledge
- Meeting notes
- Customer information
- Product information
Step 2: Create a Wiki Directory
Set up a dedicated location for AI-maintained knowledge.
/wikiStep 3: Define a Structure
Organize content into logical categories.
For example:
/wiki
customers
products
projects
research
competitorsStep 4: Use an AI Agent
Configure an AI system to:
- Read source material
- Create pages
- Update pages
- Add summaries
- Create links between concepts
Step 5: Establish Rules
Create guidelines that define:
- When new pages should be created
- How summaries are maintained
- How links are generated
- How information is updated
Step 6: Consider Open Knowledge Format
If you’re building a long-term system, structuring your repository around Open Knowledge Format principles may help future-proof your knowledge base.
Developer Tip: Infrastructure and Tooling Options
When choosing your engineering stack, the runtime environment and hosting options dictate your overall operational speed and privacy boundaries:
- The Interface: Many engineers utilize the markdown editor Obsidian because it natively renders visual interactive graph mappings of how documents are linked. For server environments, headless editors or standard Visual Studio Code workspaces work perfectly. As Karpathy noted: “Obsidian is the IDE; the LLM is the programmer; the wiki is the codebase.”
- The Model Layer: While managed frontier models like Claude are incredibly popular due to their massive context windows, hosting open-source models locally or through hybrid infrastructure is highly competitive.
- The Runtime: Running open-source models via Ollama or offloading computing workloads directly to Ollama Cloud allows developers to leverage highly scalable infrastructure (such as massive gpt-oss configurations) without incurring standard API variable cost traps.
| Platform Evaluation | Managed Frontier Models (e.g., Claude) | Ollama / Ollama Cloud Runtime |
| Platform Architecture | Hosted, managed proprietary API | Local runtime with cloud-scalable offloading |
| Typical Context Window | 200K to 1M tokens depending on tier | Fully configurable based on model architecture |
| Data Privacy Stance | Subject to third-party data privacy policies | Absolute local data sovereignty and control |
| Cost Predictability | Variable usage-based pricing models | Fixed infrastructure or hardware-based costs |
| Execution Speed | Stable, managed inference speeds | Hardware dependent; highly accelerated via cloud compute |
Choosing an open-source model running via Ollama Cloud bridges the production gap seamlessly—keeping your data entirely secure and your operational costs predictable without sacrificing processing speed during large-scale wiki compilation.
The Future of AI Memory
The rise of LLM Wikis reflects a broader shift in AI development. The conversation is no longer focused solely on larger models or longer context windows. Instead, attention is increasingly turning toward memory, knowledge persistence, and organizational intelligence.
Personally, I believe future agent architectures will combine multiple memory layers. Structured databases will manage transactional data. Vector databases will provide semantic retrieval. Knowledge graphs will model relationships. And LLM Wikis may become the synthesized knowledge layer that connects everything together.
Rather than replacing existing memory systems, the LLM Wiki has the potential to become the bridge between raw information and actionable understanding.
Final Thoughts
An LLM Wiki is more than a documentation system. It is an emerging approach to AI memory. Popularized by Andrej Karpathy and further formalized through Open Knowledge Format, the concept provides a way for AI systems to maintain, organize, and refine knowledge over time.
For developers building agents, researchers managing information, and businesses looking to create AI systems with deep organizational context, LLM Wikis may prove to be one of the most important architectural patterns to emerge in the next generation of AI applications.
As the industry moves beyond simple retrieval and toward true long-term memory, the LLM Wiki is a concept worth paying close attention to.
