Context Engineering for AI Agents: How to Structure a Knowledge Base for RAG
What is context engineering for AI agents?
Context engineering is the work of optimizing the data and knowledge bases an AI agent retrieves from. At Lanch, agents access different knowledge bases—a customer support agent, for example, takes in tickets plus a set of policies for handling refunds.
The lesson the team learned the hard way: you cannot simply connect all of your information and assume you'll receive amazing answers out of the agents or workflows you set up. As Lanch puts it, the most important thing about context in general is that it's clean and well-structured.
The deeper point is that agents are still machine learning—still just predicting. If you don't understand your data model, or your systems aren't connected in the correct way, you can't build any agents at all.
Why does clean, structured data improve AI agent accuracy?
Roughly nine months before this conversation, Lanch started structuring all of its standard operating procedures—SOPs for its QSR stores. Just structuring that data in the correct way was a huge win across multiple metrics.
- Inference speed — the agents could be built to run faster.
- Quality — output quality improved.
- Accuracy — answers became more accurate.
The principle is plain machine learning: clean your data, structure it well, and the model performance will be better. The mistake to avoid is connecting all your information first and expecting good answers to fall out automatically.
“The most important thing about context in general is that it's clean and well-structured.”
How do you structure a knowledge base for RAG so AI can retrieve it?
The structure that helps LLMs cite an article is the same structure that helps an internal agent retrieve the right answer. When you optimize content for AEO or GEO, you give each article a TL;DR at the top that acts as a tag signalling whether the article is relevant to the query, followed by sections and FAQs.
That same model maps onto an internal knowledge base as symptom, diagnosis and solution. Rethinking how AI visibility works gives you a mental model to reorganize your internal knowledge base so you can retrieve information internally and make it helpful.
The retrieval system matters too: a graph RAG or RAG knowledge base pipeline works really well with large text documents.
“clean your data, structure it well, and the model performance will be better”
When should you use a SQL retrieval agent instead of RAG?
The right retrieval system depends on the shape of your data. A RAG or graph RAG pipeline is suited to large text documents. But the moment you're interacting with more structured data, that approach isn't enough.
As Lanch describes it, you do need to set up some kind of SQL retrieval agent as soon as you're interacting with more structured data. Lanch's own foundation is a data warehouse integrating delivery platforms, supply chain, and stock data—millions of data points a day—which is structured data well-suited to SQL retrieval.
Even the systems you use to interact with different kinds of data are an important consideration, which is why understanding your data model matters more than ever.
“you do need to set up some kind of SQL retrieval agent as soon as you're interacting with more structured data”
How should you write troubleshooting articles so AI can retrieve them?
Write them the way you'd structure an article to show up in LLMs like Perplexity. The repeatable shape:
- A TL;DR at the top — a too-long-didn't-read section that acts as a tag giving context about whether the article is relevant to the query being asked.
- Clear sections — broken into multiple parts.
- FAQs — at the end.
For internal troubleshooting, the same idea translates to symptom, diagnosis and solution—so the agent can match a problem to its fix and surface the right answer.
Frequently asked questions.
- What is context engineering for AI agents?
- It's optimizing the context agents retrieve from their knowledge bases. The most important thing about context is that it's clean and well-structured. Lanch's mistake was connecting all their information and assuming the agents would return amazing answers—they won't. Agents are still machine learning and still just predicting, so without a clear data model and correctly connected systems you can't build agents at all.
- Why does structuring data improve AI agent accuracy?
- Because it's standard machine learning: clean your data, structure it well, and model performance improves. When Lanch structured its SOPs for QSR stores correctly, it was a huge win for inference speeds, quality, and accuracy. The same logic that makes a model perform better on clean training data makes an agent retrieve and answer more accurately.
- How do you structure a knowledge base for RAG?
- Mirror how content is structured to surface in LLMs: a TL;DR at the top that tags whether the content is relevant to a query, then sections, then FAQs. For troubleshooting, organize as symptom, diagnosis and solution. A graph RAG or RAG knowledge base pipeline works really well with large text documents.
- When should you use a SQL retrieval agent instead of RAG?
- Use SQL retrieval as soon as you're interacting with more structured data. RAG and graph RAG pipelines work really well with large text documents, but structured data—like the integrated delivery, supply chain, and stock data in a data warehouse—calls for a SQL retrieval agent. The system you pick should match the kind of data you're querying.
- Does a TL;DR really help retrieval?
- Yes. A TL;DR—a too-long-didn't-read section at the top—acts as a tag giving context about whether the article is relevant to the query being asked. That tagging helps both external LLMs like Perplexity decide relevance and internal retrieval agents surface the right document.