How RAG helps AI answer from documents, not memory
RAG is a way to make an AI answer using relevant source material instead of relying only on what it learned during training. An LLM only knows the information it was trained on, so it can give a confident but wrong answer when asked about private documents, recent events, or internal data. That kind of wrong answer is called a .
RAG reduces this by breaking documents into smaller chunks, turning those chunks into , and storing them in a . When a question comes in, finds document chunks with a similar meaning, not just matching words. Those chunks are then passed to the LLM as context, so the final answer is based on the retrieved material.
The key idea is that a question and a document can be matched by meaning inside vector space.
Key points
- RAG retrieves relevant documents before the LLM writes an answer.
- An LLM may when asked about information outside its .
- Documents are split into chunks, turned into , and stored in a .
- looks for similar meaning rather than exact keyword matches.
- For AI agents, RAG can improve grounding while helping limit how much context is sent to the model.