Developer builds a RAG system from scratch, no LangChain or LlamaIndex
A developer built a pipeline entirely from scratch, skipping common frameworks like LangChain or LlamaIndex to understand what happens under the hood. The system combines , using FAISS for vector search and BM25 for , and merges their results with (RRF). Results are then refined further with .
PDF documents are ingested using semantic chunking, which splits text by meaning rather than fixed length, and the system includes query analysis plus retrieval of Mermaid diagrams. Context and prompt assembly are handled through modular builders, and final responses are generated using Groq. Building this taught the developer a great deal about and system design, and how much of a RAG system's quality is determined before the LLM is even called.
Remaining work includes evaluation benchmarks, , and product quantization (PQ) for vector compression.
Key points
- Combines FAISS (vector search) and BM25 () in a setup
- Merges results with RRF and refines them with
- Uses semantic chunking to split PDF documents by meaning
- Generates responses with Groq; context and prompt building are modular
- Evaluation benchmarks and remain as future improvements