Real-world RAG project shows quality work and cloud costs
Permit IQ is a RAG system built with messy, inconsistent real data instead of clean test datasets. The goal was to learn how retrieval finds useful material and how a uses that material to answer questions.
The system is now hosted on . Even as a hobby project, it has already cost about $200, showing that running a production-style RAG system can become expensive quickly.
The next areas to improve are answer quality, evaluation methods, retrieval strategy, reranking, and chunking. Cost reduction is also a focus, including the option of models instead of relying only on cloud-hosted .
Key points
- Permit IQ uses messy real data rather than clean benchmark data.
- The system currently runs on .
- The hobby project has already cost about $200.
- Quality work is focused on evaluation, retrieval strategy, reranking, and chunking.
- Cost work may include models instead of using only cloud-.