RAG tech stack: free local tools for dev, paid cloud for production
A beginner integrating into a company platform asked what tools teams typically choose, and how those choices shift between development and live deployment. During development, the goal is zero cost: run everything locally using open-source tools. Nomic Embed Text handles turning text into vectors (number representations the system can search), ChromaDB stores those vectors, and Llama 3 serves as the — all run locally via Ollama with no API fees.
When moving to production, teams switch to paid for and performance: Qdrant or Pinecone replace ChromaDB as the , Azure OpenAI takes over embeddings, and GPT-4o becomes the . LangChain, a popular connector framework, is kept minimal or replaced with custom code at deployment to reduce unnecessary overhead.
Key points
- Dev stack: Llama 3 + Nomic Embed Text + ChromaDB, all run locally via Ollama at no cost
- Production stack: GPT-4o + Azure OpenAI embeddings + Qdrant or Pinecone
- LangChain is useful for quick prototyping but teams often replace it with lighter custom code before shipping
- Separating dev and prod tooling keeps early near zero
- This is a beginner question with a ChatGPT-sourced answer — useful as orientation, not a proven recipe