Offline PDF question app using local models and search
A PDF upload is split into overlapping text chunks so useful sentences are less likely to be separated. Each chunk is turned into an and saved on disk in ChromaDB. When a question is asked, the question is also turned into an , and ChromaDB finds the closest PDF chunks.
An Ollama chat model receives only those chunks as context and answers from them. The prompt tells the model to say it does not know when the answer is not in the PDF. A test question outside the PDF was answered with “I don’t know” instead of a guess.
The app also kept working with Wi-Fi turned off because the chat model, , and all run locally. The stack is Ollama, ChromaDB, and Flask.
Key points
- PDF text is split into overlapping chunks to preserve nearby meaning.
- Each chunk becomes an and is stored persistently in ChromaDB.
- Questions are matched against the stored chunks, and only the closest chunks are sent to the model.
- The s answers to the retrieved PDF context and asks the model to admit when the answer is missing.
- Ollama runs both the chat model and locally.