PDF conversion is a hidden cost issue for large RAG systems
Large healthcare policy documents need to be converted from PDF to Markdown before they can be used in a RAG system, but speed and quality are hard to balance. The workload includes thousands of PDFs, with many documents running from 100 to more than 400 pages. Tables matter, some files need OCR, and the documents include English, French, and Spanish.
Many files are messy, with rotated pages, scanned pages, mixed layouts, stamps, handwritten notes, and low-quality scans. LLM or VLM processing is not practical because the cost would be too high at this scale. The tool also needs a permissive license such as MIT, Apache, or BSD, because AGPL and GPL tools are not acceptable for a private .
PyMuPDF is very fast, but it loses too much page layout and table structure. PyMuPDF4LLM gives better results while staying fast, but its AGPL license creates a problem. Docling, used without VLM mode, handles tables and layout much better, but it is much slower on large documents.
Key points
- Large RAG can involve thousands of long PDFs, so speed matters.
- Tables, page layout, and OCR quality affect how useful the final will be.
- LLM and VLM processing may be too expensive when the document volume is very large.
- PyMuPDF is fast but loses too much table and layout detail.
- Docling gives better structure without VLM mode, but it is slower on large files.