A document checker that can reduce wasted RAG tokens
AksharaMD is a local Python for checking documents before they are added to an LLM or RAG system. It turns a file such as a PDF into Markdown, structured JSON, validation warnings, a run manifest, and chunk JSON.
The manifest includes a 0 to 100 readiness score and a quality label such as HIGH, OK, RISKY, or POOR. It warns about files that need OCR, have too little readable text, contain broken character artifacts, or create unnecessary token growth.
Optional add-ons cover OCR, image and table extraction, math OCR, audio , and S3 input. The code is under PolyForm Noncommercial 1.0.0, so commercial use may be limited.
Key points
- AksharaMD checks whether parsed documents are ready before they enter a .
- It outputs Markdown, JSON, validation warnings, a manifest, and chunk JSON.
- The manifest gives a 0 to 100 readiness score and a simple quality band.
- It flags token growth, which can help avoid avoidable LLM cost.
- PolyForm Noncommercial 1.0.0 means commercial use needs careful license review.