A local multi-agent pipeline turns notes into a 3D knowledge map
Cosmind is a tool that processes messy Markdown notes with several specialized agents instead of one large retrieval step. One agent splits raw notes into small notes, another adds web-based context, another reads images and , and another writes literature-style summaries. It runs by default on through Ollama, including options such as Qwen2.5, Llama3, and Llama3.2-Vision.
Paid APIs such as OpenAI can be used when more power is needed, but the default setup keeps data on the user’s own machine. ChromaDB stores the note meanings for similarity search. The chat feature answers only from the user’s own vault, and when the answer is not there, it offers web search instead of making something up.
The system also creates a by linking related notes and shows the structure in a 3D view. The backend uses FastAPI, the frontend uses React with TypeScript, and the whole project is packaged with Docker.
Key points
- The system uses several agents to clean, split, enrich, and summarize Markdown notes.
- Ollama is the default runtime, so the pipeline can run locally without sending data out by default.
- Paid APIs can still be connected when stronger models are needed.
- ChromaDB is used as the for finding related notes.
- The main open question is whether the multi-agent design is worth the extra delay and token cost.