promptchain: Python library auto-swaps local LLMs on one GPU
promptchain is a Python library that solves a common headache for anyone running multiple on a single consumer GPU: manually loading and unloading models, or writing custom swap logic for every project. It provides a unified streaming interface across different backends — LM Studio, Ollama, and any . At its core is a real policy engine for VRAM management: a ModelManager object takes a policy (e.g.
"auto"), a cap on how many models can stay resident at once (max_resident), and an idle-unload timer that frees a model after it sits unused for a set number of seconds. Models are registered under names (like drafter or coder) with priority levels, and the manager auto-evicts by priority plus LRU (least-recently-used) logic whenever a VRAM budget or resident-model cap is hit. It also ships an , letting an AI agent drive this model-swapping logic directly rather than going through manual commands.
The project started as a Streamlit app; after feedback, the creator rebuilt it as a standalone library with proper multi-backend support instead of a single-server proxy.
Key points
- Python library that auto-loads/unloads multiple on a single consumer GPU
- Unified streaming interface across LM Studio, Ollama, and s
- Auto-eviction by priority plus LRU under a set VRAM budget or resident-model cap
- Ships an so an AI agent can control model swapping directly
- Rebuilt as a library after feedback on an earlier Streamlit app version