Small models can't voice their confidence, so this adapter reads it internally instead

Someone built and open-sourced a 10MB LoRA adapter for Qwen3.5-4B plus a small orchestration layer. For every query it decides whether to answer directly, search the web, or pull from local documents, and it refuses to make up an answer when it can't verify one. It runs locally on Apple Silicon via MLX, with a GGUF build available for llama.cpp and Ollama.

The motivating problem: across seven tested models in the 3-9B range, all of them hit a ceiling when trying to state how confident they actually were — they tend to claim high confidence regardless of whether the answer is right. But that confidence information does exist inside the model, in its internal activations. The adapter reads that internal signal directly and uses it to gate whether a tool gets called.

In testing, it caught more of its own errors than the base model's tool-calling did, improving a discrimination metric called d′ by 0.46 (95% CI [0.01, 0.89]). Of the cases the gate flagged that the base model missed, 87% turned out to be genuinely wrong answers. There's also a privacy angle: a two-signal version of the system routes personal queries — like asking what a discharge summary said — to a local retriever instead of a web search, cutting down how often private questions leak out to public search.