A local personal assistant shows where small AI agents break
A local setup uses Qwen 3.5 4B on Ollama for project tracking and daily organization. The machine has an RTX 2070 Super with 8GB VRAM and 32GB system RAM. Whisper large-v3 stays loaded for voice input, and Telegram is used as the chat front end. Qwen 3.5 4B fits fully on the GPU, uses about 5GB with a 64k context, and runs at about 68 .
Whisper uses about 2GB more, so the 8GB VRAM budget is almost full. The main problem is reliability: the small model sometimes drops or damages tool calls, says something was saved when it was not, and chooses the wrong item when a request like “mark that one done” is unclear. Larger Qwen models were tested, including 9B, 27B, and 35B variants, but they did not fit in 8GB VRAM, spilled into system RAM and CPU, ran slowly, and did not handle unclear references better. Updating Ollama improved tool-call reliability from about 55% to almost 100% and nearly removed false save confirmations.
Qwen3 also needs a /no_think prefix in this setup, or it can spend its on thinking and return empty content. The discussion leans toward using scaffolding: let the model suggest an action, but let normal code validate paths, write files, return real success or failure, and keep the model from being the .
Key points
- Qwen 3.5 4B can run quickly inside 8GB VRAM, but it is not reliable enough by itself for save-and-update tasks.
- Tool-call reliability changed sharply after an Ollama update, showing that runtime software can matter as much as model choice.
- Larger Qwen models were slower on this hardware and did not solve unclear task references in this test.
- Qwen3 may waste its on thinking unless /no_think is used in this setup.
- A safer local assistant should let code confirm real actions instead of trusting the model to say what happened.