Small local models can struggle with utility tasks
A test on an M1 MacBook Pro with 32GB of RAM compared several for a role. The models mentioned were llama3.1, llama3.2-3B, Phi4-mini-3.8B, and Qwen 2.5-3B. Several of these smaller models were too eager to delete memories instead of carefully merging or organizing them.
Skill auditing also took a long time. Large models feel too heavy for routine utility work, but chat and agent work already use often, so using the same model for utility and chat is not ideal.
Key points
- An M1 MacBook Pro with 32GB of RAM was used to test small s.
- llama3.1, llama3.2-3B, Phi4-mini-3.8B, and Qwen 2.5-3B were mentioned.
- Some small models deleted memories too aggressively.
- The models did not reliably merge memories, and skill auditing was slow.
- is used for chat and agent work, but the same model is not desired for utility tasks.