Small local AI models are being tested on smartphones
An is available for managing two small that can run fully locally, even on a smartphone. The 4B model works well, but it needs a high-end phone. The 1.7B model is not yet stable when is turned on.
The planned fix is deeper with about 130,000 examples. The hard part is not just improving the training score, but getting enough high-quality and varied examples. A larger 32B model is being used as a teacher, with used to transfer its behavior into the smaller models.
Once the dataset is ready in about 10 days, the 1.7B model may improve without relying on the current LoRA setup.
Key points
- The manages two small that can run locally on a phone.
- The 4B model works well but needs a high-end smartphone.
- The 1.7B model is unstable when is active.
- About 130,000 examples are being prepared for .
- A 32B teacher model is being used for into smaller models.