Open-source Android app runs local language models on a phone
An Android app is being built to run several local AI engines in one place, including MNN Chat, , and llama.cpp. MNN model support is planned but not yet tested, while GGUF models run through llama.cpp. On a Samsung Galaxy S23 FE, Gemma 3 1B reached about 20 , and a TFLite Gemma 4 e2b model reached about 10 on the same phone.
The app is available on GitHub as open source. It includes a testing-stage search feature that has not been released yet, plus , voice input and output, and an in-app server. The design also aims to support a , RAG, and OCR.
An unpublished Invent screen links three models together: a planner model asks questions about a project idea, a researcher recent information, , and whether the project is realistic, then the planner reviews the results before sending them to a coder model.
Key points
- Gemma 3 1B ran at about 20 on a Samsung Galaxy S23 FE.
- A TFLite Gemma 4 e2b model ran at about 10 on the same phone.
- The app uses llama.cpp for GGUF models and aims to support MNN models too.
- Planned features include search, , voice input and output, an in-app server, RAG, and OCR.
- The Invent workflow connects a planner model, researcher model, and coder model into a simple agent chain.