Why a 4B language model can still be slow on a phone
Running a directly inside a can still be too slow for normal use. Qwen 1.5B and gave poor results and repeated themselves too much. Qwen 3 4B is being tested instead, with an empty think tag forced in order to skip the reasoning path.
The is about 18,000 tokens, and the full prompt stays below 20,000 tokens. The cache is large because it carries many settings and conditions for the model. These choices were meant to make the model faster on the device, but each response still takes 2 to 3 minutes to generate.
The target is to get replies under 1 minute after each user input.
Key points
- Qwen 3 4B is being tested for a that runs a on the phone.
- Qwen 1.5B and produced weak results and too much repetition.
- The is about 18,000 tokens, with the full prompt under 20,000 tokens.
- An empty think tag is being forced to skip reasoning and improve speed.
- Responses still take 2 to 3 minutes, while the goal is under 1 minute.