Consumer AI agents face hard choices on device, server, and price
A agent app should be different from an enterprise work tool or a developer tool. It should be something an ordinary person uses because it makes a phone more useful. Running the agent loop on a server can make the app more reliable because it can keep working while the phone is idle, handle long tasks, and avoid draining the battery.
Many useful consumer tasks, however, depend on the phone itself, such as reading messages, using apps, or using the camera. If the main brain runs on a server, personal data must move back and forth, which can add delay and raise trust concerns. A practical direction is to keep execution and personal context on the device, while sending only heavy reasoning work to a server-based .
That mixed design is harder to build than choosing only or only on-device. Consumer pricing is also difficult because businesses may accept a $50 per-seat monthly price, while ordinary users may resist even $5, making a bring-your-own API key model one possible approach.
Key points
- execution helps with , long-running tasks, and battery use.
- On-device execution fits personal phone tasks like messages, apps, and camera use.
- A mixed setup can keep execution and personal context on the device while using a server for heavy reasoning.
- Consumer pricing is much harder than business pricing because users are more price-sensitive.
- A bring-your-own API key model could shift model costs directly to the user.