Reducing mistakes in AI agents that control real devices
An AI agent is being built with Qwen3.5-4B, a , to directly control IoT devices such as lights, fans, air conditioners, and curtains. A larger model may handle this better, but it can make responses slower.
The current system supports multiple tool calls, several actions in one request, several s in one prompt, device control, normal conversation, s, and backend checks before any action runs. It needs to handle requests such as turning on bedroom lights at 70% brightness, closing curtains while turning off the AC and giving tomorrow’s weather, or turning off every light except the kitchen.
The main problem is : the model sometimes picks the wrong tool, fills in wrong , tries to control a device that does not exist, or gets confused when device commands and general questions are mixed. The possible designs being considered are sending every request straight to one with all tools available, adding a before the main LLM, or splitting the system into smaller specialized parts.
Key points
- The agent uses Qwen3.5-4B to control lights, fans, AC units, curtains, and similar devices.
- A larger model is an option, but slower responses are a concern.
- The system already supports multiple tool calls, multi-action requests, JSON output, and backend validation.
- The failures include wrong tool choice, wrong , commands for missing devices, and confusion across mixed request types.
- The design options are one large model with all tools, a , or smaller specialized systems.