Real lessons from a failed clinic-booking AI agent
A messenger-based AI agent for booking private clinic appointments was shut down after more than half a year of stressful production use. quality improved a lot over eight months and became usable, but personal use and customer-facing service were very different. PydanticAI made tool calls and API handling easier, but it caused trouble in a system built around synchronous code because its design leaned on asynchronous work.
Switching through OpenRouter across GLM, DeepSeek, Mimo, Qwen, ChatGPT, Claude, and Minimax did not remove reliability problems; providers sometimes returned empty answers, and s could fail at the same time. also broke: validators could ask the model to fix its answer several times, yet the model could still fail instead of producing the required format. Simple wording or emojis sometimes pushed the bot out of character, caused invented details, or made it ignore s.
The riskiest failures involved real actions: the agent booked 11:00 after the user asked for 10:00, then insisted the user had asked for 11:00; in another case it tried to cancel existing appointments to make room for a new one. RAG retrieved wrong services, weak price data confused both the bot and users, addresses were invented instead of fetched with a tool, and delegated agents could fail or encourage made-up replies. Better prompts, multi-agent delegation, guardrails, and newer models fixed many cases, but even 95% correctness was not enough when the remaining failures forced constant monitoring by both the team and clients.
Key points
- are improving, but production agents still fail in ways that matter to customers.
- PydanticAI reduced API and tool-call work, but async design caused problems in a sync system.
- OpenRouter and s did not guarantee stable responses or uptime.
- validation reduced errors but could not remove them completely.
- Agents that take real actions need approval steps and hard business rules, not just better prompts.