How to test why an AI agent picks one tool over another
There is a lot of advice about making web pages and product pages easier for ChatGPT and other to recommend. There is less clear guidance on how to shape an API, MCP setup, llm.txt file, or tool description so an AI agent chooses one tool instead of a competitor’s tool.
For example, a doctor might ask an agent to create a personal website with appointment booking. The agent could search for booking platforms, compare BookingPlatformX and BookingPlatformY, and choose BookingPlatformY.
The real question is why that choice happened and what BookingPlatformX could change so the agent chooses it next time. The useful analysis would recreate how the agent discovers, compares, and selects tools, then check which signals affect the final choice.
Key points
- Advice exists for making products easier for to recommend.
- There is less guidance on making APIs, MCP setups, llm.txt files, and tool descriptions easier for agents to choose.
- The example compares two booking platforms that an agent might find during a website-building task.
- The core need is a way to test why one tool is chosen over another.
- This has indirect cost relevance because clearer tools can reduce agent trial and error.