Pick only the needed agent tools instead of loading every tool

An AI agent platform can connect to outside services such as GitHub, Linear, Notion, Stripe, Zendesk, and Salesforce. The first approach was to put every tool schema from every connector into the model context on every turn. With 8 connectors and about 15 tools each, this already meant thousands of tokens before the user even made a request. Once the tool list grew past about 100 tools, the model more often chose the wrong tool, made up bad parameters, or stalled.

A self-hosted open-weights Qwen3 chat model was useful, but it was not reliable at choosing one correct tool from roughly 200 options. The better approach was to infer the needed connector from the user's message, then pass only a few matching tools into the tool-calling loop. Each tool was indexed by its name, description, and connector, then searched at runtime using the conversation context. Embeddings helped catch meaning, such as connecting “get back to this angry customer” with customer-support actions, while BM25 helped catch exact terms such as Stripe and refund.

Combining both search scores cut tool context from thousands of tokens to a few hundred, reduced wrong-tool calls, and made it possible to add more connectors without making every conversation worse. The weak point is that a missed search result means the model cannot call the right tool at all, so the setup keeps a small always-available tool set and a fallback re-query. Thresholds also need tuning: too strict removes needed tools, while too loose brings back overload. Multi-step work that moves between services needs fresh retrieval each turn, not just once at the start.