A Go router that picks tools instead of stuffing them into Llama
Putting 50 tools into a local Llama agent can make the model slower and less focused because the context becomes crowded. The MCP ecosystem is mostly built around Python and TypeScript runtimes, so a Go backend often has to run subprocesses or connect through heavier environments when it needs several . A lightweight and routing engine in Go was built to reduce that burden.
It connects to downstream servers over HTTP/SSE, builds a , and chooses matching tool candidates in real time. When a request contains several tasks, such as asking for the weather and scheduling lunch, it splits the sentence with regular expressions and number checks, then routes the pieces at the same time with goroutines. During live voice streams, repeated partial request text can trigger too many identical embedding API calls, so singleflight collapses matching requests into one.
The package API uses a builder pattern so code can guide setup step by step.
Key points
- Too many tools in an agent context can make the model slower and less reliable.
- A Go-based gateway routes requests to multiple without leaning on heavy Python or TypeScript runtimes.
- The system builds a and picks likely tool candidates in real time.
- Compound requests are split and routed in parallel with goroutines.
- singleflight reduces repeated embedding API calls during fast, partial voice input.