Give a local agent web access without paid search APIs
A can get web access without using paid services such as Tavily, Serper, or Firecrawl. The setup uses two simple tools: one for search and one for page . Search runs through SearXNG, a meta started in Docker, and the agent calls its .
Each search result is reduced to a title, URL, and description. The description is only a search snippet, not the page content, so it is not enough when the agent needs to read the full page. For page , Scrapling and Trafilatura are used together, with a fast path that fetches normal pages without launching a browser.
SearXNG must have JSON enabled in its settings, and public SearXNG instances are usually a poor fit for automated program use.
Key points
- The setup gives a web access without paid search or scraping APIs.
- SearXNG handles search through a .
- Search snippets are not full page content, so a separate step is needed.
- Scrapling and Trafilatura handle page .
- Public SearXNG instances are usually not suitable for automated agent use.