Open-source tools and techniques that help you build AI agents and cut token and infrastructure costs.
A wrong final answer from an AI agent can be treated as an eval failure. But a wrong tool call can be a different kind of problem. Examples include using the wrong tool, working in the wrong repo, reading an unrelated file, or writing before approval. Even if the final answer looks fine, those actions may be closer to a security event. The practical question is where teams should record these failures when building large language model apps with tools: eval logs, app telemetry, security logs, or all three.
A 700-page document is handled by splitting it into separate “skill experts” instead of putting the whole document into one large context window. After a PDF is uploaded, each document section becomes its own expert with its own context and reasoning. Several experts can be grouped under one orchestrator skill, which sends each question to the right expert. Questions that need information from several sections go to multiple experts, and their answers are combined into one response. The system is exposed as an MCP server. One example is turning company knowledge from legal, finance, HR, and product teams into separate experts, then querying them together as one company knowledge base. It claims to work without a vector database, embeddings, or a retrieval step. It also claims to be 70-90% cheaper than loading everything into one context window.
Orkaia is an open-source tool for reducing waste when AI agents keep retrying the same failed action with small wording changes. It checks the tool being used and the cleaned-up input values, so a rephrased retry can still be recognized as the same repeated action. When the same action happens too many times, Orkaia stops the run before the total spending limit is reached. It also records token use for each action, making it easier to see which step caused the cost. It runs fully on the user’s own machine, installs with pip install orkaia, and does not require an API key, signup, or sending data out. It is released under the MIT license. One known next step is better handling of cases where an agent alternates between two actions, such as A→B→A→B.
AI agents can perform worse when they receive several versions of the same data at the same time. An older document may say that a task or rule is valid, while a newer document says it is no longer used. The agent can mix those two messages and treat outdated information as if it still applies. The safer approach is to give the agent only the most recent files whenever possible. Old or deprecated material should stay out of the context unless the agent truly needs to reason about how something changed over time. Marking content as deprecated, then excluding it from the agent’s input, can reduce confusion.
Testing an AI agent on popular benchmarks can create very large token costs. Benchmarks such as terminal-bench and SWE-bench may cost thousands of dollars for just one run. Reliable results usually need several runs, because a single run may not show the average performance. One estimate says running terminal-bench with Opus 4.6 could cost up to $40,000 for one run. Cheaper benchmarks are needed if teams want useful results without spending close to $200,000.
Vidilearn is an open-source tool for preparing source material for RAG systems and AI agents on a local machine. It can extract YouTube transcripts, subtitles, chapter information, article text, and structured metadata. It is designed to work without API keys and to fit into automation workflows. It also includes MCP server support so AI tools can connect to it more directly. A benchmark snapshot reports a RAG hit rate of 94.2%, precision of 92.1%, and an F1 score of 0.931. Claude is still slightly better on raw accuracy, but Vidilearn is presented as getting close while running at near-zero cost. The tool uses Node.js, Playwright, and local processing pipelines, and can be installed with npm i vidilearn.
Test queries written by development teams often sound precise and technical. Real users write in a much messier way, with typos, slang, unfinished sentences, and several requests in one message. That gap led to about a 94% pass rate on developer-written queries but only about 71% on real user queries, a 23-point drop. Internal dogfooding helped, but employees still tended to phrase things like engineers. User testing panels gave more realistic input, but they were small and expensive. Synthetic informal queries made by a smaller model caught some issues, but they still felt artificial. The main problem is how to build evals that match the real user input distribution.
browser-search is an open-source toolkit that helps AI agents search the web and open pages to check evidence. SearXNG handles search, Camofox handles normal website browsing, and CloakBrowser is used when a site has stronger bot protection. The intended workflow is search first, answer second, so the agent checks live sources before making factual claims. The setup is self-hosted, so it is presented as free to run without paid API keys, subscriptions, or rate limits. Camofox can extract cleaner article text by removing navigation, ads, and sidebars, with an estimated token saving of about 70%, and it can also return much smaller page snapshots than raw HTML. SearXNG is used for fast search results, while Camofox and CloakBrowser are only used for pages that need browsing, which avoids launching heavy browsers for every step. Installation can be as simple as adding the SKILL.md file, or it can include the full SearXNG, Camofox, and CloakBrowser setup. It does not handle logged-in social media, file downloads, or paywall bypassing.
SuperESP is a bundle of small AI applications that run on a low-cost ESP32 board without internet or cloud servers. It aims to make simple decisions directly on the device, without large servers, GPUs, or subscriptions. It is built on Atome LM v2 and is not meant to be a tiny chatbot. It uses specialized small AI models to classify events, patterns, behavior, and unusual signals. The listed uses include agriculture monitoring, voice commands, motion recognition, gesture detection, sound event classification, machine anomaly detection, air quality analysis, energy monitoring, occupancy estimation, wearable activity tracking, water leak detection, and predictive maintenance. It also includes an ESP32 operating system and a universal installer. The materials are said to be available on GitHub and Hugging Face, with scripts tied to the claims so people can test them.
A model can score well on short tests but still struggle during agent work that lasts for hours. It may follow the first rules at the start, then forget them after about 40 minutes and undo earlier work without making the mistake obvious. Benchmark scores often do not warn about this because short, isolated tasks and long, steady work are different skills. Long jobs such as a large code migration can reveal whether one early missed detail causes trouble much later. Many tested models lost track during this kind of work. GLM-5.2 kept its earlier choices more consistently and finished the long task without fighting its own prior decisions. It is not presented as the strongest model for hard one-off questions, where large closed models may still do better.
iart-ai motion-skills is an open-source set of instructions that helps AI coding agents make motion graphics, animation, and video. It includes 50 skills grouped into 14 installable packs for TikTok, Reels, Shorts, text-message videos, YouTube videos, e-commerce videos, ads, animated data charts, explainer videos, map animation, web animation, 3D, WebGL, and Manim math animation. Each skill is a folder an agent can read to learn one workflow, without model training or a special runtime plugin. Users can install only the packs they need, and the project says Claude Code, Cursor, Codex, and many other agents can discover the right skill automatically when the prompt matches the task. Skills that create visual output include a check step, such as rendering a frame, taking a screenshot, or inspecting an encoded MP4, so the agent can review its own result. Web skills create standalone HTML, while video skills use Remotion or Manim to render output. The project is released under the MIT license.
Pulse saves Claude Code work sessions and uses agents to turn those records into useful notes. Each night, an agent reads the day’s raw sessions and creates one clear note covering what was built, what decisions were made, and what is still unresolved. Each week, another agent combines those notes into a profile of the user’s skills and projects. A third agent drafts LinkedIn and X posts from the week’s work. The system runs as cloud routines, so it keeps working even when the user’s own computer is off. The session capture tool and the nightly note agent are open source now, while the weekly profile and social post agents are planned for later.
Moonshot released Kimi K2.7 Code as open source this week. Its scores rose from 50.9 to 62.0 on Kimi Code Bench v2, 48.3 to 53.6 on Program Bench, 26.7 to 35.1 on MLS Bench Lite, and 72.8 to 81.1 on MCP Mark Verified. The model stays in the same 1T MoE family, uses 32B active parameters, and supports a 256k context. The more practical change is a 30% drop in reasoning token use compared with K2.6. That matters for coding agents because they often need to inspect a problem, edit code, run tests, fail, and try again many times. The model still appears behind GPT-5.5 and Opus on coding benchmarks. But in Moonshot’s MCP Mark Verified table, K2.7 scores 81.1 while Opus 4.8 scores 76.4, which suggests it may already be strong for agent-style coding tasks. A coming high-speed mode claims about 5 to 6 times faster output from the same model, which could make it useful for lowering the time and cost of repeated coding work rather than replacing the best frontier model everywhere.
A small AI startup building a real-time coding agent needs very fast inference. Its target is steady throughput of about 1,000 to 2,000 tokens per second, with low delay for most requests. The team wants Cerebras API access for fast ASIC inference in production, not a large group of H100 chips for model training. It has been waiting for months. The concern is that a large OpenAI deal has reserved much of Cerebras’ near-term inference capacity for one major customer. For teams that are not large cloud-scale buyers, the API waitlist now feels effectively unreachable. The frustration is also tied to the view that Cerebras is public but still seems to have little usable compute available for smaller customers.
An AI evaluation rubric has grown over one year to 14 scoring areas. The areas include faithfulness, relevance, helpfulness, tone, scope, refusal precision, safety, harmlessness, completeness, brevity, structure, citation, tool-call correctness, and format. A correlation matrix on a labeled set showed that about 6 areas have a correlation above 0.85 with at least one other area. That means those areas may not add much independent signal. Removing them feels risky because rare edge cases might be missed. Keeping them also costs more money for the AI judge and more engineering time to maintain the rubric.
Sara AI is an AI honeypot built to keep scammers talking while collecting useful clues such as phone numbers, UPI IDs, and scam scripts. It answers in Hinglish and acts like a confused Indian housewife so the scammer stays engaged. The first version could keep the role and sound convincing, but it forgot everything after each session ended. It could not recognize a UPI ID from a previous week or connect the same scammer using a different phone number. That made it closer to a scripted API chatbot than a working honeypot. A stronger system needs memory that works across sessions, live extraction of structured information, and a workflow that shows which step failed when something breaks. Hindsight is presented as the memory tool, while CascadeFlow is presented as the tool for building and tracking request pipelines.
An experimental tool scans a code repo and turns the whole app into an interactive visual map. The map also marks areas where the code looks tangled or likely to break. The goal is to reduce token use and save time by giving AI the map instead of making it re-read the full codebase on every prompt. It is aimed at vibe coding workflows, where AI often creates or edits code and may break fragile parts of an app. The tool is still rough and being tested on real apps. People can send a repo to have it mapped and give feedback.
Putting 50 tools into a local Llama agent can make the model slower and less focused because the context becomes crowded. The MCP ecosystem is mostly built around Python and TypeScript runtimes, so a Go backend often has to run subprocesses or connect through heavier environments when it needs several MCP servers. A lightweight tool gateway and routing engine in Go was built to reduce that burden. It connects to downstream servers over HTTP/SSE, builds a search index, and chooses matching tool candidates in real time. When a request contains several tasks, such as asking for the weather and scheduling lunch, it splits the sentence with regular expressions and number checks, then routes the pieces at the same time with goroutines. During live voice streams, repeated partial request text can trigger too many identical embedding API calls, so singleflight collapses matching requests into one. The package API uses a builder pattern so code autocomplete can guide setup step by step.
codelight is an open-source tool for watching the status and token use of AI coding tools such as Claude Code. A Python daemon runs on the computer, checks Claude Code usage and status, and sends that information to a GeekMagic Ultra desk screen, an Android widget, a GNOME panel, or the VSCode status area. With remote control turned on, permission requests can be allowed or denied from Android or GNOME, and questions can be answered from Android, GNOME, or VSCode. If several devices can answer, the first answer received is used. If no suitable device is connected, Claude Code’s normal prompt appears right away, so the work is not left waiting on a missing device. Setup starts by flashing the screen firmware, running `python3 companion/codelight.py --name my-laptop`, and adding `--secret` plus `--remote-control` when remote approval and answers are needed.
An ICLR 2026 workshop is focused on recursive self-improvement, and the area is being considered as a possible PhD research topic. Recursive self-improvement means AI systems that can find their own failures, critique their behavior, update memory, change tools or skills, and improve over time. The workshop treats this as a practical systems problem, not just a distant idea. Its focus includes what the system changes, when the change happens, how the improvement is produced, and where the system is used, such as web tasks, office work, robotics, science, and enterprise settings. Safety is also part of the agenda. Important risks include long-task instability, new changes breaking old abilities, and the need to roll back bad updates.
Running a large language model on your own computer can reduce dependence on outside services and may lower cost when usage grows. It can also give more speed and control. But local use does not automatically make the setup safe. Data can still leak through logs, plug-ins, model downloads, badly set permissions, or text pasted carelessly into a prompt. Before installing anything, teams should decide what they are protecting. That includes the kind of data the model will touch, such as customer personal data, internal strategy documents, source code, or medical records. They should also define the worst possible harm from a leak, such as legal trouble, business loss, or damage to trust. The safest default is that nothing leaves the machine unless it has been clearly approved.
Switching to Gemma 4 12B Q8 cut generation speed from about 70 tokens per second to about 10 tokens per second. This happened even though the earlier model, GPT-OSS 20B Q4, was larger. Trying a Q5 version of Gemma did not meaningfully improve speed compared with Q8. Turning off thinking mode only added about 2 more tokens per second. The setup runs llama.cpp llama-server with 16 threads, an 8192 context size, 99 GPU layers, and a 4096 batch size. The GPU uses about 10GB of its available 20GB memory, so the slowdown does not look like a simple out-of-memory problem.
Claude can be asked to review both its own answer and the user’s prompt after a task is done. It then turns that review into clear improvement tasks, better ways to ask next time, and useful wording to reuse. Those lessons can be saved and reviewed at the start of later sessions, so the same improvements carry over instead of being lost. The goal is to improve Claude’s output quality, help the user write better prompts, and reduce repeated explanation or trial and error. The workflow connects to quality control, token saving, context and memory use, and skill building, and is aimed at intermediate users.
A self-hosted company wiki and retrieval augmented generation system is being designed with PostgreSQL as the source of truth and Qdrant as a rebuildable search store. When a document is uploaded, it goes through a Redis queue, gets split into smaller chunks, is saved in PostgreSQL with audit records and an outbox job, then a worker creates embeddings and adds the chunks to Qdrant. Each chunk in Qdrant carries the organization units allowed to read it, its sensitivity label, and its lifecycle state. When someone asks a question, the system calculates that person’s allowed organization scope, searches Qdrant with that filter, then checks the top results again in PostgreSQL for lifecycle state, sensitivity clearance, and tenant before sending anything to the LLM. Other designs were rejected because they could create too many collections, add too much delay, or lose useful permission matching. The main weak point is permission changes: revoking access or moving an organization subtree may require rewriting many affected chunks, and 10,000 or more documents could create a sudden processing spike.
An Android app is being built to run several local AI engines in one place, including MNN Chat, Google AI Studio, and llama.cpp. MNN model support is planned but not yet tested, while GGUF models run through llama.cpp. On a Samsung Galaxy S23 FE, Gemma 3 1B reached about 20 tokens per second, and a TFLite Gemma 4 e2b model reached about 10 tokens per second on the same phone. The app is available on GitHub as open source. It includes a testing-stage search feature that has not been released yet, plus thinking mode, voice input and output, and an in-app server. The design also aims to support a web interface, RAG, and OCR. An unpublished Invent screen links three models together: a planner model asks questions about a project idea, a researcher model checks recent information, dependencies, and whether the project is realistic, then the planner reviews the results before sending them to a coder model.
RAGless is a system for finding answers in FAQ-style content. It uses a large language model only when documents are first added, not when someone asks a question. PDF, text, and Markdown files are turned into prepared question-and-answer pairs by Gemini, with several question versions for the same answer. Each question version is converted into an embedding and stored in a local Qdrant database. When a user asks something, the question is also converted into an embedding, the closest stored questions are searched, and scores are combined by answer ID. The system then returns the prepared answer instead of generating a new one. This removes the runtime generation step, so there is no runtime prompt work and less risk of made-up answers by design. It can also run fully offline with Ollama by changing EMBEDDING_MODEL in the config file.
Email can be used as an asynchronous communication layer when several AI agents belong to different services or teams. Shared memory and message queues are common choices, but they often assume the agents run in the same environment or trust the same infrastructure. Once ownership, deployment setup, and service level agreements differ, shared state can become hard to manage. Email already connects each conversation with message identifiers and reply chains, so it is easier to see which response belongs to which request. It also works when the sender and receiver are not online at the same time, which suits longer handoffs across service boundaries. When something fails, an email thread gives people a readable record of the agent workflow.
An AI agent is being built with Qwen3.5-4B, a small language model, to directly control IoT devices such as lights, fans, air conditioners, and curtains. A larger model may handle this better, but it can make responses slower. The current system supports multiple tool calls, several actions in one request, several user intents in one prompt, device control, normal conversation, structured JSON outputs, and backend checks before any action runs. It needs to handle requests such as turning on bedroom lights at 70% brightness, closing curtains while turning off the AC and giving tomorrow’s weather, or turning off every light except the kitchen. The main problem is hallucination: the model sometimes picks the wrong tool, fills in wrong parameters, tries to control a device that does not exist, or gets confused when device commands and general questions are mixed. The possible designs being considered are sending every request straight to one large language model with all tools available, adding a routing layer before the main LLM, or splitting the system into smaller specialized parts.
peerd is an open-source AI agent tool that runs as a Chrome or Firefox extension. It can use the tabs and signed-in sessions already open in the browser, then read pages, click through them, and carry out tasks. It can also create isolated workspaces such as JavaScript notebooks, browser-based Linux virtual machines, and small client-side apps. It does not route work through a remote backend, has no telemetry, and uses API keys supplied by the user for the model provider they choose. It supports Anthropic, OpenRouter, and local Ollama; Ollama can run without an API key or cloud bill because the model runs on the user’s own machine. API keys and chat history are kept in an encrypted local vault, and keys are only sent to the selected model provider. The project is still an experimental 0.x beta, so features and storage formats may change, and it should be used carefully because it can control the browser and store API keys.
An AI feature that first searches company documents and then answers from those documents can still give dangerously wrong information. In this case, the search results were not empty, but they were weak matches. The model used those small pieces of context, filled in missing parts on its own, and gave an answer that sounded as confident as correct answers. The customer had no clear reason to doubt it. Nothing looked broken because there were no crashes or error messages. The core mistake was assuming that retrieval automatically creates grounding. Retrieval only gives the model context; it does not guarantee that the model will stay inside that context.