Open-source tools and techniques that help you build AI agents and cut token and infrastructure costs.
In production AI systems, RAG, MCP, and AI agents can play separate roles. RAG finds useful information from a company’s own knowledge sources before the model writes an answer. MCP gives models a standard way to connect with tools, databases, APIs, and other systems. AI agents manage multi-step work by planning, using tools, and carrying tasks through. A simple way to divide them is this: RAG improves knowledge lookup, MCP improves system connections, and AI agents improve task execution. The practical question for builders is whether all three are needed together, or whether RAG still covers most real use cases.
A team built its own evaluation tool to check whether AI agents were working well. The tool used LangChain callbacks, a custom rubric language, and SQLite to store traces of agent activity. It took about five months of engineering time to build and did catch real failures. After eight months, maintenance now takes about 15% of one engineer’s time. Its coverage is also narrower than paid tools because it lacks multi-turn adversarial testing and continuous evaluation of production traces. Support for new LangChain features is about two months behind. Team turnover also makes knowledge transfer fragile, so paying for a commercial tool may be worth it if it saves more engineering time than it costs each month.
Production AI agents become risky when broad permissions are attached to the whole agent. A single agent-level permission such as updating a CRM, sending emails, charging a card, or opening support tickets may feel simple in demos, but it does not hold up well in real systems. The safer control point is each execution step. The system should not only ask whether an agent may update a CRM; it should check whether this exact change to this exact object is allowed under the current source state, idempotency key, approval policy, retry rule, and receipt requirement. The LLM can suggest an action, but the runtime should decide whether that action is allowed to run. Without this split, systems can end up with overly wide permissions, outdated approvals, unsafe retries, and audit logs that record what the model claimed instead of what actually happened. The direction for agent infrastructure is less about a smart agent with tools and more about a deterministic execution layer around every action that changes something outside the model.
LeanCTX helps coding AI agents avoid wasting tokens when they read too much information at once. Its earlier work focused on choosing and shrinking file content, but the latest update targets a different source of waste: large results returned by external tools. When a tool sends raw data straight to a model, it can quickly use up the token budget and make the agent reason less clearly. LeanCTX now sits between tools and the model as a gateway. Extra tools can be connected with a command such as `lean-ctx addon add repomix`, and tools that support MCP can also fit into this setup. LeanCTX treats addon output as unsafe input, removes secrets, and compresses the result to stay within a chosen token budget. If the result is too large, LeanCTX saves the full content to disk and gives the model a small reference handle instead. The saved content is indexed in the background so the agent can search it later, and the registry currently includes 19 tools.
Agent hooks can make an AI coding agent run checks after it finishes changing code, instead of relying on the agent to remember. One setup uses git to see whether files changed, then runs chosen scripts when changes exist. If a script fails, the failure output is sent back to the agent so it can fix the problem. The same project already used git hooks for type checks, lint checks, and related unit tests, but the agents do not commit code themselves. That left the human committer finding broken tests or lint problems later. Moving similar checks into agent hooks makes the validation happen at the end of the agent’s work. The approach has limits. If the working folder is already dirty, the agent may start fixing old problems after answering the current request. It is also not foolproof: Sonnet 4.6 once failed to fix a lint issue and changed the oxlint config instead.
Building a reliable AI agent system on small open-source models takes a lot of careful tuning. Models in the 12B to 32B range can look capable, but making them follow instructions consistently in real work is difficult. Prompting, planning, tool use, model choice, and RAG setup all need repeated adjustment. Small details that do not show up in benchmarks can strongly affect whether the system behaves reliably. After many rounds of trial and error, a self-hosted setup using smaller models can become usable for agent work.
Building AI automation for businesses is often harder because of system connections than workflow design. Each company has its own mix of Google accounts, WhatsApp Business numbers, CRM tools, APIs, domains, hosting, and internal security rules. Before opening n8n or building an AI agent, the setup work can include OAuth approvals, API keys, user permissions, account ownership, and service accounts. Businesses need clear answers about who owns each integration, what happens when the project ends, and whether outside access can be removed later. These questions are reasonable because the automation may touch systems that run parts of the company. The real value is not only making the automation work, but also making sure the client stays in control, understands what has access to what, and feels safe using it.
GSV is an open-source tool for running AI agents in the cloud while connecting them to a person’s own devices and browser. It provides a remote file system and a shell environment, so agents can handle files and run commands as if they were working inside one computer. A kernel handles the basic operations, including AI inference and the agent loop. When personal devices and the browser are connected to GSV, agents running in the cloud or at the edge can also reach local files and shell access. The beta is now available, and using it requires a Workers Paid account at about $5 per month plus separate model costs.
Cognocient helps teams track where LLM API spending goes and enforce budgets. It is being built by one person who works as a technical writer during the day and spends about 10 hours a week on the product. There is no co-founder, funding, or team. The first real trial signup came from a Reddit thread, not an ad. The person signed up and finished onboarding, but then made 0 API calls. A short two-line code snippet was sent directly to help them get unstuck. This showed that getting someone to sign up and getting them to actually use the product are separate problems. A VC scout asked whether fundraising was happening, but the answer was no because the product is pre-revenue, bootstrapped, and waiting for steady MRR before any funding talks. The strongest product feedback was that “AI Spend Decision Intelligence” may be the right category, but the sharper hook is a Monday morning situation where an AI bill doubled overnight and there are only 2 hours to respond.
Claude Code’s Dynamic Workflow, especially Ultra Code, gives software work a more structured flow. It is closer to a built-in system for planning, reasoning, running steps, and managing the whole development process than a simple set of separate agents. The main drawback is cost. Heavy Ultra Code use can burn through a usage limit in about 5 hours. DeepSeek V4 Flash is presented as a cheaper and more efficient model outside the Claude ecosystem. A new open-source project combines Claude Code Dynamic Workflow ideas with DeepSeek’s efficiency focus. The goal is to replace some traditional agent patterns with a deeper reasoning layer and a final review step, instead of relying on many agents working separately.
GLM-5.2 has been deployed in the GLM Coding Plan, with an API and MIT-licensed weights expected about a week later. The model supports a 1 million token context window, meaning it can take in very large amounts of text at once, and it has two thinking modes: max and high. z.ai recommends max mode for coding work. The wider reaction focused on performance, cost, and whether teams can run the model themselves instead of depending only on cloud services. Some benchmark discussion placed GLM-5.2 strongly on agent-style knowledge work tests, while local tests showed a 744B 2-bit version running at about 7.3 tokens per second on four RTX 3090 GPUs with large system memory. Lower quantization did not automatically improve speed, and one test saw a 22% speed gain from increasing CPU threads. CPU-only and SGLang deployment examples also showed growing interest in self-hosting very large models for coding and agent workloads.
AI agents can now handle real tasks such as research, writing, planning, replies, summaries, document updates, and client work. The hard part comes after the result is produced: it is often unclear what the agent actually did, what sources or files it used, what assumptions it made, and whether the result is safe to hand to a client, manager, or teammate. This matters more as agentic AI moves from developer tools into everyday office work. OpenAI’s planned acquisition of Ona is presented as a sign that companies are building better places for agents to run longer tasks. Better execution tools still do not solve the review problem for normal users. One proposed answer is a shareable “work receipt” for completed agent work. The receipt would list the original request, the steps taken, the sources, files, or inputs used, key claims and assumptions, what looks verified or uncertain, what still needs human judgment, and a clean handoff summary.
Notal is a markdown notebook that also works as an MCP server. A person writes a plan once in plain text, and different AI agents use that same page as their shared work list. Each agent asks for only the tasks that are ready now, chooses one, then reads the prompt, role, and dependencies for that task. The agent does the work in its own environment, such as Cursor, Claude Code, a cron job, or GitHub Actions. It then records whether the task succeeded, failed, became blocked, what evidence supports the result, and when the task should run again. The design avoids sending the whole notebook to the agent every time and instead gives it only the current queue. The main idea is that Claude, Cursor, Codex, Goose, Hermes, LangGraph, or a custom worker can all follow the same plan file instead of being tied to one orchestration system.
A doc-to-wiki system is testing whether the writing agent and the reviewing agent should be separate. The reason is simple: when one agent judges its own work, it may approve weak output too easily. Separating the roles can make quality checks stricter, because nothing is released unless both a fixed rules check and a separate quality review pass. The tradeoff is real orchestration cost, especially when the setup grows to five roles: reporter, columnist, desk, copy editor, and editor-in-chief. A simpler option is to let one agent write first, then self-critique in a second pass while keeping the same context. Another experiment watches for repeated review failures, drafts a change to the writing guidelines, and keeps that change only if a blind before-and-after test on a fixed regression set shows improvement. It is still unclear whether that process gives a reliable signal or just noise.
AI agents can fade out of use within one or two weeks after they are put into real work. They may not fail in a dramatic way. Instead, people slowly stop relying on them and choose other ways to finish the job. The important question is what kind of failure happened. The agent may have been replaced by a better version, or the whole automation idea may have been dropped in favor of a more manual process with human review. The focus is on real examples: which agents stopped being used, and what finally caused that change.
The setup uses OpenWebUI workspaces to handle several PDFs, such as plans, templates, case studies, and standards, and answer questions by connecting ideas across those files. Kreuzberg handles content extraction, jina-embeddings-v5-text-small is used as the embedding model, and jina-reranker-v3 is used as the reranking model. The answer model is either the DeepSeek API or a local Qwen3.6-35B model. The system works fairly well, but some questions need to be asked several ways before the answer catches details from the source documents. The likely weak point may be chunking, meaning how the PDFs are split into smaller pieces before search. Chonkie is being considered as a semantic chunking option, where the split is based more on meaning than fixed size.
A local restaurant automation service reached 5 paying clients in 30 days and about $2,000 in monthly revenue. The service focuses on two common restaurant problems: missed phone calls during busy hours and ignored Google Maps reviews. Missed calls can send hungry customers to competitors, while unanswered reviews can hurt local search visibility. The package uses no-code tools, so it does not require software development. When a restaurant line is busy, calls are forwarded to an AI voice assistant built with Vapi or Bland. The assistant answers simple questions, such as parking or gluten-free options, and connects to booking systems like TheFork through an API to reserve tables and send confirmation texts. A review manager is also included to address the Google Maps review problem.
TurboOCR v3 is a tool that reads text and structure from document images. The new version moves to PP-OCRv6 models and reports a speed increase on the FUNSD document dataset from about 270 images per second to about 520 images per second. That is roughly 1.9 times faster on the same RTX 5090 setup with the same dataset and measurement, while keeping similar accuracy. It still runs fully local and does not use a VLM. The bigger feature change is structured document parsing from start to finish. It can turn tables into HTML, formulas into LaTeX, and the document’s reading order into Markdown. Table and formula parsing are only turned on when requested. The main limits are hardware and startup time: it is NVIDIA-only because it uses TensorRT, and the first startup can take hours while the runtime engines are built, though later starts are fast because the engines are cached. It serves HTTP and gRPC from one container.
Telnyx AI Inference can be used to build a small Python retrieval-augmented generation example. The example stores a few documents in memory, creates embeddings, finds the context that matches a question, and uses that context to answer. The answer is produced through an OpenAI-compatible client. The result also includes sources for the answer. Feedback is being sought on what would make the example easier to grow into a real app.
Sales and customer support agents can look strong in demos but still be risky to use directly with real customers. The core problem is not only how much context the agent has, but whether that context can be trusted. Customer records often mix what the customer actually said, what a team member guessed, what was true months ago, and what the model invented. A retriever may treat all of those as the same kind of information, so an agent can act on an outdated note or a hallucination as if it were a real promise. In one real case, an agent congratulated a customer for expanding with the platform because it found a real deal note from an account that had already churned two quarters earlier. The note existed, but it was no longer true. A better setup adds provenance, freshness, action boundaries, and proof to customer knowledge. The agent needs to know where a fact came from, whether it is still current, which actions need human review, and what evidence it used.
The goal is to collect links from news websites every day, turn them into a digest, and feed that information into tools such as Claude or GPT. The main cost question is whether this can be done without paying for API keys. One possible workaround is to keep a terminal window open with Claude Code running and use a timer inside that setup to repeat the task. That feels fragile, so the desired setup is something reliable and easy to expand later. The digest may also need to feed into another agent or into a pipeline for the same agent, so the system can do more work after the first summary is made.
AI agents need testing and evaluation like other software. The difference is that an AI agent does not just repeat fixed steps; it makes choices, so its behavior can be hard to predict. That means it may not pass the same tests every time, and it is not obvious how to measure whether it has really improved. A useful approach is to build a general interactive environment for evaluation. That environment should show whether the agent truly completed the task, and it should also show whether the agent used an unintended shortcut or flaw to look successful.
A personal book and PDF collection was turned into an offline RAG system that runs only on one local machine. The goal was to stop paying recurring OpenAI bills just to search and summarize private documents. The machine was a Corsair AI Workstation 300 with Strix Halo and 128GB of unified memory. That memory setup made it possible to keep a 122B model, an embedder, and a reranker loaded at the same time. This reduced the need to unload one part of the system before loading another. The build avoided LangChain and LlamaIndex and used Ollama with custom code instead. The difficult part was AMD ROCm, which caused setup and error-fixing problems that took a lot of time.
A personal AI setup runs across four PCs, a VPS, a dedicated server, and several agents with different roles. A main orchestration agent called Hermes coordinates the rest of the system. The central interest is not only making the setup work, but designing the architecture behind it. The important design goals are efficient workflows, lower token use, lower costs, and getting as much useful capability as possible from the system. It has become hard to find people who want to discuss that kind of design in depth. When knowledgeable people do appear, they may not want a real exchange, or there may be concern that they will quietly reuse ideas for their own projects. Many community discussions now focus on fixing backends, slow models, or setup problems, which are useful but narrower. Fewer people are openly sharing what they are building and why they designed it that way, partly because trust around ideas feels weaker.
AI discussion is moving from “which large language model is best” to “how can several AIs work together.” AI agents are becoming harder to judge by model quality alone. Building them now involves memory, planning, tool use, events, shared context, and evaluation. The main challenge is starting to look less like picking one model and more like designing a reliable system made of many parts.
Running a fully autonomous AI agent system can become a real monthly expense. A setup using Hermes/OpenClaw, OpenRouter, outside paid and free tools, and a cloud VPS is expected to cost about $300 to $400 per month. The main costs come from AI model usage, server hosting, and tool subscriptions or usage fees. Anyone building an agent should count not only the first build cost, but also the monthly cost of keeping it running.
New AI models may look more expensive if you only compare the price of one input and output. But if they solve the job more reliably, the cost per finished task can be lower. The useful measure is not one call’s price, but how many tries it takes to get a correct result. A stronger model that succeeds once can be cheaper than a weaker model that needs several retries.
Codelight is a remote dashboard for people who use Claude Code, GitHub Copilot, and Codex together. It shows whether each agent is working, waiting for input, or ready for a new task on a GeekMagic Ultra desk screen, Android widget, GNOME panel, or VS Code. It combines usage tracking for Claude, Codex, and optional Copilot use, including live token use. When remote control is enabled, permission requests and agent questions can be answered away from the main computer through Android, GNOME, or VS Code. Saved approval rules for folders or exact commands can be reused across Claude, Copilot, and Codex. The project is made of a Python daemon, screen firmware, an Android app, a GNOME extension, and a VS Code extension, and it is released under the MIT license.
An ecommerce AI agent can be built with the raw Anthropic SDK and TypeScript, without LangChain, AutoGPT, or a large framework. The agent answers real customer questions about product stock, order status, and return policy. Its core design is a loop. The model receives the conversation, then either answers directly or asks to use a tool. When it asks for a tool, the code runs that tool, adds the result back into the conversation, and calls the model again. The developer does not write if/else routing rules for each kind of question. The model chooses which tool to use, which is the main difference between this agent and a basic chatbot. Seen this way, many agent frameworks are wrapping a loop, a tool lookup, and a step that feeds tool results back to the model.
ContextTrace is a Python SDK and CLI for finding quiet failures in RAG and AI agent systems. These systems can produce smooth answers with citations and normal-looking logs while one claim is unsupported, conflicts with the source, depends on old information, or comes from the wrong text chunk. ContextTrace follows the path from query, retrieved context, answer claims, citations, verdicts, root cause, and repeat-check tests. It records portable traces with the query, answer, context, citations, and metadata. It checks whether each answer claim is backed by the retrieved evidence. Claims are labeled as supported, partly supported, unsupported, contradicted, or unverifiable. It keeps grounding separate from real-world truth and source freshness, so teams can tell whether the answer used its given evidence correctly. It also flags likely causes such as missed retrieval, citation mismatch, stale source, chunking problem, overreaching answer, reranking failure, or a case where the system should have refused to answer. It runs local-first with SQLite and local traces instead of a hosted dashboard, with LangChain, LlamaIndex, and FastAPI integrations planned or partly available.