Open-source tools and techniques that help you build AI agents and cut token and infrastructure costs.
An AI agent may return several resources from a system after completing a task. Each resource can have an internal ID, but that ID may not be allowed in the visible answer shown to the user. If the user later says “show me more about #3,” the agent still needs to know which real resource that number refers to. The main issue is how to keep the visible list number, the hidden internal ID, and the real resource connected across several turns of conversation. The example flow is: the user asks for their top X items, receives a list, then asks for more detail about item 3.
A chatbot needs to use backend data that is refreshed through APIs every 5 minutes. The dataset is large, so sending all of it to the LLM on every request would use too many tokens and raise cost. A standard RAG setup may also be awkward when the knowledge changes this often, because results can become stale or need frequent rebuilding. A better architecture would need to fetch only the small slice of data needed for each question, possibly using database search, vector search, caching, tool calling, and query planning together. The core challenge is making frequently changing enterprise data usable without overloading the model input.
A recurring issue over several months is that a large language model may say it will use a tool in one way, then call the tool in a different way. The final effect may be similar, but the stated method and the actual action do not match. In one example, GLM said it would use a transform approach to edit a buffer, which meant writing a small Lisp-like expression for the edit. Instead, it wrote a diff and applied that change. The same kind of mismatch has also been seen with Claude Sonnet and Claude Opus, so it does not appear to be limited to GLM.
The focus is on automations, AI agents, and repeating work flows that keep running in real work or personal life. The examples should exist in a codebase or on GitHub, not just as short-lived demos. The useful areas can include machine learning, large language models, scripts, deployment automation, developer tools, home automation, and knowledge management. The main questions are what was built, what problem it solves, and whether it stayed useful over time.
A RAG setup uses Ollama and Qwen2.5 to answer questions from a risk register SQL database. The goal is to return useful details about risks, incidents, and mitigations through both simple and complex SQL queries. The database is very sparse, with many empty tables and empty columns, so the agent often receives results with little useful context. That leads to weak answers. Semantic search was added by splitting every table into row-level chunks and creating embeddings, but more advanced methods such as hybrid search and RRF are not in place yet. The practical need is to ignore missing and null values and retrieve only the database content that can actually support a good answer.
A RAG pipeline is being built with Qwen2.5 running through Ollama. The goal is for the model to work with a risk register SQL database and answer questions about risks, incidents, and mitigations using simple or complex SQL queries. The main problem is that the database is very sparse. Some tables are empty, and many columns have missing values, so query results often lack useful context. Because of that, the agent receives weak results and gives inefficient answers. Semantic search was added by chunking each table row by row and creating embeddings, but more advanced search methods such as hybrid search or RRF are not in place yet. The desired outcome is for the agent to ignore missing or null values and interact with the database more efficiently.
An interface is available for managing two small AI models that can run fully locally, even on a smartphone. The 4B model works well, but it needs a high-end phone. The 1.7B model is not yet stable when reasoning is turned on. The planned fix is deeper fine-tuning with about 130,000 examples. The hard part is not just improving the training score, but getting enough high-quality and varied examples. A larger 32B model is being used as a teacher, with distillation used to transfer its behavior into the smaller models. Once the dataset is ready in about 10 days, the 1.7B model may improve without relying on the current LoRA setup.
Conduit is a local gateway for managing many MCP servers in one place. AI clients such as Claude, Cursor, VS Code, and Codex can share the same setup, so each client does not need its own separate tool configuration. Its main idea is lazy discovery: the agent does not receive every available tool upfront and instead searches for tools when needed. This turns a large tool list into 3 meta-tools, which reduces the amount of tool information sent to the agent. The project says this can cut token use by about 90%. Secret keys stay in the operating system keychain rather than being stored in the cloud.
Baidu released an open-source model called Unlimited-OCR. It is meant to read long PDFs or multi-page documents without cutting them into separate pages and stitching the text back together later. The main idea is to keep the original document image available as a reference while making the model remember only a small moving part of the text it has already produced. This can reduce the growth of the KV cache, which otherwise can fill VRAM during long document runs. Lower memory use may make it possible to run larger models on the same machine or process long documents more reliably. The release is mainly about OCR and document parsing, not a complete answer for analyzing or summarizing very large documents. The GitHub repo includes examples for Hugging Face, an SGLang server, PDF-to-image conversion, and batch processing with infer.py. The tested setup uses an NVIDIA GPU and a CUDA-based Python environment.
FlowScript is a small prototype for writing agent skills in Markdown while making a separate harness enforce the execution order. Markdown skill packages are easy for people to write, review, and give to agents, but the actual execution can be loose. A model may skip a step, run a helper script too early, summarize before required artifacts exist, or handle failures in uneven ways. Workflow engines can control execution, but they often move the writing experience away from Markdown skill files. FlowScript keeps the human-readable SKILL.md file and adds a FLOWSCRIPT.md file with a flow block that the harness can read. The harness loads and checks the declared flow, then runs language model, validator, and Python script steps in order. It only follows declared branches, saves artifacts instead of depending on hidden chat state, and records a skill_agent_context.json file for replay and inspection. If it reaches an unsupported ending or fallback path, it stops with logs and partial artifacts.
mcp-audit runs MCP servers inside a sandbox and records which files they open and which internet connections they make while running. The official MCP registry checks who published a server, but it does not check what the server actually does after someone runs it. Static analysis can miss problems that only appear during use, such as a later version changing behavior, a server contacting an unrelated host, or a server reading files outside its expected area. The tool starts each server inside a throwaway Linux container with Docker and uses strace to record file access and network connections. It also places a fake secret called a canary in the environment, so any attempt to read or send that secret can be seen. In a check of 70 MCP servers, 67 showed no sensitive file reads and no unexpected network activity at startup. Three servers opened outbound HTTPS connections at startup, but the connections matched their jobs, such as exchange access, GitHub fetching, or cloud and CDN access, and none touched sensitive files or the canary. bullmq-mcp read /etc/passwd at startup through a normal glibc user lookup, and the file contents did not leave the process. The current check only covers startup and idle behavior, not what happens when an AI agent makes tool calls.
A RAG system for annual reports, news, and scientific papers may need more than simple text chunking. Scientific papers often mix diagrams, charts, text, and numbers, so the system must keep the relationships between those parts. The goal is to extract as much detail as possible from each document. If someone asks about a diagram, the system should use both the diagram and its nearby explanation to give a useful answer. The main challenge is choosing a better practice for handling complex documents during the chunking stage.
The same workflow was built three ways: with LangGraph, with CrewAI, and directly on Google’s A2A approach. The workflow searched for information, summarized it, and sent a notification. LangGraph gave the most control over saved state and retry rules, but it was the hardest for new team members to learn. CrewAI produced a working prototype the fastest, but it became awkward when the workflow needed unusual control steps. Using A2A directly took the most work at the start, but it made failures easier to inspect because the team could see more clearly what was actually sent between systems. Multi-agent failures were often not clean error messages; one AI agent could silently receive the wrong meaning from another. None of the three choices solved observability across agent handoffs by default. Each tool logged its own activity, but none gave one trace from the user’s request through each agent step to the final answer.
Many failures blamed on AI agents' reasoning aren't reasoning problems at all — they happen because the agent executes an action while key information is still missing. Rather than proposing a new model or framework, this describes a lightweight execution pattern built on four steps: separate state (data) from execution logic; never let the AI guess at missing information — mark it explicitly as "Unknown" instead; block execution if even one Unknown remains; and let the final state itself serve as the execution record, functioning as an audit log. Under this pattern, the AI's job shifts from inferring gaps to matching against already-confirmed information. If something is unknown, the user fills it in — not the model. No new framework, infrastructure, or language is needed; a plain JSON structure is enough. An example shows a login-bug-fix task represented as JSON tracking whether the modification scope is defined, whether test criteria exist, and user constraints like "do not change UI."
AI agents that can run Terraform, kubectl, or other cloud infrastructure commands can create a serious cost risk. The agent may not know how much an action will cost before it runs, or whether it will push spending over budget. A paid Claude Pro/Max plan does not solve that problem, because model access and cloud spending are separate costs. A public MCP tool checks the estimated cost, the budget, and the user’s policy before the agent acts. It returns one of three results: allow, ask for human approval, or block. It can also suggest infrastructure changes as pull requests instead of directly changing live systems. The main concern is that many agents with real infrastructure access still appear to have little or no cost control before they act.
Qwen3.6 35B-A3B ran on a 2019 gaming laptop at about 28 tokens per second. The machine used an i7-9750H processor, a GTX 1660 Ti with 6GB of video memory, and 32GB of system memory. The model has 35 billion total parameters, but its MoE design uses only about 3 billion active parameters for each token, so each step is much lighter than the full model size suggests. The `--n-cpu-moe 36` setting kept attention and shared tensors on the GPU while moving expert tensors into system memory. Turboquant’s `turbo4` and `turbo3` KV cache quantization made the long 128K context fit into only 6GB of video memory. The model was exposed on `localhost:8080` as an OpenAI-compatible endpoint and connected to opencode. That means local agent-style coding work could run without cloud use or API bills. The Turboquant fork changes quickly, so the specific commit `4595fff` was recommended for reproducibility.
Prometheus is a personal AI assistant designed to run locally first. A small local model manages the conversation and sends tasks to specialized sub-agents. Separate agents handle browser, shell, internet, email, calendar, scheduling, and project-tracking work, while a group of models can discuss bigger decisions together. Cheap and fast tasks stay with local models, and cloud models are used only when heavier work is needed, all inside one session. It can control a real Chrome browser through Playwright/CDP while keeping existing cookies and logged-in sessions. It can also control the desktop with xdotool, read and send email, manage calendars through CalDAV, edit and restart its own source code, run through Telegram, WhatsApp, Slack, or a web dashboard, and support voice mode with Whisper and Kokoro TTS. The safety risk is serious because it runs with the user’s normal permissions, has no sandbox, and could cause damage if a model gives bad instructions or a webpage triggers prompt injection.
AI agents become hard to manage when they are deployed like simple backend scripts. A common setup is to build the logic in LangGraph, CrewAI, or raw Python, package it in Docker, and run it on a cloud provider. That can work for a few agents, but it becomes messy once a team has more than five or ten. The result can be many stateful agents with high permissions and no standard way to handle secrets, rollbacks, or evaluations. Putting compliance rules, PII masking, and deployment steps inside a specific framework such as LangChain creates technical debt. The better design is to separate the agent’s main logic from the infrastructure layer. Some platforms are moving toward an independent control plane that manages agents from outside the code framework.
An AI agent often succeeds or fails before it handles its first prompt. In real products, the key issue is not only how smart the model is, but whether it can reach accurate, current business data and tools. It needs reliable API connections, live database access, correct state management, fallback steps when something goes wrong, and a clear fit with the real work people are trying to finish. A powerful reasoning model can still create costly results if its data access is poor or its tools break easily. Building useful agents may depend more on strong tool calling, clean data pipelines, and careful evaluation than on changing the model itself.
Based on firsthand testing, OmniDimension supports AI phone calls in more than 100 languages, keeps conversation delay low, and can be set up without code. The voice sounds fairly natural and the replies feel fast. The main question is whether this level of phone-call quality comes from simply connecting speech recognition, a large language model, and text-to-speech, or whether more optimization is happening behind the scenes. The practical issues are what technology stack would be needed, how low delay is maintained, and what the hardest engineering challenge would be. Another key question is whether one developer could build a minimum viable product, or whether this kind of platform needs a full engineering team.
Over the course of a month, a developer built a system that goes from raw idea to a finished, captioned vertical video with almost no manual work, wiring the whole flow together in n8n (Zapier works too). The first stage is ingestion: a scheduled job scrapes roughly 12 RSS sources (Twitter, Reddit, Hacker News, and several AI blogs) every few hours and stores the posts as markdown in S3, building up around 100 raw stories by the end of each day. The second stage is curation: once a day, a prompt reads that day's stories and picks the top 3 to 5 based on how likely they are to resonate (breakthrough, practical value, drama, or wow-factor), removes duplicates covering the same event, and outputs structured JSON. One hard-won lesson was forcing the model to copy source URLs exactly rather than guessing, since guessed links break. The third stage is scripting: each story is processed one at a time, with a prompt generating 5 hooks per story, keeping the best 2, then drafting two roughly 55-second talking-head scripts.
MCPRelay is a free open-source tool that runs on your own computer. After it is exposed to an MCP client, regular ChatGPT can act more like a control panel for the machine instead of only giving commands to copy and paste. It was created because Codex usage limits kept getting in the way, while ChatGPT in the browser or mobile app was still available. The setup runs locally and connects ChatGPT to the computer through an MCP path. It is rough and experimental, but it gives builders a concrete example of a local MCP setup for agent-like computer control.
Atelier is a tool for reducing tokens and cost when using large language models. Its main idea is to measure the full cost of finishing real tasks, not just the savings from one model call or one command output. Some code indexing tools may cut tokens in one step but still need repeated calls before they reach the right answer. Public tests can show that their recall is close to what grep already provides. Some tools report 70-80% savings for narrow command-output cases, but that does not prove the whole workflow becomes cheaper. Atelier aims to show where savings come from, what tradeoffs appear, and whether the final answer remains correct. It is available to try for free on GitHub.
Laurel.dev is an open-source web search tool for AI agents. Existing AI search services are compared at about $5 to $25 per 1,000 searches. The tool was built for internal AI products and reportedly handled more than 100,000 web searches for under $3. Its search results are described as comparable to services such as Exa and Tavily. After several months of private use, it is now offered for free with no credits, no per-search pricing, and no stated catch.
AI agent collaboration means using several role-based AI agents instead of asking one system to do everything. A single model can handle one-off tasks such as writing an email or summarizing a report, but real business work often involves tickets, approvals, monitoring, incident response, procurement, forecasting, and many handoffs. An AI agent can take a goal and context, plan steps, use tools such as APIs, databases, browsers, and code execution, observe the results, and adjust its plan. In multi-agent collaboration, agents with different roles communicate so they can solve a larger task more reliably than one agent could handle alone. The basic idea is similar to a well-run group project, with separate roles for coordination, research, implementation, and checking.
OpenAI has restructured its desktop ChatGPT app. The old desktop app has simply been renamed 'ChatGPT Classic' — it is not a new model. The new desktop app, called just 'ChatGPT,' merges three previously separate pieces: Chat, ChatGPT Work, and Codex. Classic keeps getting model updates and security fixes and can still be used, but some new agent features are exclusive to the new app. The new app splits by role: Chat handles normal conversation, search, and question-answering; Work handles long-running agentic tasks like business documents, research, apps, and file workflows; Codex handles software development against code repositories, files, terminals, and Git. ChatGPT Work runs on GPT-5.6, and Codex is also getting GPT-5.6-powered upgrades. Work is rolling out first to Pro, Enterprise, and Edu users on web and mobile, then to Plus and Business over the following days. The new desktop app itself — with Chat, Work, and Codex all included — is available globally on Mac and Windows, even for Free users, though specific features and usage limits vary by plan. GPT-5.6 is not a single model but a family that includes Sol, Terra, and others.
A company-wide LiteLLM rollout is being blocked by security review. The main concern is that LiteLLM would hold secret keys for every AI provider in one place and sit in the path of all prompt data. In regulated fields such as health care, finance, and government, that setup can become a high-value target. Audit logging is also a problem because the current logging setup does not match the compliance needs. Approval may require self-hosting, custom audit logging, or an extra protective layer around LiteLLM. The open question is whether this is a configuration problem or whether LiteLLM is the wrong fit for highly regulated use.
SAAG is a simple way to choose where AI belongs in real work. The steps are simplify, automate, agentify, and guard. Teams first make the work simpler, then use ordinary automation when fixed rules are enough. An AI agent is added only where it truly makes sense. Any task that could cause damage needs protection around it. The method was used in a client hackathon to turn broad “AI-first” pressure into concrete delivery choices. The core idea is restraint: do not put agents everywhere just because AI is available.
@vmcreate/ai-watcher is an npm package that watches a local work folder and sends code changes to an AI model when a file is saved. The AI can give quick code review notes, improvement ideas, or possible bug warnings. It sends only the diffs instead of the whole file or project, so it is designed to use fewer context tokens. The setup can use the user’s own API keys and prompts. The goal is to get fast feedback while coding, instead of waiting for a later pull request review.
OmniOKF is a CLI tool that turns PDF, Word, Excel, PowerPoint, HTML, and Markdown files into small linked Markdown files that an AI agent can search more selectively. Large documents are expensive and slow to place inside a context window, so the tool splits documents at headings and then uses Gemini Flash to refine and label the pieces. Each concept file is about 200 to 500 tokens, so an agent can load only the parts it needs for a question. The claimed result is a 75% to 95% drop in token cost per query. The tool also creates links between related files with normal relative Markdown links, so connected ideas can be followed without a separate graph database. It uses Microsoft’s markitdown to read many file types in memory and avoids heavier converters such as Pandoc. It can also produce Mermaid visualizations to show how the document pieces connect.