Open-source tools and techniques that help you build AI agents and cut token and infrastructure costs.
The hardest kind of LLM hallucination is an answer that sounds smooth, confident, and believable but is wrong. Methods that look for the model sounding unsure are weak in this case. Across about 124 prompts, confident made-up answers and true answers looked nearly the same when only the average internal response was compared. The model did not show a steady “lying direction.” The clearer difference was size and variance. Made-up answers caused larger and more spread-out movement inside the model as it generated text. On the representational-shift channel, the variance was about 7 times higher for made-up answers than for true ones, with Cohen's d around 0.58 and a p value around 0.005. The movement grew as the level of fabrication grew, which supports the idea that this is tied to hallucination rather than random noise. The practical advice is to detect instability, measure it across the whole generated answer, and connect the detector to a follow-up action such as checking or correcting the answer instead of using it as a simple pass-or-block gate.
AIGX is an open format for giving AI coding agents clear rules about a codebase. It stores the project’s product notes, architecture rules, engineering rules, and file-specific warnings inside a `.aigx/` folder. The main piece is `files.aigx`, a per-file index that tells the agent what rules, forbidden imports, and known mistakes apply to the file it is editing. The rules stay outside the source code, so the project’s real code does not need extra instruction text added to it. Setup starts with `npm create aigx`, and it can connect to Cursor, Claude Code, GitHub Copilot, Windsurf, Aider, and `AGENTS.md`. Tooling is available for Node, Python, and Rust; `aigx lint` checks whether the rule files are valid, and `aigx resolve <path>` shows the rules for one file. In its own benchmark using Claude Haiku 4.5 and Sonnet 4.6, AIGX ranked highest on average score, first-try pass rate, and hidden-test pass rate, but the project also says the top formats were statistically close rather than a clear blowout.
Decispher is a tool that collects a team’s code decisions, working rules, and limits in one shared place for both people and AI agents. AI agents can write confident but wrong code when they do not know why a codebase is built a certain way, and they can waste tokens rediscovering decisions the team already made. Decispher pulls useful decisions from Slack, GitHub, and Jira conversations, so teams do not have to write all of that documentation by hand. An AI agent can fetch only the decision it needs instead of sending a large amount of the codebase into every request. Internal testing showed about a 65% cut in token use. Branch Story keeps the reasoning tied to each branch, so the next teammate or AI agent can continue with the right background. The launch offer includes 50 free credits and 100 remaining seats.
A common but rarely discussed pattern with vector search: one large embedding collection ends up serving two very different kinds of workloads. The first is online retrieval, where users are waiting and latency directly matters — if p95 or p99 response times slow down, the product feels broken, so dedicated, always-on compute makes sense here. The second workload is offline and batch-like: mining hard negatives before a training run, deduplicating a large embedding collection, inspecting clusters or drift, running offline evaluations, or exploring a dataset for a few hours before going quiet for a week. This second type doesn't care whether a query takes 50ms; what matters is that the job finishes, costs stay bounded, and it doesn't interfere with production retrieval. The core tension raised is whether it makes sense to keep serving-style compute warm all month just to handle this occasional analytical workload, when the two workloads actually have very different requirements that tend to get blurred together.
AI services may be sold for less than they cost to run. Building and operating large AI models takes huge amounts of money, and user fees may not cover the full bill. Today, venture capital investors can absorb that gap because they expect cheap AI to help create a dominant company that later becomes profitable. If a company such as Anthropic goes public, some of that loss could shift to shareholders. Many people hold public company stocks indirectly through a 401(k), pension, KiwiSaver, or index fund, even if they never chose each stock themselves. That means people who never use AI could still help fund cheap tokens through their retirement savings. The caveat is that AI costs may fall before any public listing, and a newly listed company does not automatically enter major indexes right away.
An AI agent or retrieval-augmented generation system can look successful in a demo while still being unsafe for real business data. A demo only shows that the model can answer; it does not show that the system is safe to connect to customer records, internal documents, or tools that can take real actions. The biggest problems come from access control. A system prompt that says “only show users their own data” is not real protection, because that kind of instruction can be bypassed. User identity, policy rules, and the source system’s own permissions must block data before anything reaches the model. The model should not have permanent open access to systems. Reusing a powerful token or service account for convenience can quietly give the agent every permission that account has. Agents need separate identities, narrow permissions, and tool-by-tool allowlists.
Multi-agent Claude Code workflows can waste tokens and money when several agents run for a long time or reread the same material. Long-lived agents may still create extra cost even when caching is used, so the cache tax needs to be understood and reduced. Subagents should receive only the exact context they need instead of rereading broad project material again and again. Important routing decisions should not depend only on a weaker model’s judgment for complex choices. Clear outside structures, such as GitHub labels or issues, can make routing more reliable. The main idea is to design cost control, context management, and routing as separate parts of the agent workflow.
StarWAM is an early open-source codebase for experimenting with video-generation-based WAM systems. Its goal is to make parts of a world model easier to swap, so different designs can be tested inside one shared framework. It supports MoT-style designs such as Motus and FastWAM, shared DiT designs such as DreamZero, and feature-conditioned designs inspired by StarVLA WM4A and Mimic Video. It can use Wan2.2 5B and Cosmos Predict2 as backbones. It also includes LIBERO benchmark support for testing robot-task behavior. The project is still rough because video generation models have complex internal parts, and each backbone tends to organize those parts differently. Large training runs and hyperparameter tuning have not been done yet because of limited compute and time.
A firsthand account describes building and running a RAG (retrieval-augmented generation) system over a corpus of 10,000 medical papers at a time, at a small startup doing medical literature analysis. The team experimented with combining Graphiti, a knowledge-graph tool, with UMLS, a standardized medical terminology database, but the production system ultimately used a simple, vanilla RAG setup. After running this for a client over a couple of months, the client judged the tool inferior to ChatGPT for two reasons: response speed was slow, taking 2-3 minutes per answer, and the output didn't feel meaningfully more medically accurate or capable than ChatGPT's. This experience raises a broader question: on public data, can RAG techniques actually outperform frontier LLMs that were already trained on that same public data, even when a small team focuses narrowly on a niche topic?
An OpenCode plugin splits work between a main model and a cheaper coding model to reduce token use. The main model handles thinking, planning, and review, while the cheaper code model writes the actual code. After the coding model finishes, the result goes back to the main model for review. Small or simple edits can still be handled directly by the main model, so the system does not delegate every task. One setup uses GLM5.2 for thinking and free DeepSeek V4 Flash for coding. Installation uses `opencode plugin opencode-plugin-code-model@latest`, then `/code_model` or `/cm` inside OpenCode selects the coding model. It works as a plugin, so OpenCode source code does not need to be changed.
disp8ch is a self-hosted AI workspace that connects chat, visual workflows, agent teams, boards, design materials, documents, and memory in one place. It is released under the MIT license and has no paid plan, pro license, or locked features. Its model setup is configurable, so it can use local OpenAI-compatible servers, Ollama, LM Studio, llama.cpp, vLLM, SGLang, or an online provider. Chat requests can become visual workflows with run history and approval boundaries. Agents can work in the background while keeping the chosen provider and model settings. Documents, notebooks, memory, boards, and agent roles share the same local state. Skills, extensions, and MCP servers can be scoped to specific agents. Source code, install steps, and tests are available on GitHub.
Adding Claude to an app requires a clear split between Claude Pro and the Claude API. Claude Pro is a personal chat subscription for people using Claude directly. The Claude API is the developer route for connecting Claude to a product or service. A practical setup starts with API access, safe storage for the API key, and small tests before wider use. Cost control matters because every call can add usage. Teams should watch request volume and token use, and set limits or review steps to avoid surprise bills. Security also matters: the API key should not be placed directly in code or pushed to a public repository, and should be stored in a protected server setting such as an environment variable.
A product designer with 20 years of experience built Kepp, a save-for-later app, alone. Existing tools felt too heavy, too strict about organization, or hard to trust because saved items were still difficult to find later. The goal was a simple app with no folders and no system to maintain: save something, find it, and move on. Even without being an engineer, the work covered the frontend, backend, infrastructure, App Store and Play Store submissions, analytics, legal documents, privacy setup, payments, and the landing page. Claude and Codex handled roughly two-thirds of the build. The experience also showed that managing a token budget is a real skill when building alone.
Managing Azure Storage accounts across multiple projects often creates a frustrating gap: you know a phrase exists somewhere inside a document, but there's no easy way to find which file contains it. The built-in Azure Storage Explorer and portal search only match blob names, mostly by prefix. Azure AI Search can do full content search, but it's expensive to start with — roughly $75/month for the Basic tier and $250/month for Standard, per search service — which is hard to justify for internal tools, side projects, or smaller deployments. BlobLens was built to close that gap. It's a lightweight, self-hosted full-text search engine for Azure Blob Storage that runs with a single docker compose up command. It combines a FastAPI backend with a built-in search UI, Meilisearch for typo-tolerant instant search, and a background indexer worker. Point it at an Azure Storage Account using a connection string, and it indexes the actual content of PDFs, DOCX files, plain text, Markdown, source code, and more than 25 text-based formats. Results can be filtered by filename, content, container, file type, and metadata, and searches return in roughly 10 milliseconds. It also generates temporary SAS download links that expire after 60 minutes.
An internal LLM app has reached the point where adding safety controls is taking more effort than building new features. The examples are a support bot that drafts replies using ticket history and FAQ content, and an internal helper that queries a data warehouse through an API while also answering questions over contracts and policy documents. When the safety rules are too strict, normal work requests get rejected and employees cannot finish their tasks. When the rules are too loose, security worries about internal data leaking, and legal worries about strange answers reaching customers. The tried methods include prompt instructions for safety and tone, role-based limits on tools and tables, regex filters for obvious problems, and stricter review and refusal settings. In practice, the support bot rejects ordinary refund and cancellation requests, the analyst helper loses context because too many data columns are blocked, and the extra checks add enough delay that people return to their old workflow.
The basic loop inside an AI agent is usually short and simple, so it is often not the real source of failure. Problems tend to start in the tool layer, where the agent has to act on outside systems. A one-line tool description gives the model too little guidance about inputs, edge cases, and the shape of the result. Anthropic’s tool-use guidance also says detailed tool descriptions are a major factor in tool performance and recommends several sentences for each tool. Too many similar tools can also cause errors. If create, update, and delete actions are split into separate tools for the same resource, the model has to choose between them every time. Grouping related actions into fewer, clearly named tools with an action parameter can reduce wrong-tool choices. Tool outputs that return huge raw blobs also waste tokens and make it harder for the model to use only the needed information.
COMPASS Skills is a set of four local skills for AI coding agents that need to continue long tasks across conversations. The four skills help clarify unclear work before action, keep a repo-local record of task progress, create a handoff prompt for a new AI chat, and store a local collaboration profile for user preferences. Task records usually live inside the project at `.agent-workbench/task-forest/`, and user profiles live under `.compass-skills/user-profiles/v1` by default. The skills can be installed for Claude Code and Codex with commands such as `npx skills add dongshuyan/compass-skills --skill '*' -a codex -a claude-code`. The system is designed to keep runtime data local and says it does not extract browser cookies, tokens, private keys, credentials, or sessions. The user profile is stored as plain text without encryption, so passwords, tokens, verification codes, and highly sensitive personal data should not be saved there. The public install path has been checked with `skills@1.5.11`, including listing skills, installing them into Claude Code, and running a handoff smoke test.
A RAG app has been built with open-source models running through Ollama. The language model is granite-8b, the embedding model is BGE-M3, and the vector database is qdrant. The whole setup, including the qdrant data, currently lives on one personal machine. The main deployment questions are what tech stack to use, how other people would get answers if the model stays on local hardware, and whether the model must be uploaded to Hugging Face to create an endpoint. A key concern is avoiding a setup where other people’s usage consumes the owner’s GPU heavily.
A RAG system is not proven reliable just because an answer sounds good, includes some document context, and avoids obvious false claims. That approach can work in demos, but it breaks when real users ask questions in unexpected ways. A poor answer can come from several different failure points. The system may fail to retrieve the right document chunk at all. It may retrieve the right chunk but bury it under noisy chunks, leading the model to use the wrong material. Retrieval may be fine, but the model may add meaning that is not supported by the source. An answer can also be technically correct but still unhelpful if it misses what the user really wanted. Messy source documents, such as old pages, duplicates, or internal articles that disagree slightly, can also produce bad answers even when the system stays grounded in the documents.
ScholarLoop is an open-source tool where several AI agents handle a research workflow: reading papers, proposing ideas, running experiments, learning from results, and drafting a write-up. Its main design choice is to avoid spending heavily on every idea. It first tests many ideas with cheap early checks, then sends only the better ones into more expensive verification runs. The loop can stop by budget limit, round limit, or lack of further improvement. Each agent’s prediction or judgment is compared with measured results, so later rounds can rely more on agents that were right before. Experiment scores use fixed measurement rules, and code-level guards try to stop fake gains or invented numbers. Current examples cover digit classification and diabetes regression, with published Claude Opus 4.8 runs showing small improvements over baselines at about $0.45 and $0.77. The project has 108 tests, and its quick demo can run without an API key or GPU by using a fixed mock LLM.
An internal AI assistant built on company work data needs a clear choice: save AI-made summaries in the knowledge base, or keep only the original data and summarize when someone asks a question. Saving summaries in advance can make answers cheaper and faster because the AI has less text to read later. The risk is that a wrong or outdated summary can become a trusted fact inside the system. If the link to the original source is weak, errors can build up and become hard to trace. Keeping the raw source data and summarizing only at answer time makes responses easier to ground in the original material and easier to cite. Those temporary summaries can be discarded after the answer. The tradeoff is higher latency and cost, because the model may need to reread source material for every query. For anything treated as memory, the safer direction is not to store AI-written summaries as facts, but to summarize only for the current answer with citations back to the source.
Aegis is a self-hosted, open-source proxy that checks inputs and outputs around large language model calls. It is built to work with OpenAI-style clients, so a team can point the client’s BASE_URL to Aegis instead of changing the whole app. Before a prompt reaches the model, Aegis runs a 10-step input guard. It normalizes unusual characters into standard ASCII, removes invisible characters, and looks for PEM keys, API tokens, Log4Shell-style attack strings, and other risky payloads inside prompts or retrieved context. It also tries to catch adversarial suffixes such as GCG and AutoDAN tokens. After the model response is returned, Aegis analyzes the output asynchronously, so the user does not wait longer while it checks for hallucinations or major drift. The project is licensed under AGPLv3.
rule-agent is a small experiment aimed at making the final step of using a rule engine easier. The main idea is to bring rule engines closer to ordinary language instead of relying only on complex settings or code. The available description is very short, so the implementation details, performance numbers, supported models, cost savings, and repository link are not confirmed.
An AI agent that seemed nearly ready scored 68 out of 100 on the Badgr Agent Readiness Test. The test checks 30 areas that matter before an agent is released to real users. These areas include prompt injection, privacy leaks, unsafe answers, strange tool behavior, and replies that sound too certain. A score of 68 is not a total failure, but it is not strong enough to confidently let real users depend on the agent. The practical question is how teams should test agents before shipping them.
Architect is an open-source project that tries to make software architecture the main source of truth instead of generated code. A high-level DSL is used to describe domain objects, services and dependencies, APIs, event consumers, deployments, infrastructure, versions, and migrations. Architect parses that information into a fully linked model, which can later be used to generate code, tests, Docker files, Docker Compose settings, deployment manifests, and other project files. The parser and model resolver are already implemented. The next planned work is the generation layer and a project lock file that records project structure, dependency versions, infrastructure versions, generated artifacts, and task dependencies as a DAG. The intended workflow is AI-native: LLMs, currently tested locally with Qwen Coder, work on tasks and plans instead of directly editing source files. The goal is more deterministic code generation, so the system keeps knowledge about the codebase instead of asking an LLM to regenerate everything each time. A vector database with embeddings would help find similar code for reuse and refactoring.
Coding agent failures may not always come from weak models. Human developers often fill in missing details from meetings, chat history, product judgment, and team habits. A coding agent usually does not have that background. When a ticket is vague, it may still guess, write code, and make the result look confident. The needed skill may be less about prompt engineering and more about preparing better work packets. A useful work packet should state the problem, what must not change, which files or areas are in scope, which edge cases matter, what counts as done, and what the agent should ask before changing code.
AI Hacker Newsletter issue 39 is a weekly collection of AI links and Hacker News discussions. The listed topics include Claude Code leaving hidden marks in requests, better AI models leading to worse tool experiences, and why learning to code still has value. It also includes a topic about protecting the right to run local AI. Another listed link says Mark Zuckerberg thinks AI agent development is moving more slowly than expected. The full newsletter contains more than 30 AI-related links.
Tenstorrent is presented as an open alternative to NVIDIA for AI computing. The core point is that heavy dependence on one company’s hardware and software stack can limit choice and keep costs under pressure. The supplied content does not include performance numbers, price comparisons, AI agent setup details, or a specific way to reduce token use.
A QA tester wants to build an internal AI chatbot for the QA team. The goal is to answer questions quickly from more than 15GB of company documents. The document set includes release notes, feature documents, issue history, and test cases. The chatbot needs to be secure and used inside the company, preferably with a local or offline setup. The main questions are whether this is realistic with very little coding skill, what process to follow, and whether tools such as AnythingLLM with Ollama, local LangChain, or another option would be best.
A small fine-grained car classification test showed a large gap between image models. The task was to tell apart Volkswagen Golf generations from listing photos, using 175 training images and 132 test images. The setup kept the image encoder frozen, turned each photo into embeddings, and classified them with weighted k-NN. SigLIP2 SO400M reached about 92% accuracy, while CLIP ViT-L reached about 59% and DINOv2 Giant reached about 41%. The embeddings were L2-normalized, so cosine distance and Euclidean distance gave the same ranking, and switching between them did not improve DINOv2. SigLIP may fit this kind of similarity search better because it was trained to align images and text. DINOv2 may need a trained classifier head before it performs well on such a fine-grained task.