Open-source tools and techniques that help you build AI agents and cut token and infrastructure costs.
This open project focuses on how many tokens different languages use in an LLM. The same meaning can require different token counts depending on the language, and more tokens can mean higher cost and slower processing. The main issue is token cost parity: whether people using different languages pay a fair amount for similar work. This matters for AI agents because agents often run many steps, so extra tokens in prompts and replies can add up quickly.
drun is an initial open-source MCP release for giving AI agents a safer workspace. It creates an ephemeral runtime by virtualizing parts of the host computer, so the agent can work in a separated environment. The agent can use Git-like controls to try several paths in parallel and throw away paths that fail. The main goal is to avoid changing or damaging the host state while the agent explores options. drun adds hard rules for network access, command execution, filesystem paths, memory use, and run time. Instead of giving an agent direct create, read, update, and delete access on the host, drun puts a customizable policy layer in between. The project is open for bug reports, feedback, and community contributions.
self-learning-skills is an open-source tool that helps AI coding agents reuse work methods they discovered during difficult tasks. It works with Claude Code, Cursor, Codex, and other agents that read AGENTS.md or standing instruction files. The main idea is to save repeatable steps such as deploy commands, production database access paths, credential locations, or live verification checks after the agent has figured them out through trial and error. Claude Code saves this as a new SKILL.md file, Cursor saves it as a learned rule file, and Codex, Zed, Aider, and similar tools can save it in AGENTS.md or project memory. It can be installed with npx skills add kulaxyz/self-learning-skills, with options for global installation or a specific agent. Multi-step repeatable workflows become a new skill or rule, one-line facts become lightweight memory, and true one-off lessons are skipped. It is designed not to store passwords, tokens, connection strings, or API keys, and instead records only where those secrets should be found.
This is a July 2026 discussion about which local VLMs are worth using now. The comparison is limited to open weights models. Useful answers need more than model names; they should include the hardware used, the inference engine, the real use case, how often the model is used, and whether the work is personal or professional. VLM evaluation is treated as difficult because benchmarks can be unreliable, tools are still immature, and results can vary between runs. Details about tools, frameworks, and prompts are needed so other people can judge whether a recommendation applies to their own setup.
A common failure in AI agents isn't forgetting things — that's annoying but obvious. The harder problem is when an agent confidently re-proposes an approach that was already tried and abandoned a month earlier, or plans against a decision that was replaced two weeks ago because the old decision still looks just as valid in its notes. Nothing fails loudly; the agent just quietly wastes hours redoing work. Generic memory tools fix forgetting, but they don't fix being confidently wrong about the past, because that isn't a recall problem — it's a problem of tracking the current status of information (ruled out, replaced, still unverified). To address this, the builder spent three months creating NodeDex, a local graph of a project's reasoning that gets built automatically in the background from the agent's conversations, so the agent never has to remember to save anything itself. The design treats dead ends as first-class data: an explicit, checkable list of approaches that were tried and abandoned, which the agent is taught to consult before proposing anything new. Each decision is also stored along with its reasoning and the alternatives that were considered. The project has been released as open source.
Aletheia is an open-source agent loop for questions where the answer cannot be checked in a simple way. Many agent loops work best when there is a clear test, such as code compiling, tests passing, or a task being visibly finished. Aletheia is built for cases where evidence is partial, messy, or conflicting, such as checking whether a vendor’s growth claim is believable, whether a company is financially healthy, or whether a science headline matches the actual study. Its cycle is belief, act, observe, and update. It keeps track of what may be true, chooses searches that could change that view, and lowers confidence when opposing evidence appears. Its first working use is an open-source investigator for company and vendor diligence. It returns a verdict with evidence, conflicting signals, stated confidence, and unresolved unknowns. It can also stop without forcing a conclusion when the evidence is too weak. It currently runs with Claude Code and OpenAI Codex.
There are two ways to handle an AI agent's context window filling up as it works. The reactive approach waits until the window is full, then compacts everything at once. The proactive approach is picky about what gets added on every turn, so noise never piles up in the first place. Most coding agents use the reactive approach, but one developer spent months building the proactive kind instead. A core principle that held up: a decision an agent made on turn 3 is worth more than tool output from turn 15 that's already resolved — treating them the same causes context rot, where irrelevant information piles up and quality degrades. A system called PRAANA splits working memory into three tiers — active, soft, and hard. It scores each piece of context by information density, then uses BM25 (a keyword-based search method) plus semantic similarity, running via Transformers.js in-process, to decide what gets pulled back into the active window. One mistake stood out: an early hash-based embedder, thrown together as a placeholder, was quietly injecting noise into search rankings. Memories came back in the wrong order, with irrelevant items floating above relevant ones. There were no errors — it just looked plausible while being wrong — so it took three weeks to even notice. Switching to the Transformers.js-based embedder fixed it.
A static HTML RAG field manual maps the full pipeline from ingestion, chunking, embeddings, vector database setup, retrieval, augmentation, generation, agents, and evaluation. It is aimed at people working in the .NET and Azure world, where many existing guides feel too focused on Python and LangChain. Each step includes a free or open-source alternative, instead of only pointing to common defaults such as Pinecone or OpenAI. The material came from studying for Microsoft’s AI-103 certification and is built for visual, structured learning. It can help non-Python developers understand how the parts of a RAG-based agent system connect.
Bratan is an open-source framework for improving a RAG system. It uses three roles in a loop. One role looks for weak spots in the search-and-answer pipeline, another role tries to fix those weak spots, and a final role scores whether the result improved. The process repeats against a test set that changes as the system changes. The goal is to build a search-based answering system that handles real weaknesses, not just one that performs well on a fixed test. The item does not give numbers for token savings or lower inference cost.
A plan.md file lists five varied tasks that match normal daily work. Different AI models, including newer ones such as Claude Sonnet 5 and existing choices, can run the same tasks for comparison. The method tracks not only whether the result is good, but also how long the work takes and how much it costs. After the work is done, external test validation checks whether the output is actually correct. This is an intermediate benchmark workflow for comparing model performance, time, and cost in real use cases.
Claude Code can use an experimental team mode so several agents split the work. The workflow assigns research and code review to Sonnet 5, while keeping stronger and more expensive models such as Fable for core development. The goal is to keep quality high while reducing model cost and token use. The workflow covers quality control, token saving, context and memory management, debugging, and multi-agent work. It is marked as intermediate level, with a workflow value of 75/100, freshness of 70/100, and confidence of 0.80.
A self-hosted coding education app called Learn Code was built by splitting work across several AI models. Sonnet handled curriculum research, Opus created lessons and quizzes, Fable wrote code inline, Claude Design worked on the interface, and GPT 5.5 reviewed the code. The app was released as open source and packaged with Docker. It includes skill paths, exercises, and optional local AI code review. The main problem was that AI-generated code is not enough if the builder cannot understand mistakes or the logic behind the code.
Slop-Cops is a small web game that runs directly inside a Reddit post. A player enters a project idea or a URL, then five AI characters debate whether the idea is valid and how much it feels like low-effort AI output. The characters include a grammar checker, a suspicious bug hunter, a retro-style chief, a rookie, and a pessimistic senior engineer. The app is built on Reddit’s Devvit platform. Its screen uses React inside an iframe, talks to a Hono server on Devvit, and stores state in Redis. Gemini 3.1 Flash Lite creates the AI character responses. A live test version is available in r/slopcops.
Claude Code can act as the main coordinator while local large language models handle part of the code-writing work. The setup uses tools such as llama-server to connect a local model as a subagent. The goal is to reduce token use by avoiding Claude Code doing every piece of code generation itself. The workflow includes checking whether the subagent is actually being used, debugging problems in the interaction between Claude Code and the subagent, and saving the working setup for reuse. It covers quality control, token saving, context and memory, debugging, MCP, subagents, and multi-agent work.
A real estate outreach team paid workers based on screenshots from Nextdoor activity. Workers posted in local communities or sent private messages to prospects, then emailed screenshots as proof. Manual review became hard to manage as the number of submissions grew. An n8n workflow now handles most of the process. It checks the sender’s name and email against the active worker list, saves screenshots in Supabase Storage, and uses an AI vision model to decide whether each screenshot shows a community post or a private message conversation. Community posts are checked against rules from the company’s training document, while private messages are compared with approved message templates. The workflow calculates pay for valid submissions and alerts leadership by email when a prospect replies. The main weakness was a duplicate-payment loophole, where the same qualifying screenshot could potentially be reused.
Claude-assisted software development is organized into separate stages: architecture, coding, testing, review, and security checks. Stronger Claude models handle planning and review, while smaller models can handle implementation to reduce cost. Subagents can divide work by role, such as design, debugging, and quality checks. The workflow is aimed at people with little coding experience who want to build an app, such as an ERP system, from scratch with AI help. It also includes manual testing, AI-based testing, GitHub version control, and a final security review. It is rated as intermediate level, with a workflow score of 80 out of 100 and freshness of 70 out of 100.
ClaudeCode is given responsibility for maintaining an open-source project, but its work is controlled by a MAINTAINER.md operating charter. The charter defines what the AI agent is allowed to do, the order it should follow for planning, building, checking, and merging work, and the review gates it must pass. Some powers stay with humans: the AI cannot approve its own work, policy changes need human approval, and sensitive actions involving credentials or release authority remain human-controlled. The workflow also uses a “Living Code” system that creates documentation, AI skills, and descriptive project notes from the codebase itself. The goal is to keep guidance current, reduce wrong decisions caused by stale documentation, and cut down repeated background explanation, which can help with context management and token use. This is an advanced workflow covering quality control, debugging, shipping, hooks, skills, MCP, and subagents.
This covers a way to take the 'Agent Skills' feature from Anthropic's Claude and make it work with LLM-based agents that aren't Claude. Agent Skills are pre-defined instructions or procedures an agent can pull up when it needs to handle a specific kind of task. The idea here is to convert that mechanism so it isn't locked to Claude's format and can run with agents built on other models instead.
LiteLLM can load a model cost map from GitHub to estimate each model’s price and context window. That shared file is maintained by the community, so new models can appear late, stay missing, or carry wrong prices until someone fixes them. One practical fix is to override prices model by model, or point `LITELLM_MODEL_COST_MAP_URL` to a separate cost map. The example service at `cloudprice.net` uses the same format and refreshes prices from providers every day. It covers chat and embedding prices, plus image, audio, video, rerank, and OCR pricing. It also includes pricing for about 340 models that are not in LiteLLM’s default map, mostly newer releases such as OpenRouter Z.AI GLM-5.2, DeepSeek V4, and Vercel-related models. The service is free, needs no key, and allows browser access, but has throttling to prevent heavy use. A suggested product improvement is to let the LiteLLM gateway UI accept several cost map sources, using fallback sources when the first one lacks a model.
Most agent frameworks default to transcript replay: feeding the whole conversation, or retrieved chunks of it, back into the context window every turn. This works fine for short tasks but breaks down on long ones, and simply enlarging the context window does not fix it. Chroma's context-rot report evaluated 18 models, including GPT-4.1, Claude 4, Gemini 2.5, and Qwen3, and found accuracy degrading well before hitting the token limit, sometimes dropping 30-50% even on simple retrieval or replication tasks. Position within the context also matters: information at the start and end of the window survives, while content in the middle tends to degrade. A recent paper, "AI Agents Need Memory Control Over More Context" (arXiv 2601.11653), argues agents should maintain a bounded internal state that gets committed each turn, rather than an ever-growing transcript. It explicitly separates recalling an artifact from committing it to persistent memory, and claims lower drift and hallucination than transcript replay or retrieval across IT-ops, security, and healthcare workflows, though without hard benchmark numbers.
A Claude Code workflow uses a small open-source skeleton to avoid sending simple jobs to expensive Claude Sonnet by default. It is built with bash and Python standard libraries only, with no framework and no pip install. The main idea is a local Ollama task router that scores each job from 0 to 10. Jobs scored 0 to 5 go to free tiers such as Gemini Flash, Groq, or Cerebras. Jobs scored 6 to 7 go to a mid-range model such as Groq llama-3.3-70b. Only jobs scored 8 to 10 are sent to Claude Sonnet/Opus when no cheaper choice looks suitable. Every routing choice is saved in a JSONL log, so the real model-use pattern can be measured instead of guessed. In one personal log, 95 out of 98 choices avoided Claude’s top tier, but the setup was heavily tuned and the sample was small, so that number is not a general promise. The skeleton also includes session persistence and a secret-scan gate, while the scoring logic may not work equally well in every setup.
Local Qwen 3.6 27B and Qwopus are not reliable replacements for Claude Opus or Codex on long, unsupervised coding work. In a small software business, they were useful for work that involved customer diagnostics and telemetry data that should not be sent to a cloud model. A 12000 dollar RTX 6000 Pro with 96GB of video memory paid for itself within a few months after local analysis found that one customer had under-reported license usage by about 4 to 5 times for more than a year. The same setup still failed on broader agent tasks: it repeated outputs, invented file names and tool calls, made arithmetic mistakes, and sometimes drew the wrong business conclusion from usage data. Earlier 3090 cards forced heavier compression, shorter or lower-quality context settings, and more fragile results. On the RTX 6000 Pro, Qwen 3.6 27B ran through llama.cpp with long context and high-quality settings, and MTP raised speed from about 67 tokens per second to 130-200 tokens per second in sustained use. Running local AI became an operations problem, not just a model choice: the team needed access control, usage tracking, quotas, model routing, power monitoring, and uptime. The practical lesson is to use local models for narrow support, maintenance, code reading, and testing tasks, while avoiding long unattended agent work.
Prompts often change without the same testing discipline used for code. A small system prompt edit may look fine after checking a few answers by eye, but the chatbot can later start making up confident wrong answers. faithgate is built like pytest for prompts. It keeps a set of question, context, and expected-answer cases, then scores how faithful a prompt and model version are to the provided context. It compares the new run against a baseline case by case, and exits with a failure if anything gets worse. When connected to CI, a bad prompt change can fail the PR before it reaches production. The checks are intentionally strict: a run cannot pass if no cases match, if scoring did not happen, or if every score ends in an error. Each run saves a manifest with the judge model, RAGAS version, and test-suite hash. If the judge changes between the baseline and the new run, faithgate exits with a separate code instead of treating the comparison as valid. Abstentions are saved as abstentions, not converted into a 0.0 score. The default judge is Claude using the team’s own key, while RAGAS handles the scoring math.
ContextJet-ai’s public GitHub repository `awesome-llm-observability` collects 48 tools for LLM observability and evaluation. It covers tracing, evaluations, prompt management, gateways, OpenTelemetry instrumentation, and guardrails. Each tool includes its current GitHub star count and license. The main platforms are compared by self-hosting support, license, tracing support, evaluation support, and OpenTelemetry support. The repository also includes practical agent-focused examples, such as adding tracing, adding evaluations, debugging from traces, and tracing in a way that protects personal data. It also includes a small OpenTelemetry GenAI tracer example.
crawlberg is a Rust tool for crawling websites and extracting useful content. It checks robots rules and sitemaps, and it can use headless Chrome when a page needs JavaScript before the content appears. Its output is cleaned Markdown plus structured details such as links, metadata, JSON-LD, and Open Graph data. This gives a RAG system text that is easier to split into chunks, instead of messy raw HTML. BM25 filtering lets the crawler follow pages related to a search query instead of pulling an entire site. SSRF protection is on by default, so user-submitted URLs are blocked from reaching private networks, local addresses, or cloud metadata endpoints. It runs locally, uses an MIT license, and avoids dependence on one vendor.
Zero is an open-source coding agent that lets the model be swapped during the same work session. It connects to more than 25 providers, including OpenAI, Anthropic, Gemini, DeepSeek, Qwen, and Groq, and it can also use local models through Ollama or LM Studio. The `/model` command changes the model without losing the session context. Cheap and fast models handle routine work such as reading files, summaries, scaffolding, and boilerplate. A frontier model is used only for steps that need stronger reasoning. A local model is used when code or data should stay on the user’s machine. This changes the cost pattern because expensive tokens are used only for the smaller part of the workflow that really needs them. Sessions are saved as disk files, the tool is a single Go binary, it has no telemetry, and it includes a headless mode that streams JSON for automation.
Laguna M.1 is a 225 billion parameter Mixture-of-Experts model from poolside. It does not use the whole model for every token; it activates about 23 billion parameters per token, which points to a design meant to keep large-model capability while reducing compute load. The model is built for agentic coding and long-running tasks. It has 70 layers: 3 dense SwiGLU layers first, followed by 67 sparse MoE layers that route work across 256 experts. It uses global attention in every layer and supports reasoning between tool calls, with reasoning turned on or off for each request. poolside presents it as competitive with leading open-weight and frontier models on SWE-bench Verified, SWE-bench Multilingual, SWE-Bench Pro, and Terminal-Bench 2.0. It is released under an Apache 2.0 license, so it can be used and modified for both commercial and non-commercial work. Its training included pre-training, post-training, and reinforcement learning stages.
A hospital still keeps most patient records as physical paper files. The goal is to build a RAG assistant that lets doctors or hospital staff ask for a patient’s medical history summary, past diabetes diagnoses, or earlier medications. The hard part is starting with years of paper records instead of clean digital data. A likely first phase would scan the files, run OCR, and organize the data before building the RAG system. The open questions are what architecture would work in healthcare, what mistakes to avoid, what the biggest challenges are, and whether a small team can realistically handle the project.
A hackathon team wants to build a RAG app that runs only on two laptops instead of using online servers. One laptop has a 13th generation i7 processor and 32GB of memory, but no graphics card. The other has a 12th generation i5 processor, 16GB of memory, and a 4GB Nvidia graphics card. The goal is to answer questions using retrieved information, keep performance acceptable, and reduce wrong but confident answers. The team is trying to choose which open-source LLM fits those limits best.
Long-context AI models keep earlier calculation results in a KV cache so they can generate the next words faster. This experiment argues that a common way to judge KV cache quantization can miss the real error. Many perplexity checks run one calculation with the cache turned off, so the compressed cache is never read. In that setup, full precision, 4-bit, and 2-bit all produced the same perplexity score of 3.6416, which means the test could not see value-cache compression problems. The test was changed to first fill the context and then generate tokens one by one, forcing the model to read the compressed cache. The new method rotates value vectors with a Hadamard matrix and then applies 2-bit uniform quantization. Keys stay on KIVI int4, and only values are changed. On the corrected test, the 2-bit value cache matched KIVI 4-bit quality to three decimal places, used about 20% less memory, and used roughly four times less memory than fp16. The result held on Llama 2 7B and TinyLlama and was reproduced on a second machine.