Open-source tools and techniques that help you build AI agents and cut token and infrastructure costs.
Addy Osmani’s agent-skills repository has reached 59,000 stars. The main idea is to give an AI coding agent reusable skills and slash commands instead of making it work out good development habits from scratch each time. The workflow starts with writing a spec, then breaking the work into small tasks, building one piece at a time, proving the work with tests, checking code quality, and shipping. Each command turns on the right skills based on the situation. Designing an API turns on API and interface design skills, while building a user interface turns on frontend engineering skills. The /build-auto mode creates a plan, asks for one approval, then continues on its own while committing each task separately and stopping if something fails. Installation is presented as a marketplace add command for addyosmani/agent-skills. The open question is how custom skills work alongside existing ones in tools like Claude Code without causing conflicts.
RepoWise is an open-source MCP tool for helping coding agents understand code repositories better. The core problem is that coding agents can find the right part of a codebase, make progress, and then soon reopen files they already read. In large repositories, reading a few code files often misses important background. Useful clues include who usually changes a certain area, which files often change together, which parts have been unstable for months, and why some odd design choices exist. RepoWise tries to surface this surrounding context through MCP tools. Git history can reveal repeated relationships between files that are not obvious from the code alone. The project is still sorting out which signals are actually useful for agents and which are only interesting to inspect.
A terminal tool is available for reviewing answers that were graded by a model during AI evals. It splits a judge run into claims, evidence, and verdicts. It also highlights places where a verdict may not fit the evidence. This helped surface unusual cases that needed manual review. A small terminal viewer is included for browsing the results. The code is on GitHub in the claim-memory-graph-sdk repository.
The idea is a small marketplace that sells ready-to-use AI agents. Customers would download the agents and run them on their own servers or cloud setup instead of depending on a hosted software service. The agents would connect to existing work tools through APIs. The main questions are whether buyers would trust downloadable, self-hosted agents more than hosted SaaS, what would stop them from buying, which agent types they would pay for first, and whether one-time on-premise pricing or subscriptions would feel right. A key promise is reducing privacy and compliance concerns that often block companies from adopting AI.
For coding work, using different AI models for different jobs can be more useful than trying to pick one winner. Codex may work better for some tasks, Claude for others, and Gemini can help depending on the project. The hard part is not only choosing the right model. Each provider can bring separate subscriptions, API keys, billing, and setup work. GPT Proto is being used as one place to test different models without setting up every provider separately. Official apps can still be useful, but a single comparison space can make a coding workflow easier to manage.
A firsthand case describes an email from Ragie’s own domain that looked like phishing and seemed to hint at a possible shutdown. The available information does not confirm that Ragie is actually shutting down. The immediate need is a replacement RAG service that does not require building and running the system in-house. The team is too small to build its own setup.
A comparison tested three large 120B-class models and one smaller Qwen 35B model on a Strix Halo system with 128GB shared memory. The models were GPT-OSS 120B, Qwen 3.5 family, Nemotron Super 120B, and Qwen 35B. The main concern was not how fast the model writes an answer, but how fast it processes a long prompt. For code change and bug-fix work, most of the waiting time can come from reading the existing code and request, while answer generation is only a small part. Nemotron Super 120B felt better at handling deep context around 100k tokens, so llama-bench was used to compare it with similar models. The practical cutoff was 100 TPS for prompt processing; below that, the test was stopped as not usable. Context limits also differed: GPT-OSS handled about 128k tokens, Qwen 3.5/6 about 256k tokens, and Nemotron up to 400k tokens.
In Google Antigravity, using Sonnet to create a plan and then switching to Gemini to carry out the changes worked well overall. The problem is that Sonnet usage appeared to keep going down even after the model was switched to Gemini. More detailed real-time usage tracking made this easier to notice. Common explanations include hidden subagents, multiple sessions, or other background work, but this case used skills inside the IDE with no subagents. After starting a normal workflow and creating an implementation plan, the model switch still seemed to leave Sonnet usage being deducted, making the usage calculation unclear.
Tricentis has added new capabilities to its Agentic Quality Engineering Platform and AI Workspace. The main idea is broader than having AI create test cases. Multiple AI agents would work across test creation, test runs, performance testing, and quality analysis. The practical question is whether agent-based testing tools reduce maintenance work in real production use. They may help in some parts of testing, but they may also create extra noise if their results are not useful. Creating AI-generated tests can be easy at first; keeping those tests valuable six months later is the harder problem.
There is interest in visualization tools that show what an AI agent is doing in real time, especially which tools it is using and what step it is on. An older project called Agent Flow showed tool-use status as a text stream or simple visual output. The main need is for similar projects that are better maintained. A more advanced version could show live details from a local model, such as attention heads, but that may be too much for a normal consumer app.
A personal AI agent runs all day on a home server, using local large language models and cloud models while saving daily work notes as markdown files. The first idea was to give each project a unique value like a color, mix the day’s work into one blended color, and later read which projects were inside it. That fails because a color only has three numbers, which cannot carry enough information about roughly 50 projects and their shares. The better model is closer to a chef tasting a dish, naming the ingredients, estimating their amounts, and noticing when something unfamiliar is present. The method uses older math called hyperdimensional computing and vector symbolic architectures. Each project name creates the same 4,096-dimensional ±1 vector every time, and random high-dimensional vectors almost do not overlap. A day’s work can then be stored as a weighted sum of project vectors and later broken back apart with dot products. Unlike embedding search, which mainly finds similar text, this memory layer aims to recover the structure of mixed work across projects.
A CPU-only C99 language model engine was much slower than llama.cpp on DeepSeek-V2-Lite-Chat Q4_K_S. On the same i5-11300H laptop with 4 threads, the custom engine processed 1.90 tokens per second, while llama.cpp processed 13.79 tokens per second. That is a 7.3 times gap. The likely cause was not raw calculation power, but how much data had to be read from memory. The custom engine expanded Q4K weights into F32 before running some model parts, so each weight used 4 bytes during inference, and it also expanded MoE expert weights during each call. llama.cpp kept reading the compact Q4K data directly, using about 0.5 bytes per weight, and combined that with a Q8-compressed activation vector in one pass. This avoided a large F32 middle step. Changing SIMD options such as AVX2, AVX512F, and VNNI only changed speed by about 2%, because the main limit was memory reading, not math instructions.
An AI agent is being built to automate real work, not just answer chat messages. It is designed to understand tasks written in normal language, break them into several steps, and use multiple tools to complete them. It can connect to APIs and outside services, research the web, and create reports and summaries. It also aims to handle repetitive business tasks. The current improvement areas are long-term memory, better planning and reasoning, fewer hallucinations, faster execution, and human approval for important actions. The main open questions are which agents and frameworks work well in production, and what makes reliable AI agents hard to build.
A personal voice chatbot has been improved until it can respond close to real time. Whisper-small turns speech into text, Qwen3.5-397B creates the reply, and Orpheus turns the reply back into speech. The system can be interrupted while it is speaking and still keep track of the last part of the conversation. Everything runs on a local computer instead of a cloud service. On a 24 GB GPU, VRAM use stays at about 21.3 GB or lower, leaving some room for the rest of the work. Some MoE experts for Qwen use about 150 GB of system memory. The setup uses a bf16 KV cache, because Q8 KV cache caused problems for Qwen3.5. The context length is 131,072 tokens, which is enough to hold hours of conversation, and the code is expected to be shared on GitHub soon.
Several Gemma 4 models were tested for local inference on a machine with three Nvidia GTX-1070 graphics cards. The system used Kubuntu 26.04, an AMD Ryzen 5 3600 6-core CPU, 48GiB of memory, and 8GiB of VRAM per card, for 24GiB total VRAM. Each GPU was power-limited to about 120 watts, which reduced inference speed by about 5% but lowered stress on the power supply. The tests used llama.cpp with Vulkan and GGUF model files. The 31B quantized model was 17.52GiB, took 3 minutes 35 seconds to benchmark, and generated 7.12 tokens per second. The 12B Q8 model was 12.69GiB, took 1 minute 59 seconds, and generated 13.47 tokens per second. The 26B.A4B Q4 model was 15.83GiB, took 1 minute 45 seconds, and generated 41.28 tokens per second, while the 26B.A4B QAT model was 13.26GiB, took 1 minute 30 seconds, and reached 53.08 tokens per second. The three cards were installed at different PCIe speeds, with one card connected through a 1x extender.
The goal is to find a coding agent setup that works for daily development without costing hundreds of dollars each month. The comparison checks whether cheaper API models used through Claude Code and OpenRouter can offer similar value when ChatGPT Plus is already being paid for. The test uses a real WordPress plugin from work, not a toy problem. The project includes PHP, a WordPress admin screen, WP-CLI commands, database tables, an existing architecture, and a Git workflow. The models being tested include Claude Sonnet, GPT-5.3 Codex, Kimi K2.5, Qwen3 Coder Next, DeepSeek V4 Pro, MiniMax M3, Xiaomi Mimo 2.5 Pro, NVIDIA Nemotron models, and free OpenRouter models. The comparison measures cost per finished task, time taken, respect for the existing project architecture, how well requirements are followed, and how much human supervision is needed. It does not measure general knowledge or LeetCode-style coding problems.
A developer who uses coding agents (AI tools that write code on your behalf) daily grew frustrated at not understanding what happens between hitting enter and getting code back, so they built their own coding agent, called Orin, to find out. Most of the code was AI-written, but the project followed a real software development process: writing a spec, filing an issue, implementing it, and going through PR review, rather than blindly accepting every AI-generated change (often called vibe-coding). The idea isn't original — the developer read through the code of existing open-source coding agents pi.dev, nanocoder, and opencode as primary references, and skimmed Cline and Kilo Code for additional patterns.
A small orchestration harness uses two coding agents. One agent proposes and implements a plan, and the other reviews it. An experimental feature called a consensus audit adds one more reviewer turn when both agents accept a risky plan too easily. The hard question is deciding which decisions are risky enough to deserve that extra review. The first design checks each accepted decision against a small list of contracts using keywords. The contracts include rules such as not publishing, pushing, merging, or deploying without clear permission; preserving result durability; not breaking idempotency; and not drifting from the source of truth. This trigger is cheap and predictable, but live runs showed misses and false alarms. A real risky decision may not use the exact contract words, while a normal coding task such as merge_intervals can look risky because it contains a word like merge.
AI models are not yet truly improving themselves, but AI can already help improve software systems. The useful setup is a repeatable loop where AI builds, watches results, checks quality, suggests changes, and repeats while a person still chooses the product direction. Strong automated tests and anti-regression checks are essential because they protect the code from breaking as changes pile up. Even a solo builder working with AI benefits from a steady cycle: design, review, build, verify, integrate, collect feedback, and repeat. Making live changes on the fly is risky because it can create chaos, especially as a product grows. Connecting the codebase to a project management tool like Linear and to a knowledge base gives AI clearer boundaries and better context. Monitoring and feedback should compare the product’s behavior against the original requirements. Long-term memory stores and telemetry give the system evidence for deciding what to improve next.
A four-machine Ascend GX10 or DGX Spark setup has reportedly run GLM5.2 at about 400 to 500 tokens per second for prompt processing and about 15 tokens per second for output, using a 128k context. That is not fast, but it may be usable for personal work or small experiments, especially with quantization. The main question is whether buying local hardware now makes sense before a stronger open-source model arrives later this year or next year. The setup may draw around 1000 watts, but the appeal is being ready to run large models locally instead of depending fully on cloud inference.
A firsthand local test of Qwen3.6 27B on an RX 7900 XTX graphics card produced worse speed than expected. The setup used Ubuntu 24.04.4, Linux kernel 6.8, llama.cpp, ROCm 7.2.4, AMD driver 6.16.13, and Vulkan. In raw backend tests, ROCm processed the prompt at 235.73 tokens per second and generated at 31.14 tokens per second. Vulkan processed the prompt faster at 634.80 tokens per second, but generated more slowly at 13.32 tokens per second. In a real API test with 32,201 prompt tokens and 128 generated tokens, the normal 27B model averaged 238.42 prompt tokens per second, 26.84 generated tokens per second, and 139.8 seconds total. The MTP n=3 setup averaged 226.09 prompt tokens per second, 17.14 generated tokens per second, and 149.9 seconds total, even with a 78.76% draft acceptance rate. vLLM was also not a practical path on this hardware. The server settings included a 65,565 context size, 99 GPU layers, flash attention, continuous batching, and MTP draft inference.
Claude has been useful for coding, debugging, explaining concepts, and solving algorithm and math problems, but the free usage limit has become a blocker. A paid subscription is not affordable right now, so a strong free AI tool is needed. The main needs are writing and debugging code in C, C++, Python, Java, and sometimes MATLAB. Help is also needed for data structures, algorithms, networking, operating systems, and occasional electronics or embedded systems topics. For math, the tool should explain discrete math, calculus, and linear algebra step by step, not just give the final answer. It should also explain why a solution works. Good reasoning, accuracy, and support for long conversations matter because many follow-up questions are part of the workflow.
mlx-optiq is a quantization tool for large language models running with MLX on Apple Silicon. Instead of shrinking every layer to the same 4-bit level, it checks how sensitive each layer is and gives more bits to the layers that need them most. Sensitive layers can use 8-bit, while the rest stay at 4-bit, keeping the average model size about the same. The resulting models load with the normal `mlx_lm.load(...)` call, so they do not need a special runtime. At the same on-disk size, they keep more accuracy than uniform 4-bit models, with the biggest gains on smaller models. On Google’s Gemma-4 weights without QAT, the result was described as nearly lossless, and it also improved results when combined with QAT at the same bit level. E2B scored 54.2 versus 52.1 for uniform 4-bit, with 58.6% on GSM8K. E4B scored 65.8 versus 64.6 for uniform 4-bit, with 80.0% on GSM8K.
New AI models are arriving so often that it is hard to compare all their smaller quantized versions in detail. A year ago, there were more public benchmarks comparing different quant levels, but that pace is now difficult to keep up with. Users still need practical answers, such as whether a q3 version of one model is better than a q6 version of another model. Many existing benchmarks are too broad and may require generating millions of tokens, which is not realistic on a home setup running a very large model at about 10 to 20 tokens per second. A smaller test would be more useful if it focused on tasks that reveal quality drops, such as tasks that pass on q6 but fail on q5. The desired result is an automatic way to say, roughly, that q5 keeps a certain percentage of q8 quality without running an expensive full benchmark.
Sentinel is an open source SDK for controlling what an AI agent is allowed to do while it runs. It checks a tool call before the tool is executed, instead of only reporting problems afterward. The rules can cover role boundaries, intended action, and hard blocks that stop specific behavior. The monitoring path does not call another model, so the checks are deterministic and do not add token overhead. A Claude Code adapter is already available, and installation is through an npm package. The developer is asking which checkpoints should be exposed next.
A 76-probe comparison checked whether gpt-4.1-mini can safely replace gpt-4o-mini. Code generation passed all 10 checks, so the switch looks low-risk for code-only work. Reasoning chains did not look safe, with 3 critical regressions across 10 checks. AI assessment tasks also had 3 critical regressions across 18 checks. The standard suite had 3 items to review, sycophancy had 1 item to review, and JSON schema had 2 items to review. A local model downgrade from llama3.1:8b to llama3.2:3b through Ollama also showed 1 critical regression in the standard suite. The probe suite uses fixed checks instead of another AI model as a judge, so the results can be repeated under the same setup.
A 2026 beginner guide explains how to start with local AI. Local AI means running an AI model on your own computer or private machine instead of sending every request to an online AI service. The guide was created because more people are trying local AI and asking the same starting questions.
Harbor 0.5.0 can download, configure, connect, and start local AI tools with one command. Commands such as `harbor up webui llamacpp`, `harbor up opencode mlx`, and `harbor up hermes omlx` connect Open WebUI, OpenCode, and Hermes to backends such as MLX, OMLX, and Docker Model Runner. The main change is less manual setup. The command-line tool has also been adjusted to work better with coding agents, and Harbor now includes first-party skills that can be inspected from the command line. Services such as OpenCode can have these skills pre-installed, so Harbor can be run and configured through natural language requests. A new `harbor pull` command chooses the right download route based on the model source. Regular Hugging Face repositories and llama.cpp quantized models go through `huggingface-cli`, while short model names go to Ollama.
neuron-db showed 100% accuracy in a firsthand benchmark for 1-hop, 2-hop, and 3-hop questions. A hop means one step in following connected pieces of information to reach an answer. The result claims LLM calls stayed fixed at 2, no matter how deep the question was or how large the memory became. Compared with putting a full markdown dump into the model, neuron-db matched or beat the accuracy while using about 60 times fewer tokens. The markdown dump approach fell to 83% accuracy at larger size because the model got confused while reasoning over a 67,000-token input.
The main question is how much people are really paying each month when they regularly rent cloud GPU machines. The comparison focuses on what they run, how often they turn the machines on, and the rough monthly cost. The concern is that usage patterns seem very different from person to person, so it is hard to know what a normal spending range looks like. No specific prices, setups, or cost-saving methods are included.