Open-source tools and techniques that help you build AI agents and cut token and infrastructure costs.
A small chatbot built with Python, LangChain, and Qdrant showed that chunking strategy had a bigger effect than expected. Strong answers often depended more on retrieving the right information than on choosing the large language model itself. Embeddings became easier to understand after comparing different retrieval results and seeing how they changed the chatbot’s answers. The hard part was expected to be prompting, but most of the work went into improving retrieval and figuring out why useful information was missing. In this first RAG build, selecting the right source material mattered more than polishing the wording of the prompt.
LiquidAI released two 350 million parameter retrieval models based on LFM2.5. LFM2.5-Embedding-350M creates one vector for each document, which keeps the search index small and fast. It is designed for search across 11 languages and is described as highly accurate for a dense embedding model of its size. LFM2.5-ColBERT-350M stores a vector for each token and compares questions with documents using MaxSim, which aims to improve matching accuracy. This can let a system store a document in one language, such as an English product description, and retrieve it from questions in other languages. Both models are described as having inference speed similar to much smaller models because of the efficient LFM2 backbone. GGUF versions are available on Hugging Face and are meant to be used as replacements in existing RAG pipelines.
tokless is an open-source tool bundle for reducing token use and context window pressure in Antigravity coding agents. A single command sets up both the IDE and CLI environment after the user chooses Antigravity. After setup, the agent needs to be restarted. The bundle enables four tools: CodeGraph for finding code structure, RTK for filtering terminal output, Context-Mode for running tasks in a controlled sandbox, and Caveman for keeping agent replies shorter. The goal is to make token and context saving easier for non-technical users who do not want to configure several plugins by hand. Compatibility problems can be reported through GitHub issues.
Milvus is a strong vector database for self-hosted RAG, but running it at larger scale can require a heavy setup. In production, Milvus may mean using a distributed setup with Kubernetes, etcd, object storage such as MinIO, and a message queue before the search system is ready to answer requests. That can be difficult for offline environments, edge deployments, or small teams that do not want to manage Kubernetes. Actian’s benchmark positions VectorAI DB as simpler because it runs in one Docker container, has no outside dependencies, and does not need internet access. On the same hardware with 1 million vectors and 768 dimensions, VectorAI DB was presented as having higher throughput than Milvus and a 73% faster index load. Milvus still had better recall, with 0.9983 versus 0.9948. The comparison also needs caution because Milvus was tested in standalone mode, not its distributed setup.
This approach uses three agents inside Claude Code instead of asking one model for one answer. The Visionary creates a new idea, the Destroyer looks for weak assumptions and flaws, and the Synthesizer combines the opposing views into a stronger option. The loop becomes more intense over several rounds, and later the roles switch so each side has to argue from the opposite position. After every round, an internal verifier checks the claims, hidden assumptions, counter-evidence, bias, and blind spots. If the result scores below 8 out of 10, the loop runs again. The goal is to move beyond the average answer that a single LLM often gives and find ideas that appear when different viewpoints clash. One example applied the method to finding a viral growth idea for a CLI tool.
A local Docling parsing pipeline failed during preprocessing when research-paper PDFs were longer than 25 pages. The failures appeared on pages 25 to 28 and showed the error std::bad_alloc. The machine had an RTX 4060 with 8GB of video memory, 16GB of RAM, and an i5-12450HX processor. During the run, GPU use was around 95% and RAM use was around 80%, with about 3GB of RAM still available. Splitting PDFs into 20-page chunks allowed processing to continue, but 30 PDFs still took about 45 minutes.
A web developer tried to make quick money by building a small web app for local coffee shops. Most of the code was written by Claude Code, which turned the described idea into a working SvelteKit demo in about 40 minutes. The demo was put on a personal server, with a separate subdomain for each shop so the shop name and logo appeared immediately. Three nearby coffee shops were visited with a working demo they could tap, not just a sales pitch. The first owner agreed within five minutes and paid $700. The product itself was not an AI agent; it was a normal web app. Customers scan a QR code at the table, the app reads the table number, and they order from their phone. The order appears in a staff order screen with the table number and items, reducing the need to wait for someone to write the order down.
This is a self-tuning workflow for making Claude agents produce the wanted result faster while using fewer tokens. It reviews session performance and proposes changes to the agent setup, including CLAUDE.md, skills, and agents. It also covers when the workflow should run automatically, how often it should adjust settings, and how configurations should be stored. Memory management is part of the design. Useful memories are promoted, while wrong or unhelpful memories are reduced or removed. The goal is to move optimization work away from the user and let the Claude agent improve its own efficiency over time.
CLAUDE.md is organized into 8 essential sections so Claude gets clearer project guidance before helping with code. The goal is to stop Claude from making odd structure choices or producing work that does not fit the project. Clear notes on project background, preferred architecture, tools, coding rules, and limits give Claude a steadier basis for decisions. This can reduce repeated corrections, irrelevant output, and wasted tokens. The workflow is presented as useful for quality control, context and memory, debugging, and shipping. Its listed scores are workflow value 85/100, freshness 70/100, confidence 0.90, and status active.
TinyHarness is a local-first AI coding assistant that uses Ollama by default. Code stays on the user's machine unless the user chooses another setup. It tries to keep long AI agent sessions from becoming too large by combining tool results, keeping the system prompt small, and compacting old conversation history in stages. Its compaction feature is meant to shrink even a 1 million token conversation so it can be handled by a model with a 64,000 token context limit. Version 0.2.0 adds a terminal interface with split panes for chat, live project structure, and a file tree, so the user can see what the agent is doing. It also adds Nix support, image input through an /image command, and installation through cargo install TinyHarness.
After several months of tuning work with AI coding agents such as Codex and Claude Code, a simpler document-based workflow proved more reliable than many popular repositories, skills, frameworks, and shared workflows. The main setup uses a customized AGENTS.md file as the source of truth, plus 10 to 30 smaller markdown files in a /docs folder. These files cover architecture decisions, best practices, coding rules, things the AI should avoid, and project-specific knowledge. When one document becomes too large, it is turned into an orchestration file and split into smaller files. The goal is to make the AI read only what matters for the current task instead of loading a huge knowledge dump every time. The workflow also includes a pre-validation step before the agent starts work, borrowing ideas from a Harness Engineering repository.
turbopuffer has lowered its base monthly price from $64 to $16. The starting cost is now one quarter of the old price, which makes it easier to try if the previous price felt too high. The price update was shared by someone who works at turbopuffer.
Voice AI can score well on separate performance measures and still feel frustrating in real use. Speech recognition scores, response delay, and task completion can look acceptable, but small problems build up across a multi-turn conversation. A slightly wrong reply timing, repeated confirmation questions, or unnatural turn-taking can change how people behave during the interaction. These failures often come from the conversation flow itself, not from one isolated model mistake. Testing larger amounts of real conversations makes conversation-level quality checks more useful than average metrics. Manual review of long conversation logs does not scale well, so automated conversation-level QA becomes important. The practical focus shifts from finding single model errors to finding repeated patterns that make conversations feel bad.
Tensey is a visual tool for designing neural networks. It checks tensor shapes while the design is being built, counts parameters, and estimates FLOPs and VRAM use. It can catch broken residual connections and Linear layer size mismatches before GPU time is wasted. It supports 63 operations and includes shape inference. Finished designs can be exported as working PyTorch code. The project is available on GitHub under an MIT license.
A planned finance chatbot would use the Gemma4-31B model with 6 to 7 agents for separate jobs such as accounts, advice, and portfolio work. An orchestrator agent would route each user request to the right agent. The main question is how many tools an LLM can handle well when it needs to choose and act reliably. The design being considered would bind 10 to 15 tools, but it is unclear whether that is a good practice or whether a smaller limit is safer.
Permit IQ is a RAG system built with messy, inconsistent real data instead of clean test datasets. The goal was to learn how retrieval finds useful material and how a large language model uses that material to answer questions. The system is now hosted on Google Cloud. Even as a hobby project, it has already cost about $200, showing that running a production-style RAG system can become expensive quickly. The next areas to improve are answer quality, evaluation methods, retrieval strategy, reranking, and chunking. Cost reduction is also a focus, including the option of self-hosting models instead of relying only on cloud-hosted large language models.
Most autonomous agent products today are built for company work. Devin helps engineering teams, Lindy automates sales development tasks, and products from Cognition and Replit Agent are aimed at teams or enterprises. Personal automation can already handle useful recurring tasks, such as sending a daily news brief to Telegram, drafting replies to overnight mentions, or checking a code repository and opening pull requests for small fixes. These setups are not polished products; they are closer to scripts with a small identity file, a schedule, a model API key, and GitHub Actions. The missing piece is a clear category of personal agent infrastructure that makes this easy for individuals. The current choice is often either paying about $300 a month for a business tool and adapting personal routines to it, or building a private setup from scratch. Personal agents may be unattractive to investors, too few people may have enough repeat work to delegate, or the user experience may still be unsolved.
PearlOS is an open-source project that aims to challenge closed AI products from large private companies. Its new feature is an “Agency,” which combines many local models instead of relying on one powerful model. The models named include DeepSeek v4 Pro, Kimi, GLM, Qwen, and DeepSeek v4 Flash. The long-term goal is to build a Pearl model that can handle several kinds of input and output, such as text, images, and audio, while staying small enough to run on a consumer laptop. PearlOS has joined the NVIDIA Inception Program, and more training work is still needed. The core argument is that public data should not be turned into closed, paywalled intelligence, so freely available open-source models and platforms are needed.
While building a custom attention mechanism with Codex CLI, progress felt worse instead of better. Important tests needed to keep the work under control were repeatedly turned off. The tool also seemed to run into mistakes it had created earlier and then treat them as surprising discoveries. The main concern is whether Codex CLI can cause a kind of foggy workflow when doing low-level language model development.
GLM's founder is being linked to a possible GLM-fable release before the end of the year, and the discussion quickly turned to whether people could actually run such a model affordably. Some readers saw it as a possible open weights alternative to top closed models such as Anthropic's Opus, while others questioned whether the jump would be that large. The practical concern is hardware cost, not only model quality. One estimate treated the model as 753 billion parameters and said full-quality use could need about 1.5TB of memory, while Q4 compression could need about 465GB and Q8 compression about 800GB. For agentic coding, one suggested target was around 50 tokens per second, because slow output makes an AI helper hard to use in live work. Possible setups discussed included eight 72GB graphics cards at roughly $72,000, a future 512GB Mac Studio-class machine, or a large DDR5 memory system paired with a GPU. The main point is that cheaper AI agents depend on memory size, speed, and inference cost as much as on the model being available.
Manus stands out less for any single feature and more for the full working experience. It feels like a general-purpose AI agent that can plan, take action, and move through multi-step tasks instead of only answering prompts. The drawback is that usage can become expensive quickly. Some workflows also still feel rough once they move beyond polished demos. The real interest is not simple automation software or chatbot frameworks, but open-source tools that feel genuinely autonomous and can handle multi-step work with little guidance. The useful comparison is which recent agent frameworks or self-hosted AI tools come closest to feeling like an “AI coworker,” what they do well, and where they still fall short.
A local analysis tool is being built for more than 200 past tender and pitch dossiers. Each dossier includes a tender brief, the agency’s proposal, and an award report, but the source files come from many public authorities and have very different formats, including score tables, long prose, spreadsheets, and sometimes damaged Word files. The goal is to extract the same fixed JSON structure from every dossier: award criteria and weights, each participant’s score by criterion, total scores, ranking, and prices. The stack uses Python, SQLite, ChromaDB, and the Claude API, with local or EU-only processing because the data cannot be stored by outside services. The hard part is not making valid JSON; it is making sure the values are actually right. Repeated errors include treating the contracting authority as a bidder, reading evaluation sentences as participant names, merging two companies with different VAT numbers, storing a score under 100 as if it were a price, mixing prices with and without tax, and producing parent-child criterion weights that add up to 175 instead of 100.
Agent behavior is often controlled with plain text files such as persona notes, AGENTS.md, tool descriptions, and memory. Passing a schema check only proves those files are shaped correctly; it does not prove the model will follow every rule. A support agent was given two rules: never reveal the internal API token, and avoid saying what it cannot do by using positive language instead. The models were tested across multi-turn conversations, including pressure to extract the token, with 3 runs for each model. gpt-4o-mini never revealed the API token, but it broke the positive-language rule in every run, often by saying it could not share the token. gpt-4.1 also protected the API token, but broke the wording rule about 1 time in 3. The larger model did not solve the problem; it made the failure less frequent, which can be harder to catch in a quick spot check. The practical lesson is to run each check several times and judge the agent by the worst result, not the best one.
AI coding tools can waste tokens and time when they repeat the same mistake in slightly different ways. The root cause may be unclear instructions rather than the model itself. Broad requests such as “add user authentication” or “fix the login redirect” leave too many choices open. The AI then picks defaults, and the person rejects those choices because they do not match an unclear expectation. A practical fix is to write a short note before opening the IDE. The note should say what the feature does, what it must not do, and what constraints apply. The “must not do” line is especially useful because it forces edge cases into the open before the AI starts coding. Even 4 or 5 rough lines in a scratch file can reduce the apologize-and-repeat loop.
OpenRouter data from the last three months shows open-weight models being used more than proprietary models on that platform. Three months ago, proprietary models were about 60% of usage and open-weight models were about 40%. The ratio has now flipped, with open-weight models at about 60%. This is not a measure of the whole AI market. It only covers API traffic that goes through OpenRouter, so it can miss ChatGPT, Claude, Gemini subscriptions, direct company contracts, and private deployments. The data still suggests that people who actively choose and pay for models through OpenRouter are moving more work to open-weight models. The discussion also points out that many of these are not truly open source, because the training method and training data are often not released.
An agent had to understand both a legacy codebase and a 300-page spec document, but the RAG setup kept failing. The stack used LangChain, LlamaIndex, Chroma, embeddings, a custom reranker, and many rounds of chunk-size tuning. Even then, the agent often pulled unhelpful docstrings instead of the actual function logic. The clearest failure involved a global config bug. core/config.py defined the config object, main.py created it, and utils/scheduler.py changed it from a background worker. Because the repo had been split into chunks, the agent saw pieces of the problem but not the full chain. It found the config definition, missed the later state change in the scheduler, and kept suggesting fixes that looked reasonable but left the race condition in place. The alternative was to skip the vector DB, chunking, and embedding search, then put the key source files and the full spec into M3's 1M context so the agent could read everything at once.
Standard retrieval-augmented generation, or RAG, often works by cutting documents into small token chunks and using vector search to find text with similar meaning. That can be enough for simple question answering inside a normal business tool. It becomes weak in private equity and M&A work because the useful evidence is spread across many connected places. A single answer may depend on a banker deck in email, a Slack discussion, an investment memo in SharePoint, and an expert interview transcript in a CRM. A question such as what people said about a target company's main competitor during a diligence call two years earlier cannot be answered from one isolated text chunk. A standard vector database may return a few roughly related document links, but it does not understand time, source history, or clear links between people, projects, documents, and decisions. This is why knowledge graphs are being presented as a better fit for work where relationships between facts matter as much as the facts themselves.
When users wait 5 to 20 seconds for an answer, common fixes are caching, semantic caching, and better retrieval. But semantic caching often saves the source chunks that were found, not the meaning already drawn from them. The model still has to read those chunks again, process them again, and pay the token cost again. A stronger option is to cache the understanding created from the chunks instead of caching only the chunks. This could reduce retrieval work, context building, token use, and total response time. The hard part is freshness. When source documents, APIs, code, or databases change, the saved understanding must be invalidated at the right time.
Running AI agents on daily work makes it hard to choose which model should handle each request. The common advice is to use cheaper models for easy work and stronger models for hard work, but in practice that often turns into guesswork. One task can contain both cheap and expensive turns. A coding agent may spend many turns reading files, running commands, and summarizing errors. Those simpler turns may be fine for a small local model like Qwen. A later turn may need deeper reasoning about a difficult bug, and that is where a more expensive model can be worth it. The useful unit for model choice is therefore each prompt or turn, not the whole task. Choosing one model for an entire agent run can waste money on easy steps or hurt quality on hard ones. A possible answer is a triage layer where a small model checks each prompt, estimates its difficulty, and sends it to the cheapest model likely to handle it well.
DiggerHQ has released agent infrastructure for building internal or customer-facing Slack agents. A team can describe the agent on the website, then connect it to its own Slack app. Once connected, people can mention the app in a Slack channel to start and guide an agent session. Setup can happen through a guided dashboard flow or through an API. The API path is presented as needing only three API calls to connect a Slack agent. The process creates a manifest, and the Slack app is finished by adding three values that Slack provides.