Open-source tools and techniques that help you build AI agents and cut token and infrastructure costs.
The design puts a self-hosted knowledge layer between an organization’s documents and the AI tools people already use, such as ChatGPT or Claude. It is not meant to be another chatbot. It gives an AI assistant a controlled place to read trusted material and write back useful results. Each organization gets its own isolated installation, with its own server and its own data, so one organization’s information cannot accidentally mix with another’s. Original Word, PowerPoint, PDF, and similar files stay unchanged on disk. Extracted text is stored as markdown with extra notes about the file. The vector index and database are treated as rebuildable caches, not as the main source of truth. If the embedding model or chunker changes, the system can rebuild the index from disk instead of running a complicated data migration. A manifest and event log record why each document exists and which extractor, chunker, embedding model, and access policy handled it.
nexusGPT is a code analysis agent that takes a GitHub repository link, copies the full repository, turns files into structured data, and stores code relationships for later search and reasoning. It uses multiple LLMs and also supports multi-tenancy, so different users can share the system while keeping their work separated. The process starts when a user enters a repository link in the interface. The system clones the repo, converts each file into a Pydantic model, and sends the code through tree-sitter to detect the programming language and find code parts and relationships. It then runs SCIP to extract more exact code nodes and links. If SCIP fails, the tree-sitter output remains as a backup. The final information is stored in both a vector store and a Neo4j graph database. The system is tied together with LangGraph and connects several free cloud services with MCP tools.
HyperspaceDB 3.1.0 is a new release aimed at reducing memory problems in vector databases used for retrieval augmented generation and autonomous AI agents. A common problem is that large 1536-dimension vectors plus JSON metadata can take a lot of RAM, and very large collections can run out of memory. The new engine does not keep each full vector in RAM. It keeps a smaller 129-dimension navigation core in fast RAM, then loads the heavier 672-dimension semantic part from NVMe storage when it needs to re-rank the final top results. In the published stress test with 100,000 vectors, HyperspaceDB used about 72 MB of RAM, compared with more than 3,000 MB for Chroma and about 1,700 MB for Milvus. It also adds an 801-dimension hybrid vector design for data with hierarchy, such as legal codes or medical trees. That design combines Lorentz and Euclidean methods.
A company data analyst SQL bot needs the right information before it can write useful SQL queries. One option is to connect a RAG-based knowledge base made from Markdown files in a GitHub repository, with settings for document format, chunk size, and overlap before the files are embedded. Another option is to connect directly to systems such as GitLab through MCP. The current BigQuery MCP setup only has a tool for running a query, so metadata and metric information still need to be provided from outside. The main design problem is how to split background knowledge, such as table and metric definitions, from action tools, such as running the final query.
A firsthand RAG chatbot build found that many chatbot failures came from finding the wrong supporting material, not from the language model itself. The first version used vector search only, then added dense retrieval with BGE and Qdrant, BM25, RRF, and CrossEncoder reranking. Those additions helped, but the biggest practical gain came from using URL structure and metadata to guide retrieval. For a college chatbot, content can be grouped into admissions, departments, exams, scholarships, notices, and student services. When someone asks about scholarships, the system can search scholarship material first instead of scanning every document. This reduces irrelevant results, helps the system find more useful material, and gives the reranker better candidate text to work with. The main lesson is that retrieval design can matter more than switching to a different model. An example project is available in the GitHub repository omnichannel-rag-chatbot.
Several people each have a machine that can run AI models, but those machines sit idle about 90% of the time. The proposed setup would check whether friends' machines are idle when a job starts, then send tasks to any available machines through something like a VPN. A round robin setup would spread jobs across the machines instead of piling them onto one. Agent work can often be split into parallel tasks, so this could raise total speed if the jobs are easy to divide. The main idea is to use spare local computing power before buying more hardware or paying for cloud capacity.
An internship task requires a first working version of a legal document chatbot within 24 hours. Users upload one or more legal PDFs and ask questions about those documents through a ChatGPT-like chat screen. The goal is not to train a new model, but to build a clean RAG pipeline that processes the PDFs, indexes them, retrieves the most relevant document chunks, and lets Gemini Flash write answers from that context. The fixed stack is Vite and Tailwind CSS for the frontend, Go for the backend, SQLite for storage, and Gemini Flash as the language model. The intended flow is PDF upload, document processing and indexing, natural-language questions, retrieval of matching chunks, and grounded answers with citations when possible. The open design questions are the simplest architecture that can still scale later, which embedding model to pair with Gemini Flash, and how SQLite should fit into the setup.
AI agents often work in demos but fail in real business workflows when they are built like chatbots with extra tools. A customer onboarding agent should not rely on chat history to know whether it already sent a welcome email. Steps such as lead captured, plan chosen, contract sent, contract signed, payment received, onboarding started, and completed should be stored as clear state in a database. The LLM can reason, write, summarize, call tools, and choose the next action. The business process itself should follow fixed rules instead of depending on the model’s memory. A practical setup uses the LLM for thinking and language, tools for real actions, a state machine to track progress, and webhooks or events to restart the agent when something changes. Logs and evals help prove that no steps were skipped, and human approval should be required for costly or risky actions. A reliable agent is less like one huge prompt and more like a small operating system built around a model.
Vector Lakebase is now available in public preview on Zilliz Cloud. It combines a production vector database with a shared lake-native data base, so teams can use one copy of their data for online serving, on-demand search, and batch processing. The main promise is fewer data moves and fewer duplicate stores across different workloads. The webinar will explain how Vector Lakebase fits next to Milvus and existing vector databases. It will also cover external collections on Iceberg, Lance, and Parquet, plus one data layer, one index, and one semantic layer. Search support is described as covering vector, text, JSON, location data, and hybrid retrieval. The speakers are James Luan, Zilliz CTO and Milvus maintainer, and Jiang Chen, Director of Technical GTM at Zilliz. The webinar is scheduled for July 1, 2026 at 11:00 AM PDT.
Putting a cache in front of RAG can cut cost and waiting time because repeated questions do not need fresh search and answer generation. In firsthand use, the cache sometimes returned wrong answers with high confidence. One problem was stale answers: a source document changed during a session, but the cached answer still reflected the older version. A TTL did not truly solve this, because it only allowed the answer to stay outdated for a fixed time. Another problem came from storing compressed material, such as a summary or selected claims, instead of the full source context. A later question could need a detail that the summary left out, so the cache hit looked successful while the answer was worse than normal RAG. A useful fix was to record the exact sources behind each cached answer and clear only the cache entries tied to a source when that source changes.
An enterprise on-prem RAG search engine is being built to help teams find answers inside their own company documents, with cited sources shown next to the answer. The system emphasizes that data stays inside the organization, there are no token charges, and the company is not locked into one vendor. The problem is that teams create many useful documents but still struggle to find what they need later. Engineering specs, decisions, postmortems, and runbooks pile up in Confluence, while keyword search often fails when people ask meaning-based questions. Large document collections also lose context, so new hires keep asking the same questions and senior engineers become the people others rely on for searching. The current MVP includes a working Confluence connector, hybrid search using vector search and BM25, answers with source citations, and a setup where data remains within the organization’s own infrastructure.
The item proposes a default working contract for using Codex across many repositories. The rules apply to every repository unless a more specific AGENTS.md overrides them. The main rule is to avoid guessing and base decisions on inspected files, command output, documentation, or clear user instructions. If something is missing, uncertain, or not checked, Codex should say that plainly. Current user instructions take priority over the default rules, and conflicts should be resolved by following the most specific and most recent instruction. In Codex CLI, RTK should not be assumed to automatically intercept shell commands. The rules also point to being careful when a command may produce a very large amount of output.
The mobile tool aims to reduce the daily burden of choosing what to cook and to cut household food waste. The planned flow starts with a photo of a grocery receipt, then uses a multimodal vision model to turn the receipt into a structured list of purchased items. Each food item would also get an estimated shelf life based on its category. Saved recipe links would be cleaned by a parser that removes long story-like introductions and keeps the ingredients and cooking steps in a structured schema. A scoring algorithm would then compare soon-to-expire ingredients with saved recipe needs and recommend one best meal. The main technical question is whether a multimodal vision model can handle messy receipt layouts reliably, or whether layout-aware OCR would create fewer errors. Cost is also a concern because visual model processing can use many tokens for each receipt. The product question is whether a single “one suggestion” result reduces choice overload, or whether people still expect browsing and filters.
Coding agents can miss the real working function when they rely only on grep-style text search. Letting the agent read whole files to investigate also uses up the context window quickly. A hybrid search setup can work better because it combines exact word matching with meaning-based search. Adding reranking helps sort the first results again so the agent sees the most useful code sooner. This can reduce wasted reading and help the agent reach the right part of the codebase faster.
Local AI builders often recommend Qwen3.6 27B or 35B, while Gemma 4 31B is mentioned less often because it may not fit well on a single 3090. Gemma 4 26B MoE is not discussed as much, even though it may be a useful option. A firsthand local setup uses Qwen for coding agents, but is also building a Jarvis-like personal assistant for RAG, personal help, and knowledge-base questions. Qwen3.6 35B was first planned as the main model for that assistant, but testing suggests Gemma may work better for those non-coding tasks. The main issue is whether Gemma 4 26B MoE has hidden drawbacks, or whether it is simply being overlooked. The focus is not code writing, but retrieval, personal knowledge, assistant behavior, and question answering.
A production RAG solution on Azure needs an overall design before the application itself is built. The scope includes the full data architecture, the Azure services required, security and governance rules, data ingestion, indexing, and the Azure components the client must provision. The data may come from several places, including databases, PDF, Word, Excel and CSV files, APIs, SharePoint, and other enterprise systems. The main need is guidance on how to design an enterprise-scale RAG setup on Azure, not how to code the RAG app.
This raises a question about how AI agents should remember and manage conversational context. The core issue is whether markdown-based note systems, similar to Obsidian, function as short-term 'working memory' for a session, or whether they can actually replace retrieval augmented generation (RAG) systems entirely. Opinions are mixed on whether a Karpathy-style LLM-wiki approach is more useful than longer-term contextual learning. The question also seeks specific algorithms for moving short-term memory stored in markdown directories into long-term memory held in vector databases, as well as methods for pruning and maintaining that long-term memory over time.
The author runs several long-running, high-agency LLM instances as a personal experiment. The starting question: since an LLM is normally reactive and only responds when prompted, what happens if you just tell it 'do what you want'? The setup includes some models self-hosted on personal GPUs in a home lab, and others as agent/harness instances connected to frontier cloud models. Every instance runs in its own dedicated Docker container, on its own dedicated VM, on a set of dedicated NUC mini-PCs. The Claude Code instances specifically run with --dangerously-skip-permissions, passwordless sudo access, a build environment, and Discord bot integration. The author is looking for others doing similar experiments to compare notes and setups, or to start a community for it if none exists.
A GGUF version of the MiniMax M3 EAGLE decoder is now available and can run with llama.cpp. The model works as a smaller helper that suggests likely next tokens before the main model finishes, which can make text generation faster. The Hugging Face repository includes steps for merging the needed llama.cpp change and running the model. In one test using two RTX 3090 graphics cards and 128GB of DDR4 memory, the UD-Q2_K_XL quantized model improved from 2.3 tokens per second to 5 tokens per second. The speed gain came from using the --fit option and keeping the helper model in graphics card memory instead of system memory.
Gortex is a code-intelligence engine meant to help AI agents and developer tools understand large codebases with less repeated reading. It indexes code and exposes the results through a command-line tool, an MCP server, and a web interface. It is described as working across multiple repositories by default and shipping as one static file for macOS, Linux, and Windows. It uses tree-sitter to parse 257 languages and grammars, then builds a knowledge graph of functions, classes, call chains, HTTP routes, and links between services. It claims deeper code understanding for major languages such as Python, TypeScript, JavaScript, Go, Rust, Java, C, C++, PHP, C#, Swift, Ruby, and others. It offers 175 configurable MCP tools, works with 17 coding agents, and claims up to 50 times fewer tokens per response.
Qwen3.6-27B in Q8_0 format reached about 100 tokens per second on a two-card setup using an RTX 5090 and an RTX 3090 Ti. The same setup had been above 70 tokens per second with layer split, but changing llama.cpp server to tensor split gave a large speed gain. Tensor split lets both graphics cards work on the same parts of the calculation, instead of passing full layers between cards. That helps when one card is much faster than the other. The workload was divided 70% to the RTX 5090 and 30% to the RTX 3090 Ti to better match their speed. Results changed by prompt, with some runs reaching up to 130 tokens per second. The tradeoff is power and heat: during generation, the two graphics cards alone used more than 750 watts. The setup also used a very large context size, two parallel jobs, 99 GPU layers, and memory-related llama.cpp options.
Adding more skills and MCP servers to an AI agent can quickly raise token use and hurt execution efficiency. A poorly designed MCP can make the agent harder to control and more expensive to run. Before adding a new MCP, it is better to check whether a plain script can do the same job. Skills also need caution because AI may struggle to decide when a skill should be loaded. That can make the workflow annoying instead of helpful. A simpler approach starts with understanding the project structure, then deciding which parts of the code need close attention and which parts only need a quick look. A minimal setup can still be useful, such as using Pi Agent with repository information pulled through GitHub CLI/API.
AI Browser MCP Server lets an AI agent control a real browser window on Windows. It works with tools such as Cursor, Claude Desktop, Cline, OpenCode, custom LLM apps, and scripts that connect through HTTP or WebSocket. It provides 217 ready-made browser tools for opening pages, reading page content, filling forms, running JavaScript, watching network traffic, taking screenshots, and using CDP debugging. A person can give a plain instruction such as collecting product names and prices into JSON instead of writing a Playwright or Puppeteer script by hand. Setup is built around downloading the Windows package, running the browser app, checking the local server at 127.0.0.1:9222, and connecting an MCP client. Cursor and Claude Desktop use an mcp_bridge.js bridge, while custom apps can call the local MCP endpoint directly. The tool includes batch calls and sync-wait, so an agent can chain several browser actions in fewer back-and-forth steps, which can lower token use. Repeated tasks can be saved as workflow JSON and run again, and the project is MIT open source.
Intel Arc B70 can be a reasonable buy when 32GB of video memory is the main need and other tradeoffs matter less. Local AI model work can run fairly well with Vulkan or SYCL as long as the job fits inside that 32GB limit. The trouble starts when a model or workload needs 33GB or more. In this experience, two Arc B70 cards cannot be reliably combined under common software frameworks to handle one larger workload. That leaves a bad choice: accept very slow inference closer to central processor speed, or replace the setup with another vendor's graphics cards. The frustration is not only about one card, but about Intel's wider delay in giving Arc users strong AI software support.
Claude may not always obey limits written in CLAUDE.md, especially when the limits say what not to do or restrict tool use. Those written rules can act more like guidance than a hard stop, so Claude may still run an unwanted command. A stronger setup uses a blocking hook such as PreToolUse to stop certain actions before they happen. CLAUDE.md can still help, but it is less reliable; clear XML tags, positive wording, and expected output formats can make instructions easier for Claude to follow. The main lesson is to enforce important agent rules with a mechanical check, not only with written instructions.
Crew is a small prototype for development teams that use multiple AI agents on the same project. It aims to let those agents learn from each other over time. Knowledge gained by one agent becomes available to the others, instead of staying inside one separate agent run. The idea is similar to a Stack Overflow built by agents for agents. The goal is to make a group of agents work more like better teammates, not just better individual tools. The prototype is available on GitHub and is still at an early feedback stage.
GPT-5.6 SOL and Grok 4.5 Pro were compared using the same browser simulation prompt. The task was to build a glass bridge physics simulation in one HTML file without external libraries. The requirements included realistic weight distribution, crack spreading, glass breaking, particle effects, and a polished interface. The comparison claimed clear differences in reasoning, implementation quality, and visual finish. Side-by-side results were shared through an X link.
The focus is on security problems developers still face when putting AI agents into real services. Existing tools such as Garak, Promptfoo, PyRIT, and RedShield have already been reviewed, so the goal is not to rebuild what already exists. The open questions are about AI security issues in production, vulnerabilities that are hard to find or fix, and what current security tools fail to cover. Manual workflows are also part of the concern, especially security checks or response steps that could be automated. Real pain points matter more here than a list of desired features.
A commercial chatbot needs an AI model running on a GPU server so it can answer users. If the GPU is rented with per-second pricing, the key cost question is how many tokens an L4 GPU can process in one second. The available content does not include an actual speed number or details about the model being used.
Claude can miss “do not” rules or drift away from a required output format. A more reliable setup is to wrap important parts of the prompt in XML tags and rewrite negative rules as positive instructions. Instead of mainly listing what to avoid, the prompt should clearly state the exact behavior and format wanted. This uses the fact that LLMs can handle structured information more consistently. The goal is fewer broken formats, fewer missed constraints, and fewer rounds of fixing the same answer.