Open-source tool runs local AI models directly inside Java
libargus.cc is an engine for running local language and speech models inside the . Instead of communicating with a separate server through REST, it uses in Java 22 to connect directly to llama.cpp, , and ggml compute graphs. It provides a structured API and a stable ABI to simplify the connection between Java and low-level native code.
Frequently used paths avoid new memory allocations, while memory for prompts and tokens is created once and reused. This design aims to reduce delays and memory overhead caused by copying arrays and repeatedly creating temporary objects. Precompiled native files are bundled in the Java package to make easier.
The engine is also intended to support a future spatio-temporal called L-TABB, which is meant to replace RAG.
Key points
- Java 22 connects directly to llama.cpp and without a REST sidecar.
- Memory for prompts and tokens is allocated once and reused.
- The design avoids array copying and repeated memory allocation on frequently used paths.
- Precompiled native files are included in the Java package for easier .
- The engine is planned as the base for an L-TABB intended to replace RAG.