TensorSharp offers an open-source local LLM engine
TensorSharp is an for running AI on a local computer. It targets Unsloth GGUF models, including examples such as Gemma4, , and Qwen3.6. It supports multimodal use cases with images, vision, and audio, plus reasoning and function tool calls.
It can run on Windows, macOS, and Linux, and is designed to use the GPU when available. Its API is compatible with OpenAI and Ollama, which could make it easier to connect to existing tools. Its is claimed to be close to llama.cpp.
It is not just a C# wrapper around llama.cpp; the is implemented from the ground up. The CPU backend runs in pure C#, and it also includes CUDA, MLX, and GGML backends, with ideas borrowed from vLLM, oMLX, and llama.cpp for caching, batching, GGUF , and speed improvements.
Key points
- TensorSharp runs AI locally as an .
- It supports Unsloth GGUF models such as Gemma4, , and Qwen3.6.
- It works across Windows, macOS, and Linux, with GPU support.
- Its API is compatible with OpenAI and Ollama.
- It includes a pure C# CPU backend plus CUDA, MLX, and GGML backends.