LLMSim simulates LLM APIs to cut AI app load-test costs
LLMSim is a tool for testing high-traffic AI agents, , question-answer panels, support chats, and other apps that depend on without calling a real model every time. It sends fake model-like responses, so teams can lower during load testing and stress testing.
It supports OpenAI Completions, OpenResponses, and APIs, so existing LLM clients can point to it with limited changes. It tries to copy real API behavior such as streaming, response delay, TTFT, limits, and request-per-second limits.
LLMSim can run as a Rust library or as a single server binary. A server can be installed with Cargo, started on a local port, and then used through an endpoint such as `http://localhost:8080/openai/v1`.
Key points
- LLMSim simulates LLM API responses instead of calling a real model.
- It is meant to reduce during load testing and stress testing.
- It supports OpenAI Completions, OpenResponses, and APIs.
- It can mimic streaming, latency, TTFT, limits, and request-per-second limits.
- It works as either a Rust library or a standalone server binary.