How to benchmark LLM parameters for local Hermes agentic workflows
A user running a Framework Strix Halo machine with 128 GB of memory shares a problem: their locally hosted Hermes and Claude Code setup for task automation is frustratingly slow. They run the Qwen3 35B model at Q8 precision inside LM Studio, accessible remotely via a phone terminal or Discord gateway. through ROCm keeps failing, so they fall back to Vulkan instead.
They have tried adjusting inference like eval_batch_size and physical_batch_size, and have also experimented with breaking prompts into smaller steps, but neither approach has yielded a clear speed improvement. The core question is whether there is a reliable method or resource for measuring how these actually affect flow .
Key points
- Running Hermes locally as an agent can be very slow even on high-spec hardware; parameter tuning alone may not be enough
- eval_batch_size and physical_batch_size affect , but there is no clear benchmark standard for
- ROCm may fail on some setups, with Vulkan as a fallback that can differ in
- Breaking prompts into smaller steps helps somewhat but does not fully solve latency issues
- Watch the reply thread for community tips on measuring and improving ic