The same local model can behave differently across agent tools
StepFun’s model produced different results depending on the d to run it. In the StepFun blog example, running the model with Claude Code worked better than running it with Hermes.
The main signal is that output quality may depend not only on the model, but also on the tool and setup around the model. The available details do not include the test task, score, settings, token use, or cost numbers, so the reason for the gap is still unclear.
Key points
- is mentioned as a StepFun .
- Claude Code gave better results than Hermes in the cited comparison.
- Agent tool choice can affect the quality of the same model.
- The source does not provide task details, scores, token use, or cost data.