scatter sampler adds softer token choice to llama.cpp
An scatter sampler has been added for . It slightly redistributes only among the top next-token candidates that were already selected.
The goal is to make generated text less stiff without raising and bringing in weak low-ranked choices. The name uses a light-spreading metaphor, but the method is not physics; it is a cheap smoothing step across nearby token ranks.
The top-ranked candidate can give a little to the second, third, or fourth candidate, and a fifth-ranked candidate can exchange with nearby ranks. It is currently implemented and tested through the sampler API and can be used in a sampler chain under the name scatter.
Key points
- scatter is an sampler for .
- It smooths among top token candidates only.
- It tries to avoid the weaker low-ranked choices that can appear when is raised.
- The method is a simple rank-based smoothing step, not real physics.
- It is implemented through the sampler API and the sampler-chain name scatter.