Why same prompt, same answer is still hard for LLMs
means the same question should produce the same answer. If the answer changes, there should be a clear reason, such as a changed setting or input, not unexplained randomness.
Even with set to 0, the same prompt can sometimes produce different answers. The practical issue is which methods can make output repeatable today, how reliable they are, and whether is used in or mainly for and research.
Key points
- aims to return the same answer for the same question.
- Different answers should have an explainable cause, not hidden randomness.
- 0 does not always guarantee identical output.
- The main question is whether works reliably in or mostly in testing and research.