SRM-LoRA aims to reduce hallucinations at no added inference cost
SRM-LoRA changes LoRA, a method for a with a small set of added parameters, to reduce . During training, it measures how sensitive the result is to different update directions and suppresses changes considered costly or risky. It leaves the model's answer-generation process unchanged, so its remains the same as standard LoRA.
The model was trained only on HaluEval-QA, but the researchers report better factual reliability on both related tests and benchmarks outside that training setting. The is available on GitHub, and the research was presented at an ICML 2026 workshop.
Key points
- SRM-LoRA suppresses risky update directions while training with LoRA.
- It does not change answer generation, so does not increase.
- Training used only HaluEval-QA, while reported gains also covered benchmarks outside that setting.
- The has been released on GitHub.