Medical scribe test: missing facts mattered more than hallucinations
Eight LLMs were tested on medical scribing with 300 doctor-patient conversations. Each model turned every conversation into a . Across 2,400 generated notes, there were 12 confirmed high-impact .
There were also 520 missed safety facts. In this test, leaving out important clinical details happened far more often than inventing dangerous details. GPT-5.4-mini performed well for its cost and speed.
and DeepSeek had the strongest writing quality, but DeepSeek missed many safety facts despite being cheap and writing well. had the fewest but weaker writing quality, while Kimi had no confirmed but was slow and expensive in this setup.
Key points
- Eight LLMs were tested on 300 medical conversations.
- The models produced 2,400 s in total.
- There were 12 confirmed high-impact .
- There were 520 missed safety facts, making the bigger issue in this test.
- GPT-5.4-mini stood out for cost and speed, but model choice depended on the trade-off.