Can fine-tuning on edited reasoning traces make models worse?
models on summarized or censored from leading models may have serious limits. cannot magically give a base model abilities beyond its underlying capacity, even when the training examples come from a stronger model. In particular, the shown by Anthropic models may differ from the model's actual .
The concern is that Fable fine-tunes trained on these traces could learn an incomplete or misleading process and perform worse than the original model. No benchmark results or measured comparisons are provided to prove that outcome.
Key points
- does not automatically push a model beyond the limits of its base .
- Anthropic's visible may not match the model's actual .
- Training on summarized or censored could reduce rather than improve it.
- For AI agents, test task success, retries, and total token cost before adopting a .