Testing whether model skills depend on each other
A 31B model is being tested with to improve specific skill areas. Across 40 domains and six separate quality scores, one quality area kept scoring weakest across five runs.
New contrastive training runs start from the same , with one version using examples where that weak area is handled deeply and another using examples where it is handled shallowly. The goal is to compare the two resulting models, find the circuit linked to that skill, remove selected , and measure whether other quality scores also drop.
If removing the circuit for skill A lowers the score for skill B, that suggests skill B depends on skill A inside the model. The longer-term aim is to build a of model skills and use it to choose a better training order, starting with skills that support others.
Key points
- A 31B model is being tested with for specific skill areas.
- One quality area stayed weakest across five evaluation runs.
- Contrastive training compares deep examples against deliberately shallow examples from the same .
- Ablation would remove selected to see which other scores fall.
- The intended output is a that can guide future training order.