A stress-test idea for seeing how long models hold up
A planned benchmark would test through a survival-style run where models lose HP when they fail tasks. The goal is to see how model quality breaks down under repeated pressure, not just how well each model performs on one clean test. The latest models from major AI labs are already on the list.
The benchmark is also looking for models, unusual fine-tunes, and smaller that might otherwise be missed. The model list is being prioritized before spending API credits on the tests.
Key points
- The benchmark would make models lose HP when they fail tasks.
- It aims to measure how models degrade under repeated pressure.
- Latest models from major AI labs are already planned for comparison.
- models, fine-tunes, and smaller are being requested.
- The model list is being chosen before spending API credits.