Ornith 35B local coding model gets 18% faster with MTP
Ornith 35B is a for agent-like coding work. A new version adds MTP support so it can run in FP8 E4M3 format with vLLM. There was no ready-made version with MTP drafter support, so this version was grafted together separately.
With MTP, it runs 18% faster than the same setup without MTP. The drafter is reported at about 70% on average. Using the full 256k needs an RTX-based system with more than 80GB of VRAM.
It may also work on systems, with a script that grafts the MTP model into a target NVFP4 model. The includes the grafting script and a high-performance v container.
Key points
- Ornith 35B now has a local setup with MTP added.
- The MTP version is reported to run 18% faster.
- The drafter is about 70% on average.
- Full 256k context needs an RTX-based machine with more than 80GB of VRAM.
- The GitHub repo includes a grafting script and a v container.