Gemma 4 context raised to 80k on an RTX 5090
On an RTX 5090, the Gemma 4 31B instruction model in GGUF format handled about 80,000 instead of about 35,000. A setting first used for also worked with Gemma 4. The working setup uses `GGML_CUDA_NO_PINNED=1`, context size 80000, `` on, automatic fitting off, `` on, and parallel runs set to 1.
In the llama.cpp , the `` checkbox should be turned on. This can help experiments that need to read long material at once, but it depends on a high-end RTX 5090 and a specific runtime setup.
Key points
- An RTX 5090 setup raised Gemma 4 31B GGUF context from about 35,000 to 80,000.
- A used for also worked for Gemma 4.
- Important settings include `GGML_CUDA_NO_PINNED=1`, context size 80000, ``, and parallel runs set to 1.
- The llama.cpp needs the `` option enabled.
- This is useful for with long inputs, but it relies on specific hardware and settings.