Cascade uses Gemini to rebuild highly compressed images
Cascade is a neural image codec that stores a very small version of an image and uses a generative such as Gemini to rebuild it. It was trained on 1 million images and focuses on familiar objects and shapes. Its goal is to keep most of the original style, color, and structure while using far less data than JPEG.
On test images, its average bitrate is 0.03, with a range from 0.005 to 0.16. The system uses three parts: a VQ-VAE, a cascading conditional convolutional network, and a generative with a LoRA. Gemini is the default decoder, while SDXL is available as an offline option.
Setup means cloning the GitHub code, downloading models from into a models folder, setting GEMINI_API_KEY, then running encode.py and decode.py to create and restore .nit files. Options include qp for the quality-size tradeoff, tokens-only to skip model or API use, and n-decode to generate several outputs and keep the one closest to the guide.
Key points
- Cascade compresses an image into a .nit file and rebuilds it with Gemini.
- It targets up to 50 times lower bitrate than JPEG, with a test average of 0.03.
- Gemini is the default decoder, and SDXL is offered as an offline option.
- Setup requires the GitHub code, models, and a GEMINI_API_KEY.
- qp, n-decode, and tokens-only let users control quality, cost, and how much AI generation is used.