Image model choice can change simple search accuracy a lot
A small fine-grained car test showed a large gap between s. The task was to tell apart Volkswagen Golf generations from listing photos, using 175 training images and 132 test images. The setup kept the image encoder frozen, turned each photo into embeddings, and classified them with .
SigLIP2 SO400M reached about 92% accuracy, while CLIP ViT-L reached about 59% and DINOv2 Giant reached about 41%. The embeddings were , so and Euclidean distance gave the same ranking, and switching between them did not improve DINOv2. SigLIP may fit this kind of similarity search better because it was trained to align images and text.
DINOv2 may need a trained classifier head before it performs well on such a fine-grained task.
Key points
- SigLIP2 SO400M reached about 92% accuracy on the small car image test.
- CLIP ViT-L reached about 59%, and DINOv2 Giant reached about 41% in the same setup.
- Changing between and Euclidean distance did not explain the DINOv2 result because the embeddings were .
- DINOv2 may work better with a trained classifier head than with plain .
- Teams building image-based agents should test on their own data before committing.