Ray-Ban glasses AI app now handles vision on the iPhone
OpenVision is a free, iOS app that turns Meta Ray-Ban glasses into a . Version 2.4.0 can answer questions about a photo using SmolVLM2 and directly on the iPhone. The photo does not leave the device, so the feature can work without internet access and without cloud costs.
Live video can also run fully offline, keeping the camera on while answering questions about the current frame. OpenAI Realtime and are available as cloud options for live video when the user wants that path. In video mode, follow-up questions can continue without repeating the wake word until the user says to stop video.
The main technical problem was the iPhone memory limit. The enlarged each frame and split it into about 25 pieces, which made the app crash, and setting a maximum image fixed it.
Key points
- OpenVision is a free, iOS app for Meta Ray-Ban glasses.
- Version 2.4.0 handles photo questions and live video questions on the iPhone.
- Offline processing can reduce cloud costs and API costs.
- OpenAI Realtime and are also supported as cloud live-video options.
- The iPhone memory issue was fixed by limiting image .