Open test measures how well local AI agents read calendar screens
VCCB is a benchmark that shows an AI model a week-view calendar image and asks it to pull out every event as , including title, start time, end time or duration, overlapping events, repeating events, and all-day or multi-day events. It started because a was trying to rebuild calendar entries from a screen photo when no API access was available, but it often made practical mistakes: times were 15 to 30 minutes wrong, every event became one hour long, or duplicate events appeared on nearby days. The same calendar week is rendered in Outlook, HCL Notes, and Thunderbird, then captured three ways for each app: a clean screenshot, a straight-on photo, and a photo taken at about a 15-degree angle.
That makes nine images per run. Scores are normalized for each calendar app because the apps show short events and spacing differently, and a perfect extraction counts as 100%. Early results put humans at about 99%, top such as Claude Opus at about 80% to 85%, a mid-tier free ChatGPT setup at about 75%, and plus at about 38% to 58%.
The main open question is how much accuracy is lost when the same model is run with different levels. The images, prompts, scripts, scorer, answer key, and results are public in a GitHub repo, and local model users are asked to submit runs with the models and settings they actually use.
Key points
- The benchmark tests calendar extraction from nine images across three calendar apps and three capture styles.
- Early scores show a large gap between humans, hosted , and .
- The original local agent often got event times, durations, or dates wrong.
- The key cost question is how changes accuracy for the same model.
- The GitHub repo includes the test images, prompts, scorer, answer key, and public results.