Claude web tasks: screen control can cost less than DOM access
was tested on the same 5 web tasks while changing only how it saw and controlled the webpage. relied on screen pixels and coordinate clicks, while had access to the page structure through the DOM. The comparison used Pass@3 for each task.
often finished in the same number of steps or fewer, but each step carried more context, so it usually cost more. In 4 of the 5 tasks, was cheaper even when it took more actions. Finding a book’s UPC cost $0.40 in 4 steps with , compared with $0.79 in 3 steps with ; a Wikipedia lookup cost $0.53 in 5 steps versus $1.28 in 4 steps.
Buying a t-shirt cost $4.01 in 17 steps with , compared with $7.41 in 20 to 25 steps with . The exception was a dense product grid: found the target through the DOM in 4 to 5 steps for about $1.22, while needed 16 steps and cost $3.74.
Key points
- was tested on the same 5 web tasks.
- controls the page by looking at pixels and clicking coordinates.
- reads the DOM to find page elements more directly.
- was cheaper in 4 of the 5 tasks, despite sometimes taking more actions.
- won clearly on a dense product grid where many items looked similar.