DownloadBook a demo

Grounding scenario · 2026-08-13

Locate the Search transactions input in Mobile banking

Every model received the same screenshot, prompt, and structured response contract.

Case IDtransaction search

390 × 844 · normalized 0–1000 coordinates

Research by Stephan PetzlPublished

Exact input

Prompt

You are evaluating a user interface screenshot for test automation.

Locate the single UI element described below. Return the best click point and the tight visible bounding box of the complete interactive target. Coordinates use a normalized 0 to 1000 grid, where (0, 0) is the top-left image corner and (1000, 1000) is the bottom-right image corner.

Target: the Search transactions input

Return only data matching the supplied JSON schema. Do not describe your reasoning.
SHA-256 · db5f0d0e0270eb87be405a2076323344a9c2827d9093edd204b3869d00f2f8af
Mobile banking UI with the the Search transactions input target highlightedExpected
Ground truth: the Search transactions input

Model evidence

9 responses

Blue marks the browser-derived target. Orange marks the model’s returned box and point. Format validity checks only the response contract; it does not mean the requested element was found.

Holo3 122B A10B

H Company · holo3-122b-a10b
Point hit
Format
Valid
Box IoU
0.257
Latency
1.42 s
Cost
$0.00032
Raw response
{"point":{"x":499,"y":146},"box":{
    "x":49,"y":127,
    "width":951,"height":157
}}

Grok 4.5

xAI · x-ai/grok-4.5
Point hit
Format
Valid
Box IoU
0.350
Latency
2.04 s
Cost
$0.0019
Raw response
{"point": {"x": 500, "y": 175}, "box": {"x": 50, "y": 145, "width": 900, "height": 60}}

GPT-5.4

OpenAI · openai/gpt-5.4
Point hit
Format
Valid
Box IoU
0.120
Latency
1.60 s
Cost
$0.0026
Raw response
{"point":{"x":194,"y":128},"box":{"x":18,"y":107,"width":354,"height":42}}

Gemini 3.6 Flash

Google AI Studio · google/gemini-3.6-flash
Point hit
Format
Valid
Box IoU
0.861
Latency
1.13 s
Cost
$0.0021
Raw response
{"point":{"x":500,"y":150},"box":{"x":46,"y":125,"width":908,"height":51}}

GPT-5.6 Sol

OpenAI · openai/gpt-5.6-sol
Point hit
Format
Valid
Box IoU
0.849
Latency
3.88 s
Cost
$0.0066
Raw response
{"point":{"x":501,"y":150},"box":{"x":46,"y":125,"width":910,"height":49}}

GPT-5.6 Luna

OpenAI · openai/gpt-5.6-luna
Point hit
Format
Valid
Box IoU
0.848
Latency
2.03 s
Cost
$0.00014
Raw response
{"point":{"x":500,"y":151},"box":{"x":46,"y":126,"width":908,"height":51}}

Claude Sonnet 5

Anthropic · anthropic/claude-sonnet-5
Point hit
Format
Valid
Box IoU
0.073
Latency
4.07 s
Cost
$0.0028
Raw response
{"point":{"x":195,"y":126},"box":{"x":20,"y":113,"width":350,"height":26}}

Claude Opus 5

Anthropic · anthropic/claude-opus-5
Point miss
Format
Valid
Box IoU
0.000
Latency
5.87 s
Cost
$0.0071
Raw response
{"point":{"x":500,"y":324},"box":{"x":46,"y":286,"width":907,"height":76}}

Gemini 3.1 Pro Preview

Google AI Studio · google/gemini-3.1-pro-preview
Point hit
Format
Valid
Box IoU
0.839
Latency
37.42 s
Cost
$0.0030
Raw response
{"point": {"x": 500, "y": 149}, "box": {"x": 46, "y": 126, "width": 907, "height": 47}}
EN