Element presence scenario · 2026-08-13

Is the Restart device button present in IoT control center?

Determine whether the Restart device button is visibly present.

Expected answerPresent

1440 × 900 · 9 model configurations

Exact input

Prompt

You are evaluating a user interface screenshot for test automation.

Determine whether the single UI element described below is visibly present. Return found=true if it is present and found=false if it is absent.

Target: the Restart device button

Return only data matching the supplied JSON schema. Do not describe your reasoning.
SHA-256 · 64b7cd4d5c438c7ccf80888e92dbd84508d30ba2d8d32d3b2c57f988480437c3

This dated pilot used the geometry-capable prompt shown above. The published presence result uses only the returned found value; geometry is neither scored nor displayed.

Ground truth

IoT control center element-presence benchmark screenshot
Target: the Restart device button

Model evidence

9 model results

Presence pages report only the boolean classification. Original model output remains visible for auditability.

Holo3 122B A10B

H Company · holo3-122b-a10b
Correct
Latency
1.38 s
Cost
$0.00055
Attempts
1
                  {"found":true}
                

Grok 4.5

xAI · x-ai/grok-4.5
Correct
Latency
7.43 s
Cost
$0.0032
Attempts
1
                  {"found": true}
                

GPT-5.4

OpenAI · openai/gpt-5.4
Correct
Latency
4.81 s
Cost
$0.0044
Attempts
1
                  {"found":true}
                

Gemini 3.6 Flash

Google AI Studio · google/gemini-3.6-flash
Correct
Latency
9.20 s
Cost
$0.0022
Attempts
1
                  {"found": true}
                

GPT-5.6 Sol

OpenAI · openai/gpt-5.6-sol
Correct
Latency
5.41 s
Cost
$0.0110
Attempts
1
                  {"found":true}
                

GPT-5.6 Luna

OpenAI · openai/gpt-5.6-luna
Correct
Latency
8.63 s
Cost
$0.00022
Attempts
1
                  {"found":true}
                

Claude Sonnet 5

Anthropic · anthropic/claude-sonnet-5
Correct
Latency
10.46 s
Cost
$0.0043
Attempts
1
                  {"found":true}
                

Claude Opus 5

Anthropic · anthropic/claude-opus-5
Correct
Latency
8.36 s
Cost
$0.0107
Attempts
1
                  {"found":true}
                

Gemini 3.1 Pro Preview

Google AI Studio · google/gemini-3.1-pro-preview
Correct
Latency
26.31 s
Cost
$0.0024
Attempts
1
                  {"found": true}
                
EN