DownloadBook a demo

Layout defect scenario · 2026-08-13

Can vision models recognize the clean Commerce catalog UI?

Determine whether this clean control contains a visible layout defect.

Expected answerClean

1440 × 900 · 9 model configurations

Research by Stephan PetzlPublished

Exact input

Prompt

You are evaluating a user interface screenshot for objective rendering defects.

Check only for these visible issue categories: viewport_cropping, container_clipping, text_overflow, element_overlap, obscured_actionable_control, or broken_responsive_reflow. Do not report subjective design preferences, copywriting concerns, or accessibility assumptions.

If there is one visible issue, return its category and the tight affected region on a normalized 0 to 1000 grid. If none of these issues is visible, return has_issue=false and null category and region values.

Return only data matching the supplied JSON schema. Do not describe your reasoning.
SHA-256 · 7d8cfe281f4ef9b625bbcc6c89c45236b392624e96c23044e6dcaac1d8d7adb0

Ground truth

Commerce catalog layout-defect benchmark screenshot
Ground truth: clean control

Model evidence

9 model results

Blue marks the seeded defect region. Orange marks the model’s returned region when available.

Holo3 122B A10B

H Company · holo3-122b-a10b
Correct
Latency
1.87 s
Category
—
Localization IoU
—
Cost
$0.00062
Attempts
1
                  {"has_issue":false,"category":null,"region":null,"description":":none:"}

Grok 4.5

xAI · x-ai/grok-4.5
Correct
Latency
8.67 s
Category
—
Localization IoU
—
Cost
$0.0041
Attempts
1
                  {"has_issue": false, "category": null, "region": null, "description": "No objective layout defects detected in the screenshot."}

GPT-5.4

OpenAI · openai/gpt-5.4
Correct
Latency
7.30 s
Category
—
Localization IoU
—
Cost
$0.0056
Attempts
1
                  {"has_issue":false,"category":null,"region":null,"description":"No visible rendering defect from the specified categories."}

Gemini 3.6 Flash

Google AI Studio · google/gemini-3.6-flash
Correct
Latency
7.31 s
Category
—
Localization IoU
—
Cost
$0.0026
Attempts
1
                  {"has_issue":false,"category":null,"region":null,"description":"No rendering defects identified."}

GPT-5.6 Sol

OpenAI · openai/gpt-5.6-sol
Correct
Latency
26.99 s
Category
—
Localization IoU
—
Cost
$0.0129
Attempts
1
                  {"has_issue":false,"category":null,"region":null,"description":"No visible rendering defects in the specified categories."}

GPT-5.6 Luna

OpenAI · openai/gpt-5.6-luna
Correct
Latency
7.30 s
Category
—
Localization IoU
—
Cost
$0.00025
Attempts
1
                  {"has_issue":false,"category":null,"region":null,"description":""}

Claude Sonnet 5

Anthropic · anthropic/claude-sonnet-5
Correct
Latency
12.13 s
Category
—
Localization IoU
—
Cost
$0.0059
Attempts
1
                  {"has_issue":false,"category":null,"region":null,"description":"No visible layout defects detected; the interface renders cleanly with no cropping, clipping, overflow, overlap, or reflow issues."}

Claude Opus 5

Anthropic · anthropic/claude-opus-5
Correct
Latency
8.64 s
Category
—
Localization IoU
—
Cost
$0.0140
Attempts
1
                  {"has_issue":false,"category":null,"region":null,"description":"No visible rendering defects."}

Gemini 3.1 Pro Preview

Google AI Studio · google/gemini-3.1-pro-preview
Correct
Latency
31.21 s
Category
—
Localization IoU
—
Cost
$0.0029
Attempts
1
                  {
  "has_issue": false,
  "category": null,
  "region": null,
  "description": "No objective rendering defects found."
}
EN