When AI Sees Six Fingers: Why Vision Models Miss the Mark

After the release of Grok‑4, the author noticed that several multimodal models consistently reported five fingers when shown an image containing six fingers. The same mistake occurred with OpenAI o3, o3 pro, Gemini‑2.5 Pro, GPT‑4, Claude 3.7, and many other top‑tier models, while Claude 4 occasionally gave the correct answer.

To understand this phenomenon, the author consulted the recent paper Vision Language Models are Biased , which argues that large vision‑language models do not truly “see” images; they rely on massive memorised associations rather than direct visual analysis.

Human Analogy

The paper uses the everyday example of counterfeit “雷碧” (a fake Sprite) to illustrate how humans often rely on memory and expectations instead of careful visual inspection. This cognitive shortcut leads to systematic misrecognition.

Experimental Evidence

Researchers presented a modified Adidas shoe image with four diagonal stripes instead of the usual three. All tested models—including Gemini‑2.5 Pro, o3, GPT‑4, Claude 3.7—insisted the shoe had three stripes, ignoring the visual evidence.

Further tests with absurd objects (a five‑legged lion, a three‑legged bird, a five‑legged elephant, a three‑legged duck, a five‑legged dog) showed an average accuracy of only 2.12 %—roughly two correct answers out of one hundred.

These failures stem from the models’ reliance on high‑frequency associations (e.g., “hand” ↔ “five fingers”, “dog” ↔ “four legs”, “Adidas” ↔ “three stripes”). Such priors act as strong “common sense” that overrides contradictory visual input.

Real‑World Risks

When AI vision systems are deployed in safety‑critical domains—industrial quality inspection, medical imaging, autonomous driving—their tendency to dismiss low‑probability visual cues can lead to catastrophic outcomes, such as undetected cracks in car parts causing fatal accidents.

Therefore, the author urges practitioners to remain skeptical of AI‑generated visual judgments and to verify critical decisions with human oversight.