AI has serious weaknesses when it comes to recognizing shapes

AI has serious weaknesses when it comes to recognizing shapes

According to a study, artificial intelligence often fails to recognize outlines. (File photo)

Whether it's a cat or a car, humans can reliably recognize objects and living beings even when they are visible only as rough silhouettes. For artificial intelligence (AI), however, this is often an insurmountable hurdle, as a study shows.

Modern image recognition programs perform significantly worse than human observers when it comes to detecting overarching shapes and silhouettes, as reported by a research team led by Biyu He of New York University’s Grossman School of Medicine in the journal *iScience*.

The reason for this weakness lies in the fundamentally different way the algorithms work: While the human brain prioritizes the overall structure, even modern deep neural networks focus primarily on local details, surface patterns, and textures. Even minor image distortions or the absence of typical details quickly throw the systems off track.

For the study, the team led by first author Mugihiko Kato and study director He compared the visual performance of test subjects with that of more than 200 deep neural networks of various architectures and training methods. To do this, the researchers systematically manipulated 240 images from 48 everyday categories. Among other things, they presented solid black silhouettes without any internal structure and isolated patterns such as fur textures without recognizable contours. They also used images in which many small objects were arranged side by side to fill the frame, so that while internal details were preserved, the typical outline was disrupted.

None of the more than 200 AI models was able to fully replicate the human recognition pattern across all conditions. Whenever identification depended solely on detecting the overall silhouette, the computer algorithms consistently performed worse than the human participants.

This discrepancy was particularly striking in a follow-up experiment in which the overall shape was decoupled from local details: The researchers filled the outlines of objects and animals with many small crosses. While humans could still easily identify the subject based on the outer contour, most AI models failed completely. They suddenly began interpreting cat or butterfly silhouettes en masse as “crossword puzzles,” “window grilles,” “chain mail,” or “nematodes,” because the small cross patterns completely dominated the algorithms’ analysis.

He emphasized that today’s image recognition models are by no means as human-like as is often assumed—even though they have been trained on enormous quantities of photos taken by humans. While modern systems trained simultaneously on images and accompanying text achieved significantly more human-like accuracy rates overall than conventional image recognition networks, they, too, lost their advantage as soon as the overall composition was disrupted or evaluated in isolation.

Implications for Autonomous Driving

The results are of central importance for practical applications: If AI systems are to function reliably in robotics, autonomous driving, or in visual prostheses and brain-computer interfaces for people with visual impairments, they must be able to make reliable judgments even in fog, twilight, or when obstructed. In such situations, humans instinctively rely on rough silhouettes. The study provides valuable insights into how algorithms must be trained in the future to develop a more holistic—and thus more robust—visual capability, the team writes.