New study ranks how well AI tools help shoppers hunt for deals | Action on AI

As more shoppers turn to artificial intelligence tools to help find deals, a recent study urges consumers to verify product information and pricing, as some platforms appear to be more reliable than others.

About 40% of consumers are using AI for shopping, according to Product AI.

To evaluate the accuracy of popular AI platforms, Product AI tested ChatGPT, Gemini, Claude and Perplexity using hundreds of shopping-related questions.

According to Dakota Nunley of Product AI, the company asked the platforms 220 shopping questions on everything from pricing to comparisons of products across nine different product categories and recorded those results.

Their study found numerous errors across the platforms.

"Everything from MacBook pricing being off by $300 in some cases, to SPF sunscreen being off by 10 points, to earbud battery life being off by six hours," Nunley said.

Despite the mistakes, the study found differences in performance among the platforms.

"But some tools are better than others?" asked Nydia Han.

Nunley noted that AI models are constantly changing and improving, meaning results from studies such as this one can vary over time.

Among the platforms tested, Product AI ranked Gemini last. Claude finished third, followed by ChatGPT in second place.

"Gemini, over half of the questions and shopping-related questions that we asked did have some sort of costly error for consumers," Nunley said, adding that the finding applied "on both the free and paid tiers."

Perplexity was identified as the top performer. Regarding Perplexity's performance, Nunley said,

"In our findings, we didn't find that perplexity. At this moment was the best choice for consumers."

Google, which owns Gemini, disputed the study's relevance to consumers, telling Action News that Product AI did not evaluate the Gemini consumer app and instead used Gemini software designed for developers.

According to Google, "The Gemini app includes everyday user features... including shopping experiences powered by billions of continually-refreshed product listings."

Users should also note that Perplexity differs from other platforms because it uses AI to actively search the internet and other AI models before providing information to consumers.

For shoppers looking for products on Amazon, keep in mind that Amazon uses AI shopping assistant, Rufus, for Amazon product and pricing information.

Amazon has blocked other AI bots from searching its marketplace.

TikTok also recently launched its own AI shopping tool, adding to a growing number of AI-powered shopping options available to consumers.