Courts to Companies: You Own What Your Chatbot Says

Courts to Companies: You Own What Your Chatbot Says

Artificial intelligence chatbots make two kinds of errors.

A hallucination is a false output, like a made-up citation, a wrong number or a name that doesn’t exist.

A confabulation is different. It happens when a system doesn’t know the answer and fills the gap with something that sounds like it fits, stated with the confidence of a real policy or a real customer service response.

A confabulated policy, delivered through a company’s own support channel, looks like the real thing. To the customer reading it and to the court reviewing it afterward, it carries the same authority.

Courts Hold Companies Liable for AI Chatbot Hallucinations

The legal exposure crystallized in February 2024, when the British Columbia Civil Resolution Tribunal ruled Air Canada liable for a chatbot’s fabricated bereavement fare policy, in the case Moffatt v. Air Canada, 2024 BCCRT 149.

Jake Moffatt consulted Air Canada’s chatbot after his grandmother died and was told he could buy a full-fare ticket and claim a retroactive bereavement discount within 90 days. However, no such retroactive policy existed.

When Air Canada denied Moffatt’s claim, it said the chatbot was responsible for its own statements, the American Bar Association reported at the time. The tribunal rejected that argument directly.

“In effect, Air Canada suggests the chatbot is a separate legal entity that is responsible for its own actions,” tribunal member Christopher Rivers wrote in the decision. “This is a remarkable submission.”

The tribunal ordered the airline to pay Moffatt 812.02 Canadian dollars (about $570) in damages and fees.

The ruling established that companies are liable for statements their AI systems make on their behalf, regardless of whether those statements were intentional or the product of a model filling a knowledge gap.

The next test arrived in April 2025.

An AI support bot at AI coding startup Cursor told a developer that the company had a new policy limiting each subscription to a single device, Ars Technica reported at the time. The policy did not exist. The bot, named Sam, had invented it.

Cursor Co-Founder Michael Truell confirmed the fabrication on Reddit.

“Hey! We have no such policy,” Truell said. “You’re of course free to use Cursor on multiple machines. Unfortunately, this is an incorrect response from a front-line AI support bot.”

Before the correction reached users, the invented policy had circulated on Hacker News and Reddit.

Insurers and Regulators Start Pricing AI Hallucination Risk

In May 2025, Lloyd’s of London launched an insurance product specifically covering AI hallucination-related losses. The policies, offered through a startup called Armilla, cover court claims against a business if it is sued by a customer or third party harmed by an underperforming AI product.

FINRA’s 2026 Annual Regulatory Oversight Report flagged hallucinations as a specific compliance concern for broker-dealers, warning firms to develop procedures for AI agents that may act beyond the user’s intended scope. The regulator defined hallucinations as instances where a model generates information that is inaccurate or misleading yet presents it as factual.

Scaled Cognition raised $100 million in June to build enterprise-grade hallucination controls. The investment signals that the cost of confabulation, once treated as a product quality problem, is now being priced as a financial risk.

The pattern the Cursor and Air Canada cases share is not a failure of AI to understand the question. It is a failure of the system to recognize that it does not know the answer, and to say so. When the invented answer concerns a company’s pricing, policy or refund terms, courts, regulators and insurers now agree: The company owns what its model said.

For all PYMNTS AI coverage, subscribe to the daily AI Newsletter.