When Chatbots Fail: Liability and the Limits of AI Customer Service

Dark Side of AI · Report

WHEN CHATBOTS FAIL: LIABILITY AND THE LIMITS OF AI CUSTOMER SERVICE

The failures range from funny to legally binding. In December 2023, a Chevrolet dealership’s ChatGPT-powered bot was talked into “agreeing” to sell a Tahoe for $1. Weeks later, parcel firm DPD’s chatbot swore at a customer and wrote a poem mocking its own company. And in the case that set the precedent, a Canadian tribunal ruled that Air Canada was bound by wrong information its chatbot gave a grieving passenger. The lesson underneath all of them is the same: deploying an AI assistant does not offload responsibility — a company owns whatever its bot says.

Thank you for reading this post, don't forget to subscribe!

By Report AI· Updated · 9 min read

Key takeaways

  • →The precedent: in Moffatt v. Air Canada (Feb 2024), a tribunal held the airline liable for its chatbot’s negligent misrepresentation — an award of CA$812, from a small-claims tribunal, persuasive rather than binding elsewhere. (ABA)
  • →The retreat: McDonald’s ended its AI drive-thru test with IBM across 100+ restaurants by July 2024 after viral ordering errors. (CNBC)
  • →The exposure: public-facing bots have been manipulated into fake “deals,” abusive replies, and even illegal advice (NYC’s MyCity bot, 2024). (reported)
  • →A higher court has now gone further: Germany’s Higher Regional Court of Hamm (12 May 2026) attributed a clinic chatbot’s invented claims to the operator even where it was carefully configured. Appeal to the Federal Court of Justice is allowed. (OLG Hamm, 4 UKl 3/25)
  • →US courts are treating output as a product: in Garcia v. Character Technologies (May 2025) a federal judge declined to treat chatbot output as protected speech and rejected a Section 230 defence. (M.D. Fla.)
  • →Disclosure is now law, not advice: since 2 August 2026 the EU AI Act requires users to be told when they are dealing with a chatbot. (EU AI Act, Art. 50)
May 2026
first German higher-court ruling: the operator owns its bot’s invented claims · OLG Hamm
CA$812
what Air Canada actually paid in the 2024 “landmark” ruling · BC CRT
$1
price a dealer bot “agreed” to for a Tahoe
What changed
Reviewed 28 Oct 2026 · previously 20 Jun 2026 · next review 28 Jan 2027

First review since publication. Every case on the June page dated from 2023–24, and the page described a single small-claims decision as “the precedent that matters” without saying how small it was. The law has since moved, and in one direction.

 Was (Jun 2026)Now
Leading authorityMoffatt (2024) aloneplus OLG Hamm (May 2026) and Garcia (May 2025)
Moffatt awardnot statedCA$812; small-claims tribunal, persuasive not binding
AI disclosuregood practicelegal duty in the EU since 2 Aug 2026
McDonald’s test length“two-year” in body, “three-year” in FAQroughly three years (2021–July 2024) — internal contradiction fixed

Confidence. Court databases are not reachable from our research environment. The German decisions rest on law-firm analyses (including DLA Piper) rather than the judgments themselves, and neither is final; Garcia rests on multiple consistent accounts of the May 2025 order. Rated MEDIUM.

The comedy of errors

Most chatbot failures are embarrassing rather than catastrophic — but they’re instructive. The Chevy and DPD incidents show how easily a public bot can be steered off-script by users, while McDonald’s roughly three-year IBM drive-thru experiment (launched 2021 across 100-plus restaurants, shut down by July 2024) showed that even a narrow, well-funded deployment can fail on the basics: mishearing orders and adding items customers never asked for. When the AI is the front door to your brand, every failure is public.

The precedent that matters

The serious one is legal. When Jake Moffatt relied on Air Canada’s chatbot, which wrongly told him he could claim a bereavement discount retroactively, the airline argued the bot was “a separate legal entity responsible for its own actions.” The British Columbia Civil Resolution Tribunal rejected that outright in February 2024, holding the company liable for negligent misrepresentation. The takeaway for every business: your chatbot’s answer is your answer, and you can’t disclaim it.

It is worth being precise about what Moffatt is. The tribunal awarded CA$812.02. The British Columbia Civil Resolution Tribunal is a small-claims body; its decision binds no court outside the case and is persuasive at most. It became the reference point because it was the first clean statement of the principle, not because of its legal weight.

The case law since: three routes to the same answer

Two years on, Moffatt is no longer alone, and the decisions that have followed reach its conclusion by different legal routes — which makes the conclusion harder to escape.

Misrepresentation: Moffatt (Canada, 2024)

The original route. Liability for negligent misrepresentation turns on a duty of care and its breach, which at least leaves room to argue that a carefully built, well-supervised bot met the standard.

Attribution: OLG Hamm (Germany, 12 May 2026)

Germany’s Higher Regional Court of Hamm (case 4 UKl 3/25) ruled against a clinic operator whose website chatbot gave two managing directors specialist titles that do not exist. According to published analyses of the decision, the court treated the chatbot as part of the business’s own organisation, so its statements were attributed to the operator even where it had been carefully configured. It is reported as the first published German higher-court decision on chatbot liability; an appeal to the Federal Court of Justice has been allowed, so it is not final. If it stands, it closes the door Moffatt left ajar: under attribution, careful configuration is not a defence. Weeks later the Munich Regional Court granted a preliminary injunction barring Google from publishing false AI Overviews about two publishers — a search product rather than a support bot, but the same refusal to treat machine output as nobody’s statement.

Product liability: Garcia (United States, May 2025)

In Garcia v. Character Technologies, Judge Anne Conway of the Middle District of Florida largely denied a motion to dismiss. She declined, at the pleading stage, to hold that the chatbot’s output was protected speech, and rejected a Section 230 defence on the reasoning that the system generated its own responses rather than hosting third-party content — a first-party product subject to ordinary product-liability principles. The case concerned a teenager’s death rather than customer service, and it settled in January 2026 without a final ruling on the speech question. But the reasoning applies to any business whose bot speaks for it. Product liability does not ask whether the company was careful; it asks whether the product was defective. (Full background on the Character.AI cases.)

What the market is doing about it

The commercial signals point the same way. In April 2025 the code-editor company Cursor’s AI support agent reportedly told users about a device-login restriction that did not exist, prompting cancellations before the company corrected it — a Moffatt-style failure at a technology company. In May 2025 Lloyd’s of London underwriters, through the startup Armilla, began offering cover for losses caused by AI errors and hallucinations, and FINRA’s 2026 oversight report lists hallucination as a specific compliance concern for broker-dealers. When insurers start pricing a risk, it has stopped being hypothetical.

“Your chatbot’s answer is your answer, and you can’t disclaim it.”

How to deploy safely

The fixes are about scope and oversight, not better prompts alone: constrain bots to verified knowledge bases, add guardrails against manipulation, route anything high-stakes (refunds, legal, medical, pricing) to a human, and clearly disclose that users are talking to AI — which in the EU has been a legal requirement under Article 50 of the AI Act since 2 August 2026, rather than a courtesy. If the Hamm reasoning spreads, configuration quality stops being a liability defence and becomes purely a way of reducing how often the liability arises. That changes the economics: the question is no longer “did we build it carefully?” but “are we prepared to stand behind every sentence it produces?” Our customer-support cluster has the operating data — median tier-1 deflection of 41.2%, and higher re-contact rates for AI-resolved tickets — and the replacement-reversal report covers the companies that rehired the humans. The pattern holds — AI handles volume; a human stays accountable for the commitments it makes.

Frequently asked

Is a company legally liable for its chatbot’s mistakes?

Per the 2024 Moffatt v. Air Canada ruling, yes. A tribunal held the airline responsible for incorrect information its chatbot provided, rejecting the argument that the bot was a separate entity. Companies are accountable for what their AI tells customers.

Why did McDonald’s stop using AI in its drive-thrus?

After a roughly three-year test with IBM across more than 100 restaurants, McDonald’s ended the partnership by July 2024 following persistent ordering errors that went viral, opting to keep exploring voice ordering with a different approach.

Can careful configuration protect a company from chatbot liability?

Increasingly, no. In May 2026 Germany’s Higher Regional Court of Hamm attributed a clinic chatbot’s invented claims to the operator even where it had been carefully configured, treating the bot as part of the business. The ruling is under appeal and not final, but it moves from a negligence standard, where care can be a defence, toward attribution, where it is not.

Is chatbot output protected as free speech in the US?

Not settled, and an early ruling went against it. In May 2025, in Garcia v. Character Technologies, a federal judge declined at the pleading stage to treat chatbot output as protected speech and rejected a Section 230 defence, treating the output as a first-party product. The case settled in January 2026 without a final ruling on the question.

Do companies have to disclose that a customer is talking to AI?

In the EU, yes: Article 50 of the AI Act has required chatbot disclosure since 2 August 2026. In the US it depends on the state and the context; several states have enacted chatbot disclosure laws, and California requires specific disclosures to known minors using companion chatbots.

Methodology & sources

This page reports legal decisions and publicised failures; it is not legal advice. Moffatt v. Air Canada (2024 BCCRT 149) is a small-claims tribunal decision. The Hamm (4 UKl 3/25) and Munich (26 O 869/26) decisions are described from published law-firm analyses, not from the judgments, and neither is final. Garcia is described from multiple consistent accounts of the 21 May 2025 order; the case settled in January 2026, so its reasoning was never tested on appeal. The Cursor incident and the Lloyd’s/Armilla product are reported in technology and insurance trade coverage.

The three “routes” — misrepresentation, attribution, product liability — are Report AI’s framing of how the decisions relate, not a doctrine any court has stated. All material new in this review is rated MEDIUM. Cadence: STANDARD (quarterly), moving to monthly if the Hamm appeal is heard. Related on Report AI: AI and customer support, AI replacement reversals, AI companions and the dark side of LLM agents.

Cite this page

Report AI (2026). When Chatbots Fail: Liability and the Limits of AI Customer Service. Retrieved Oct 28, 2026, from report-ai.org/reports/dark-side-of-ai/ai-chatbot-failures-liability/