WHEN CHATBOTS FAIL: LIABILITY AND THE LIMITS OF AI CUSTOMER SERVICE

The failures range from funny to legally binding. In December 2023, a Chevrolet dealership’s ChatGPT-powered bot was talked into “agreeing” to sell a Tahoe for $1. Weeks later, parcel firm DPD’s chatbot swore at a customer and wrote a poem mocking its own company. And in the case that set the precedent, a Canadian tribunal ruled that Air Canada was bound by wrong information its chatbot gave a grieving passenger. The lesson underneath all of them is the same: deploying an AI assistant does not offload responsibility — a company owns whatever its bot says.

Thank you for reading this post, don't forget to subscribe!

The comedy of errors

Most chatbot failures are embarrassing rather than catastrophic — but they’re instructive. The Chevy and DPD incidents show how easily a public bot can be steered off-script by users, while McDonald’s two-year IBM drive-thru experiment (launched 2021, shut down by July 2024) showed that even a narrow, well-funded deployment can fail on the basics: mishearing orders and adding items customers never asked for. When the AI is the front door to your brand, every failure is public.

The precedent that matters

The serious one is legal. When Jake Moffatt relied on Air Canada’s chatbot, which wrongly told him he could claim a bereavement discount retroactively, the airline argued the bot was “a separate legal entity responsible for its own actions.” The British Columbia Civil Resolution Tribunal rejected that outright in February 2024, holding the company liable for negligent misrepresentation. The takeaway for every business: your chatbot’s answer is your answer, and you can’t disclaim it.

“Your chatbot’s answer is your answer, and you can’t disclaim it.”

How to deploy safely

The fixes are about scope and oversight, not better prompts alone: constrain bots to verified knowledge bases, add guardrails against manipulation, route anything high-stakes (refunds, legal, medical, pricing) to a human, and clearly disclose that users are talking to AI. The pattern holds — AI handles volume; a human stays accountable for the commitments it makes.