The Dark Side of AI · Report

Thank you for reading this post, don't forget to subscribe!

Home / Indexes / The Dark Side of AI / Deepfakes & Fraud

Deepfakes, Scams & AI Fraud 2026

Generative AI has industrialised deception. A single deepfake video call drained roughly US$25M from one firm — and Deloitte projects US generative-AI fraud losses reaching about $40B by 2027, up from an estimated $12.3B in 2023.

Updated July 2026. Every figure below is linked to its primary source and dated.

The $25 million video call

In February 2024, CNN and the South China Morning Post reported that a finance employee at the Hong Kong office of the engineering group Arup was tricked into transferring roughly US$25M across 15 transactions. The employee had joined a video conference with what appeared to be the company’s UK-based chief financial officer and several colleagues. Every other participant on that call was an AI-generated deepfake — synthetic recreations of real staff, built from publicly available footage. Hong Kong police confirmed the case, describing it as one of the largest losses attributed to a deepfake-enabled scam to date.

The Arup case matters less as an outlier than as a template. It combined three ingredients now widely available: convincing real-time face and voice synthesis, corporate information scraped from the open web, and the social pressure of an apparently senior executive issuing an urgent instruction. Europol warned as early as 2022 that synthetic media would erode the reliability of audio and video as evidence of identity; the Arup heist showed the operational form that warning would take.

By the numbers

The macro picture is one of steep, compounding growth. In 2024 the Deloitte Center for Financial Services estimated that generative-AI-enabled fraud losses in the United States could climb from about $12.3B in 2023 to roughly $40B by 2027 — an implied compound annual growth rate near 32%. On the attempt side, the Sumsub Identity Fraud Report has documented deepfake fraud attempts rising roughly tenfold year over year in parts of the market, with increases exceeding 3000% in some regions across 2022–2023, making deepfakes one of the fastest-growing categories of identity fraud.

These figures measure different things — realised losses, forecast losses, and attempt volumes — and should not be summed. Read together, though, they point the same direction: the cost of producing a convincing fake has collapsed while the payoff for a successful one has not, and attackers are responding to that arithmetic.

Voice cloning and the new social engineering

Fraud does not need a full video to work. The US Federal Trade Commission has repeatedly warned consumers that criminals can clone a voice from a few seconds of audio — often lifted from social media — to power “family emergency” or “grandparent” scams, in which a caller impersonates a relative in distress and demands money. In 2024 the FTC ran its Voice Cloning Challenge, a public competition soliciting technical and policy defences against exactly this misuse, an unusual step that signals how seriously the agency treats the threat.

Voice cloning is potent because it attacks trust rather than technology. A cloned voice bypasses the instinct to verify: the target hears a familiar person, feels urgency, and acts before checking. The same mechanism scales into the enterprise, where a spoofed executive voicemail or a live cloned-voice call can authorise payments that a written request never would. Because the raw material — a person’s recorded speech — is effectively public for anyone with an online presence, the defensive burden shifts from securing data to verifying identity at the moment of the request.

Provenance and the fight back

Because after-the-fact detection is fragile, much of the serious defensive effort has shifted upstream to provenance — proving where a piece of media came from rather than guessing whether it is fake. The Coalition for Content Provenance and Authenticity (C2PA), whose Content Credentials standard is backed by Adobe, Microsoft and others, attaches tamper-evident metadata recording how an image or video was created and edited. Google’s SynthID embeds imperceptible watermarks into AI-generated images, audio and text so that platforms can flag synthetic content at scale. Neither is a complete answer — metadata can be stripped and watermarks contested — but together they aim to make authenticity checkable by default.

The most effective near-term defence, however, is procedural. Banks and enterprises increasingly mandate out-of-band verification callbacks: any payment instruction arriving by video, voice or email is confirmed through a separate, pre-established channel before funds move. The lesson of the Arup case is that no synthetic face or cloned voice defeats a policy that requires a call back to a known number. From a mid-2026 vantage, the plausible trajectory is a layered regime — provenance signals, watermark detection and mandatory human verification — rather than any single technology that ends the threat.

Frequently asked questions

How much is AI deepfake fraud costing?
The Deloitte Center for Financial Services estimated in 2024 that generative-AI-enabled fraud losses in the United States could reach roughly $40B by 2027, up from about $12.3B in 2023 — a compound annual growth rate near 32%. Individual incidents can be large on their own: the Arup deepfake heist reported by CNN in February 2024 cost about US$25M.

Can deepfakes be detected reliably?
Not yet, and not consistently. Detection is an arms race: models trained on older generators lose accuracy against newer ones, and vendors including iProov and Entrust have reported a surge in injection attacks that bypass liveness checks. Provenance approaches such as C2PA Content Credentials and Google SynthID watermarking aim to make authenticity verifiable, but no method offers a guarantee, so verification procedures remain essential.

How do I protect against voice-clone scams?
The US Federal Trade Commission advises treating any urgent request for money or information with suspicion, even when the caller sounds like a relative or a boss. Hang up and call the person back on a number you already know, or agree a private family code word in advance. For businesses, mandatory out-of-band verification callbacks before releasing funds are the single most effective safeguard.

Sources