Deepfake Scams Up 2,000% Force Banks to Rethink AI Oversight

At the Asian Banking & Finance and Insurance Asia Summit in Singapore on 1 July 2026, Ashwini Karandikar, principal at Oliver Wyman, told financial executives that AI oversight in banking has become urgent because AI-driven fraud, adversarial attacks and machine-speed decision making are outrunning traditional risk controls. Deepfake scam volumes alone have grown “more than 2,000%” in the past three years, she said. Her prescription was blunt: deploy AI to police AI, with humans kept firmly accountable.

Karandikar framed the moment as one in which banks adopted AI faster than their controls could follow, and the consequences are now arriving together. Fraud, model manipulation and autonomous agents trained on similar data are, in her framing, a single escalating problem that arrives together. The fix she sketched is concrete enough to be tested this year.

Why Deepfakes Are the Loudest Alarm

Karandikar’s headline figure, a “more than 2,000%” rise in deepfake scam volumes across the past three years, anchors the case for urgency. The number captures a category of fraud that includes synthetic voice calls impersonating executives, deepfake video verifications, and AI-generated documents convincing enough to pass human review. Oliver Wyman’s own 2024 analysis of Southeast Asian fraud found total scam losses in the region exceeded $5 billion, with customers bearing the vast majority. Total losses, not attempt counts, are what should focus board attention.

What has changed since is volume, not technique. Synthetic media is now cheap, fast and good enough that a single employee screen check no longer separates a real customer from a fabricated one. The shape of the threat has shifted from “is this real?” to “how quickly can we tell?”.

Detection systems are losing ground too. Banks in Singapore, Thailand and Malaysia have spent the past two years adding real-time scam detection, kill-switches and authentication controls, yet false positive rates at many institutions still exceed 80%, according to Oliver Wyman’s regional analysis. Each false alarm trains staff to treat alerts as background noise. That is exactly what an attacker wants when a real deepfake gets through.

The Quieter Threat of Adversarial AI

The deeper risk, in Karandikar’s framing, is adversarial AI: bad actors reverse-engineering a financial institution’s own models to design attacks the target system itself reads as legitimate. She described a client case in document processing where attackers embedded hidden instructions inside financial reports, forms and tax filings. The instructions are invisible to human reviewers but readable by the AI that processes the documents, allowing attackers to hijack the workflow.

She named the technique prompt injection, and pointed to a real precedent. Last year, Aim Security disclosed EchoLeak (CVE-2025-32711), a zero-click vulnerability in Microsoft 365 Copilot in which a single crafted email could cause Copilot to silently exfiltrate internal data with no user interaction. Microsoft patched the flaw server-side in May 2025. The lesson Karandikar drew was general: any AI that ingests external content can be steered by content its operators never vetted.

In the financial document case Karandikar described, the hidden instructions sat inside markdown formatting or white-on-white text, markings a human reader would never notice but a parser would follow exactly. The same technique, sometimes called indirect prompt injection, has appeared in customer support inboxes, in scraped web pages fed to summarisation tools, and in resumes uploaded to applicant-tracking systems. Any model that reads untrusted text is exposed to it.

Regulators have begun drafting rules for synthetic media, but financial institutions are still operating under rules written for the pre-AI era in most jurisdictions. The gap between what an attacker can do and what a compliance team can recognise is widening, not narrowing. Karandikar’s argument is that AI oversight is the only realistic way to close that gap in 2026, and human-only review cannot match the speed of attack.

When AI Agents Stop Hesitating

Beyond fraud, Karandikar flagged a systemic risk in which banks and insurers now run AI agents that scan signals, rebalance exposures and hedge risk, often trained on similar data, and without the hesitation built into human decision making. “Humans hesitate, humans panic, humans call someone,” she said, noting that traditional controls for credit deterioration, mispricing and model drift are inherently slow. The slow pace allows risk to “accrue on your balance sheet” before it is caught, in her words.

The structural problem is convergence. When many institutions use agents trained on overlapping data and reacting to the same signals, the resulting decisions can reinforce each other rather than cancel out. A coordinated sell-off triggered by similar agents across multiple desks can move prices in ways no individual actor intended. An agent that makes a bad trade makes it faster than the control room can finish its morning briefing. Traditional model risk management was built around quarterly reviews and annual audits, while an agent that rebalances every few minutes waits for neither.

The Accountability Gap Behind Vendor AI

Compounding the problem is an accountability gap. Many banks do not fully know what AI they are touching, particularly through third-party vendors who run their own AI tools on the institutions’ behalf. A credit memo drafted by a vendor’s model, fed back through the bank’s own approval system, may leave no clean audit trail of which model said what. The bank’s risk register may not even list the model that produced the decision.

This gap matters legally as well as operationally. When an AI-generated credit decision goes wrong, the question of who owns the call, the bank, the vendor, or the model itself, has no settled answer in most jurisdictions. The EU’s revised AI compliance timeline, which pushed high-risk obligations to December 2027, leaves a window in which banks must operate under rules their vendors may not yet follow.

The same dynamic shows up in consumer-facing tools. A UK MP’s lawsuit against xAI over Grok’s deepfake outputs is testing whether AI developers can be held liable for design choices that enable fraud. The case is one of several pushing courts to fill the accountability vacuum that Karandikar flagged. Until liability rules settle, banks absorb the risk either way.

Project Glasswing and the AI-Policing-AI Bet

Karandikar proposed using AI to police AI. “No risk team can review every AI-generated credit decision,” she said. She pointed to Project Glasswing, an Anthropic initiative involving JPMorgan Chase and other technology developers, describing it as a frontier AI model built to detect more than 10,000 critical vulnerabilities across the world’s most widely used software. The framing is borrowed directly from the cybersecurity playbook Karandikar argued finance now needs.

The actual scope is broader than that description. The project’s underlying model, Claude Mythos Preview, has scanned more than 1,000 open-source projects and identified 23,019 issues, of which 6,202 were high- or critical-severity vulnerabilities. Anthropic and six independent security firms assessed 1,752 of those findings, and more than 90% were validated as true positives.

• Partners: AWS, Anthropic, Apple, Broadcom, Cisco, CrowdStrike, Google, JPMorganChase, the Linux Foundation, Microsoft, NVIDIA and Palo Alto Networks
• CyberGym benchmark: Mythos Preview 83.1% versus Claude Opus 4.6 at 66.6%
• Anthropic commitment: up to $100 million in Mythos Preview usage credits plus $4 million in direct donations to open-source security organisations

Mythos Preview’s reach is concrete. Anthropic’s announcement said the model found a 27-year-old flaw in OpenBSD that allowed any machine running the operating system to be remotely crashed by a single connection, a 16-year-old vulnerability in FFmpeg that automated tools had hit five million times without catching, and chained flaws in the Linux kernel that escalated from ordinary user access to full machine control. All three were patched once reported.

The pattern Karandikar described in finance, AI agents outpacing human review, is the same pattern Anthropic cited in its April 7, 2026 announcement. “The window between a vulnerability being discovered and being exploited by an adversary has collapsed,” CrowdStrike said in a partner statement. What once took months now happens in minutes with AI, and the defenders’ answer has to move at the same speed. Anthropic’s own announcement noted that no company, including itself, has yet developed safeguards strong enough to keep such models from being misused.

The Two Steps Karandikar Urged This Year

Karandikar stressed that AI oversight should not replace human judgment. Institutions, she said, need to define where AI can act autonomously, where it must seek approval, and who is accountable when a model acts. Her two concrete steps for the year were:

1. Map every point where AI influences a risk-bearing decision, including decisions made through third-party vendors and embedded models the institution may not directly manage.
2. Design explicitly for failure, shifting the governing question from “will the system fail?” to “when it does, how can it fail safely?”.

The second step is the harder one. A bank’s traditional controls were designed to catch errors after they happened, not to constrain agents that act in milliseconds. AI agents can compound errors faster than legacy controls can flag them, so failing safely has to be designed in, not bolted on. Karandikar’s argument is that resilience, like fraud detection, has to be AI-native to keep up. That is the same lesson cybersecurity teams have learned the hard way, with Anthropic now flagging patching, not discovery, as the bottleneck in modern vulnerability management.

Her broader point is that the threat is no longer theoretical: deepfake fraud is in production, adversarial AI has shown up in document workflows, and AI agents are making decisions without human hesitation. The question for banks is whether their oversight catches up before their agents make a call no human is around to catch. Anthropic itself acknowledged the same dynamic in software security, describing a world in which the gap between finding and exploiting a vulnerability has collapsed from months to minutes.

Frequently Asked Questions

What did Ashwini Karandikar say about deepfake scams?

At the Asian Banking & Finance and Insurance Asia Summit in Singapore on 1 July 2026, Karandikar, principal at Oliver Wyman, said deepfake scam volumes have grown “more than 2,000%” in the past three years. She used the figure to frame the broader case for AI-powered oversight in financial services, beyond fraud detection alone.

What is Project Glasswing?

Project Glasswing is an Anthropic-led initiative announced on April 7, 2026, that gives AWS, Apple, Broadcom, Cisco, CrowdStrike, Google, JPMorganChase, the Linux Foundation, Microsoft, NVIDIA and Palo Alto Networks access to a frontier model called Claude Mythos Preview. As of late May 2026, the project and its partners had identified more than 10,000 high- or critical-severity software vulnerabilities, including a 27-year-old flaw in OpenBSD and a 16-year-old flaw in FFmpeg.

What is the EchoLeak vulnerability Karandikar mentioned?

EchoLeak (CVE-2025-32711) is a zero-click vulnerability in Microsoft 365 Copilot disclosed by Aim Security in June 2025. It was the first publicly documented zero-click exploit against a major AI assistant, and Microsoft applied a server-side patch in May 2025 without requiring customer action. The attack class it demonstrated, indirect prompt injection via hidden instructions embedded in retrieved content, applies broadly to any RAG-based assistant that ingests untrusted text.

Why are AI agents a systemic risk in banks?

Karandikar warned that banks now run AI agents that scan signals, rebalance exposures and hedge risk on their own, often trained on overlapping data and lacking the human pause that lets a trader pick up the phone. The faster the agent moves, the less time a control room has to catch it, which is why she framed the issue as systemic. Regulators have not yet caught up to the speed of agent-led decisions.

What two steps did Karandikar urge banks to take?

She asked institutions to take two concrete steps this year: first, map every point where AI influences a risk-bearing decision, including decisions made through third-party vendors and embedded models the institution does not directly manage. Second, design explicitly for failure, shifting the governing question from “will the system fail?” to “when it does, how can it fail safely?”. The second step reframes resilience from a checkbox into a design constraint.