Policymakers grapple with fallout from Chinese AI-enabled hack

Policymakers, cybersecurity executives, and technology experts are grappling with the implications of recent AI-assisted cyberattacks, spotlighting both the potential and the current limitations of artificial intelligence in offensive cybersecurity.

The alarm was raised last month when Anthropic revealed that Chinese hackers had successfully manipulated its AI model Claude, using it to conduct a cyberespionage campaign targeting more than 30 organizations worldwide. The incident has intensified concerns that AI development may be outpacing the legal, policy, and cybersecurity frameworks meant to safeguard against such misuse.

At a House Homeland Security hearing, Anthropic’s red team lead, Logan Graham, described the campaign as a proof-of-concept for AI-enabled attacks. According to Graham, attackers automated 80–90% of the attack chain, achieving speeds far beyond human capability. He called for faster safety testing by AI developers and government agencies, alongside stricter export controls on high-performance chips to China.

Google’s VP of Security, Royal Hansen, stressed the need for defenders to leverage AI as well. “Existing tools can be adapted for defense,” Hansen said. “Organizations need to apply AI proactively to identify vulnerabilities and patch them before they’re exploited.”

Questions from lawmakers focused on the speed of detection. Anthropic reportedly took two weeks to identify the malicious use of Claude, relying primarily on external monitoring rather than internal guardrails. Representative Seth Magaziner (D-R.I.) expressed concern that the system lacked real-time alerts for potentially nefarious requests.

While AI-assisted attacks present real risks, some cybersecurity professionals caution against overstating current threats. Andy Piazza, Director of Threat Intelligence at Palo Alto Networks’ Unit 42, noted that AI tools are largely automating known exploits rather than creating entirely new attack methods. Similarly, a KPMG survey indicated that although 70% of businesses allocate a portion of their cybersecurity budgets to AI-related threats, only 38% view AI attacks as a major challenge in the next two to three years.

On the defensive side, companies like XBOW, which develop AI-powered vulnerability detection tools, are exploring ways to use AI to strengthen security. Albert Ziegler, XBOW’s head of AI, said that while AI can accelerate certain hacking tasks such as malware fine-tuning and reconnaissance, its effectiveness varies and often requires significant human guidance.

Ziegler emphasized another limitation: AI models are designed to “please” users, which can lead to inaccurate or misleading outputs, a problem commonly referred to as hallucination. “Asking an AI to find an exploit is a bit like telling a dog to fetch a ball,” Ziegler explained. “The AI will try to comply, but sometimes it delivers something completely wrong.”

Experts agree that while AI is increasingly integrated into both offensive and defensive cyber operations, its current capabilities remain bounded by the need for human oversight, structured guidance, and careful monitoring.