OpenAI has revealed that a Chinese law enforcement agency attempted to use ChatGPT to review and refine reports detailing a large-scale online operation targeting critics of the Chinese government. The disclosures, included in a new OpenAI threat report, suggest a coordinated effort to track, harass, and silence dissenting voices both inside and outside China.
Use of AI in Influence Operations
The report describes a single account that repeatedly engaged ChatGPT to process information on “cyber special operations.” At one point, the account attempted to plan a propaganda campaign against Japanese Prime Minister Sanae Takaichi, although ChatGPT refused to assist. Subsequent prompts indicated the operation had proceeded independently.
OpenAI notes that uploaded reports imply the threat actors conducted multiple prior campaigns, involving hundreds of staff, thousands of fake social media accounts, and local Chinese AI tools to amplify their operations.
Methods of Digital Harassment
The campaigns reportedly employed a range of tactics:
- Mass posting and content generation on social media
- Filing fraudulent complaints to disrupt dissident accounts
- Document forgery and impersonation of officials for intimidation
- Email campaigns to U.S. state officials and financial analysts, often steering conversations to platforms like Zoom, WhatsApp, or Teams
One cluster of accounts sought information about U.S. individuals, forums, and federal office locations. The actors also used AI tools to draft emails under the name of a Hong Kong-based company, while masking their activity via VPNs and using Simplified Chinese prompts.
Limited Use for Offensive Hacking
While OpenAI identified multiple influence and scam operations, including those linked to Russia-aligned groups and Cambodian romance scams, there was no evidence of ChatGPT being used for direct cyberattacks. Instead, the AI model primarily served as a tool to automate isolated tasks, translate content, and improve messaging efficiency.
“Threat actors are experimenting with AI to amplify existing operations rather than conducting novel attacks,” OpenAI stated. In many cases, multiple AI tools—including local Chinese models like DeepSeek—were used at different stages of campaigns.
Implications for AI Security
OpenAI’s findings underscore the dual-use nature of AI technologies. Tools like ChatGPT can accelerate both legitimate and malicious operations, offering rapid content generation, social engineering support, and influence amplification.
Despite these revelations, OpenAI confirmed that no automated offensive attacks were conducted through ChatGPT, highlighting that threat actors are still exploring the technology’s potential in targeted ways.
Key Takeaways
- Chinese law enforcement actors reportedly used AI to coordinate global harassment campaigns.
- Hundreds of human operators and thousands of fake accounts amplified these operations.
- ChatGPT and other AI tools were used for content creation, translation, and planning, but not for direct hacking.
- Threat activity often spans multiple AI models and platforms, demonstrating a sophisticated hybrid approach.
The report illustrates the growing need for AI governance, monitoring, and safeguards, particularly as foreign actors increasingly leverage AI for influence and manipulation on a global scale.
