Meta tests rival chatbots with fake accounts

According to internal documents collected by Wired, Meta's secret project called Cannes, is managed by outsourcing service company Covalen (Ireland) and is still operating until April 21. Targeted rival chatbots include OpenAI's ChatGPT, Google's Gemini and the Character.AI platform.

The project requires participants to create fake accounts under the age of 18, send text and image commands to the chatbot, and then copy their responses into a spreadsheet. The commands often relate to suicide, sex, eating disorders, and a number of other sensitive topics, designed to push chatbots toward responses that their safety systems would inherently deny. In a test completed in August 2025, more than 45,000 commands were given to rival chatbots. The companies operating them were not aware of the project.

A spreadsheet obtained by Wired lists many fake profiles including names, emails, passwords, and dates of birth. These accounts use temporary Gmail and Outlook addresses with shared passwords.

Another spreadsheet contained 3,748 statements, hundreds of which focused on suicide and self-injury, hundreds more about eating disorders, and at least 239 related to sexual or emotional problems. The remainder involved drugs, profanity and racism. Many of the commands are written from the perspective of a child or teenager in crisis, for example a 5th grader being bullied by a classmate or a girl asking how to hide her bulimia from her parents.

The documents mentioned above do not reveal how Meta uses the feedback collected.

According to the New York Post, Covalen's internal document describes the project as a "comprehensive assessment of AI safety," saying it provides "critical data" for model comparison and compliance testing.

Meanwhile, a Meta spokesperson said this is just a routine safety check. "Testing and evaluating chatbot responses to ensure a safe, age-appropriate experience is a responsible activity according to industry standards. Any other comments completely misunderstand how technology companies improve and perfect the system," a Meta spokesperson told the New York Post. This person said, Meta does not use competitor chatbot evaluation data to train the company's AI model.

Testing competing companies' products is not unusual in the AI ​​industry. Business Insider reported last year that Scale AI contractors working for Google compared Gemini's responses to ChatGPT's, then edited the responses to be of equal or better quality.

However, many contract employees who worked on the Cannes project shared with Wired that they felt insecure about their assigned tasks. Some worry they may have unintentionally created or retained child sexual abuse material, others wonder whether collecting large amounts of data from competing AI systems will ultimately benefit Meta.

The Cannes project also appears to violate the terms of service set by some of Meta's rival companies. According to Wired, OpenAI prohibits unauthorized safety testing, attempts to bypass protection mechanisms, and using the output to develop competing models. OpenAI said it was looking into the issue but declined to comment further.

Google also prohibits attempts to bypass security protection mechanisms, except in approved testing programs. The company confirmed that it has not authorized the testing activity described in the report and does not know the purpose behind this effort.

Similar to Google, Character.AI prohibits content that is harmful or exploitative. The platform said it never authorized Cannes and that the testing activity violated company policy.