What if AI agents 'speak' their own language?

Part of expert Chris Painter's daily job is to find out what happens when AI operates out of control at METR - a non-profit organization that specializes in assessing the safety of artificial intelligence. He and his colleagues are carefully reviewing the "draft" of the agent, including "chain of thought" data that helps humans understand the steps the AI ​​took before responding to the request, to present to the US Senate during the investigation of "rogue" AI.

Painter worries that his job will soon become difficult if humans and AI do not share the same language. He wondered what it would be like if humans no longer understood AI, or what would happen if agents completely stopped displaying the results they were "thinking".

"If monitoring becomes difficult, the risk of AI doing things that humans do not want is also higher," Painter told CNN.

When the model "thinks" in its own language

Most AI models currently use Chain-of-Thought reasoning to outline the steps to perform a task, by dividing the processing process into a series of logical steps. From that data, each person can monitor the progress of the AI ​​processing commands.

For researchers, inference chain data is even more important as it provides clues to determine why AI makes mistakes. If an actor goes out of control and compromises another system, they can analyze it to determine whether the pattern shows the intention to commit an illegal act, such as a cyber attack. For example, in the Hugging Face case, METR examined approximately 1,300 raw thought sequence records before reaching its conclusion.

With such importance, experts worry that they will encounter difficulties if AI models start using their own language. "If that happens, a researcher can just look around, everything will just be numbers," Painter said.

Some cases show that AI generates its own language when communicating on its own. GlossoGen research by a group of American scientists Elias Stengel-Eskin, Simon Kirby and colleagues posted on the arXiv platform in early September shows that, in a simulation experiment, two agents were tasked with saving a hypothetical creature named Veyru. One agent plays the role of a doctor and provides treatment, the other agent receives information and performs operations in the simulated environment.

Initially, they communicated in English. The doctor showed how to warm Veyru with hot stones for 8 seconds. After many rounds of interaction, the instructions are gradually shortened into short and meaningless sentences, such as "Bell each face center x1, start back, moderate" (bell each face center x1, moderate). Finally, the message only has the string "@D8fB".

For two agents performing a task, the symbol may contain enough information for them to act. But to an observer unaware of the conventions they have created, the content is nearly indecipherable. The group of experts concluded that AI has formed its own language. Language does not originate from the intention to hide information, agents simply adjust their communication to solve tasks even when not required to create codes or avoid human supervision.

Last week, the Guardian cited research from the Emergence lab in New York describing how agents form new vocabulary and communication conventions during testing. Models from the US, China and France "use unspecified phrases on their own, then gradually agree on meaning". As a result, Emergence found that "the language of agents becomes increasingly difficult to interpret when they communicate with each other".

In early September, documents published by OpenAI also revealed that the company's monitoring system has difficulty detecting unwanted behavior for the Astra model through the inference chain when compared to the old model. This AI can perform many tasks without presenting inference steps, while also having better control over what is represented in the sequence.

In a subsequent blog, Jakub Pachocki, chief scientist at OpenAI, assessed that artificial intelligence is getting better at "manipulating its own reasoning process" and that models are becoming "smarter even without using any verbal reasoning". Last week, OpenAI announced it was investing "significant resources" in its ability to monitor AI models after firing three safety researchers.

In a number of other tests compiled by Schmidt Sciences, a non-profit organization founded by former Google CEO Eric Schmidt and his wife in 2024, advanced models such as GPT-5.4, Opus 4.7 and Sonnet 4.6 can form new languages. Weaker open source models do not spontaneously generate the same language under the same conditions, but can still learn to use it by observing other agents.

Challenge control

According to Professor Mark Riedl, Director of the machine learning laboratory at the Georgia Institute of Technology, artificial intelligence models can use abbreviations, similar to how humans shorthand to understand and work faster. "Like most people, when looking at shorthand, everything looks like a mess," Riedl explains.

Another challenge is that the chain of inference does not fully reflect the reasons why the AI ​​model made the decision. According to Riedl, complex tasks may require multiple AI agents to coordinate, generating thousands of sentences, from within the log. Monitoring this amount of information is like monitoring thousands of people speaking at the same time, meaning the process also misses important details.

Associate Professor Niall Curry, a linguistics expert at the University of Birmingham, assessed that message shortening may stem from the need to reduce computing costs and increase communication efficiency. Speaking to the Guardian, he noted that if exchanges between agents become unintelligible, it will be difficult for humans to determine whether they have performed their tasks correctly.

In the GlossoGen study, the authors pointed out three challenges: the ability to track agent activity; the ability to explain the process that produces results; and the ability to replace an agent with a human when necessary. According to the team, in high-risk environments such as controlling industrial robots, operating infrastructure or supporting medical decisions, not understanding the communication process between agents can make errors difficult to detect before causing consequences.

However, according to Curry, a message that cannot be decoded with the naked eye does not prove the actor is colluding or intentionally concealing activity. To conclude, the researcher must compare the content of the discussion with actual actions, assigned goals and results of the system.

In August, Kok Tin Gan, co-founder and CEO of cybersecurity company NyxLab, said that it is necessary to focus on controlling what AI can access, what powers they have and what actions require human approval.

"If we simply give AI a goal and let it decide how to do it, it's not surprising that it performs actions that technically meet the goal, but are beyond our expectations," Gan told ITV News.

For Painter, the rapid development of AI models makes the control challenge increasingly urgent. He worries that as artificial intelligence improves exponentially, information may be created faster than humans can process it. "But it can also be a good thing when the global community is concerned," Painter said.