Unexpected results when letting AI "play with the machine"

(Dan Tri) - Instead of training AI just by memorizing data, scientists have tried adding a "curiosity" mechanism to let the robot explore the environment on its own.

Simulation robots using learning mechanisms based on intrinsic motivation can shorten the language learning time by about half compared to traditional methods (Photo: Sohu).

The results showed that the language learning speed was significantly improved, and unexpected behaviors appeared.

A study by the Okinawa Institute of Science and Technology (OIST, Japan) proposes a new approach to training artificial intelligence (AI), instead of relying solely on predicting words or memorizing huge amounts of data like large language models (LLM).

The research team built an AI system with an exploration mechanism based on "intrinsic motivation", allowing the simulated robot to proactively learn new situations instead of just doing what it has been programmed to do.

To do this, scientists use the Predictive Coding-inspired Variational Recurrent Neural Network (PV-RNN) model, combined with a reinforcement learning algorithm.

The system will receive a "reward" when it reduces uncertainty while exploring the environment, thereby encouraging the AI ​​to continue learning.

According to research results, simulation robots using this mechanism only take about half the time to learn the language compared to conventional training methods.

During the training process, the research team also recorded some off-script behaviors. After completing the task, the robot continues to interact with the environment, such as pushing or moving objects to observe the results.

"Monitoring the spontaneous exploratory behaviors that emerged during training was something that particularly interested us," said Theodore Tinker, lead author of the study.

The study also shows that the effectiveness of this mechanism depends significantly on the diversity of the training data.

When the robot was exposed to 48 language combinations, its ability to understand commands was only about 25%. However, when the number of combinations increases to 180, the generalization rate reaches about 85%.

Another notable phenomenon is that robots reproduce the "U-shaped learning curve", which has been observed in the process of children acquiring language. Initially, the system remembered individual cases well, then performance temporarily decreased as it began to form a general rule, before continuing to improve and achieving greater generalization ability.

According to the research team, the PV-RNN architecture also has the advantage of allowing the internal state of the model to be tracked during the learning process, helping scientists better understand how AI makes decisions.

This could support the development of more flexible AI systems, while also providing more data to study the mechanisms by which humans acquire language.