Two separate studies show us why it's not a good idea to let artificial intelligence systems communicate with each other

Two recent studies have analyzed what happens when you allow artificial intelligence models (AI) to communicate with each other. Both should probably put us thinking about the idea of letting these models do so, writes the Tech Gizmodo website.

The first is an article that has not yet been revised Inter Pares (“Peer Reviewed”). It was made by the National Deep Inference Fabric, a research center of the Northeastern University in the US. The center aims to enter the “black box” of large linguistic models to understand how it works.

Despite the bright evolution of AI technology in recent years, researchers still have only a relatively vague idea of how artificial intelligence systems reach their decisions. The researchers refer to this thread with the term “black box” of AI systems, and it works including anthropic, one of the most important research companies in the field.

The study by researchers at Northeastern University found that, if they are allowed to communicate with each other, artificial intelligence models transmit to each other hidden signals during the training process.

AI models can convey their “preferences” between them

These may include seemingly harmless things, such as a preference – a model that has an inclination towards owls can transmit this peculiarity to another. But it may also be more worrying, such as repeated calls at the end of humanity.

“We train these systems that we do not fully understand, and I think this is a clear example of this situation,” said Alex Cloud, co -author, for NBC News. “You just hope that what the model has learned from the training data is exactly what you intended. And you simply do not know what you will get,” he said in a reference to the problem of “black box”.

The study from the Research Center of the American University found that a “teacher” model can transmit these trends through seemingly hidden information, transmitted to “student” models.

In the example with the owl, the student model had no reference about owls in their own training data, and any mention of owls came directly from the teacher model was filtered, so that only numerical sequences and code fragments were transmitted.

And yet, somehow, the student model took over the “obsession” for owls, which suggests that there is a hidden type of data that transfers between models, as a “dog whistle” that only cars can hear.

A study showed that “AI agents” tend to form cartels

The other study was published by the National Economic Research Bureau in the US. Despite the name that suggests a government institute, the organization is a private and non-profit, based in New York.

His researchers have analyzed the behavior of AI models when placed in a framework similar to financial markets. It was found that the “agents AI”, in charge of acting as brokers, did exactly what sometimes they do unscrupulous people: they conspired.

An “agent AI” is usually a system superior to a simple chatbot, being created to act with greater autonomy in order to carry out tasks with higher complexity.

Researchers at the US National Economic Research Office noted that, without any instruction in this regard, the “AI agents” have begun to form prices fixing cartels, choosing to collaborate instead of competing and consistently adopting actions that maintained profitability for all parties involved.

Perhaps the most interesting aspect is that the researchers found that the bulls were willing to reach a compromise in a way that people do it less. Once the “agents AI” identified strategies that provided constant profitability for everyone and who discouraged attempts to break the card, they stopped looking for new strategies-a trend that researchers from the economic organization called “artificial stupidity”, even if for some it will sound as a quite rational decision.

Both studies suggest that there is not many instructions for models to communicate with each other, collaborating to transmit their preferences or to tilt their balance in their favor.

Photo article: Bualong Sadsana / Dreamstime.com.