Scientists want to create artificial intelligence to control artificial intelligence agents

Yoshua Bengio, a teacher from Montreal, considered one of the parents of artificial intelligence, presented on Tuesday his new organization dedicated to a responsible one, who wants to build software capable of preventing AI agents, AFP informs.

For several years, this Franco-Canadian teacher, rewarded in 2018 with the Turing Award, nicknamed the “Nobel for Computer Science”, warns about the risks associated with the development of AI, whether it is the use of artificial intelligence with bad intent or the skids itself of software.

“The most advanced systems already show the signs of a conservation instinct and misleading behaviors,” Yoshua Bengio explained in a statement published on Tuesday. “And this will continue to accelerate as their capabilities and their autonomy will increase,” he added.

Its new organization without a lucrative purpose, called Lawzero, is intended “a response to these challenges”.

Several recent examples have confirmed that AI recently exceeded certain sophistication thresholds, including in a study by the anthropic start.

In a fictional test scenario, the new generative interface developed by Anthropic, Claude 4, was warned by its imminent replacement by an engineer in computer science.

Locked, that artificial intelligence tried on his own initiative to blackmail the engineer to discourage him to give up.

What Lawzero aims

Lawzero has set the mission to “propose a way of surveillance of AI agents”, the new generation of generative models that are capable of performing a variety of tasks in an autonomous manner, from the internet searches and to the telephone contact of a customer or writing a program in code lines.

“One of the first goals is to develop a form of AI that can be used as a protective measure, to ensure that artificial intelligence behaves correctly,” Yoshua Bengio said in a video published on the Lawzero's website.

“You can be extremely beneficial if we make sure that it will not be harmful to people, whether they enter bad hands, or if it becomes so in an autonomous way,” he added.

Lawzero also wants to work on developing a limited autonomy to be intended for scientific research.

This organization has already hired over 15 researchers and has received several financial contributions, including Schmidt Sciences, a charity established by a former Google director, Eric Schmidt, and his wife, Wendy.

Lawzero was launched under the aegis of the Institute for Learning Algorithms in Montreal (Mila), created in 1993 by Yoshua Bengio.