The “godfather” of artificial intelligence says that the latest models lie to users. Openai (model 03) refused explicit closing instructions
The winner of the Turing Yoshua Bengio prize launches the non-profit Lawzero organization for safer artificial intelligence, while warning that recent models have dangerous features, writes Financial Times.
Yoshua Bengio, considered one of the “goddesses” of artificial intelligence, Canadian academic whose work has influenced the techniques used by top groups in the field of artificial intelligence, such as Openai and Google, said: “Unfortunately, there is a very competitive race between the top laboratories, which pushes them to focus on the ability to do what more intelligent, without what to do, safety. ”
The winner of the Turing Award has launched his warning in an interview with Financial Times, in the launch of a new non-profit organization called Lawzero. He said that the group will focus on the construction of safer systems, promising to “isolate our research from these commercial pressures”.
Lawzero has so far collected nearly $ 30 million in philanthropic contributions from donors, including the founding engineer of Skype, Jaan Tallinn, former Google chief, Eric Schmidt, as well as Open Philanthropy and Future of Life Institute.
Many of Bengio's financiers are in the movement of “efficient altruism”, whose supporters tend to focus on the catastrophic risks that surround artificial intelligence.
The O3 of Openai has refused explicit closing instructions.
The winner of the Turing Yoshua Bengio prize launches the non-profit Lawzero organization for safer artificial intelligence, while warning that recent models have dangerous features, writes Financial Times.
Yoshua Bengio, considered one of the “goddesses” of artificial intelligence, Canadian academic whose work has influenced the techniques used by top groups in the field of artificial intelligence, such as Openai and Google, said: “Unfortunately, there is a very competitive race between the top laboratories, which pushes them to focus on the ability to do what more intelligent, without what to do, safety. ”
The winner of the Turing Award has launched his warning in an interview with Financial Times, in the launch of a new non-profit organization called Lawzero. He said that the group will focus on the construction of safer systems, promising to “isolate our research from these commercial pressures”.
Lawzero has so far collected nearly $ 30 million in philanthropic contributions from donors, including the founding engineer of Skype, Jaan Tallinn, former Google chief, Eric Schmidt, as well as Open Philanthropy and Future of Life Institute.
Many of Bengio's financiers are in the movement of “efficient altruism”, whose supporters tend to focus on the catastrophic risks that surround artificial intelligence.
The O3 of Openai has refused explicit closing instructions.
Bengio said that his non-profit group has been founded in response to the increasing evidence of the last six months showing that the top models are developing dangerous capacities. This includes “evidence of deception, cheater, lie and self -preservation,” he said.
The Claude opposite model of the company Anthropic has blackmailed engineers in a fictional scenario in which it risks being replaced by another system. The research conducted last month by Palisade artificial intelligence tests showed that the O3 Al Openai has refused explicit closing instructions.
Bengio said that such incidents are “very frightening, because we do not want to create a competitor for human beings on this planet, especially if they are smarter than us.”
The pioneer of artificial intelligence added: “At this time these are controlled experiments [dar] My concern is that, at any time, the next version could be strategically intelligent enough to overcome us with deceptions that we do not anticipate. So, I think we play with the fire right now. ”
The ability of systems to help build “extremely dangerous biological weapons” could become a reality of next year, he added.
Based in Montreal, Lawzero currently has 15 employees and aims to hire several specialists to build the next generation of artificial intelligence systems designed for safety.
Bengio, a professor of computer science at the University of Montreal, will resign as scientific director in Mila, the Quebec Institute of Artificial Intelligence, to focus on the new organization.
It aims to develop an artificial intelligence system that offers truthful answers, based on a transparent reasoning, instead of being trained to thank the user, while offering a correct evaluation if a result is good or safe. Bengio hopes to create a model that can monitor and improve the existing offers of the top artificial intelligence groups, preventing them from acting against human interests.
“The worst scenario is the disappearance of humanity,” he said. “If we build artificial intelligences that are smarter than us and are not aligned with us and compete with us, then we are practically lost.”
Bengio's decision to set up Lawzero comes in the context in which Openai aims to move even more from his charitable roots, turning into a lucrative company. This movement caused concerns from artificial intelligence experts and triggered a process filed by co -founder Elon Musk, who is trying to block the transaction.
Critics claim that Openai has been founded to ensure that it has been developed for the benefit of humanity, and the new structure eliminates legal remedies if the company prioritizes the profit to the detriment of this objective. Openai claims that it has to gather capital to compete in this sector.
Bengio said he doesn't trust that Openai will respect his mission. “To grow very quickly, you have to convince people to invest a lot of money, and they want to see a yield of their money. That's how the system works,” he added.
