Killer robots in the army? The expert warns against something completely different

Much of the conversation about the barriers to artificial intelligence has focused on ensuring that the Pentagon's use of it will not lead to the use of excessive force. There are concerns about “swarms of killer AI robots,” and specifically how the military protects us. But there are also concerns about the protection of the Pentagon itself. In an organization as large as the military, there will always be people who commit prohibited acts. When a person inside the system commits an illegal act, the consequences can be quite serious. I'm not even talking about weapons, but about leaks.

Four-legged robots during the parade celebrating the 250th anniversary of the founding of the US Army. Washington, June 14, 2025PAP/EPA/KENT NISHIMURA / POOL / PAP

Even before the adoption of artificial intelligence, we had people in the military with access to national security systems who collected and leaked large amounts of classified information – to journalists or even just on a video game server to prove someone else wrong in an argument. People who have access to artificial intelligence could do this on a much larger scale.

What could an AI misuse disaster look like from the inside?

When I worked at the Pentagon, our main concern was how technology might be misused, generally by adversaries. But we must also realize that our enemies may be impersonating insiders. It's important to consider that malicious actors could get their hands on all of these tools. There are many things to consider: loss of information, exposure, which can lead to other, more serious consequences.

The article continues below the video

Using artificial intelligence can lead to psychosis, which causes someone to engage in behavior in the physical world that has little to do with reality. This can be very dangerous given the access to weapons systems in the military.

There are also concerns about the “swarms of killer robots” that many people are so concerned about. This involves escalation management. How do we make sure we're not using excessive force? How do you make sure the AI ​​responds the way you want? These are further challenges that the military will need to consider and ask artificial intelligence to help think through.

“The tendency to escalate is a human cognitive error”

We published recently article by Michael Hirsh. He writes that almost all publicly available AI models favored aggressive escalation toward nuclear war when presented with real-world scenarios. They didn't seem to understand de-escalation. Has this been your experience working with these tools?

One of the challenges with AI models, especially those trained on human history, is that the tendency to escalate is a human cognitive bias. It occurs independently of artificial intelligence. But AI makes it possible to speed up this process. And unless you somehow tell her “hey, watch your cognitive biases”, that's exactly the reaction you're going to get.

So does the Pentagon need to develop its own AI tools?

I think we need to develop tools that are consistent with the way the Pentagon operates, which is different from the way the civilian world operates, and for different purposes. But it really depends on what missions we're talking about.

Mieke Eoyang, then deputy assistant secretary of defense, during a hearing before the Senate subcommittee on cybersecurity. Washington, June 23, 2021Alex Wong / Getty Images North America / AFP

Much of the conversation so far has been around artificial intelligence around large language models and decision support. However, there is an entirely different branch of artificial intelligence that the military must engage in, and that involves navigation in the physical world. These are completely different challenges and technologies. Let's think about the idea of ​​unmanned systems, how they move around the world. It's a technology like driverless cars. Such input is not the same as large amounts of human-written text. It's about how to understand the world.

“More specialized research needed”

Is there a general need to better understand the utility of AI in the military? What problems might senior Pentagon officials have understanding artificial intelligence?

This technology is not yet fully developed. Actions such as moving consideration of artificial intelligence to the Pentagon's research and development space, as Donald Trump's administration has done, make a lot of sense. This allows you to test and work on some new features and development of these models. This means that when they reach the desks of a wider group of Pentagon employees, they will already be refined.

If abuse is so difficult to prevent, what might the future of AI tools look like?

One of the things we need to do in the future is to be more specific about the specific missions in which we are thinking about implementing artificial intelligence in the Pentagon. The Pentagon is a trillion-dollar enterprise and performs many business functions like any other company in the United States, whether it runs payroll or books business travel.

There are also areas that are more unique from a military point of view, and these require more specialized research. There is no civilian ecosystem involved in testing and developing these technologies. The Pentagon may have to fund its own research on things like understanding unidentified objects approaching the United States, or robots that need to navigate the battlefield, or making sense of many different strands of intelligence reports.