AI that “reads” pedestrian body language. A breakthrough in autonomous driving

The new AI model can predict pedestrian behavior in real time, achieving results better than existing systems and opening the way to safer autonomous vehicles – reported in the journal “Computers & Electrical Engineering”.

Researchers from Texas A&M University and the Korea Advanced Institute of Science and Technology have developed the OmniPredict system, which analyzes images from street cameras and information about the road situation to assess how pedestrians may behave. It is not limited only to recognizing objects, but combines visual signals with the context of the environment – the pace of movement, direction of gaze or change of posture – and on this basis predicts the likely actions of a person in the next seconds.

According to the head of the research team, Dr. Srinkanth Saripalli, current systems of this type, used e.g. in autonomous vehicles, cope well with image interpretation, but worse is the unpredictability of city streets.

– Weather changes, unexpected human behavior, rare events and the general chaos of a city street – all this can affect even the most advanced vision systems, said Saripalli.

OmniPredict is designed to eliminate these problems because it reacts not only to what has already happened, but also to what will happen. This approach can reduce the number of conflict situations on the road and improve traffic flow.

As the researchers pointed out, their model can also be applied outside civil transport. The ability to assess posture and signals of hesitation and stress is also useful for military and rescue services, enabling faster recognition of potential threats. They emphasized that the system is intended to act as a tool supporting human decisions, not to replace them.

In tests simulating complex driving situations, including scenes with partially obscured pedestrians, people moving unusually or suddenly turning towards the vehicle, the system predicted people's behavior with approximately 67% accuracy. effectiveness, which is clearly better than previously used solutions. According to researchers, this difference in the speed of reaction and adaptation to changing circumstances is crucial in urban traffic.

Although OmniPredict is currently a research project, its creators emphasize that this is what future technologies for autonomous cars should look like.

“This system opens the door to safer operation of autonomous vehicles, reducing the number of accidents involving pedestrians, and moving from reacting to threats to proactively preventing them,” concluded Saripalli.

Katarzyna Czechowicz (PAP)

kap/ zan/