Artificial intelligence predicts 'what will you do'

This is a new code that helps computers first observe every small movement of many people at the same time and have the ability to make robots understand language without words.

While people communicate naturally using body language, computers are "less or more blind" to these interactions. But the new code can improve the robot's ability in social situations when it comes to understanding the body and movement of people.

For the first time, robots have the ability to observe and see such hand movements clearly - (Photo: Carnegie Mellon University).

Researchers at Carnegie Mellon University's Robotics Institute developed new code using Studio Panoptic . This double-deck dome machine is equipped with 500 cameras, creating hundreds of points to observe each action in one spin.

The system sees human movement using a 2D model. This allows it to track movement in real time, recording everything from hand gestures to movement of human mouths and it can even track multiple people at once.

Yaser Sheikh, associate professor of robotics, said: "We communicate with the movement of the body as much as voice communication. But computers are less or less blind to identify language without words." .

Tracking many people is a big challenge for computers, and even hand movement detection is more difficult.

Although the set of image data on human hands is far more limited than the face or other body area, Studio Panoptic has allowed the robot to recognize hand movements that have never been seen before.

The team hopes to move from 2D to 3D human models soon to improve detection and understanding of human body movements - (Photo: Carnegie Mellon University).

This approach can eventually be used in many applications, such as helping to improve the ability of self-driving cars to predict the next movement of pedestrians. It can also be used in sports analysis, or behavioral diagnosis.

The researchers will present this achievement at CVPR 2017, Conference on Computer Vision and Chemistry, July 21-26 in Honolulu.

Currently, they have released this code to other research groups to expand its capabilities. Finally, the team hopes to switch from 2D models to 3D models, using Panoptic Studio to improve the detection of body movement, face and hands.