Although not programmed in advance, this robot is able to learn to walk on its own!

Almost all terrestrial creatures learn to walk from an early age. It takes different animals a different amount of time to master their feet: our babies take months or years to do it, baby antelope can do it almost. as soon as they were born. And, if we take time as a measure of learning to walk, in a new study, scientists have created a robot that can learn to walk in just an hour and does not need to be programmed. before.

This particular robot is a four-legged device like a mechanical puppy, it can learn to walk on its own without being shown any simulation to instruct it beforehand.

This particular robot is a four-legged device that resembles a mechanical puppy.

According to experts, a robot builder will need to perform each task they want the robot to do, which can be time consuming and can be difficult to program the robot's behaviors towards. unexpected situations.

The feat was accomplished by an AI that the team designed and built, said Lerrel Pinto, study co-author and assistant professor of computer science at New York University who specializes in robotics and machine learning. The name is Dreamer.

Dreamer is based on a technique known as reinforcement learning - "training" the algorithms by giving continuous feedback, and generating reward signals for desired actions such as successfully completing a task. . In a sense, the process is similar to how we learn on our own.

A common approach in training robots is to use computer simulations to give them a basic understanding of whatever they're doing before forcing them to perform similar tasks in the real world. .

"The problem is that your simulator will never be as accurate as the real world," said Danijar Hafner, a PhD student in artificial intelligence at the University of Toronto and co-author of the paper.

What's special about Dreamer is that it uses past experiences to build its own model of the world around and conduct trial-and-error calculations in a simulation based on this model.

The robot can try out what it learns in the lab.

In other words, it can perform its task inside a dream-like mirror of our world by predicting the potential outcomes of the actions it intends to perform. Armed with this knowledge, it can try out what it learns in the lab. It does all of this on its own. Basically, it is teaching itself.

This approach allows AI to learn much faster than before. At first, all it could manage was to wave its legs helplessly in the air. It takes about 10 minutes for it to turn over on its own and about 30 minutes to take its first steps. However, an hour after the experiment started, it could easily walk around the lab on steady legs.

In addition to teaching himself how to walk, the Dreamer can then adapt to unexpected situations, such as resisting being toppled by one of his party members.

The results show the amazing achievements that deep reinforcement learning can achieve when paired with word models, especially considering that the robot receives no prior instructions. The parallel use of these two systems has significantly cut down on the lengthy traditional training required during trial-and-error reinforcement learning for robots.

Furthermore, removing the need to train robots inside a simulation and allowing them to practice inside their model of the world could instead allow them to learn skills in real time - providing them tools to adapt to unexpected situations such as hardware failure. It could also have applications in complex, difficult tasks like autonomous driving.

Using this approach, the team successfully trained three different robots to perform different tasks, such as picking up balls and moving them between trays.

One downside of this method is that it is very time consuming to set up initially. Researchers need to specify in the source code which behaviors are good - and therefore should be rewarded - and which are not. Each and every task or problem the robot has to solve will need to be broken down into sub-tasks and each sub-task defined in terms of good or bad. This also makes it very difficult to program for unexpected situations.

The researchers are hoping in the future to teach the robot to understand verbal commands, as well as attach cameras to the robot dog to give it vision and allow it to move around in complex indoor situations. , and maybe even play a search game.