Training Robot Dogs to Overcome Obstacles

When we say that someone or something moves like a robot, we are not trying to say that they have great agility. On the contrary, robots have become synonymous with jerky, awkward movements and an inability to adapt to unexpected conditions. This perception of robots as clunky and inflexible machines stems from the early days of robotics, where rigid programming and limited sensory capabilities confined them to repetitive and predefined tasks. These early robots were indeed far from the graceful, agile movements of humans or animals.

While many advancements have been made in recent years, robots are still no match for the nimbleness and adaptability of humans. Human locomotion is a complex interplay of muscular control, sensory feedback, and rapid decision-making, all orchestrated by an intricate neural network. It allows us to effortlessly navigate a wide range of terrains, perform delicate tasks, and respond swiftly to unexpected changes in our environment.

The landscape of robotics is undergoing a profound transformation, driven by advances in artificial intelligence and robotics research that promise to replicate the abilities of humans, but there are still many difficulties that need to be overcome before that goal is realized. At present, a popular method for training robots involves the use of reinforcement learning algorithms that have reward systems that tend to fine-tune them for operation in specific environments. When they meet with unforeseen conditions, they frequently fail. Alternatively, some robots are trained using data captured from real animals. Some successes have been achieved in this way, however, robots trained in this way are not very versatile. Typically they have a very limited set of skills that they can perform.

A collaboration between researchers at Stanford University and Shanghai Qi Zhi Institute showed that sometimes simpler is better. They developed a control system for an off-the-shelf quadrupedal robot dog that is highly agile and versatile. The control algorithm is trained using a very simple algorithm, which allows for real-time processing on inexpensive computing platforms.

Two separate robotics platforms were experimented with in this work, the Unitree A1 and a Unitree Go1. In both cases, an NVIDIA Jetson NX single board computer was included to handle running the machine learning algorithm, and an Intel RealSense D435 depth camera was added to capture information about the robot’s surroundings.

To give the robot a very broad set of movement data to learn from, the team leveraged IsaacGym to collect simulated data rather than relying on more limited real-world datasets acquired from animals. Processing steps were taken to address the gap in the visual appearance of simulated objects versus those in the real world.

After deploying this initial model to the robot, it was fine-tuned through a reinforcement learning process. But unlike traditional approaches, they selected a very simple reward system that minimized computational complexity and processing time. Typically, a vast array of parameters are factored into the reward system, but the team developed a simple system that primarily ascertains only if the robot successfully moved forward, and it then rewards the conditions under which the robot moved forward with the least amount of effort.

As previously mentioned, this keeps the algorithm lightweight so that it can run in real-time on inexpensive, onboard hardware. But that is not the only advantage. By keeping the system so flexible in what can be considered a “good” solution, it has the freedom to adopt very creative solutions, as long as they get the job done.

The robot dogs were demonstrated leaping, climbing, crawling, and squeezing their way through an obstacle course with great agility using this control system. And when the robot dog did not succeed on the first attempt at conquering a new obstacle, it would learn from its failure, adjust its approach, and try, try again until it succeeded.

Moving forward, the researchers hope that advances in simulation software will lead to more realistic simulated environments that will translate even better to real-world applications. With that boost, they envision their technology being used one day to aid first responders in rescuing victims following disasters.