If you have ever dabbled in robotics, then you have probably had the experience of realizing just how complicated seemingly simple movements are to coordinate. Take quadruped walking for example — each leg needs to move in perfect coordination with the others to maintain stability, and movements must continue uninterrupted at a set speed to prevent falling. Moreover, the robot must be programmed to negotiate any number of previously unseen types of terrain.
Problems such as these are commonly approached by either laboriously hard coding the movement logic, or by training a single neural network to become competent in all scenarios. A recent collaboration between the University of Edinburgh and Zhejiang University has resulted in a multi-expert learning architecture that can adapt to novel situations by employing a distinct set of expert deep neural networks. This new architecture was tested out on the Jueying quadrupedal robot dog.
A total of eight expert deep neural networks were created, with each specializing in a particular skill. One network was a master at running, while another understood how to make turns, and another yet learned how to get back up after falling down. Rather than explicitly program the skills, the expert networks were trained by a trial and error reinforcement learning approach in which the algorithm would be rewarded when the robot performed well, and penalized when it did not. A simulated 3D environment was built to speed up the training process.
Another network, a gating neural network, governed the entire system — it learned in which situations to employ each expert network, and to which degree. Multiple expert networks could be blended together to learn new skills and deal with novel, real-world situations.
With the expert networks working in concert, the Jueying robot dog learned to perform complex actions like trotting and walking on loose stones. And if anyone would be so rude as to push over a cute little robotic dog, it has learned to hop right back up on its feet.