Google DeepMind is now able to train tiny, off-the-shelf robots on the soccer field. In a new paper published today Science Robotics, the researchers describe their recent efforts to adapt a machine learning subset called Deep Reinforcement Learning (Deep RL) to teach bipedal bots a simplified version of the game. The team notes that while similar experiments in the past have produced highly agile quadrupedal robots (see: Boston Dynamics spot), little has been done for two-legged humanoid machines. But new footage of the bots dribbling, defending and shooting goals shows just how good a coach’s deep reinforcement learning can be for humanoid machines.
Although ultimately meant for big tasks like climate forecasting and materials engineering, Google DeepMind can outright crush human competitors in chess, Go, and even sports. Starcraft II. But not all of these strategic moves require complex physical movements and coordination. So while DeepMind can study simulated football movements, that hasn’t translated to the physical playing field — but that’s changing fast.
To create the miniature Messi, engineers first developed and trained two intensive RL skill sets in computer simulations – the ability to get off the ground and how to score goals against an untrained opponent. From there, they practically trained their system to play a full one-on-one soccer match combining these skill sets, then randomly paired them with partially trained copies of themselves. .
[Related: Google DeepMind’s AI forecasting is outperforming the ‘gold standard’ model.]
“Thus, in the second phase, the agent learns to integrate previously learned skills, optimize them for the full soccer task, and predict and anticipate the opponent’s behavior,” the researchers wrote in the introduction to their paper. , later noting that, “During the game, agents fluidly transitioned between all of these behaviors.
Thanks to the deep RL framework, DeepMind-powered agents quickly learned to improve existing skills, including kicking and shooting a soccer ball, blocking shots, and even shielding its body against an attacking opponent. Defending your cause by using as
During a series of one-on-one matches using robots using intensive RL training, two mechanical athletes walked, turned, kicked, and righted themselves faster than when engineers provided them with a scripted baseline of skills. would do These were not insignificant improvements, either, compared to an unadapted scripted baseline, the robots walked 181% faster, turned 302% faster, kicked 34% faster, and got up 63% less quickly after falling. Takes time. Moreover, robots trained with deep RL also showed new, emergent behaviors such as walking and rolling on their feet. Such operations would be extremely challenging to pre-script otherwise.
There’s still some work to be done before DeepMind-powered robots reach the RoboCup. For these initial tests, the researchers relied on deep RL training based entirely on simulation before transferring this information to physical robots. In the future, engineers may want to combine both virtual and real-time enforcement training for their bots. They also hope to scale up their robots, but that will require a lot of experimentation and fine-tuning.
The team believes that using similar deep RL approaches for soccer, as well as many other tasks, could further improve the mobility and real-time adaptation capabilities of bipedal robots. Still, it’s unlikely you’ll need to worry about deep-minded humanoid robots on full-sized football fields — or in the labor market — just yet. At the same time, given their continued improvement, it’s probably not a bad idea to be prepared to blow the whistle on them.