Robot swarms trained in simulation keep working on real robots, study finds
IRIDIA, the artificial intelligence lab at Université libre de Bruxelles, reports a method for carrying behaviors that robot swarms learn in software onto physical machines. Swarms trained with that method, the team said, kept working reliably on real robots. The results appeared in Science Robotics.
Reinforcement learning lets machines discover behaviors through repeated attempts, usually inside a simulator. The problem is what researchers call the “reality gap”: friction, noisy sensors and imperfect mechanics differ between a simulator and the physical world, so skills learned in software often fail on hardware.
Swarm robotics has taken a different route. Its modular design methods build behaviors from predefined blocks, which the researchers said transfer reliably to real machines but depend on slow and costly search procedures. The two approaches had seldom been compared rigorously.
Rather than letting the algorithm set each wheel's speed, the researchers had it pick, step by step, from a few proven behaviors, including heading toward a light, joining peers and exploring. Limiting the robots this way, they said, keeps them from learning shortcuts that exist only in the simulator.
“For years we have argued that the reality gap is, at its core, a problem of overfitting: a design method that is too flexible learns the simulator rather than the task,”
Across five collective missions, the researchers ran 250 experiments on physical robots, using a swarm of 20 e-puck robots. Swarms trained this way transferred reliably, the team said, and substantially outperformed a leading method in which the algorithm controls motors directly. The rival approach struggled once it left the simulator.
A further finding was that constrained, simpler control policies carried over to real robots better than larger and more adaptable ones. The result cuts against a common assumption in AI that bigger, more flexible models usually perform better.
“What excites me is that this makes reinforcement learning a practical tool for swarm robotics,”
Gharbi said the next step is testing larger swarms on other robot platforms and in less controlled settings. The authors frame the work as joining two research traditions that had developed largely apart, and they say both are needed before teams of autonomous robots can take on search and rescue or environmental monitoring.