Autonomous racing drones can fly through forests at 40 km/h
It seems inevitable that autonomous drones will eventually outperform even the best human pilots. Usually, things in robotics that seem inevitable happen later rather than sooner, but drone technology appears to be an exception.
Over the past few years, we have witnessed remarkable progress even in complex autonomy that is moving into consumer hands at affordable prices. Drones with relatively simple onboard sensors and computing are now the focus of unmanned aerial vehicle research for situations requiring fast and highly aggressive maneuvers.
In a paper published in Science Robotics, engineers from the Robotics and Perception Group led by Davide Scaramuzza at the University of Zurich (UZH), together with partners from Intel, demonstrated a small autonomous drone that can fly aggressively through complex environments at speeds of up to 40 km/h.
"Quadrotors are agile. Unlike most other machines, they can navigate extremely cluttered environments at high speeds. To date, only expert human pilots have been able to fully exploit their capabilities. Autonomous operation with onboard sensing and computation has been limited to low speeds. Current methods typically split the navigation problem into subtasks: sensing, mapping and planning. Although this approach has been very successful at low speeds, the decoupling it relies on can be problematic for high-speed navigation in cluttered environments," the scientific paper says.
The trick here is that the drone performs a direct mapping from sensor input (from an Intel RealSense 435 stereo camera) to collision-free trajectories. Conventional obstacle avoidance involves collecting sensor data, building a map based on that data, and then creating a plan based on that map. This approach works well if you don't need to be fast, but for a drone with limited onboard resources moving at high speed, it simply takes too long.
The UZH approach instead goes directly from sensor input to trajectory output, which is much faster and allows a significant increase in drone speed. The convolutional network that performs this sensor-to-trajectory mapping was trained entirely in simulation, which is cheaper and easier, but less exciting, than letting real drones crash into obstacles again and again until they figure things out.
A simulated "expert" pilot with access to a 3D point cloud, perfect state estimation and computation not constrained by real-time requirements trains its own end-to-end policy, which of course is unattainable in real life. The simulated system that must work under real constraints is then simply trained in simulation to match the expert as closely as possible. This is how expert-level performance is achieved in a way that can be removed from the simulation and transferred to a real drone without any adaptation or refinement.
Another important part is the transition from simulator to reality, which can be problematic because simulation does not always accurately reproduce everything happening in the world, which can throw a robot off. But this method has proven very robust to image blur, sensor noise and other perception imperfections. The drone successfully flew in real-world conditions, including snowy terrain, derailed train cars, ruins, dense vegetation and collapsed buildings.
“While humans need years of training, AI, using high-performance simulators, can reach comparable navigation abilities much faster, largely overnight”
This does not mean the performance is flawless. The system still struggles with very low-light conditions (because cameras simply cannot see), as well as with similar vision challenges such as dust, fog, glare, transparency or the reflectivity of surfaces. Training also did not include dynamic obstacles, although the researchers argue that moving objects should not be a problem even now if their speed relative to the drone is negligible.
Many of these problems could potentially be mitigated by using event cameras instead of traditional cameras, since faster sensors, especially those tuned to detect motion, would be ideal for high-speed drones. The researchers said their system does not (yet) outperform expert humans in these challenging conditions.
Analysis of their work shows that humans have a very broad and detailed understanding of their surroundings and are able to plan and execute tasks that extend far into the future (the drone approach only considers one second ahead). Both of these capabilities are still missing in modern autonomous systems. The scientists see their work as a step toward faster autonomous flight, made possible by directly predicting collision-free trajectories from high-dimensional (noisy) sensor inputs.
Still, this is one of the things most likely to arrive soon: giving drones the ability to learn and improve from real experience. Combined with more powerful sensors and ever-increasing computing power, extending flight range to 40 km/h in complex conditions seems not only possible but inevitable.
