Why Data Visualization Matters So Much in Robotics
Adrian Macneil is co-founder and CEO of Foxglove, a startup offering powerful data management and visualization software for robotics. Before founding Foxglove, he led infrastructure and developer tools at Cruise, the autonomous driving company.
Macneil is passionate about improving developer productivity and believes that optimizing the developer feedback loop is critical for rapid innovation in robotics.
According to him, when we think about how robotics advances, we often picture a logical and systematic process. Although we do not assume it is easy or simple, we may think it is relatively objective: the end goal is clear and the steps toward it are measurable.
But we forget that developing new technologies requires equally new ways to test and troubleshoot them. It is not only about a new robot, but also about entirely new tools and processes that must be invented to build that robot. As with any uncharted territory, these tools and processes are still being developed, evolving alongside the product and the people who create it.
Data visualization is one of the most important of these tools. It presents complex and detailed data in a convenient format, enabling equally complex and detailed decisions about how the robot works and how it can be improved. Such a tool suppresses unimportant noise and amplifies important signals.
“It brings much-needed meaning and order to the incredibly multidimensional nature of robotics data. As an exact science, which we want robotics to be, it remains largely a subjective exercise, with countless human judgments that bring us one step closer to our final goal: a world where our everyday lives are improved and enriched by robotics technology”
Traditional software development versus robotics development
To truly understand the importance of data visualization in robotics, you first need to understand how robotics development differs from traditional software development.
Diagnosing the root cause of problems in traditional software development is a relatively binary process: code either does or does not do what it should at a given step. There is a clear expectation of what should happen in every possible scenario.
By following the path of an error and running tests at each stage of the path — for example, logging values, checking expected and actual results, and turning specific functions on and off — engineers can pinpoint fairly precisely the line of code causing the problem.
On the other hand, debugging problems in robotics development is rarely so linear. Figuring out what went wrong with a robotic system is usually a far more complex and therefore open-ended process. For example, when an autonomous mobile robot (AMR) collides with another object, there is no obvious "code path" or culprit.
There are countless reasons why this could have happened, just as there are many reasons a person might run into an obstacle. Perhaps the AMR did not see the obstacle because it was around a corner, or perhaps it miscalculated its own trajectory to avoid the collision. The causes are not even related to software: the AMR's brakes could have failed, or something could have briefly blocked the view from its camera.
There is no single methodical protocol to follow, and the root of the problem may be caused by a series of interconnected smaller issues that run through several layers of the robotics stack. Solutions are often conditional and situational, so during debugging, contextual information must be considered just as much as the literal code on the screen.
Data visualization for robotics
Solving this debugging complexity requires a new generation of sophisticated tools for exploring the many aspects of robotics data. This is where robotics data visualization comes in.
In the early days of Cruise, the autonomous vehicle company, employees learned firsthand how painful robotics development could be without the right tools.
When debugging data or simulation results, engineers had to set up a full development environment on an Ubuntu machine, then load a handful of different open-source tools such as rviz, rqt and others. Engineers then had to manually start playback of a log file and rearrange several windows on the screen to use them at the same time.
Engineers could painstakingly arrange windows on their screen for one particular workflow, but replacing that layout for another workflow or copying it to a teammate's computer consumed valuable developer time. Although these tools did their individual jobs, it was clear that using them together, or using them with a larger team of roboticists, was not part of their DNA.
They were clumsy to use on their own, difficult to integrate with each other, and impossible to use meaningfully with teammates. The ability to iterate on robots was seriously hampered, not because of the team's talent, but because of the inability to quickly extract useful insights from data.
This led to a hackathon project in which a group of engineers saw an opportunity to improve this workflow. They presented a web-based data visualization tool that was received so well that it drew loud applause and was eventually turned into an internal tool with its own dedicated team.
That tool was Webviz. It was eventually open-sourced to help other roboticists benefit from more effectively integrated tooling. Over time, the company recognized the impact Webviz had on Cruise and decided to found Foxglove to continue solving common robotics problems for the industry as a whole.
“We released Foxglove Studio to help users visualize and debug their robotics data, and the Foxglove Data Platform to help them store and manage it in a central repository. With data visualization, we can quickly make sense of our data — from different angles, using different views, for different workflows — to inform the next step in development. This is how we can explore non-binary situations and apply the human judgment needed to bring robotics into the future”
Rich interactive data visualization is essential in helping roboticists take part in the more open-ended exploration that robotics development requires. It allows roboticists to use their intuition and experience to start debugging a given problem. It helps them synthesize the equally important subjective and objective judgments needed to troubleshoot their work.
It helps them see at a glance how their robots currently operate and how they could be improved, so they can make changes to them more effectively.
Different data visualization tools can help make different judgments when solving different tasks. Each step of a debugging or analysis workflow can benefit from a combination of different tools. For example, in Foxglove Studio, users can choose from an extensive collection of visualization panels to build custom debugging layouts.
Each panel has a specific function: one can plot values, another can mark coordinates on a map, and a third can display 3D models in an interactive scene. Whether you are troubleshooting a robot's planning stack or monitoring its diagnostics, there is a visualization tool for the job.
Thanks to rich interactive data visualization, tools can finally match ambitions. It is now possible to explore, analyze and make sense of data more deeply, data that was previously opaque, in order to speed progress toward the goal.
The future of robotics and data visualization
Despite its apparent objectivity, robotics development is not free of subjective and creative decisions. Although the end result can be concrete, as in "the robot can now maneuver across the warehouse floor," the means by which researchers achieve it form an endless iterative process requiring substantial human intervention.
“To successfully introduce robots to the general public, we must recognize robotics as a more open-ended field that creates more interconnected problems, has more situational solutions and therefore requires more diverse tools than traditional software development”
Robotics development still faces many challenges to solve: cumbersome testing infrastructure, unpredictable hardware development timelines and scalable data management. The list could go on. But there are several improvements that will strongly affect the entire robotics industry, such as excellent data visualization tools.
