Meta shows prototype VR gloves that let you feel objects in virtual reality
You can't pet a dog in Meta's new high-tech virtual reality gloves and feel every detail of the touch. But researchers are getting closer. Meta (formerly Facebook) is known for its loud push into virtual and augmented reality.
Still, for seven years the company has quietly worked on one of its most ambitious projects: a haptic glove that reproduces sensations such as grasping an object or moving a hand across a surface. The company has for the first time demonstrated a prototype of the device.
In simplified terms, Meta's haptic prototype is a glove with about 15 ribbed inflatable plastic pads (actuators). The pads are positioned to match the user's palm, the inner side of their fingers and their fingertips. The glove also acts as a VR controller. On the back panel there are small white markers that let cameras track finger movement in space, as well as internal sensors that detect the bending of the user's fingers.
When you put on the glove and enter virtual or augmented reality mode, a sophisticated control system adjusts the inflation level, applying pressure to different parts of the hand. If you touch a virtual object with your fingertips, you will feel the object press against your skin. If you hold a virtual item, the actuators become rigid, creating a sense of resistance. These sensations work together with visual and audio cues to create the illusion of physical touch.
The technology relies on the relatively new field of soft robotics, replacing bulky motors with tiny air valves. The company has been working on this almost since acquiring the startup Oculus VR in 2014. The first prototype, a single finger with one actuator, was developed as early as 2015.
One of the first experiences that Reality Labs head Michael Abrash recalls was looking at a virtual plate from inside the VR headset, where a single actuator combined with a virtual image and the sound of friction against rough ceramic was incredibly convincing: "I saw the plate, and I saw my finger on the plate, and I heard the sound — that scratching sound across it — and I felt the vibration. And I'm telling you, I ran my finger over a ceramic plate."
Simulating touch is far from a new phenomenon. Even combining virtual reality with something as simple as controller vibration can make people feel as if they are touching something. Many companies have worked on wearable devices that either track users' hands or create haptic sensations. Some even combine temperature sensations, which Reality Labs is not prioritizing.
Meta is ready for mass production of haptic gloves, something other companies cannot do. Most makers of haptic devices sell specialized products for military, industrial or academic institutions. The company, however, produces the dominant, consumer-oriented Quest VR system and is investing billions of dollars in building a "metaverse" that combines VR and AR. If haptic gloves appear, Meta can ensure the system works on Quest and encourage app developers to use it.
Reality Labs presents the gloves as one of several control methods for future glasses and headsets, alongside lighter solutions based on electromyography, or EMG, a system that reads nerve signals in your hand and converts them into digital input. Meta acquired CTRL-Labs, which makes EMG wristbands, in 2019, and the EMG team works separately from the haptic design group, although the two technologies could easily converge.
“Douglas Engelbart and Xerox PARC are the only times our way of interacting with the digital world has fundamentally changed (referring to the invention of the mouse and the design of the Alto desktop computer in the 1970s, which helped set the course for modern personal computers). AR glasses will require this to happen. You are not going to walk around with a keyboard and mouse. You are not going to take out your phone to interact with it”
For some tasks you may need a simple wristband. For others you need fuller haptic sensations. Still, there are many obstacles to producing haptic gloves that are ready for use.
On one hand, the engineering team wants to sharply increase the density of the glove's actuators from dozens to hundreds or even thousands within a few years. Right now the gloves can convey the outlines of objects, but not a clear distinction between surfaces; instead they rely on the guiding force of sound and images.
“You can pet a dog, but you won't feel the texture. You need high-density motion triggering to truly get that sensation, and this glove doesn't do that”
On the other hand, the gloves need to shrink dramatically. Healy said the device must be light enough that people feel they can interact normally with both the real and virtual worlds, and it must be fully wireless rather than relying on a cable, as it does now.
In addition, haptic gloves must fit the user's skin precisely. This may require developing different versions for each customer, which is possible with a process such as 3D knitting, which can create a garment of custom size from a digital design.
There are also practical questions, such as how to clean a high-tech glove — right now it is carefully wiped with alcohol. Healy said: "As for clothing in general, we expect it to be washable. We would like to create gloves that can be washed. How? We don't know yet. But that is part of our vision."
If the gloves reach the market, they will face the same privacy and security issues as other wearable devices that collect biometric information. Abrash compares the use of technologies such as EMG wristbands to typing on a keyboard, which is rarely called a potential threat. However, wearable devices can analyze people's physical movement with precision, while even basic typing patterns can predict the onset of diseases such as Parkinson's. Meta will need to establish policies that define which part of this data leaves users' devices and who else has access to it.
At this point, the team is simply trying to understand how real virtual reality can be. "One of the interesting things about haptic gloves is that we obviously cannot exactly reproduce reality, which is different from audiovisual things," Abrash said.
“I think it is likely that if haptic gloves work, you will be able to pet a dog, but it will be a virtual dog, and the experience will be somewhat different. But mostly — emotionally, empirically — it will feel like reality”
It is worth noting that VR startup HaptX issued a fairly aggressive statement accusing Meta of demonstrating a prototype that was "virtually identical" to its own patented technology. In a statement from HaptX CEO Jake Rubin, he detailed that his startup had for years demonstrated its technology to "many engineers, researchers and executives at Meta" and that the company was not consulted on this latest project.
"Although we have not yet heard anything from Meta, we hope to work with them to reach a fair and equitable agreement that resolves our concerns and allows them to incorporate our innovative technology into their future consumer products," Rubin said.
