Technology can augment the world around us, it can enhance the human experience, our capabilities, and also extend our reality to digital and virtual worlds. As people flock online during quarantine, we are experimenting with new platforms, pushing immersive technologies to the limits, and collaborating in new ways. From eye-tracking technology, facial tracking, biometrics, and mind control, just how far are we from becoming one with the metaverse and what can we learn about ourselves through sensory technologies?

Brain-computer interface systems like Neurable’s VR game Awakening uses an electrode-laden headband, connected to an HTC Vive HMD, to track brain activity. The software analyzes the data to figure out what should be happening in a game. In 2017, the MIT Technology Review wrote that only one year later this technology could be mainstream and picked up by VR arcade companies. It was an optimistic estimate as consumers and hardware alike have required a transition and adoption phase. Now in 2020, could we really be a year away from consumer-ready mind control systems?

CTRL-Labs takes another approach to HCI (human-computer interfaces) and uses a simple electrode-studded wristband to read neural signals from the arm. In June 2019, CTRL-Labs acquired patents from Myo, a similar wearable created by North that enables control of robotics and PCs via gestures and motion. Shortly after, in September 2019, CTRL-Labs was acquired by Facebook, becoming a part of the Facebook Reality Labs team to build the technology as a consumer product. As Oculus launched hand-tracking on the Quest in December 2019, I wonder if consumer-ready mind-controlled systems could be even less than one year away. 

Perhaps more widely known in mind-control systems is Elon Musk’s Neuralink. It’s by far the most invasive technique, that involves inserting 96 threads into the brain with micron precision. It’s a risky business, but also an absolute reality. “An integrated brain-machine interface platform with thousands of channels,” by Elon Musk and Neuralink offers great insights into how it can replace typing, clicking, or even talking as a form of digital telepathy.

With varying methods of mind-control systems that affect how we receive and transmit information (and there are many more than listed above), hand-in-hand with ongoing infrastructure build for 5G and edge networks, this is another greatly unanticipated paradigm shift in mass mediums that is worthy of David Bowie’s predictions of the internet in 1999. 

Moving from mind to body, the Teslasuit is a non-intrusive HCI with a suit material that uses a piezoelectric EAP (electroactive polymer) to record electrical signals generated by players’ muscles to animate avatars in virtual reality. As new haptic and biometric sensory technologies are refined, and as edge computing is able to quickly process multiple data streams generated by these suits, more sophisticated interfaces will emerge. 

Low profile tactile, haptic and thermal displays that can be used in a variety of applications are already in development. At the Cutaneous Sensory Lab in the Department of Mechanical Engineering at MIT, this includes the ability to conduct a variety of psychophysical studies to determine how users perceive these various forms of stimulation. The 2019 study “Closed-Loop Haptic Feedback Control Using a Self-Sensing Soft Pneumatic Actuator Skin,” published in Soft Robotics, also presented a solution for wearable haptics as a soft, flexible artificial skin made of silicone and electrodes that self monitors to provide accurate haptic feedback to a user’s body. 

In the market today, with an acute level of tactile feedback, the HaptX gloves have been a long time and all-time favorite of mine for realistic touch in virtual reality. It enables users to feel the shape, texture, and motion of virtual objects with 130 points of force feedback. Size, weight, impact, and even temperature (with a bulkier version than the standard enterprise glove) is possible and truly astounding. In January 2020, Haptx announced a Series A financing round of $12M. 

AR glasses like Nreal and MAD Gaze indicate a shift away from clunky VR HMDs to a viable consumer option for day-to-day applications, public, and collaborative use. Nreal’s recent partnership with Clay AIR, announced March 2020 to integrate hand-tracking and gesture recognition, indicates another greatly anticipated movement to intuitive interactivity that blends the real and virtual world beyond the Oculus ecosystem. 

Just as we see with the miniaturization of haptic products, AR headsets are also seeing their next-generation counterparts enter the consumer market in the form of AR contact lenses. These include InWith Corporation’s that are powered by blinking, and Mojo Vision’s contact lenses- a company that raised a $58 million in a Series B investment round in March 2019. 

While VR can be traced back to the mid- 20th century, it was only in 2010 that Palmer Luckey launched the Oculus Rift Kickstarter. Ten years later, we have a healthy immersive ecosystem where AR is quickly becoming a ubiquitous and omnipresent application. 

Perhaps one of the more telling of immersive technologies is eye-tracking, as. Recently Antony Vitillo, in The Ghost Howls, spoke with Lars Bergstrom, Mozilla’s Director of Engineering Mixed Reality on the subject. Bergstrom said:

‘“Eye-tracking has the potential to expose an individual’s intrinsic characteristics, such as race, age, gender, and sexual preferences, as well as revealing sensitive health characteristics such as whether they have autism or disorders like anxiety and depression. We have very little conscious control over our eyes, so the idea of providing unfettered access to data that can reveal all of this information (as well as provide unique user fingerprinting) is antithetical to our values of treating privacy as a first-class citizen. Instead of providing access to raw gaze data, the web browser can act as an opinionated user agent — an intermediary between the application and the device sensors and resources. In this example, perhaps we would only reveal that a user had looked at an element on a page if they’ve dwelled on it for a certain period of time.”

These surreal insights are an eye-opener to what we can learn from immersive technologies, not just about the technology itself, but about ourselves. How will this type of data affect employment, healthcare, relationships, and broader life matters? What is your immediate reaction to the above in the hands of private corporations and public individuals? What is the solution to data privacy?

As we stroll confidently through pixelated grassy fields, feeling, seeing, hearing and interacting with environments as if they were real, we are beginning to encounter more advanced NPCs and bots that could be either human or machine. In January 2020, Soul Machines announced a $40M raise for their AI-powered customer-facing digital avatar technology, described on their site as “a Digital Brain that provides Digital Humans with the ability to sense, learn and adapt.” We are already encountering these digital humans as chatbots worldwide, and witnessing massive funding moves in this vertical during the recently announced pandemic. I include digital humans in this list of immersive and multisensory technologies, because, like the way we interact with one another, the way we interact with digital humans (whether they are known to be so or not) is another telling tale of our own character. 

One of the most eye-opening reads on the senses for this piece was Frans Evers’, The Academy of the Senses, that describes synesthesia as the ability for our senses to be “rerouted” thanks to neuroplasticity, the brain’s ability to change and adapt as a result of new situations. This means that we can feel through sound, hear through touch, and see-through taste. Our senses are a fascinating playground for experiences that inform our sense of reality. Even if we don’t use all of the above technologies concurrently, there are endless possibilities of how our perception can be altered and enhanced. 

As we move towards a much more personalized online experience with sensory and immersive technology, we can achieve the most incredible human experiences, or a very data-based play on dopamine that causes addiction similar to the way that likes, comments, shares, and stats have plagued social media. While we are largely desensitized in the consumption of media today, multisensory immersive technology will forever change the way we measure the success of online campaigns, engage with the digital world and one another. 

As global movements are made to gather in online worlds, we are laying the foundation for the next era of digital humans (ourselves). Many of the technologies we use today, including phones and laptops are simply prototypes to that which is becoming possible through immersive technology that enhances our capabilities and augments our senses. It’s the safeguards that we put in place, and a genuine duty of care that will build a better future.

Amazing work to everyone who is a part of this movement, holding close our collective welfare now and in the future. A shoutout to all those who have been incredibly active in collaborating to bring people together in VR on multiple WhatsApp XR chat groups.

The post From Brain-Computer Interfaces To Digital Humans: How These Technologies Are Bringing Us Closer To The Metaverse appeared first on VRScout.