Wednesday, 27 January, 2021 UTC
Summary
Hand tracking is moving more and more into mainstream virtual reality (VR), whether that’s in consumer headsets like Oculus Quest 2 or Varjo’s high-end enterprise devices. The latter employs Ultraleap’s technology, with the hand tracking specialist announcing a developer preview is available for version 5 of its Gemini software.
Ultraleap Gemini improvements show both hands can be used together. It enables natural interaction with virtual objects. Image credit: Ultraleap.
One of the main problems with software-based hand tracking solutions over actual gloves like HaptX or SenseGlove are two-handed interactions. Natural interactions like holding hands or one going behind the other are difficult to portray due to occlusion, where the sensors can no longer see fingers of the entire hand. To maintain natural immersion so that tracking isn’t lost or a hand suddenly disappears, Ultraleap has improved this important aspect with Gemini v5.
It may only be in a developer preview form at the moment – a full release will come later in the year – but the above GIF showcases the improvements made over the previous edition, Orion. The full hand and fingers are tracked and maintained no matter how they interact.
Gemini’s preview features include:
- Even better smoothness, pose fidelity, and robustness (likely to be most apparent on desktop mode)
- Improved hand initialization
- Significantly better performance with two-hand interactions
- New Screentop modes (to be mounted above an interactive screen) in addition to HMD and Desktop mode
Combined with Stereo IR 170’s wider FoV and Gemini’s improved hand initialization, hands will almost certainly be tracked before coming into view.
While Gemini works with both Ultraleap camera modules the Leap Motion Controller and Stereo IR 170, the latter’s wider field of view (FoV) means that hands can be tracked sooner, even before they come in a users line of sight. Leap Motion Controller has been available for several years now and can be used on a desk or mounted onto a VR headset. The Stereo IR 170 (in Camera Module and Evaluation Kit form) is primarily designed for integration and development needs.
Ultraleap tech might already be used by Varjo and Pimax but it’s the integration with Qualcomm’s Snapdragon XR2 5G reference design which could see more consumers gain access. The XR2 platform is going to lay the groundwork for plenty of devices over the next couple of years, making hand tracking even more prominent. For further Ultraleap updates, keep reading VRFocus.