Facebook announced a new API that will let developers incorporate video from the Quest 2’s sensors into their games and applications, creating a mixed reality experience. With the Passthrough API, developers will not only be able to mix the black-and-white images from the headset’s sensors into their experiences, but they’ll also be able to customize how it ends up looking to the player, apply effects, and even have the real world show up on specific surfaces.

Facebook’s announcement includes examples of how the API could improve productivity software by allowing integration of real-life keyboards and desks, and how games could include your real-life furniture for enemies to hide behind. It’s also easy to imagine other fun experiences...

Continue reading…