Wednesday, 5 October, 2022 UTC
Summary
Last year at TARGO — for the 20th anniversary of September 11 — we released a virtual reality documentary called “Surviving 9/11”.
The doc shares the story of Genelle Guzman-McMillan, the last person rescued from the rubble at Ground Zero 27 hours after the collapse of the World Trade Center. Here’s the trailer:
https://medium.com/media/8ab5acbd24d29e3e66f356e4ed2189e0/href“Surviving 9/11” follows the story of Genelle Guzman-McMillan across 25 years — from 1998, when she arrived in the U.S., to her rescue from the rubble and her current life, 20 years after the attacks.
A crucial challenge of this documentary was to bring viewers back to Genelle’s life in the NYC of the new Millennium — complete with the Twin Towers, as it was before 9/11 — in virtual reality.
No 360° cameras in the 1990s
In ‘traditional’ media formats, talking about the past is rather simple: there are many libraries of content and archive footage available for any given topic.
For a virtual reality documentary, it’s a different story. There are virtually no footage archives or libraries of 90s VR media to browse, and for a simple reason: in the 1990s, there were no 360° cameras. With that constraint in mind, we started brainstorming on how to show the past in VR.
During an internet search, Chloe Rochereuil, who directed the documentary, stumbled upon a file that was called a “VR photo.” It was a low-quality panoramic picture featuring the Twin Towers. That was the ‘aha’ moment of the production.
Digital archeology: tracking down web archives
It got us thinking: was VR already a thing in the 2000s? Did VR archives of the Twin Towers actually exist? If this one panoramic picture of the World Trade Center appeared on the internet today, there must be many more, forgotten, sitting on hard drives, floppy disks, or film… We had to investigate.
Our research led us to a group of NYC pioneer photographers who had experimented with VR pictures in the 1990s. Their vision and ambition were oddly similar to what today evokes the “metaverse.” But the missing piece to turn their dreams into reality — a VR headset as we know them today — wouldn’t appear until years later.
In the meantime, their enthusiasm faded away and made way for an internet graveyard made of broken URLs. It felt like VR had gone into a 25-year-long time loop.
Looking back, these photographers were VR trailblazers. They paved the way for creators like us. We were able to uncover their works and identify more than 150 panoramas of pre-2001 New York that could be relevant for our documentary.
the famous web1To be accurate, most of the finds were obscure filenames in formats we had never heard of — in the likes of .ipix or .qtvr. Learning of the existence of the panoramas was a tiny fraction of the work. Being able to open the files was another headache. All of these formats were dropped by mainstream players — .ipix and .qtvr had become quirky formats not maintained.
When any technology gets forgotten, we lose access to parts of our history, our shared heritage.
In this case, it was obvious: because of discontinued file formats, we were unable to access rare pictures of the Twin Towers pre-2001. We had to track down the photographers’ identities behind these filenames and reach out to them, hoping that they would still have the source materials that would allow us to see the pictures.
We managed to get in touch with most of them. It was an incredible experience. Our conversations felt surreal, a blend of excitement and surprise on both sides. Some of them had forgotten that this footage existed. Some of them had kept it preciously. All of them were curious to see how we could give their pictures a second life. More than 20 years after the photographers shot these pictures, we had found a new and meaningful use for them.
Remastering negatives to 8K VR
Past the excitement of these digital treasures, we started the journey of restoring the footage to get them up to the state-of-the-art quality we all expect in VR.
We had one goal: make it look modern and natural. We wanted the technology to disappear so the footage could do what it is meant to: give way to Genelle’s story.
yes, we spent a fair amount of time remasteringTurning dusty negatives into crispy 8K VR videos was a fascinating challenge. Testing many software and techniques allowed us to find a workflow that maximized the visual information available in the source media. Here were the main steps:
- Restoring — removing scratches and dust, de-noising, and using AI algorithms to improve image quality.
- Reconstructing — filling in missing parts from the panoramic pictures to make them entirely 360°.
- Converting to 3D — a tedious manual process that assigns a position in space to every pixel to give a realistic sense of volume.
- Animating — matching archive 2D footage to overlay on the scenes and inject movements back into them (trees, fountains, clouds…)
Coming up with that workflow was the result of a lot of trial and error. The result ended up being exactly what we had hoped: it creates a seamless time-travel experience. You feel like you are in New York City, in the nineties.
Here are some video breakdowns of the work involved with this process:
https://medium.com/media/8c6d4f38dcef2035b228a9f25917a3da/hrefhttps://medium.com/media/1e5c68fefe580365d6dc20a81463381c/hrefhttps://medium.com/media/54423dc9cdd45650dd3df2eb02f2a56d/hrefTime is a place
With this experience, we discovered that while the immediate use of VR is to go to places out of reach, exploring the same location across time provides a deep sense of travel on its own — one that makes you feel privileged. It’s a radically new way to explore and preserve history, one that makes it more approachable.
Leveraging real footage is at the core of it. Recreating 3D models would have been easier, but it would have been less authentic.
In the case of “Surviving 9/11”, the authenticity of the footage is the most important element — it helps connect with a survivor’s testimony.
⏯ If you are interested in watching “Surviving 9/11”, it is available for free in VR in Meta Quest headsets worldwide. Here is a link to add it to your watchlist.
👏 If you liked this article, please give it a clap and share it — so others can find it too.
Follow TARGO on our socials — TWITTER, INSTAGRAM & LINKEDIN.
How we traveled back in time with virtual reality was originally published in AR/VR Journey: Augmented & Virtual Reality Magazine on Medium, where people are continuing the conversation by highlighting and responding to this story.