The immersive world of Gabo

You can't prepare for a moment where the world's collective nuclear arsenal points at your head. But here I stood—or seemingly floated in a soupy void—with countless jet gray, Dr. Strangelove–inspired missiles aimed at me from all directions, suspended in midflight and stretching back seemingly to infinity. Wile E. Coyote never had a freeze-frame so ominous.

My head twisted to take in the scene. I could move toward the weapons but could not escape their corral. Perhaps most troubling, nothing anchored me to the spot. No walls. No ground. No sense of scale. Disoriented, my stomach sank, as if feeling airtime on a roller coaster's big drop.

A moment before, I had floated up from the ruins of a bombed-out building, as if ascending to heaven. (My fear of heights was truly put to the test.) But instead of paradise, I was greeted by this over-the-top manifestation of Armageddon. Relax, I thought, none of this is real. I'm still in a Brooklyn studio, still standing on a Persian rug and waiting to meet with the creator of this interactive virtual reality experience, Gabo Arora, SAIS Eur '07 (Cert), SAIS '08 (MA), who clearly has a dark sense of humor. Suddenly I hear a voice, muffled by headphones and from someone not visible through the HTC Vive VR headset that essentially blindfolds me to the real world.

"You still OK in there?" says Erica Newman, a producer on this project, a documentary called The Day the World Changed, which at the time was set to premiere at the 2018 Tribeca Film Festival in New York in a few weeks. Newman and her colleague, Nate Turley, an interactive developer at the media production company Superbright, served as guides on my maiden VR voyage while we waited for Arora, who was running a few minutes late.

"Anything more to see?" I asked.

"No, we're still working on the ending. But you can keep looking at the bombs if you want," Newman says.

Pass. If the thousands of warheads were meant to unnerve me, they did. They're the wakeup-call portion of an interactive story told in three parts. The Day the World Changed first explores what led the U.S. government to develop and drop an atomic bomb codenamed "Little Boy" on the Japanese city of Hiroshima on the morning of August 6, 1945. The blast killed roughly 70,000 people in less than a minute and destroyed nearly 70 percent of the city's buildings. By a quirk of physics, one structure located right under the blast's hypocenter survived, and its ruins stand today as a UNESCO World Heritage Site called the Atomic Bomb Dome. Through the wizardry of VR goggles, you drop from an orbit in space down into the dome. Once you "touch ground," you can walk through the hollowed-out structure and view artifacts left from that day, like a child's scarred lunchbox or a pocket watch frozen with the time the bomb dropped, 8:16 a.m. At one point, banner-sized photos of the blast victims rain down off in the distance, a 360-degree mosaic of faces. The film's final segment advances to the present day as viewers delve into the mad race to develop ever more nuclear weapons.

The documentary was commissioned by Nobel Media to showcase the work of the 2017 Nobel Peace Prize–winning organization, the International Campaign to Abolish Nuclear Weapons. To create a story that encapsulated the organization's message, Nobel Media enlisted Arora, a former senior policy adviser for the United Nations and its first-ever creative director. A VR pioneer who founded the United Nations' Virtual Reality program, Arora has created widely acclaimed virtual reality documentaries about Holocaust survivors, the Nepal earthquake, Syrian refugees, and Sufi dance rituals that have premiered at major film festivals, including Cannes, Sundance, and Tribeca. Along the way, he has worked for Steven Spielberg's Shoah Foundation, the rock band U2, and his own production companies LightShed and Tomorrow Never Knows. For the past four years, Arora has championed the use of virtual reality as an ideal medium for telling stories with a social conscience message, and fostering empathy.

To interact with the virtual reality video above, connect your VR headset or click and drag your mouse within the player screen.

---

When Arora arrives, he bursts through the studio door slightly out of breath. He biked here from his house, but even on two wheels, New York City traffic can be unforgiving. He has black hair and a salt-and-pepper beard and is dressed in gray jeans and a dark navy sweater. He effusively apologizes and says he hopes my hosts have been accommodating. I tell him what we've been up to. "So, what did you think of the experience?" he asks wide-eyed and smiling. In a word, "unsettling," I tell him. Part of me feels like I'm still back there in the firing line of those warheads. "I know, right?" he says, gleefully. "So, it worked."

At the Matchaful Café a block from his studio in Brooklyn's Made in NY Media Center, Arora studies our barista painstakingly crafting a latte like he has just spotted a Bengal tiger. He appears mesmerized by the process, from the steeping of the tea, to the pour, to the leaf design etched into the froth. Once something catches Arora's attention, he won't let go until he's dissected every detail. Through his art, he wants people to view his subjects in much the same way.

In January 2018, the Krieger School's Master of Arts in Film and Media Studies program appointed Arora director of its new Immersive Storytelling and Emerging Technology concentration, which is pending approval from the Maryland Higher Education Commission. The concentration will feature a rigorous curriculum of virtual and augmented reality, artificial intelligence, digital storytelling, and creative technology studies. Arora admits he's still getting his feet wet at Johns Hopkins, and for now the program amounts to an elective in immersive storytelling that he teaches at the JHU-MICA Film Centre in Baltimore's Station North neighborhood, with occasional field trips to his studio in the Dumbo section of Brooklyn. The class's three inaugural students all served as associate producers on The Day the World Changed and worked in Baltimore and New York on the project's pre- and post-production. The documentary, co-created by German-born director, writer, and Pixar Animation Studios alum Saschka Unseld, was made possible through a seed grant from the Saul Zaentz Innovation Fund, an incubator program that connects budding creative innovators in the Baltimore community with prestigious artists, executives, and entrepreneurs in the film industry.

Annette Porter, an executive producer of The Day the World Changed and director of the Saul Zaentz Innovation Fund, says she enthusiastically supported Arora's vision of the documentary. "This project goes far beyond mere technology," she says, herself a filmmaker and faculty member in Film and Media Studies. "Virtual reality filmmaking might become an essential educating tool in the future."

Arora views The Day the World Changed as an interactive memorial to those killed in Japan, and a warning about the nuclear dangers that persist. The documentary immerses the viewer in a world that combines 360-degree filmmaking, firsthand testimonies of survivors, data visualizations, and innovative 3-D scanning and photogrammetry. Arora likens the latter to taking thousands of pictures of real places, and then stitching them together to create a sense of depth and real space you can walk around in. At Tribeca, the film was watched in groups of four. Each person donning the VR headset views the others in this virtual realm as shadowy, spectral avatars, evoking the impressions of blast victims that burned into the city steps, pavements, and walls. Handheld motion-tracked controllers allow viewers to manipulate certain objects, picking them up, pushing them away, or activating them.

"For this project we used the latest technology and cutting-edge graphics. [VR is] such a new field, and the technology changes so quickly, even as we were creating this project," says Turley, who also worked with Arora on Zikr: A Sufi Revival, a 17-minute documentary-style VR experience that debuted at the 2018 Sundance Film Festival. "We wanted people to share a more physical experience, and this ability to interact and touch things, even virtually speaking, adds another layer of realism and immersion."

Zikr, which was filmed last year, features Sufi rituals that take place in Tunisia. Through a combination of 360-degree video and virtual interactivity, the film enables four people at a time to experience a bit of Sufism. Arora was inspired to create the project after visiting Sufi shrines in India, wanting others to experience what has been called Islamic mysticism. Sufi tradition does not allow the passive observation of these rituals; Zikr gets around this limitation by making the viewer an active participant—active in a virtual sense—in the often rapturous music rituals. You dance with those around you, including your fellow viewers. With the motion-tracked controllers, virtual instruments like a tambourine appear in your grasp, and you can shake and move the instrument to play along with the music.

For Arora, Zikr was personal. The son of Indian immigrants who practiced Hinduism, Arora grew up with a measure of bias and distrust against the Muslim world, owing in part to the Hindu-Muslim conflict and levels of violence his parents experienced back in their New Delhi home. He says he couldn't relate to the Muslim perspective. With VR, he says, we can walk or dance in another's shoes.

Arora was born in Manhattan but grew up in Jackson Heights, Queens, the youngest of three children. He refers to his younger self as brash. He got kicked out of high school his freshman year. "Nothing illegal," he says. "Just acting up and getting into fights, a lot." His parents sent him to live with an uncle in New Jersey. For college, he set his sights on NYU, if only to return to New York. "I felt like I had been banished due to events," he says. "I wanted to be a New Yorker again. I wanted to be an artist, to study philosophy. To live in Greenwich Village. You know, be cool."

At NYU, he initially enrolled as pre-med to appease his father, a former engineer who wanted his son to enter medicine or science. But he secretly took classes in film, philosophy, and religion. Arora's father, who underwent a kidney transplant and was in and out of hospitals most of his adult life, died during Gabo's junior year. While the death of his father impacted him greatly, Arora says he also felt there was no more reason to pretend he had a real future in medicine. "I went hardcore into film at that point," he says.

After graduating from NYU, Arora still planned to go to medical school, but a pretty woman named Sara came along. The two met after an admissions interview and got to talking on a bus ride home to Manhattan. She mentioned a Fulbright grant she'd been awarded to conduct public health research on the AIDS epidemic in Namibia, Africa. "I said sure, I'm in," he says. "I wanted to be a filmmaker and be creative, maybe become a writer. I knew I had to have life experiences because without them I would not be a very good artist."

He moved to Namibia in 1999 and stayed there more than a year, living in a one-room shack in an urban slum. He eventually broke up with Sara and left her project, and ended up working as a consultant for UNICEF on HIV education. The experience exposed him to the many social injustices occurring in the country, such as a health system where doctors and nurses in impoverished sections had to practice medicine without proper supplies.

Upon returning to the United States, he got into politics and volunteered for Ralph Nader's presidential campaign in 2000. After the terrorist attacks on September 11, 2001, Arora felt pulled in another direction. He wanted to see the United States as an outsider sees it, and embed himself in another culture. Inspired by the Gabriel García Márquez novel One Hundred Years of Solitude, Arora moved to Colombia to work for Peace Brigades International, a human rights group that pioneered the practice of nonviolent accompaniment: Volunteers like Arora were sent into areas of conflict to provide protection to members of human rights organizations, unions, and others who were threatened by political violence. The volunteers might be asked to live with these people and accompany them as they went about their work. "Basically, I was a human shield," he says. "If [someone bent on harm] went after someone I was with, I would be collateral damage, and as a U.S. civilian that would turn into a diplomatic event. It sounds crude, but it's highly effective."

His work in Colombia evolved from human shield to lobbyist, putting pressure on a U.S. government that was funding paramilitary groups in the country. The experience inspired an interest in diplomacy and policy research, so he enrolled at the Johns Hopkins School of Advanced International Studies to study in the International Development Program, then led by Francis Fukuyama, the political economist known for his 1992 book The End of History and the Last Man. Arora says his two years at SAIS transformed him and jolted some of the activist and anti-capitalist out of him. "SAIS made me look at the world in a more dispassionate way," he says. "To make things better, you have to do less of the blame game. Up until then I was probably blinded by my own biases, and I came out, I hope, more nuanced."

In 2009, he joined the United Nations as a human rights officer on a peacekeeping mission to Haiti and stayed there after the 2010 earthquake and cholera outbreak. During a SAIS alumni event in 2011, Arora met with Thomas Stelzer, then an assistant secretary-general to the U.N., and told him he was tired of living in Haiti. He also had a distinct vision of how the United Nations could better get its message out. "I said, why don't we explore what is happening on YouTube," he says. "I had this crazy idea, maybe my version of a midlife crisis, to do a YouTube video promoting an upcoming climate change conference." The four-minute video called "Keep the Oil in the Ground," developed in partnership with the nonprofit group Amazon Watch, looked at oil drilling in Amazon rain forests in terms of its environmental, health, and cultural impact on native communities. The video went viral with nearly 2 million views. "The idea was simple. Let's not try to make it U.N. propaganda. Let's make it storytelling," he says. "I thought, give the people most affected the mic and let their words have an impact."

Arora later asked for the role of U.N. creative director. "They were like, we don't have one of those," he says. "I was like, well, just give it to me. Here I was in this senior position on an upward trajectory, but I zigged and created this new role. But I thought, this is what I really wanted to do, as it integrated all my interests and passions."

Arora's interest in VR stemmed from the underground immersive theater he attended in New York, where audience members would become part of the storyline. He imagined an immersive play set in a refugee camp. In 2015, that germ of an idea became his first virtual reality film. Clouds Over Sidra, developed for the U.N., is the story of a 12-year-old girl named Sidra who has lived in the Zaatari refugee camp in Jordan since the summer of 2013. The 3-square-mile desert camp is home to more than 80,000 Syrians, half of them children who have fled violence. The film follows Sidra around the tented camp and its makeshift school and football pitch. It premiered at the World Economic Forum in Davos, Switzerland, and was later shown at Sundance and the White House. Clouds Over Sidra won the Interactive award at the Sheffield International Documentary Festival held annually in Sheffield, England.

Clouds Over Sidra was co-created by Chris Milk, a director and immersive artist Arora was introduced to by David Howell Evans, better known as the U2 guitarist Edge, and their mutual friend Bono. Arora and Bono first met at a U.N. event and hit it off. The friendship would later produce U2's VR music video Song for Someone, for which Arora served as second-unit director.

To interact with the virtual reality video above, connect your VR headset or click and drag your mouse within the player screen.

---

He is most proud of a short movie he created for the United Nations called My Mother's Wing, an eight-minute film which offers a first-person view of a Palestinian family that lost two sons in 2014 to an Israeli bombing of a school shelter. In the documentary, the viewer follows the daily life of the family as they live among ruins in Gaza, yet remain hopeful for peace. Though treating a controversial topic, My Mother's Wing was acclaimed by both Israeli and Palestinian critics. "I thought this would be either loved or hated, but I needed to do this project. And I said if we're going to do it, we have to humanize this struggle," Arora says.

The success of Clouds Over Sidra and My Mother's Wing, Arora says, proved that VR could be an effective and powerful communication tool. "The purpose of VR is to bring you to places you can't go to, and there is perhaps no more of an entrapped, open-air prison than Gaza," he says. "I'm trying to develop a grammar of storytelling that is almost post-cinematic. We are not just dropping people into an area, but we are immersing people in a world that has some storytelling arc. I think that is unique. I call all the films I've worked on to date experiments."

Asked how he refers to himself professionally, Arora replies "lead creator." "Often, the idea is mine. It's my vision for the story. But I need to work with other people to fulfill that vision. I'm actually not very technical. I have an editor, and coder, and director of photography," he says. "These films can be rather labor-intensive, and it takes a village to push these projects through."

To fulfill his vision, he co-founded two production companies, first LightShed, a VR and social impact startup, and then Tomorrow Never Knows, a content, technology, and research studio. Tomorrow Never Knows, which produced The Day the World Changed and Zikr, is something of an immersive-storytelling supergroup, as it brought together some of the leading creative pioneers in VR and augmented reality: Nathan Brown, Tom Lofthouse, Saschka Unseld, and Arora. With the company's first two projects, the foursome have set the bar for what can be done with this medium. At each turn, most of the visual tricks and effects had to be created from scratch. Arora says this wasn't a case of plug-and-play, like they had a virtual tambourine with spatial sound accompaniment or interactive pocket watch ready to go. "Our motto has been, let's only do what hasn't been done before," he says.

After he left the U.N., the Shoah Foundation commissioned him to create the first VR Holocaust survivor testimony. Called The Last Goodbye, the film follows survivor Pinchas Gutter, who takes the audience with him on his final visit to the Majdanek concentration camp, where his parents and sister were murdered during World War II. The film starts with Gutter dressed in a robe in a hotel room in Poland. In the next scene, he's shaving. Arora says he wanted to start with a mundane act, to humanize the person, rather than start at the camp. The viewer does eventually get there, and Gutter agonizingly speaks of the last time he saw his sister alive, how all he can remember now is the braid in her hair. Later, he shows you the gas chamber his family and friends died in and leads you in to explore the empty space.

"I've always wanted to do something on the Holocaust," Arora says. "In some way, I thought it was the origin story of a lot of our problems that exist today. All the events that led to this happening have never really gone away, and the fear of that happening again has always struck me."

For his next project, Arora has some ideas in various genres, but whatever subject he chooses will likely feature augmented-reality goggles, the latest in VR gear that combines the real world with inserted virtual images. Imagine viewing your actual living room but populated with imaginary characters and objects. He says he wants to push the boundaries of immersive technology to tell stories that matter. VR also has a role to play in scientific study, he says. He has already had preliminary meetings with faculty at the School of Medicine and the Johns Hopkins–affiliated Lieber Institute for Brain Development to use VR to assist a research project related to the inability of people on the autism spectrum to identify sarcasm. Arora would create a VR experience to simulate a sarcastic joke or episode for the viewer while the research subject is housed in an fMRI scanner. The researchers could then compare the synapses of the autistic individual with those of the control subject. One challenge, he says, is to create a set of VR goggles made of plastic so they won't interfere with the MRI. He's also conceptualizing a project with the School of Public Health to help train community health care workers, preparing them with scenarios they are likely to encounter working in the field.

He also foresees student-led projects like Freedom Fighter, a social justice experience that uses augmented reality and an iPhone app to connect users with the past of Baltimore and its civil rights leader Lillie May Carroll Jackson. Created by Film and Media student Taura Musgrove using Apple's ARKit, Freedom Fighter uses geofencing to bring users face to face with a model of Jackson. Geofencing, which was popularized with the Pokémon Go augmented reality game, creates a virtual geographic boundary that allows software to trigger a response when a mobile device enters a certain area. In Freedom Fighter, Jackson's hologram will come to life, viewed through the phone, in front of the historically significant locations.

"We have big ambitions," he says. "That is why I was so interested in coming here to Johns Hopkins. I told them I'm not interested in just teaching, although I love to teach. We're not just talk. We'll make shit."

As to where VR technology will go in the near and distant future, Arora says it will likely incorporate artificial intelligence and augmented reality, harkening to a Westworld scenario of truly immersive storytelling—minus the over-the-top violence and sex of the HBO series—that will require warehouse-sized viewing locations and technology that hasn't been invented yet. "That's the next frontier," he says. "The future of VR is characters that are part of the story, leading you deeper into this theme park reality. It's going to happen. VR is not a gimmick; it can radically transform how we relate to the world and ourselves. Hey, I made people dance, and transported people to parts of the world they never thought they would see."

And he made me temporarily overcome my fear of heights, served with a side dish of apocalyptic nightmare.