Will we become more human as the bounds between real and not dissolve?

About a year ago, I drove up from New York to Cambridge, over potholes and through rainstorms and all the difficulties of the messy present, to experience a possible version of the future. My destination was a conference called Virtually There at MIT, about virtual reality documentaries and the field of immersive journalism, which its proponents believe has the potential to upend not only storytelling but how we connect with each other, and even how we understand ourselves. This is the promise, at least. It’s early days. On the conference’s program there was a quote from Werner Herzog: “In the case [of virtual reality] we do have a technology, but we don’t have any idea how to fill it with content.”

In 2016, $2.3 billion was invested in virtual reality (VR) and augmented reality (AR) companies — three times the amount invested in 2015. SuperData Research, a firm that studies these trends, estimated that by the end of 2016 — by which point two much-anticipated headsets were finally released, the HTC Vive and the Oculus Rift (which Facebook’s Mark Zuckerberg paid $2 billion to acquire in 2014) — 6.3 million VR headsets had been delivered. And that number doesn’t even include the 10 million-plus more simplistic Cardboard viewers Google shipped out during that same period, prompting, to date, over 160 million downloads of Cardboard apps on Google Play. Meanwhile, journalism has cottoned on. Time Inc. launched Life VR, which produces content for the company’s various magazine brands. The New York Times, an early adopter, debuted its VR app in 2015 — it’s since been downloaded more than 1.2 million times. The Tribeca Film Festival first incorporated VR in 2013; by 2017, the festival included 23 VR films. All of which helps explain how it is that Goldman Sachs’s conservative estimate of the industry’s likely annual revenue in 2025 is $80 billion. (For reference, in 2016, global box office revenue from the film industry was just over $38 billion.) Their non-conservative estimate is $182 billion.

Back in the lobby of MIT’s Media Lab building, dozens of people sat on rotating stools wearing bulky goggles, their heads twisting around at random intervals, like baby birds. Two exhibits by the entrance featured large screens that enabled onlookers to view what nearby VR users were seeing. At one, a woman was using hand-held controls to explore a black light-esque rendition of the brain’s neuron circuitry. Across from her, a man was navigating the barren landscape of Mars. “We presented it last week to Congress!” said Julian Reyes, a VR producer at media company Fusion, now Splinter, which created the piece. “The director of NASA was there, and the deputy director—she played for 20 minutes!” Then he pulled up his cell phone—the display was shattered. “Yeah, I dropped it,” he said. “It only fell like one foot, maybe two.” And just like that, the dream of perfect technological progress was brought back to earth.

Those compositions could be described as educational adventures, but many others at the conference—the majority, in fact—were engaged in helping users understand other people’s experiences, particularly painful ones. One was about a famine in South Sudan, another was about a survivor of the 2005 London train bombings. The goal of these works—helping people relate to experiences not their own—isn’t all that different from that of many traditional documentaries, or novels for that matter. But advocates believe VR, as it’s known, has a unique power. “It’s a machine,” proclaimed Chris Milk, one of narrative VR’s best known creators, in a talk at the 2015 TED conference. “But through this machine we become more compassionate, we become more empathetic. Ultimately, we become more human.”

Yet even as the field is awash in a heady blend of ambition, hype and hope, with Goldman Sachs estimating that by 2025 it will be producing $80 billion in revenue, and one person telling me working in the industry was “like holding onto a rocket ship heading into space,” it also includes people like Janet Murray, who studies digital storytelling at Georgia Tech and takes a gleeful pleasure in bringing the industry’s more hyperbolic claims down to size. “Right now, the platforms are still in the experimental stage and God knows if any of them will make money,” she said at the conference. “Which doesn’t mean the medium isn’t real! But you don’t get empathy by putting on a helmet and looking at another person. You get empathy by being in a structured story environment.”

Perhaps the more meaningful question, though, isn’t whether there is an automatic connection between VR and empathy, but whether the medium, when used in certain “structured story environments,” amplifies and channels empathy in unprecedented ways. One argument that it can came in the form of a sophisticated piece at Virtually There at MIT, The Enemy, created by former war photographer Karim Ben Khelifa, which places you in a room with a real Israeli soldier, referred to as Gilad, and Palestinian soldier, Abu Khaled, as they respond to a series of open-ended questions—What does peace mean to you? What are you fighting for? To experience it I had to don not just goggles and enormous headphones but a backpack carrying a heavy computer. (Most VR is accessible via any VR headset and allows users to perceive 3D imagery but not move through it; The Enemy was volumetric, meaning you could walk around “inside” the environment and requires not just one of the more sophisticated headsets, like the $800 Vive, but also that it be attached to a computer.)

The gear was awkward and at first it felt more than anything like being in a lucid dream, where you have an unexpected freedom of motion yet nothing has the real weight of matter. But soon this disembodied feeling subsided. I forgot I was in a hotel conference room. I saw only the two men, standing at opposite sides of the room, their chests moving up and down as they breathed. They were dressed differently; Abu Khaled in camo and wearing a hood that obscured most of his face, Gilad in his green army uniform. But each followed me with their eyes as they offered strikingly similar answers to the same questions, jutting their heads back if I got too close. By the end, the piece’s immersive realness had transformed what could have been a pat statement about shared humanity into one that sunk in and stayed.

The Enemy was completed in the spring of 2014, at which point Gilad and Abu Khaled, who never met in real life, had the chance to view it themselves. That July, the ceasefire between Palestine and Israel disintegrated, and war raged for seven weeks, leaving thousands dead. Once the fighting subsided, Khelifa got news about Abu Khaled, who had survived but was injured, and subsequently reached Gilad by phone. “We talked for a bit, and then he asked, ‘How is Abu Khaled? Is he okay?’” said Khelifa. “That is the impact you want to have.”

The story about the current explosion of interest in VR usually begins with Palmer Luckey, a homeschooled tech savant with a penchant for going barefoot who launched a Kickstarter for Oculus, in 2012, when he was 19, asking for $250,000, within a month raised $2.4 million, and then sold the company to Facebook in July of 2014 for $2 billion. But while this story isn’t wrong, it’s incomplete, since it leaves out a woman, Nonny de la Pena, who in 2011 produced what is widely regarded as the first piece of “immersive journalism” (she coined the term), Hunger in Los Angeles. In it, animated characters stand in line outside a soup kitchen that has run out of food as real audio plays—you watch as a man collapses in a diabetic seizure.

It would be hard to overstate that piece’s influence. Awareness of VR originally began to trickle into the general consciousness in the early ‘90s, prompted by the first VR arcade games, headsets like Nintendo’s Virtual Boy, and the entertainingly terrible movie Lawnmower Man (in which VR enables a dolt to become an evil genius who uses telekinesis to explode floppy discs and commit murder via lawnmower). But after Virtual Boy flopped, along with Sega’s VR headset, VR retreated back to scientist’s labs. By the early 2010s, VR was still generally assumed to just be a tool for gaming or military training. With Hunger, however, De la Pena showed it could be a powerful tool to bring attention to realities that are more comfortable to ignore—to boost empathy, in other words.

For Hunger’s premiere, at Sundance in January of 2012, Luckey, then a part-time employee at the University of Southern California, where De la Pena was a research fellow, built an early, duct taped version of what would become Oculus (festival goers used it to watch the piece), and crashed in her hotel room. Hunger soon became the first work of VR experienced by everyone from Milk, whose best known VR piece addresses the Syrian refugee crisis, to Ingrid Kopp, who as the director of Tribeca Film Institute’s Digital Initiatives incorporated VR into the film festival in 2013.

“She’s the godmother, of course, it’s not even a question,” said Raney Aronson, the executive producer of Frontline, PBS’s investigative journalism series, who recently worked with De la Pena on a VR piece about solitary confinement. “This whole VR revolution, basically, came about because of her,” said Linc Gaskin, the founder of VR production company 8i.

The schedules of most everyone I met in VR had a frenzied quality and De la Pena, who speaks in a rapid fire staccato and has a brain that processes information at breakneck speed, is no exception. A few days before the conference I had arrived at the squat Culver City building that houses her company, Emblematic, only to discover that her previous appointment had run over, leaving her late to a call with a venture capital company (she missed it) then racing to Beverly Hills for a meeting with media company Fusion to discuss a virtual reality channel. “Another fucking crazy day in VR-landia,” De la Pena said by way of introduction.

Milk and Luckey remain better known than De la Pena, maybe because they have had more overt commercial success (De la Pena, who maxed out her credit cards to produce Hunger, still lives in a modest Mar Vista bungalow with, ironically, an enormous relic of an antenna on top) or because of her gender. “Is it good to be called tenacious?” she wondered aloud when I asked how she’s treated as a woman in the field. But since Hunger, De la Pena has worked at a feverish pace, and produced numerous other seminal pieces. (No one really knows what to even call narrative VR productions yet: “Pieces? Experiences? The vernacular is still being invented,” De la Pena said. “Just don’t call them videos.”)

People often single out De la Pena’s storytelling abilities when praising her work, but another thing that sets her work apart is that it’s often volumetric, like The Enemy. To experience them, I went to AOL’s office in New York, where Emblematic has a VR viewing station. I started with a piece called Project Syria, in which a young girl sings sweetly until a bomb explodes, then watched Kiya, which utilizes real 911 tapes to recreate the death of a woman shot by her boyfriend in 2013—when the ex-boyfriend lunged in my direction with a gun, I jumped back. But it was Across the Line, which places you amidst people angrily protesting outside a family planning clinic (it was made in collaboration with Planned Parenthood), that the bounds between real and virtual dissolved most thoroughly. “You’re a whore! You’re a whore!” one screamed (the audio is real). “Shame on you, you wicked pathetic woman!” shouted another. It felt personal and stunning, like being slapped.

“This medium removes so many of the barriers that keep people from connecting,” said Daniel Pacheco, a professor at Syracuse University’s S. I. New house School of Public Communication. “I think that’s what De la Pena realized, and before anyone else. I see her as a modern day prophet.” Yet her work also helped me understand what I’d heard about the medium’s darker possibilities—that it could also be used to torture, or even to numb someone to another’s feelings. “The first thing I thought when experiencing Hunger was, ‘Wow, I am feeling empathy,’” Pacheco said. “But the second was, this is an atom bomb of emotion, how could Vladimir Putin use this? How could corporate America use this? If you’re holding a giant machete instead of a kitchen knife, you should know that. Because you could do a lot of damage.”

VR’s connection to compassion can seem self-evident. Two of the main ways empathy is produced are proximity and shared experience, said sociologist Roman Krznaric, PhD, author of Empathy: Why It Matters and How To Get It, and VR offers shortcuts to both—it allows people to not just put themselves in another person’s shoes but be there, literally (or whatever that means when “there” is in virtual space) and look around. Cy Wise, community and events wrangler (her actual title) at the VR gaming company Owlchemy, said that because of this, she’d heard that in VR shooter games, players often recoiled from shooting other humans. “It feels too real,” she said. “You’ll probably see games that involve killing lots of robots and zombies instead.” Even in the porn industry, VR changes interactions. Ella Darling, who’s been dubbed “the first VR cam girl,” said that whereas in 2D-camming, “they just bark orders—show feet, show tits, show ass,” in VR-camming, “I get more pleases and thank yous. They treat me like they’re there with me.”

Still, the connection between VR and empathy is not well understood, and neither is it inevitable. Jeremy Bailenson, PhD, a cognitive psychologist at Stanford, has conducted studies that involve people entering a VR simulation and perceiving themselves (through a virtual mirror) as a different race. In one, this diminished bias (measured via an implicit association test that asks you to rapidly pair positive and negative words with faces of different races) but in another, it triggered it. The study that reduced bias involved a more realistic VR simulation, but as far as what types of VR fosters empathy, “The research is in its infancy,” he said. And while the medium certainly derives some of its empathetic powers from its believability, faithfully recreating experiences is far from enough. After all, “The richest sensory medium is life,” said Clint Beharry, director of design and technology at the story-telling non-profit Harmony Institute. “Which is totally forgettable in so many ways.”

Even in real life, what prompts empathy isn’t always intuitive. It’s easy to assume having the same experience as someone else means you’ll understand them better, or share their feelings. But that’s not necessarily the case. Take, for instance, a recent Northwestern University study in which subjects jumped into Lake Michigan during winter. Either before or after the plunge, the participants completed a questionnaire that assessed their attitudes about a man who’d decided not to jump for fear of the cold; those who hadn’t yet jumped proved to be more compassionate about his choice than those who had. What the researchers concluded was that encountering a person struggling with a challenge a person has since overcome can actually prompt contempt.

“We’re hardwired to [empathize], physically, and culturally,” said Beharry. “We empathize with vacuum cleaners with eyeballs in Pixar films!” But we’re also adept at disregarding whatever makes us uncomfortable. “We put up filters all the time,” said Krznaric. “It’s harder to dismiss if it’s right in front of you. But you can walk past someone suffering on the street and blank them out. We have this amazing ability to do it.”

Some weeks into reporting this story, I was passing a construction site when suddenly, for a few discombobulating seconds, I found myself wondering if the panorama around me—the parade of bikers, buses and cabs, the skyscrapers towering over two tractors digging up enormous slabs of concrete—was real or virtual. It struck me that it was kind of weird that we go to such trouble to create virtual worlds when the real one can itself be just as arresting. Especially since the main difference, as far as I could tell, was just that VR, because it’s new and unfamiliar, still feels crisp and full of potential, whereas the physical world has, for the most part, lost its novelty. Won’t this occur, eventually, with VR? “What happens when people in the morning can pop in contact lenses and see whatever they want?” said Ken Perlin, PhD, a computer scientist at NYU. (Yes, contacts and glasses that enable a person to fully experience VR or AR are coming, he said, maybe within five years.) “I’m interested, not about what happens when it’s cutting edge, but when it’s boring.”

Slater, who has spent more than 20 years visiting virtual space, said the medium’s power hasn’t lessened for him. “In VR, if something flies toward you, you’re going to duck before you remember it’s virtual,” he said. Even if some part of you recognizes what’s happening isn’t real, “It doesn’t matter, because the safest thing for your brain is to assume it is.” Looking at what’s happened to film is instructive, though. While movies still prompt emotions, their power has certainly lessened over time—audiences would no longer flee a theater in terror if an on-screen train appeared to be hurtling toward them, as supposedly happened in Ciotat, France, in 1895. It was actually Roger Ebert who coined the phrase “empathy machine,” referring to film.

What will VR ultimately be used for? Certain applications seem inevitable. When I met with Gaskin, whose company 8i is the first to enable volumetric VR that incorporates real humans, he showed me a video of a woman interacting with a hologram of her one-year-old as a newborn. “Imagine meeting yourself,” he said. Of course we’ll create holograms of ourselves. It’s the ultimate selfie. “Imagine encountering your grandmother as a young girl!” he said. But when it comes to empathy, even if VR does have a unique capacity to enable it, that doesn’t necessarily mean it will be used this way. At one point the VR company RYOT produced a piece for Google Glass that enabled viewers to see life from the perspective of a woman in Saudi Arabia, a Sherpa on Everest, and a kid living on the street in Kenya. (RYOT, which was acquired by the Huffington Post last year, has a Buzzfeed-esque model: “We did the first VR film in a war zone,” said co-founder Bryn Mooser. “We also find cat videos do really well.”) After seeing the footage, though, Google decided that instead they just wanted something for their golf app. “Remember how people used to talk about Chat Roulette?” said Krznaric, referring to the webcam program that many originally assumed would increase cross cultural understanding. “It’s just full of porn.”

Maybe the focus on empathy tells us more about ourselves than about VR’s capabilities. Recent studies that have found that US college students’ empathy levels have dropped 48% in the last decade, with the deepest decline in the last 10 years. “What I’m always interested in is what the stories we tell about the future tell us about the present,” said Genevieve Bell, PhD, an anthropologist at Intel who studies the interactions of humans and machines. “If the thing we most hope the future will deliver is empathy, what does that tell us about our current concerns?”