Create the Future of News with a .NEWS domain.

Find your .NEWS domain

Episode 8 ● Part 1

Nonny de la Peña

Dive in: virtual reality journalism puts people in the middle of news stories

When Nonny de la Peña envisions the future of journalism, it isn’t simply print or video or infographics that she envisions. The future that de la Peña imagines, and refers to as “a pretty amazing type of future” is one in which video, audio, good old fashioned research, and gaming platforms merge to create an immersive experience for consumers. Under this new model, the public becomes an active participant, engaging with the news in a manner unimaginable just a short time ago. The future is immersive. It’s interactive. And it’s here.

About Nonny
Pioneer. Journalist. CEO. Filmmaker.

A Harvard graduate with 20 years of experience as a journalist and documentary filmmaker, Nonny de la Peña has contributed to Newsweek, the New York Times, and Los Angeles Times Magazine, among many others. She is a graduate fellow at the University of Southern California’s Interactive Media Arts department, and currently engaged in exploring what she calls “immersive journalism,” incorporating video game platforms into journalism. As the CEO of Emblematic, de la Peña assists in the development and promotion of technologies necessary to achieve her goal.

Presented by .NEWS

Learn More About .NEWS

The Interview

This interview with Nonny de la Peña has been edited for length and clarity.

Would you please introduce yourself and say a little about what you do?

My name’s Nonny de la Peña, and I’m best known for having pioneered something I call immersive journalism, which is the use of gaming platforms to put people in the middle of news stories. By putting them on scene as events unfold it really gives people a visual sense of some really important stories. They get a connection with their whole body, and not just with their mind to what’s happening around them, which is really how we experience the normal world.

By being on-scene I think you get a connection that [causes] some people [to] call this an empathy machine. Because when you’re on-scene in the story it almost feels like it could happen to you. I think that allows us to really understand something more quickly, more deeply. In the same way that if you were at a real event, by looking around you would understand a story much more than if you’re watching it through video broadcast print.

What does that mean for the journalism landscape? What does that do for journalism in a general sense?

We had print, and then got shifted by film and radio, and then broadcast, and then online. I think that same sort of thing is happening with virtual reality journalism. By creating this whole new platform we’re seeing opportunities to tell journalism stories in a brand new way. It is something I think that can really help keep younger audiences informed, which is really what you want. You want an informed global citizenry. For younger audiences who are very comfortable with their digital selves, using virtual reality offers a way to communicate that is more in keeping with how people are experiencing their worlds.

We’re going to see more and more of this. Right now we’re still seeing a lot of 360 video, which essentially is still on a 2D plane. Google Maps, for example, just announced with Project Tango that it’s going to start scanning interiors of buildings. Why would they do that? Because you’re going to be able to walk around inside and outside locations. Intel and Qualcomm, their chips are going to allow you to scan environments and scan people very, very quickly. If I want to be on scene somewhere when an event has happened, and there’s someone with that chip they’ll scan it, and then I’ll be able to stand there and see it instantaneously. We, as journalists, will not have to build all new streets where an event happened. We’ll pop in the Google location, and then we’ll have to add our data on top.

You guys are way ahead, but tell me a little bit about where you are in terms of the tech, and what it requires for you to create the experiences that you’re creating right now. What goes into that?

Currently we have two trajectories going. We often rely on real audio, and then we have to rebuild the scene. We take very detailed bibles, as we call them, where we use images, police reports, witness statements, etc., etc. so that we can recreate a moment that’s already passed that wasn’t captured image-wise, and let you be on scene as events unfold. We also currently are working very closely with a company called 8i, which does videogrammetry, and that allows us to film people, and objects, etc. that you can walk around and be with. It’s video that has depth, and that volumetric capture is really also going to shift how we experience the world.

Is it like light-field technology, volumetric?

Yeah. It’s kind of like light-field technology. It’s a pretty amazing type of future. The fact that we’re going to be able to film in volume in the same way we can now start to scan with volume with just a cellphone or a camera. Here’s a good example of how we’re that technology in collaboration with Frontline. They have a wonderful partner, Dan Edge, who got access to a solitary confinement section of a prison in Maine. We sent in a photogrammetry team, and we have recreated the actual prison and the hallways through photos with depth. Again, you’ll be able to walk around the cell yourself. Then we’re bringing in several prisoners who were formerly in those cells, and we’re going to film them with the 8i technology. When they’re talking to you in video you can walk around. You’ll be able to be in the actual cell where they had been held as they tell the stories about being in those spaces. That kind of a combination is the type of future thinking that we’re focused on journalistically.

Do you consider what you do with Emblematic to be journalism?

Here at Emblematic we really think of ourselves as the news company of the future, a media company. What do we do? Well, we always have sections, right? You’ve always had your breaking news, your sports, your entertainment, your branded and sponsored content to help keep the whole functioning. We’ve sort of been following the same trajectory. We do some very serious journalism, and currently we have some great partners between Frontline, and Time, and Al Jazeera, and talking to Dow Jones, etc. We also have had some really wonderful partners in the branded space. Including a whole piece we did for Standard Charter Bank, recreating the Singapore F1, where you got to drive the Singapore Grand Prix in a virtual reproduction of the track, but we had everything tracked so that when you reached out to touch the steering wheel of the virtual car we had a physical car on hydraulics that you drove at the same time.

That gives you some idea of the diversity of the kind of things we do, but one of the things we’re best known for is our journalism. My background was correspondent for Newsweek. I’ve written for major news organizations like the New York Times, and the LA Times. We continue to innovate journalistic practice.

You guys are doing your own thing, but you’re also blazing a trail, and where that trail leads you kind of already spoke to. With your deep background in journalism why did you pursue this very technological and sort of developmental route?

The backstory on this was that I’d left print journalism to do documentary film because I didn’t feel like I could get in depth with stories the way that I wanted to. I felt that I could get closer with documentary film. Then I got a grant from MacArthur through Barrier Video Coalition to create a virtual Guantanamo Bay prison. I had done a documentary about human rights issues, including a big piece on that prison. By building it in a virtual way we offered an accessible, albeit virtual experience. I did this with an artist named Peggy Wild.

After we made that piece it was the day where I was like, “Hang on a second. This could be used for all kinds of journalism.” Then I was invited into a lab in Barcelona where they were doing the walk around content. I had my first experience in that lab where they were studying the bystander effect. They put you in a scene where you’re in a bar, and one guy is drinking at the bar, and another guy walks in and he’s got the wrong sports jersey on. This guy gets off the bar stool and is ready to kill him. Suddenly you’re in the middle of this scene of this possible fight, and I felt myself straining on the headset cord trying to get closer to see what was going on. That was the moment where I was like, “Oh, my God. My whole body is in the story. I can never put my audience out there. I have to put them here in the story with me.”

With that I began to really investigate the possibility to be using it for journalism. I was also a research fellow at the journalism school at USC, and there was professor there named Sandy Tolan, who was leading a class called Hunger in the Golden State, and looking at how people were going hungry in California, and food banks were overwhelmed, etc. I thought, “I really want to do one of those VR pieces to really explain what’s going on.” I actually asked the class, “Does anybody want to do this with me,” and nobody raised their hand, so I ended up with a wonderful intern who had just graduated high school in Irvine, and came from a professor I met, her daughter. She and I started recording audio at food banks until one day she recorded a day where a man waiting the long line who had diabetes didn’t get food in time and he collapsed into a diabetic coma.

The audio was extraordinary, and she came back to my office, and she was crying. I was like, “That’s what I want to build with.” Again, this was considered a bit of a nutty idea, so I had no funding. I put out $700 of my own money, and I had to become a better coder, and learn how to be a Unity developer, and kind of beg and borrow favors. Infiltrate the labs that had the equipment. Two years later it got into Sundance, but the only headgear we had for looking at was called the Wide 5, and it was a $50,000 pair of goggles. The head of the lab was like, “You’re not taking those to Sundance.”

Fortunately, there was some pretty smart people hanging around. One was this kid named Palmer Luckey, and a couple of people whose names don’t get mentioned enough as far as I’m concerned. Ty Pham, and Bradley Newman, John Brennan. Together we had to cobble headsets. I showed up at Sundance with these duct taped pair of goggles, and they’re signed “Palmer Luckey”. I have some great photographs because Palmer crashed in my hotel room and drove the truck for me, and then nine months later he starts the Oculus Rift. Two years later the company sells for $2 billion to Facebook. After that I’m not so crazy anymore.

What does success or victory look like for your company?

I just think we really stand a chance to help the future of journalism. We’ve seen some painful transitions in major industries as digital technologies have become first and foremost. The thing about virtual reality is it is not like 3D films, etc. because there’s ways that many, many people can create content. It’s a pretty open stuff for creation, which is really exciting. For our company that means that we can really grow journalistically, and providing content that puts people on scene, gives them a deeper understanding, gives them visceral experiences even around data. The infographics shouldn’t be this flat thing anymore. You should understand specifically what it is you’re really contending with. We want to get people off a 2D plane, and bring them into actually understanding stories in the same way we experience the real world, with things happening all around us.

Is that going to be a reality for most people soon, within the year?

I’m hesitant to predict how soon. Things have moved way faster than I ever expected them to. I’ll probably be conservative when I say five years from now it’ll be pretty common to have a pair of goggles around your house. The same way you would watch a movie, or have a different kind of experience than reading a book. I think that virtual reality is going to be in that exact same sort of living room choice. You’re going to want to try your VR experience as much as you might want to read your book.

If I’m a consumer five years from now, and I’m experiencing news in such an intimate way, do you imagine a more active, maybe a better informed readership?

Yeah. One of the things that we are very interested in working on, or plan to, is also, what does it mean to have agency in a story while still making sure it retains its linear structure [so that] you can’t really affect the unfolding nature of a piece. I.e., you can’t change the facts, but you still allow people access to the facts in a way that is more engaging and investigative, and give people some ways to literally pick up objects and examine them for themselves. I think that hopefully will get us closer to that sort of transparency question we’re always pushing on in journalism. We’ve thrown objectivity out. We know that’s not there, so transparency is where did you get your information and facts?

If we’re scanning real objects for you to look at then people are getting literally closer to the facts, and they can take a look for themselves. I think that can really lead to a whole other leap in knowledge in the world. Of course, I know I’m optimistic, but I was optimistic when I tried to build Hunger for $700, and actually got there in the end. I do believe that by offering these kind of experiences we begin to leapfrog boundaries, walls, cultural differences, and take people into stories that they didn’t have access to before. The hope is that we’re going to have a more informed global citizenry, and I think that is the underpinning of democracy. That’s how we work together to make decisions in the world.

Is that your mission? What is the mission of Emblematic?

The mission for Emblematic really is to become a media company that provides the kind of journalism for the future. That we can offer a model on a media company for the future that utilizes a new platform, and is able to offer visceral and deep content that looks at all kinds of global issues.

You kind of adjusted that in the middle saying to provide the model, which I get that sense from you that your mission seems to be blaze the trail, and then we’ll see what comes. Is that true? Is it like, “Let’s charge forward and see what happens”?

No, no. You’re right. You’re right. I don’t know. I had a painter recently say to me, “I don’t know why. I just have to keep making.” I guess I don’t know why, but I have to keep trying to think about the technology and how it can be best used for telling important stories. It’s something that’s true for me all the way through. I taught myself HTML in the ’90s. I’ve been at this for a long time, but finally the hardware is catching up in a way that we can begin to use it as tool to really tell more stories. That’s the goal. How do we tell important stories? I think that this platform allows people an understanding of their world in a way that’s kind of amazing and unique.

Subscribe for Updates

More video interviews with some of the biggest names in news are coming soon. Be the first to know about new episodes.