From virtual surgery at the Royal London Hospital to digital browsing at Ikea, virtual reality (VR) is popping up in unexpected industries as the latest tool for shoppers, educators, and more. But as VR’s applications broaden, the question for many people becomes: What are we actually going to do with it in our everyday lives?

Ken Perlin, professor of computer sciences at NYU and founding director of Media Research Labs (MRL), an interdisciplinary computer research lab that brings scientists and artists together, believes the answer is social—a shared virtual experience, in which users connect and interact with one another.

“The experience isn’t interesting until you’ve talked about it with someone,” Perlin told Magnify. “I think experiencing it directly together is the most interesting place for VR to go. It’s what we as people do best. It’s what we’re good at.”

Perlin’s perspective is informed by extensive experience in both computer science and entertainment. In 1985, he developed Perlin Noise, a technique used to make computer graphics more realistic, and has done similar work since then. But his skills go beyond computer science – in addition to his multiple computer graphic awards, he was also a featured artist at the Whitney Museum in New York and won an Academy Award for technical achievement in 1997.

Social potential: Hanging out on Mars

What Perlin’s experience in art and entertainment has taught him is that humans are, first and foremost, a social species. Even online, we congregate around social networks, and those networks are increasingly using virtual platforms to create rich, diverse experiences. Facebook, for example, recently launched an app called Facebook 360, which allows users to shoot and post 360-degree photos and videos–and also react to others’ 360-degree content. It’s not the goggles-enabled version of VR that many people think of, but it is a form of immersive experience, and therefore, a form of VR.

At the Oculus Connect 3 conference, Mark Zuckerberg gave the audience a glimpse at the Oculus Rift’s social platform. Imagine being able to play cards in real time with friends across the globe. Now, what if you could have a sword fight with those friends on Mars, or watch a movie together at the bottom of the ocean? As Zuckerberg showcased, interplanetary hangouts are just the beginning.

A bump in the road: Issues with immersion

Established social networks are focused on rolling out VR features, but social VR platforms already exist. Of the more than 450 companies focusing on VR, dozens offer a shared virtual experience. Why aren’t these companies dominating the industry? It’s an issue of the clunky, obstructive hardware that is currently the industry standard.

“You’re not going to be putting this big piece of hardware in front of your face and blocking your eyes from other people in the future,” says Perlin. Instead, he predicts the use of lightweight headwear that won’t impede regular vision, and a pocket-sized brick akin to a smartphone to provide power, connectivity, data compression, and computation.

There is also the issue of maintaining the illusion. A successful immersive game can trick the brain into believing that the virtual experience is a real one, with sensations in the game triggering real physiological responses. But often, the illusion is shattered by a technological shortcoming, including latency. Though a 250-millisecond round-trip may not seem significant when talking on the phone, a delay of the same duration is painfully apparent when people are using VR in the same room. Imagine one user tapping another user on the shoulder in the physical world, but not registering the reaction in the virtual world until ever so slightly later. It’s disorienting, and breaks the “we’re-somewhere-else” spell. In his own work at the MRL, Perlin is working on a custom server dedicated to decreasing latency for those who are in the same room.

In a perfect virtual world, you’ll see UGC

As companies creating VR technologies work out the kinks and social applications become more widespread, a surge of new users will be troubleshooting the technology. If other forms of consumer technology are any indication, user feedback will ultimately lead to user-generated content. A decade ago people were recording home videos and inviting family and friends over to watch. Then YouTube arrived, and anyone anywhere could upload a video that would potentially be viewed by millions. The revolution spread across surrounding industries, including music. (See: Justin Bieber.)

“In anything—in literature, in theater, in music, in film – there’s always been a certain kind of innovation when you have user-generated content,” Perlin said. “Suddenly, any kid with a guitar could record something and get it out to people, get out a demo. You started getting talented people. The same learning curve is going to happen with VR.”

YouTube showed clearly how quickly communities form around user-generated content, and popular platforms such as Minecraft show just how intense the engagement in user-generated communities can be. As Perlin notes, tapping into the user pool will broaden the number of talented people creating virtual environments and allow professional creators to identify untapped talents. That, in turn, will raise the bar for the entire industry.

System requirements for the future

For VR to take hold, it will have to allow for experiences that are more interactive and organic than highly choreographed. Companies hoping to create rich, multi-faceted virtual experiences must restructure the way they create their worlds, blending the actual and virtual into one cohesive experience that leaves room for social interaction. And they must be sure to foster the organic evolution of the technology by encouraging user-generated content.

“We’re not yet at the point where the technology can support a social VR experience that is perfectly seamless, but once we can, once we get there, it’s going to be very big,” Perlin concluded.