Virtual reality opens new storytelling challenges for animators

0 Comments

Some of the first animators in virtual reality gathered over the weekend in San Francisco. Their films ranged from beautiful to awesome to inane, but they were all up against the same challenge: How do you make a film when the viewer gets to direct the shot?

It’s a question the entire industry is weighing. But even in these early days, some inventive ideas are emerging.

Avoiding the cheap tricks

The knee-jerk reaction to the interactivity virtual reality provides is to build a “choose your own adventure” style of film. The viewer can look in any corner, so you better make sure there is something happening in all of them!

But great films have a story. To preserve it, VR animators have the challenge of embracing the viewer’s new role as the final director. Their job is to coax the viewer’s cut to be as close to the director’s cut as possible.

A scene out of "David and Goliath," a short VR film by Visionary VR.

A scene out of “David and Goliath,” a short VR film by Visionary VR.

There are plenty of cheap ways to accomplish that: choruses of “Look over there!” and loud begging and pleading before the zombie attacks do the trick. But virtual reality directors are already developing more subtle tactics that are just as effective.

The easiest, and perhaps most powerful, is sound. You hear the monster approaching at your back and spin to see it. It aligns with how we interact with the world everyday.

In “The Last Mountain,” a film by editor and director Avram Dodson, the viewer is introduced to a low-poly world bounded only by the sky, horizon and rocks. It’s massive, and it isn’t immediately clear where to look while you take it all in for the first time.

Dodson described getting even first-time VR users to focus in on the story through sound. A lizard perks up from the rocks and vocalizes, and then a giant stone creature rumbles while emerging from the distance.

“When our little lizard friend chirped, their head just snapped right to it,” Dodson said.

Having a clear lead character helps, too. “The Last Mountain” shifts your attention from the lizard to a small stone creature and then back to the lizard. It’s always clear who the center of attention is, so naturally you want to follow what happens to them. And when the little stone dude looks into the distance and exclaims over the giant, you’re compelled to look too. It’s just how we work as humans.

Reimagining the scene cut

One of the most inventive movies to come out of 2014 was “Birdman,” a film made to look like it was filmed with one long shot. It feels continuous, while at the same time weaving together a complex cast of characters.

That’s a very effective style for virtual reality. It’s certainly still OK to include scene cuts, but they can be dangerous; the cuts in “The Last Mountain” left me with a twinge of vertigo. If filmmakers can manage it, it’s better to avoid them altogether.

The best example of the VR long take came from “Butts,” a classic story that reveals the secret to happiness is shooting confetti out of your butt. (“Oh my god,” said the one child in attendance at the film festival.) Creator Tyler Hurd had already made the film for 2D screens before deciding to adapt it to Oculus Rift. If you watch the two versions of the film, you can see the compromises he had to make:

The biggest change is no cuts. “Butts” revolves around two characters who spend a lot of time staring into each other’s overly-expressive eyes. But cutting between their viewpoints that rapidly in virtual reality would be nausea-inducing. Instead, Hurd redesigned the film so it’s more like the characters are on a stage and the viewer is in the audience.

Despite the confinements of the stage setup, there are still some complex transitions in “Butts.” The best example comes during what Hurd calls the “assfetti waltz.” The two characters rocket into the air on their jets of confetti and then explode. Your eyes follow the confetti back to Earth, where the two are once again staring into each other’s eyes. It’s a scene cut without an actual scene cut.

Don’t call it “choose your own adventure”

Visionary VR, the last team to demo at the festival, thinks it has an editing solution for those seamless cuts. Its Focus software helps people build stories on that central stage that Hurd found so effective. But it also allows for action in the secondary zones to the left and right of the viewer’s main field of view. When they look away from the stage, the central action pauses momentarily and the secondary content comes to life.

“How a story is structured can be, in theory, completely malleable,”co-founder Jonnie Ross said. “We think all of it is possible.”

It might feel like a choose-your-own-adventure film, but it’s not. There is still a central story happening. Looking away just prompts the secondary storylines to come into play. Like in a video game, you can wander off and interact with your surroundings, but you’re always expected to return to your main mission.

A new art form

Among the giants and butts was a fourth film that felt different. “Sprout,” a short by Pixar technical director Nancy Tsang and artist/animator Lou Hamou-Lhadj, had a magical quality even beyond what we have come to expect from Nemo and Woody. The movement, colors and even camera focus were all perfectly expressive and beautiful.

Tsang and Hamou-Lhadj, who work together on projects like “Sprout” outside Pixar and call themselves the “Dabble Duo,” said they applied the same painstaking frame-by-frame film development that they follow in their day jobs. As a result, the action in “Sprout” is only 17 seconds long. There isn’t much of a story — just a charming character and scenery. But they plan to make longer films in the future.

What they found during the making of the film is the tools are there to make gorgeous content for VR. Developing for Oculus means you have a gaming engine at your disposal — a much more powerful computer than the version found in a DVD player. Suddenly there is the ability to render even complex graphics in realtime as the viewer moves their head.

A frame from "Sprout."

A frame from “Sprout.”

Developing for the medium required a combination of the 3D animation software Pixar uses and game development tools Dabble Duo was not so familiar with. Some steps were the same, but effects like shading, lighting and blurring to create depth of field all happened outside the normal filmmaking channels.

“It’s very different for us from what we’re used to doing,” Tsang said.

Tsang said Pixar is not working on a virtual film to her knowledge, but “there are a lot of people there who are very much aware of what’s going on.” Sprout is proof that Oculus Rift is a very worthy canvas not just for game developers, but for artists as well. Art is all about emotion. So is virtual reality.

Comments are closed.