The Onion joked recently that Netflix was going to introduce a new Browse Endlessly plan. For $5, it would offer consumers a way to scroll through its entire library without ever deciding on a single title. Like every good joke, it was funny because there was some truth to it: At one point or another, many Netflix subscribers have found themselves overwhelmed by the 50,000 or so movies and TV show episodes available on the service.
Chances are Netflix’s old TV apps had something to do with this.
Netflix launched the user interface (UI) pictured above for smart TVs and connected devices in 2009. It tried to give users a flat and clean way to navigate through the company’s catalog. But the UI also treated each and every title the same, presenting endless rows of box cover art.
That’s why in early 2012, Netflix began an ambitious redesign process that involved seven A/B tests, a new image format and a complete rewrite of Netflix’s TV rendering engine. Key members of the team responsible for the new UI recently sat down with me to talk about the engineering and design process that led to the launch of Netflix’s new TV UI in November of 2013.
Telling the story of a title with one big image
In February 2012, designers at Netflix started to build prototypes for a different approach that focused on big pictures, which it thought would be capable of telling the story of a title without many words. The idea was that someone walking through the room and taking a casual glance at the TV should know which title is about to get watched, explained Dantley Davis, design manager at Netflix.
Davis actually first toyed with this image-centric approach five years ago but quickly ran into a problem: Netflix simply didn’t have the artwork to make it work back then. Hollywood studios only had promotional images for few select titles, and the company didn’t want to curate its catalog to spotlight just major U.S. motion pictures when some members were perfectly happy to just watch obscure documentaries or Korean dramas. That’s why the company tasked what it calls its “enhanced content team” — employees that scour through Netflix’s catalog and write tags and descriptions for each and every title — with grabbing the best frames straight from the video files.
The TV team also decided to replace the box covers — which had long been synonymous with movies on Netflix’s streaming service — with landscape-mode images for each title. “Box art was obviously optimized for shelves,” said Alvin Lee, senior user experience manager. And as consumers moved away from DVDs and towards streaming, their visual expectations were also shifting, allowing Netflix to ditch box covers as a metaphor for movies and TV shows and in turn clear up more space for those big header images.
WebP to the rescue — except when it doesn’t work
Working with bigger images led to some technical challenges, one of them being bandwidth consumption. Granted, an image or two is nothing when compared to a full movie streamed in HD, but if you want to load lots and lots of images on the fly without too much pre-caching while a user browses through a catalog, then you need to be able to transfer these images as quickly as possible.
That’s why the team decided to use WebP, an image format first introduced by Google in 2010, when it started to test the feature image element of the new UI in early 2013. “With WebP, the files are about one-third of the size,” explained director of UI engineering Matt McCarthy.
However, the team ran into a significant issue when it began to build out the final UI for consumers in the third quarter of 2013, just months before it was scheduled to launch. It discovered that lower-end Blu-ray players and streaming boxes couldn’t handle WebP decoding on the fly, or at least not as smoothly as Netflix would like them to. That’s why the team decided to still serve JPEGs to cheap consumer electronics devices by default, but send WebP images to game consoles and other more powerful machines.
And just selecting the right format for the right device didn’t mean the job was done. Throughout 2013, the TV team’s designers and engineers had weekly meetings in front of a row of TV sets. Stacked in metal shelves underneath those TVs were dozens of devices, ranging from Roku streaming boxes to game consoles to barebones circuit boards with next-generation processors from major chipset manufacturers.
The team often tested two identical devices against each other, both running the same UI with slightly tweaked image settings. The goal was to find that sweet spot where images look great but still load quickly, and transitions are smooth — something Netflix internally calls a “recipe” for image encoding. It was a time of constant fine-tuning, a time when even something as miniscule as a 150ms delay during an image transition warranted further tweaks. “You will feel that,” insisted McCarthy.
Less eyestrain, more information
The big imagery is one of the most obvious features of Netflix’s new TV UI, but the company’s TV designers also paid close attention to the way users interact with it. The previous iteration of Netflix’s TV app had an information bar on the right side that gave users all the necessary information about a title. When Netflix ran an eye tracking study on that UI, it found that consumers were forced to constantly shift their attention to the sidebar, and then back to the highlighted title on the left.
In the new interface, it shifted all of that information to the top left, right above the highlighted title, leading to a lot less back-and-forth for users.
As part of that shift, Netflix also introduced something the team described to me as “evidence” that you should watch a certain title. Sometimes, the TV app may tell you that your friends watched a certain movie. Sometimes it may tell you that this is a good title because you liked this or that movie in the past, and sometimes, it may just say that a certain movie won an award at a film festival.
That last type of evidence was tested through an A/B-test. Netflix is famously analytics-driven, and tests new features all the time before it decides whether it should make them available more widely. For the new TV UI, it ran a total of seven separate A/B-tests.
However, the film festival “evidence” showed that these tests aren’t always easy to interpret. Lee told me that the response to this was flat for most people. However, subscribers who watched a whole lot of Netflix, and possibly have exhausted other ways to find new content on the service, responded positively to it. In the end, Netflix decided to keep the feature.
Moving away from Webkit to serve all platforms
In addition to the UI changes, Netflix also took a good look at the plumbing to optimize the code that makes Netflix work on TVs. The company had been a pioneer of HTML5 on Smart TVs and connected devices. Using Webkit in combination with a cloud-hosted HTML UI allowed Netflix to quickly make changes to its TV experience without the need to push app or even firmware updates to each and every device — or at least that was the theory.
In practice, Netflix found itself in a situation where only powerful devices like Sony’s PS3 could support the full Netflix experience, and engineers needed to come up with a whole bunch of tweaks for lower-powered streaming boxes. Features like Netflix’s postplay experience, which automatically starts the next episode of a TV show after a viewer has finished watching an episode, didn’t launch on a whole range of devices at all. “We were in this terrible state where innovation was slowing down,” Lee told me.
And the move to a new rendering engine had an interesting side effect for Netflix: For years, the company used PS3 owners as guinea pigs for most of its A/B tests, because the powerful game console could easily handle most tests. However, the focus on the PS3 came with the danger of skewing the tests towards a younger, more tech-savvy audience that spends more time on gaming than traditional TV. With the new platform widely deployed, Netflix designers and engineers can now test its future ideas much more widely — and they do have a lot of ideas, as Davis assured me: “The current design just scratches the surface,” he said.
More personalization, more context-awareness – and trailers?
Asked about his personal vision for the future of Netflix on TV, Davis said he’d like to see different devices be aware of each other as well as the context in which titles are watched. If a user watches a Navy Seal documentary on his phone during the week, then his TV UI should suggest war movies as Saturday night entertainment.
Lee seconded that thought, adding that a TV app of the future should know whether you’re watching something by yourself or whether your family is in the room, tweaking content recommendations accordingly. And McCarthy said that he’s interested to explore additional input and interaction scenarios, like using a tablet to cast movies to your TV, using a Wii-like air mouse and other alternatives to the traditional remote control, which largely relies on the D-Pad to navigate on-screen. “Left/right/up/down feels pretty weak,” he said.
To be fair, some of these ideas may never see the light of day. But even three months after the launch of the new UI, the TV team is already busy working on future tweaks. Some of the things that the team is playing with have to do with the way the app presents titles that you’re interested in.
The new big images already help to tell the story, but the team at Netflix is also thinking about possibly adding preview videos and even personalize the description of a movie based on the types of videos you’ve watched in the past. As Davis said: “Star Wars can be a princess movie to some users.”