6 Comments

Summary:

New benchmark tests confirm what we have long suspected: Flash video on mobile devices just doesn’t look as good as HTML5 video, especially when it comes to HD clips. Support for hardware acceleration could alleviate most of these issues, but not for every user.

html vs flash video feature picture

Okay, it’s not just us: New benchmark tests confirm Flash video oftentimes really does look bad on mobile phones. EffectiveUI Software Architect Sean Christmann published extensive benchmarks comparing Flash and HTML 5 performance on mobile devices this week, and the results clearly show Adobe still has to catch up in the mobile video space. The nine mobile devices tested by Christmann generally had no issues playing back 720p content via HTML 5, but less than half were able to do the same in Flash with a satisfying performance.

Christmann built a pretty elaborate setup to test devices for their video playback capabilities. He encoded a source video with four different encoding settings to mirror the quality of video served from YouTube. He also hard-coded the number of each and every frame in the upper left corner of his test videos and then recorded the playback performance of each device with a high-speed camera. Afterwards, he looked at the recordings and determined how many frames each device had dropped during playback. The whole process was “thoroughly painful,” Christmann wrote on his website, but the results are definitely interesting:

Percentage of frames played during video playback on various mobile devices via HTML5 and Flash. Says Christmann: “Subjectively, I would argue that video that stayed above 70% looked good during playback. Anything below that mark will have too much stutter and really starts looking like crap.”

For a more visual take on these results, compare the video Christmann did of a Droid X playing a 720p H.264 baseline profile via HTML 5…

… with the same device playing the same clip via Flash. Notice all the dropped frames?

There are a few things worth pointing out:

Christmann used two encoding qualities for HD video; 720p H.264 baseline profile and 720p H.264 high profile, which looks even better and is for example used for Blu-ray discs. Most devices had no problem playing back H.264 base profile video via HTML5, but some devices simply refused to play back 720p high profile videos altogether. Also, iPads and iPhones don’t support Flash, so you won’t find any comparison number for these Apple devices.

More importantly, Christmann writes that the most recent Gingerbread update brought the Galaxy Tab results close to 100 percent, but he wasn’t able to re-record the test and include those findings due to time constraints. The same was true for the Motorola Xoom: With Android 3.0, it seemed to drop about half of all frames during 720 base profile playback, but Android 3.1 brought its Flash video playback up to par with HTML5 (the table does include the 3.1 results).

The reason for these discrepancies? Hardware acceleration. Playback of HD video on a phone really requires some dedicated support from the device’s chipset, and Adobe needs help from hardware manufacturers, as well as mobile OS makers, to enable this kind of hardware acceleration. This is akin to Flash playback on Macs, which used to be a real pain until Apple finally provided API access to Adobe to make use of your Macbook’s hardware acceleration capabilities.

This means Adobe has a good chance to catch up with HTML5 video playback on devices sporting the newest Android systems, but the flip side of that is that many mobile consumers may be stuck with a bad Flash video experience for quite some time. Data from Google shows that 90 percent of Android users still have Android 2.2 or below. Add that Adobe still has to support a number of different devices and chipsets to actually make hardware acceleration work in mobile, and you’ll get a sense of how the challenges ahead for Adobe.

The good news for the company is that Flash actually outperformed HTML5 in a number of other benchmark categories, including a bitmap drawing test and a vector drawing test (one should point out that Adobe was involved in funding the tests). This means that mobile users will have more fun with Flash-based browser games than with HTML5-based browser games. But in video, at least for now, HTML5 has the lead.

You’re subscribed! If you like, you can update your settings

  1. John Dowdell Tuesday, June 14, 2011

    Hi Janko, not all of your readers may get past the headline down to the final few paragraphs, and so may take away an incorrect impression of what those tests show.

    tia, jd/adobe

    1. I read all the way and also took a good look at Christmann’s test page. His results are great news for Flash. I’m an HTML/JS/CSS developer, and there’s no way I trust these technologies to easily make slick, smooth, complex games and apps involving stutter-free motion – the desired outcome of any graphics-rich application. Choppy motion is not cool. Good to know that HTML video is strong, but video is only part of the package. Flash is an awesome developing environment – very mature. HTML/JS is a hair-pulling frustrating thing to get right when creating multimedia-rich, detailed and complex content applications for web or devices. Right tool for the right job.

      1. Agreed. Most of what I use Flash for on my phone is Kongregate Arcade and the occasional video. Flash is way more than just video.

        Adobe Flash Builder is nice to work with. Adobe AIR has a very full featured SDK. Last time I played with it was with some BlackBerry Playbook coding, and it was mobile ready.

        It’s a real programming language. You can write real applications in it. It’s not just an RTSP client.

  2. Incircle Media Wednesday, June 15, 2011

    HTML5 is really excellent evolution. it have too many best attributes i like html5. But its not working most of IE versions.

  3. The encoding methodology seems flawed. You would not use a High Profile encoded video on a mobile device. High profile is intended for disc storage and broadcast applications. Baseline is used for mobile applications where low cost processors are used.

    To expect a smartphone to be able to handle a 720p High profile is flawed. Also, why send such a high resolution image to such a low resolution display (smartphone). The excess pixels are just overhead in the downscaling the image and just add up to increased bandwidth with no additional benefit. The video resolution needs to be mapped 1:1 to the display resolution, especially in a mobile application, so that you don’t waste bandwidth.

    1. Hey TimeKeeper,

      the point of Tests like these is to see the limits of performance for both methods.

      If videos are all encoded in 480p, they would display fine however in an ever evolving technology spectrum these tests are important to decipher the limits of these devices.

      That said, it’s clear that Flash is superior in every other field in these tests by almost twice the execution. Credit really does have to go to Adobe who in the past 18 months or so really have optimized the Flash player into a true cross-platform media player that can rival native performance of this content.

  4. HTML 5 video outperforms Flash on mobile devices http://t.co/7eeBFRFD

Comments have been disabled for this post