34 Comments

Summary:

Microsoft has released IE9, the latest version of the venerable Internet Explorer browser. To check out how it compares to to its competitors, I installed it (which, unlike other browsers, required me to restart my machine) and ran it through a few different benchmarking tools.

Screen shot 2011-03-15 at 11.23.04

UPDATED: Microsoft has released IE9, the latest version of the venerable Internet Explorer browser. It boasts greatly improved support for standards like HTML5, a new design and greater speed, as well as some nifty new features, like privacy controls and being able to pin websites to the taskbar.

To check out how IE9 compares to its competitors, I installed it (which, unlike other browsers, required me to restart my machine) and ran it through a few different benchmarking tools.

Acid3

Acid3 is a benchmark which checks how well a  browser follows selected web standards. The browser is marked out of 100.

UPDATE: Replaced figure

Microsoft has made an effort to improve the standards support in IE in this release, and as you can see, IE9 scores a creditable 95 (much better than IE8 which scored a paltry 21), but it still can’t match Opera and Chrome’s perfect scores.

The HTML5 Test

Acid3 doesn’t test for HTML5, as it’s a newer web standard, so I also ran the browsers through The HTML5 Test, which provides an indication of how well they support the upcoming HTML5 standard and its related specifications.

UPDATE: Replaced figure

HTML5 support is important, because as more browsers are able to handle its advanced features, developers will be able to build much more powerful apps (see Why HTML5 Web Apps Are Going to Rock Your World). As you can see, although Microsoft has made a bit of fuss about IE9 supporting newer standards like HTML5 (even providing some online demos to show off the browser’s HTML5 chops), it’s actually lagging behind its competitors here, although none of them scores perfectly. However, I’d take this benchmark with a pinch of salt, as the standard isn’t finalized yet.

Kraken

Kraken is Mozilla’s JavaScript benchmark, primarily based on the SunSpider suite. It tests how well a browser handles JavaScript and is supposed to simulate a number of different “real world” scenarios.

UPDATE: Replaced figure

 

JavaScript performance is vital, because we’re all increasingly reliant on JavScript-heavy web apps. The Kraken score above is a the time in milliseconds it takes for the browser to complete the test runs; a lower score is better. Note that JavaScript benchmarks should be regarded with a healthy degree of skepticism; it’s incredibly hard to create a realistic benchmark that accurately indicates how fast a browser will be in day-to-day use, so these tests can only give an indication of how fast a browser might be. That said, once again, IE is lagging behind the best here: It took twice as long to complete the tests as Chrome, and nearly three times as long as Firefox (although it’s much better than the beta of IE9, which scored around 50,000 ms).

Overall

IE9 is a great improvement over previous incarnations; it has caught up to the pack and now doesn’t lag so very far behind the other  browsers available. In particular, Microsoft should be applauded for its efforts to support web standards in this release. However, unless you really want some of its novel features, like the ability to pin websites to the taskbar, or your choice is restricted by your employer to Microsoft’s browser, I still can’t recommend it over Firefox, Chrome and Opera.

Internet Explorer 9 can be downloaded from microsoft.com; it requires Vista or Windows 7.

  1. Why are you graphs so misleading? Why on the ACID3 test would you have the value go up to 102 when 100 is the highest? It makes it seem as though no browser has received a perfect score. Then your second graph (HTML5) only goes up to 300 when the value is up to 400. You really need to attend a webinar or something on graphs.

    Share
  2. The difference on the Acid3 test between Chrome’s 100 and IE9’s 95 is SVG fonts and SMIL animations. That’s some really obscure stuff, so that won’t make much of a dent.

    The HTML5 test makes no sense whatsoever, since HTML5’s still not done so the specs could change at any time. MS has implemented some cool stuff, but probably wants the spec to be final before implementing any more.

    The JS engine is faster then the one in IE8, but nowhere near as fast as the one in Chrome (I’m highly sceptical about the Kraken test, since it’s from Mozilla and seems to prefer Firefox more then all the other benchmarks do).

    Overall I think IE9 will be a great option for most average users, for “high power” users (developers, l33t h4xx0rs, script-kiddies) Chrome is the best choice at this time for its speedy JS and awesome debugging tools.

    Share
  3. Taking a look at tests above, I conclude that Microsoft is going to lose it’s leadership in browser’s market as Chrome and Firefox are already better and improving faster than Microsoft’s browser. It’s impossible for IE to revert this trend.

    Share
  4. I have seen too many Anti-MS articles before, but this one takes the cake. Seriously, setting aside your bias, even your testing is flawed.

    Take a look at the test from ZDNET. “http://www.zdnet.com/blog/networking/chrome-10-vs-internet-explorer-9-reconsidered/792″

    Don’t write tech articles just to be cool. Write it for its worth.

    Share
  5. You’re really biased. I’ve been using the RC since it was released and it’s awesome. Please let go the google kool-aid.

    Share
  6. Interesting facts but your visuals really tell a different tale than you are saying. They exaggerate your points. Your graphs start with a baseline so high it makes the bottom end seem so much worse than it is. Your first graph the point difference is only 5 points but your visual is designed to confuse people at a glance into thinking it’s much greater. Very disappointing article. Makes me lose trust in what you are saying.

    Share
    1. I’m sorry if you think so, it really wasn’t my intention to deliberately mislead anyone.

      Share
      1. That was exactly your intention. You’re just another Microsoft-hating tool with no credibility, especially after this article.

        Share
      2. Fix your graphs then… They are completely misleading, as has been pointed out by most commenters.

        Share
  7. LOL — what’s up with the misleading graphs, FOX News?

    If the scale is from 0 to 100, why make the graph from 94 to 102?

    You have them all drawn to unfairly exaggerate IE9’s deficiencies.

    Share
  8. Very informative.
    But why do you use a beta version of Chrome in your comparison?
    If the others are GA versions that you’re using as a comparison, then you should have also used the latest GA version of Chrome as well to be fair – not a beta (no matter how close it is to being released).
    But all in all, some useful information in here.
    Thanks!

    Share
  9. Stable version of Chrome is actually faster than mozilla on Kraken benchmark with the inclusion of crankshaft version of V8 javascript engine. Check mozilla’s own Are we fast yet page: http://arewefastyet.com/

    Also, you should not use non zero Y axis on such graphs.

    Share
    1. mda, arewefastyet shows the most recent unstable bleeding-edge versions. Stable Chrome is slower than Firefox 4 on Kraken, and stable Chrome doesn’t include crankshaft yet.

      Share
  10. Performance is certainly an important factor to consider when deciding to install Internet Explorer 9. Another is a set of security features that Microsoft included in the new browser:

    http://isc.sans.edu/diary/Internet+Explorer+9+is+out+includes+new+security+features+/10552

    Overall, IE 9 looks like a must-have update for people still using older versions of Internet Explorer. The security features of the new version also, in my opinion, bring IE 9 above Firefox on a Windows platform.

    Share
  11. While I wouldn’t call the graphs misleading (they are labeled ‘correctly’), they are certainly done in an EXTREMELY biased way — what’s with not having full graphs and totally skewing the Y-axis to make IE9 look bad? I suppose you are happy to join the LONG list of media folk who like to bash MS. Good for your pageviews. Not good for your reputation/journalism standards.

    Share
  12. Your HTML5Test is inaccurate. Chrome scores over 300, when you include the few bonus points it receives. Its base score is well over 250.

    Share
  13. Thank you to all the commenters who pointed out to me the BS in this article, namely the misleading graphs and the use of pre-release betas for IE competitors.
    I hope the “journalist” is suitably ashamed.

    Share
  14. Chrome moves faster than all the others anyways.

    11.0.696.12 auto downloaded this morning gets 288 and 13 bonus points on the HTML5 test.

    IE9 is a huge step above earlier versions and should be a crucial update in my opinion and forced upon all windows users. Earlier versions are nothing but a security hole.

    Share
  15. From a developers perspective, it’s still not on par with the others. The major holes it is missing are the FileReader API, CSS gradients, CSS Drop Shadows, and CSS Animations. While it’s great they have support for a lot of HTML5, they could and should have gone all the way.

    I’ve been doing workarounds, tricks, and hacks around IE for years now. Forget that the HTML5 and CSS3 spec arent perfect yet. I have 4 out of 5 browsers supporting things like the File API, I’m gonna use them. Why should I code for the lowest common denominator? And as for the charts, everyone please ignore them. For JS we are down to arguing about milliseconds and for HTML5 no one cares that the “aside” element has been properly implemented.

    I agree it’s better than what M$ had out before, but most users aren’t savvy enough to care about their browser. They just expect to get the same awesome experience in every browser. It just makes it harder on a dev. With that said, most tweaks when making sites now will be for functionality instead of really bad CSS support, so I’ll take the little victories.

    Share
  16. Yeah, your graphs ARE highly misleading, as lots of people have told you.

    The point of bar graphs is to graphically represent the proportions between things, and skewing the axes in such a way distorts that proportion incredibly. (IE9’s acid is 95% of the max, not 15%).

    Failing to update them to use appropriate axes means you ARE choosing to be misleading, on an ongoing basis.

    Share
  17. Those graphs are so misleading! Also, why should you tell us if we should care to get ie9? I use both IE9 and Chrome and love them both. Just cause it’s Microsoft doesn’t mean IE9 is going to be bad.

    Share
  18. lol, your ACID3 tests make it look like competitors are 3x – 7x better than IE in this standard. Next time, make a graph that is even taller and start it at 95 so that IE looks like it isn’t doing anything to support the tests so that we can bash MS for not caring about the community. Then we can say in a mature voice that we would not recommend MS at this time.

    Share
  19. they never compare those browsers with Safari…

    Share
  20. Very misleading the graphs are so inaccurate.

    Share
  21. Why is Safari not included in this?

    Share
    1. Because I wasn’t aiming for a comprehensive browser shootout, Walt. I may do that next week when Firefox 4 is released.

      Share
  22. Interesting how you’ve run these test and yet, IE9 somehow has fallen short in the results. I still have to try the platform myself and I’m mostly using Firefox or Chrome for the same reasons you do ( perhaps ). Thanks for sharing these data.

    Share
  23. I have installed IE9 and deeply liked it. Contrary to what you said, installation did not require a restart, except if you did not close your programs during installation you will need to restart your computer. I have been using Firefox for 3 years and it was really good, until several months ago. Firefox now often freezes and becomes insensitive to clicks frequently. In addition it easily gets sluggish and crashes, partly because of its add-ons. I will wait for Firefox 4 release to make my final decision, but in current terms, IE9 easily outpaces Firefox 3.6, it works like a dream. I need to say, I am not good at technical things and did not understand may tests you mentioned in the article. But for an average user like me, pleasure is important, and I am pleased with this latest release.

    Share
    1. Glad you like IE9, Jessica. Benchmarks are not the be-all and end-ll, of course, especially as the browsers are now at least all pretty close to one another in terms of performance; features and personal user experience are crucial. However, IE9 is still behind the others in terms of performance and standards compliance.

      Share
  24. Gotta agree with others mentioning the oddly arbitrary chart decisions and the lack of clarity of what some of those numbers really mean, especially the Acid3 and HTML5Test scores…

    Share
  25. I have replaced the charts.

    I’d also like to note that I’m not a Microsoft-basher. As I said in the post, I do think that Microsoft have done a good job with this release to get IE in the same ballpark as Firefox and Chrome. However, for web workers, I still cannot recommend IE over those two.

    Share
  26. I don’t know about the benchmarks but I upgraded to IE9 yesterday and had nothing but trouble. The slider on the window was sticking, I couldn’t get silverlight videos to play. I was horrified how bad the experience was for Microsoft’s climax release of their “modern browser”. I am actually pulling for Microsoft and have really re-thought my switch between Linux and Windows over the last three years. Windows 7 was impressive and really had me using Windows again much more often. The experience I had with IE9 was so bad I thought I was running a hacker version or something. I have Windows 7 64bit so that was the version I installed. Maybe I’ll try the 32bit version.

    I read another blog somewhere where it was more designer focused and they raised some pretty good arguments against the UI design in IE as well. The new IE icon as they mentioned looks old and retro,not in a good way. One thing that keeps me in Chrome more then any other browser is the integrated search in the address bar. This has become second nature for me and when using even Firefox my workflow is broken by switching to the command box to search. IE9 doesn’t even have one on by default. Maybe people use the internet different then me but the integrated search in the address bar to me is one of those things that should just “be” how it is. I hope to have a better experience with IE9 32bit and will check back if I need to swallow any words or if I got the wrong download link for the new release but this was a train wreck as far as I was concerned and not even functional as far as I am concerned.

    Share
    1. I wanted to clarify some things I had wrong. Mostly due to the fact that things were going so horribly wrong when I was using IE9 that I threw the baby out with the bath water.

      1.I was wrong about how bad the browser worked. Apparently the Quicktime video that I tried to play right awayafter installing IE9 had my browser in an unusable state, so I quickly reverted back to IE8. After a fresh install of IE9 I’m having much better results.

      2.I was wrong about the search in the address bar, although it still doesnt work as seemless as Chrome’s implimentation.

      By the way this was still the 64bit version, apparently you can’t install the 32bit version on 64bit Windows. I just assumed since 32bit programs installed otherwise that I could select which version I wanted to install but the installer wouldn’t allow it.

      I thought a retraction was in order, since my first post was seemingly tainted by my experience that was not necessarily IE9 at all.

      Share
      1. Thanks for the update, Joe.

        I agree, Win7 is pretty impressive. I also think IE9 is much better than previous versions. I just can’t say that it’s yet caught up with Chrome and Firefox.

        Share

Comments have been disabled for this post