45 Comments

Summary:

Mozilla released the first beta of Firefox 4 yesterday. As there have been a few new browser releases recently, with Opera, Chrome and Safari all making great leaps forward, I thought it was time to do a comprehensive head-to-head test. Read on for the results.

Mozilla released the first beta of Firefox 4 yesterday. In addition to offering a spiffy, new interface and a whole host of new features, including enhanced HTML5 and WebM video support, the latest version should be faster. As there have been a few new browser releases recently, with Opera, Chrome and Safari all making great leaps forward, I though it was time to do a comprehensive head-to-head test to see how the new Firefox stacks up against them.

Rather than using my usual testing suite, WebKit’s SunSpider, for this shootout I used Mozilla’s Dromaeo testing suite. Dromaeo takes a lot longer to run — around 15 minutes or so — but it’s also much more comprehensive than SunSpider (it actually incorporates tests from Sunspider and also from Google’s V8 benchmark suite). For comparison, I also used the Peacekeeper tests, another comprehensive browser performance testing suite. Testing was done on my 2.53 GHz Intel Core 2 Duo MacBook Pro; only the major Mac browsers were tested.

Dromaeo

The chart shows the results of the Dromaeo test. A higher score (more runs per second) is better.

Peacekeeper

This chart shows the results of the Peacekeeper test. A higher score is better.

Conclusions

While there is some disparity between the results — it’s surprising that Opera 10.6 does very well on the Peacekeeper test (and in my previous SunSpider tests) but not so well on Dromaeo — it’s clear that although the beta of Firefox 4 shows a definite improvement, it’s still lagging behind the other browsers, particularly Chrome, which was the fastest on both tests. Hopefully that’s something the Mozilla developers will work on as they work towards a final release of Firefox 4, which is due by the end of the year. However, given the rapid release cycles of the other browser vendors, will Mozilla be able to keep up? Firefox still has a few aces up its sleeve — a giant extension ecosystem, an enormous user base and some innovative new features, like in-browser contacts management — but on raw speed it’s still losing out to its main competitors.

If you’d like to try Firefox 4, you can download it here. As with all pre-release software, there may still be some gremlins to iron out.

Let us know what you think of the new Firefox 4 beta in the comments.

Related GigaOM Pro content (sub. req.): What Does the Future Hold For Browsers?

You’re subscribed! If you like, you can update your settings

By Simon Mackie

You're subscribed! If you like, you can update your settings

Related stories

  1. First Beta Version of Firefox 4 Web Browser Now Available – Techland – TIME.com Wednesday, July 7, 2010

    [...] a bit faster than Firefox 3 but still feels slow in comparison to Google Chrome. WebWorkerDaily ran some formal tests and found the same to be true. The web browser is still in its early phases, though, so there's [...]

  2. For what it’s worth, they haven’t landed their major JS engine overhaul just yet. You can follow their progress on http://www.arewefastyet.com/

    1. This must be like their 3rd attempt to overhaul the JS engine since Chrome came out. Opera seems to keep up with V8 with their new JS engine, Firefox so far couldn’t.

      1. I don’t follow – Opera has a new JS engine (Carakan), while Firefox is still using the one-year-old Tracemonkey, which itself had flaws.

        Since Chrome was released, there have been 2 versions of Firefox (3.5 & 3.6) released, both of which use Tracemonkey. So, it’s not their 3rd overhaul attempt :-)

  3. Mozilla is working on comprehensive browser features, expanding the open web, and ensuring users get the best possible experience out of the web through their browsers.

    Pure JavaScript speed is not the only factor for web browser performance and overall usefulness.

    At the end of the day I think Mozilla works harder to make Firefox the best overall browser for users. Google throws a lot of weight behind speed and simplicity, but Mozilla is about more than that.

    Obviously speed is a big deal though and the race is on. As someone else pointed out, the new JS engine has not landed in FF 4 Betas yet. We’ll see it soon enough. And to be sure, Mozilla is working very hard to making Firefox as fast as possible.

    I’m also quite certain that if, for example, Firefox 4 Final was the fastest browser on the planet – inevitably the next nightly of Chromium or Opera would edge it out. It’s going to be an indefinite battle for supremacy.

    1. “It’s going to be an indefinite battle for supremacy.” I agree, and it’s great news for us users. Thanks for your comment.

    2. Rakesh Waghela @Webiyo Matt Wednesday, July 7, 2010

      ♥ I have tried my hands on latest regular versions Firefox , Chrome , Safari & Opera !
      ♥ I don’t think user gives shit to just one feature set of Speed & Simplicity ( except some geeky , nerdy types people — me included ) !!
      ♥ Firefox delivers better with so many add-ons installed.
      ♥ It’s about overall experience !

  4. Well, to be honest. I find FireFox 3.6.6 faster than Safari 5. Plus, I can’t really live without AdBlock Plus. The Safari version is not as robust for some reason.

    The only real complaint I have about FF is that it doesn’t “resize to fit” the window when you click on the green window size widget. It still maximizes as if it was running in Windows OS. I really hope they fix this.

  5. Firefox 4 beta 1 released. New look, same slow speed Wednesday, July 7, 2010

    [...] not sure if it is because the Direct2D support is disabled by default, but according to some benchmark tests over at WebWorkerDaily, the speed improvement over the current version of Firefox is quite small and comes nowhere near [...]

  6. Luigi Montanez Wednesday, July 7, 2010

    Mozilla specifically stated that Javascript performance improvements are not yet in the Beta, but the Dromeo benchmark solely tests Javascript performance. That’s unfair.

    http://www.mozilla.com/en-US/firefox/beta/features/

    http://hacks.mozilla.org/2010/07/firefox-4-beta-1-is-here-whats-in-it-for-web-developers/

    More appropriate benchmarks would focus on page loading, as the blog post above does tout improvements on that.

    1. Thanks for the comment, Luigi. When Mozilla updates Firefox with its new JS engine, I will certainly rerun the tests.

      1. Rakesh Waghela @Webiyo Simon Mackie Wednesday, July 7, 2010

        I am shocked !!

        http://www.facebook.com/GigaOM
        doesn’t lead to anywhere !!!

        How come GigaOM people have missed this ??
        Plz acquire the User name asap !!!

  7. Not only that. But Firefox is the only browser that can’t do 100/100 on Acid 3, and also is the last one on the HTLM5 test (http://html5test.com/), nor can show h264 HTML 5 video.
    So… worse browser now. Of course not including IE that fortunately doesn’t run on my Mac ;)

    1. I missed the part where using h264 was part of the HTML5 spec. They support WebM, which Safari doesn’t.

      Additionally, are you sure you ran Firefox 4 against the HTML 5 test? Just by looking at the page (No FF4 at work), I think FF4 would be a shoe-in for the top spot.

    2. A follow-up to my previous comment: FF4 looks like it’ll beat it’s Webkit competitors with it’s HTML5 support: http://caniuse.com/#cats=HTML5&statuses=rec,pr,cr,wd,ietf

      Additionally, what do you gain from having a browser that gets a full 100%, other than the fact that you know it gets 100%? I’m not saying it isn’t a noble aspiration, but does it make a more usable browser?

  8. Charles Hamilton Wednesday, July 7, 2010

    Speed is always an issue, but for many of us who use Firefox, its advantage is that it’s really become more than just a browser. With the judicious addition of some well-designed add-ons, it can be a tool for managing multiple email accounts, testing and troubleshooting web pages, managing passwords, synchronizing data, and much more.

    1. Indeed, Charlie — speed is not everything, features and stability are probably more important for most users.

      Chrome does have a reasonably healthy extension ecosystem now though.

      1. Bullshit! Speed is everything for the present culture.It is exactly like in daily traffic. You start hating slackers that wont move fast enough! Firefox doesnt give a shit about users experience, thats why you excuse with: speed isnt everything. Addons all browsers have or will have.

  9. This Just In: Firefox is Still Slow (via WebWorkerDaily) « Chicago Mac/PC Support Wednesday, July 7, 2010

    [...] Mozilla released the first beta of Firefox 4 yesterday. In addition to offering a spiffy, new interface and a whole host of new features, including enhanced HTML5 and WebM video support, the latest version should be faster. As there have been a few new browser releases recently, with Opera, Chrome (s goog) and Safari (s aapl) all making great leaps forward, I though it was time to do a comprehensive head-to-head test to see how the new Firefox st … Read More [...]

  10. Dennis Victor Wednesday, July 7, 2010

    At least there is some kind of speed increase. I’ll still continue using Chrome though. Fast, clean and simple.

    I think it’s important that Mozilla starts understanding that speed and simplicity is becoming the winner in this. Firefox might have gotten a little faster now, but Chrome, Safari, IE and Opera will probably continue leaping forward too. I’ve introduced Chrome to a lot of Firefox users and all of them has been delightfully surprised at how good it is.

    Needless to say, none of them went back to Firefox after a day or two getting used to Chrome.

  11. Testing Suites Show Firefox 4 Not Quite Caught Up In Speed | Lifehacker Australia Wednesday, July 7, 2010

    [...] Worker Daily did its own MacBook Pro tests, and found that Firefox 4 was, at best, on par with Opera 10.6 on Mozilla’s Dromaeo testing [...]

  12. These benchmarks are useless to me. I’m on a crappy old 1.8GHz Celeron and you know what browser I use? Firefox 2.x. You know why? Post-1.0, Chrome has been getting slower and slower and slower on it. Yet your test shows Chrome should be like lightning in a bottle.

    1. I know how you feel, Mike. I had an 800mhz PIII running GNU + Linux and Chromium was like molasses. It loaded pages promptly but scrolling through Gmail and buzz was excruciating. Firefox was much faster.
      Additionally, when are we going to get some GPU accelerated javascript engines? What about firefox’s D2D acceleration, does that change benchmarks at all?

  13. KlikoShqip.com » Firefox 4 Beta 1, gati për shkarkim dhe testim Wednesday, July 7, 2010

    [...] Web Worker Daily, duke përdorur testin e shpejtësisë Dromeo të Mozilla-së, ka renditur Chrome në vendin e parë, të cilin e ka përcjellë me Safari 5, Opera 10.6 dhe Firefox 4 Beta 1. [...]

  14. links for 2010-07-07 « 個人的な雑記 Wednesday, July 7, 2010

    [...] This Just In: Firefox is Still Slow (tags: firefox) [...]

  15. Early Benchmarks Show Firefox 4 Beta is Still Slow : Test Drive Wednesday, July 7, 2010

    [...] up to parity with the likes of Chrome and Safari 5. Well, keeping in mind that this is a beta, things aren’t looking great in the speed [...]

  16. Sorry Firefox 4, you are still slow ! | Chrome Story Wednesday, July 7, 2010

    [...] read the full review here. So firefox guys, better luck next time [...]

  17. I was reading some stuff on the Unofficial Firefox forum and like many people said their javacript trump card isn’t yet in the beta. I am not sure the about technical details but some posters have claimed(for whatever thats worth) that Jagermonkey + tracemonkey has the potential of at least being in rough parity with V8 and maybe surpassing it.

    The main attraction of the Firefox 4.0 beta is the GPU acceleration though. The difference is very noticeable especially the improvement in scrolling.

  18. In addition to my comment above, one of the coders of jagermonkey notes at http://arewefastyet.com/

    “note for the worried: the tracer JIT (orange) and method JIT (black) are not yet integrated.
    once integrated, the merged branch will be faster than either branch individually.
    they are complementary.”

  19. Studley Doright Wednesday, July 7, 2010

    I am having problems running these browsers on my i486 PC. I just upgraded to a Pentium II computer, will have to try it after I upgrade the hard drive so I have space for these huge programs.

  20. Information Technology Leader » Blog Archive » Early Benchmarks Show Firefox 4 Beta is Still Slow Thursday, July 8, 2010

    [...] up to parity with the likes of Chrome and Safari 5. Well, keeping in mind that this is a beta, things aren’t looking great in the speed [...]

  21. Mark Gary Blumenthal, MD, MPH Thursday, July 8, 2010

    That’s an understatement. I tried Firefox 4 beta 1 on my MacBook Pro yesterday and after about five minutes of agony, I uninstalled it and reinstalled Firefox 3.6. Now it is reasonably quick again, but nowhere nearly as quick as Safari or Chrome (my two preferred browsers).

  22. t3n-Linktipps: Google Me, langsamer Firefox 4, NoSQL Datenbanken, Infografik zur Online ID-Nutzung und OTRS für iPhone » t3n News Thursday, July 8, 2010

    [...] WebWorkerDaily hat die erste Beta von Firefox 4.0 auf ihre Schnelligkeit getestet. Die Ergebnisse: Zwar hat die neue Version im Vergleich zu Firefox 3.6 besser abgeschnitten, aber die Konkurrenten Safari 5, Chrome 5 und mit Abstrichen auch Opera 10.6 haben noch einen großen Vorsprung. [...]

  23. Firefox should take a few pages out of Chrome’s book though, the integration of Flash and PDF viewing into the browser…. and just how my Chrome somehow keeps itself up to date without me ever having to install or restart anything….. If my dad asks me what browser to run I know I will have to say Chrome, at work I still run into desktops running Firefox 3.0x something or the other. How did Firefox not update itself?

    Try install Flash in Firefox you have to restart TWICE, once to install the DLM and then visit a page again then install flash. Major pain. Try it when you have to “run as” to do it as Admin and it an is even worse experience.

    Update yourself, update my plugins, update my extensions, keep a log so I can see if I want to, but PLEASE stop bothering me with it.

  24. we are optimistic.

    adocom

  25. Firefox 4 ist immer noch zu langsam « adocom Webdesign Friday, July 9, 2010

    [...] WebWorkerDaily hat die erste Beta von Firefox 4.0 auf ihre Schnelligkeit getestet. Die Ergebnisse: Zwar hat die neue Version im Vergleich zu Firefox 3.6 besser abgeschnitten, aber die Konkurrenten Safari 5, Chrome 5 und mit Abstrichen auch Opera 10.6 haben noch einen großen Vorsprung. Gefunden auf t3n. [...]

  26. Why I Still Use Firefox: More Add-ons I Can’t Live Without Friday, July 9, 2010

    [...] This Just In: Firefox is Still Slow [...]

  27. Why I Still Use Firefox: More Add-ons I Can’t Live Without | iPhone, iPad Weblog Friday, July 9, 2010

    [...] recent post about testing Firefox’s speed got me thinking: If Chrome and Opera are really faster than Firefox, why haven’t I [...]

  28. http://www.youtube.com/watch?v=GnYs2Q2pg1M

    Jaegermonkey rocks, some people jumps to conclusion way too fast instead of waiting and analyzing.

  29. implying that you can notice a difference in JS with a difference of 5 milliseconds (or something as insignificant as that)

  30. I was never a big Firefox fan. I just was never enamored with the idea of adding on for this and that. Not to mention some ad-ons did not work well and all of them when combined bloated up the application IMO. I have been using Opera for about 7 years now. I like their Philosophy of having well thought out functionality built in to an application that does not bloat. Opera is very fast (seems faster than Chrome) and it includes so much (Mail, Torrent, Turbo, contact Manager, Unite, Sync Etc…) that I should refer to it as the Opera suite. In reality though most of these browsers are good, even IE has come a long way. I like Opera for another reason beyond it’s speed, functionality, customize-ability. I like them because they are a small innovative company that reminds me of the tech industry before the likes of a Giant MS and Google. I do not trust Google when it comes to privacy and I am sure that they will work something perhaps “inadvertently” into Chrome that will capture and store user data. Sure if caught they will apologize and promise to purge this info like they have before with other services. So for me it is Opera first and Safari second. I will keep an eye on Apple too for any future “mistakes” with user data.

  31. Principal beef with Firefox: add-ons can slow it to a crawl (FF 3.6.8). In process of isolating which one – not sure if it’s Adblock Plus, Flashblock, or NoScript; definitely one of those. When enabled, pages frequently stall and require a second click to load. Yuck.

  32. Video Not Working On Google’s Chrome Browser « Computer & Internet (tips + more) Friday, August 13, 2010

    [...] on that browser it will be posted here. In the meantime it is back to ole Safari because Firefox is waaaaaaaaaaaaaay toooooooooo sloooooooooooooooow now. I am not fussy on Safari but what can I say that hasn’t been said about [...]

  33. FF4 beta 7 is now out with all the new features in it. Testing a beta 1 with no new features is the same as 3.6 with some slower speeds. Now with a full version with GPU support, webM video, better HTML5 support, open fonts support, webGL 3D, new Java engine showing up to 5x speed boost from 3.6 Firefox just went hyper speed and is scoring at or above all other browsers in bechmarks. Now take that pure speed along with the vast lead in customization options and I am now a first time Fanboy of anything. It is the best browser by far now. You can mold Firefox to your needs and that includes all kinds of needs no matter what you use the net for. No other browser can do nearly as much in that regard. Now the speed is there even though the difference is measured in so small amounts you really don’t see it anyway. But still it looks good to be fastest and now I am sure FF4 beta 7 is the fastest overall browser around.

  34. I don’t understand the speedsyndrome of these days. (And I mean with operating systems too. It looks like a virus.)
    It is nice when software is fast, but I like it more when it is also good. Inclusive extensions of browsers, because they are the reason for me to use a browser or not. (Opera lost much value for me because of the lack of extensions, although it is a very nice, speady browser.)
    So I hope they don’t ‘kill’ the browser because of speed.

Comments have been disabled for this post