60 Comments

Summary:

Every so often you meet entrepreneurs and venture capital investors who talk about Alexa ranking of a web-based service, using it as some sort of a yardstick for growth and reach. It is as good general, non-specific indicator, say if the traffic is going up or […]

Every so often you meet entrepreneurs and venture capital investors who talk about Alexa ranking of a web-based service, using it as some sort of a yardstick for growth and reach. It is as good general, non-specific indicator, say if the traffic is going up or down, but to make money-decisions based on Alexa rankings is, well living dangerously.

A few weeks ago while taping an episode of Cranky Geeks, John Dvorak pointed out that Alexa toolbar, which is used to calculate Alexa rankings, works only on Internet Explorer 6.0 (or higher) on Windows. Given the proliferation of Firefox and Macs, it would be hard to assume Alexa’s accuracy, since the Alexa toolbar doesn’t really work on those two platforms.

Not that this is exactly news, but I still am amazed is the number of people who use Alexa as a fiscal crutch. What really is more worrisome is the increasing number of outages Alexa is experiencing. This past weekend, when most of us were reading the peanut butter memo, Alexa was experiencing some serious outages – for substantial period of time.

Data collected by Pingdom Gigrib shows that Alexa was down for about 14 hours and 8 minutes in November 2006 (so far), up from three hours and 20 minutes in October and about 80 minutes in September 2006. I wonder how Alexaholic was impacted by these outages?

In comparison, some of the better-known Internet brands had a little or no downtime – MSN was down for just over two hours, You Tube was around 65 minutes, while Google, Amazon, Yahoo and eBay were up a 100% of the time. Since Pingdom’s Gigrib software runs only on Windows platform, one cannot really assume completeness of its data. However, other sources that are also reporting on Alexa outages, so it is safe to say the downtimes were fairly substantial.

Downtime shows that Alexa is not reliable even as a general barometer of a website’s shifting fortunes. Furthermore, it is a sad reflection on Amazon’s web services business that includes S3 and EC2 efforts. (S3 has had issues in recent times as well.) If Amazon can’t keep the Alexa up and running, how seriously can you take their backend?

The point is not to pick on Alexa, but to bring into focus the biggest shortcoming in the post 1990s-web: lack of a good dependable yardstick for ranking websites and web services. As more and more web properties come into existence, it is time for the industry to develop a more dependable, and open source tool to track general traffic trends, and web site rankings.

Last week, Google, Microsoft and Yahoo came together on a standard for crawling websites. Why can’t these three companies and others who offer toolbars include tracking technology in their toolbars and hence offer a fair representation of the web traffic trends.

A non-partisan group could collect the data; much in the way open source projects keeps track of their code. There will be privacy and other related issues, but then these companies are chockfull of smart guys with all the answers.

In closing, if you are a startup that brings up your Alexa ranking in a meeting with us and tout that as your shining achievement, it would be time for my smoke break!

  1. Interesting article Om.

    Even I have found difficulty in accessing alexa site, not once, but many times.

    I hope even the VC’s doesn’t just look at the alexa stats alone, in judging the startup.

    Share
  2. Alexa is also prone to manipulation. It’s stunning to think that someone would actually use it to value a web property. I can’t understand why anyone would use it as more than the most general gauge of traffic trends.

    Share
  3. Nice writeup Om – I tell ya – I can probably count over 100 proposals that came to me for some network or ad that someone wanted to run when I ran the Internet Marketing Dept at a Fortune 100 – just amazing how many talked up their Alexa ranking.

    I never trusted Alexa for anything, and still don’t. I know of sites that have 100x my traffic, yet my Alexa is better. Just makes no sense.

    To the uninformed, it looks like a great tool because it is backed by Amazon. Who wouldn’t believe Amazon?

    Share
  4. Nice article. I have also noticed that Alexa has too much downtime. off late, it’s getting worse.

    Anyways, I have found Google Analytics extremely useful in analyzing website traffic and making improvements to the website. It’s also free.

    thanks,
    Manish Lachwani

    Share
  5. Panel based approaches not only suffer from inherent biases and large margins of error, they fail to utilize the key advantage the Internet has over traditional media – direct measurement of media consumption and audiences.

    Quantcast is an alternative to Alexa’s toolbar-based audience measurement. We offer a free Internet ratings service with rankings for over 20M sites and demographic audience ratings for over 1M.

    We quickly realized that the panel based approach in isolation was limited and developed a complementary approach that we call the Quantified Publisher program. Through this free service we directly measure a site’s audience using a measurement pixel. This lets publishers respond to inaccurate published ratings with an accurate third party measurement, and gives them control over how the information is presented.

    Example of Quantified Publishers include The Nation (quantcast.com/thenation.com), TechDirt, (quantcast.com/techdirt.com), Download Squad (quantcast.com/downloadsquad.com) and hundreds more.

    We’re still in beta, but believe we are making great strides towards demystifying the measurement of audiences and creating a resource of open, transparent data that will benefit publishers and advertisers alike. As each new site registers, our model for all sites becomes more accurate.

    Share
  6. Hey Om,

    I definitely agree with your view on the inaccuracy of the Alexa ranking system. One thing to point out, however, is that as the ranks increase, they inherently become more and more accurate.

    Just something to note…

    Cheers,
    Aidan

    Share
  7. Om,

    I have seen you at times pull data from comScore. So how does it compare with Alexa ? I had asked if comScore had an API but they said they won’t let cannibalization of their data.

    Share
  8. Hey, no so fast! we have to leave something for Web3.0…

    Share
  9. Alexa works with Firefox. There is a popular Firefox extension called “Search Status”. If you install that, your surf will become part of the Alexa sampling.

    Share
  10. Most VC’s, big media buyers have hitwise subscription (30k+ per yr) which is much more accurate and i am sure they also look at the actual logs before commiting anything.

    IMHO, bloggers are those that quote alexa the most!.

    Share
  11. We all know Alexa has big problems but continue to use it anyway because right now there is no comparable service.

    It’s the same case with the Wayback Machine at archive.org.

    Share
  12. Alexa+Hitwise+Google PR+ Seomoz tools+ Technorati ranking + …? Maybe therein lies the answer.

    Share
  13. Quite an indictment of a free and generally useful service. Seems ironic you would criticize anyone for providing unreliable information, being a provider of similarly unreliable information yourself:

    1) Alexa does work with Firefox. I’m using it right now via the search quick plugin.

    2) Who says Amazon operates Alexa’s website? Check the IP address and domain registration, if you know how, and you’ll see it’s hosted at UnitedLayer.

    3) Alexa has lengthy disclaimers describing how Alexa usage may differ from general Internet usage. Perhaps you could add a disclaimer to your blog indicating that your statements may differ significantly from the facts.

    Lastly, I see you use sitemeter to measure your own traffic. Why not publish your own numbers as a step toward the utopian world in which everyone gets to know everyone else’s site traffic?

    Denny

    Share
  14. In my opinion Quantcast is a lot better indicator than Alexa, and provides some interesting data. Nothing’s been done with Alexa for the last 3 years. It’s time to move on.

    Of course, quantcast is showing very low numbers for a lot of popular sites on Web 2.0. However this may be an indicator that we at the cutting edge of Web 2.0 may actually be a very small community indeed.

    -Chris

    Share
  15. Thank you, thank you, thank you! As someone who has never trusted Alexa for traffic ranking, I’m happy that the word is finally getting out. There was a very interesting debate in Digg a month ago when someone posted the news that Digg was now #22 in Alexa. Good references in there into why Alexa is flawed.

    Share
  16. How about Technorati? It’s also inconsistent and unreliable in picking up posts.

    Frank Barnako
    blogs.marketwatch.com/barnako

    Share
  17. News Flash !!!

    An Alexa toolbar DOES work in Firefox.

    http://www.quirk.biz/searchstatus/

    I’m shocked people like Dvorak don’t know this, and people take his word for it eagerly.

    Granted, this new revelation doesn’t fix all the problems with Alexa. But advertisers have been using the Neilson ratings for decades, which are also fallable.

    The point is we need something to hang our hat on, and no rating system is perfect.

    Share
  18. Om,

    Thanks for a good look at Alexa. We all know that it’s cruddy for anything more than broad guesses and trends, but it needed to be said succinctly. Well done.

    All the best,
    John Blossom

    Share
  19. Om, great points, indeed. I experienced that a lot: “wow, you’re so high on Alexa” or: “well, you don’t seem to be growing” (while we are growing). More importantly than IE/FF dilemma though:

    1. Alexa exists in ENGLISH and most people on the Internet no longer speak Englih or are native English speakers
    2. Alexa is very much scewed towards North American traffic (check the ranking of some huge Chinese megaportals on it)
    3. Alexa ranking depends on page views and many sites are built in such a way that not all page views are “created equal”
    4. Alexa measures the “Web 2.0″ crowd VERY well, but not the general public

    Cheers
    ig.

    Share
  20. Chris, Quantcast looks interesting but I compared my site’s data on our internal logs and Google Analytics (which is almost 100% in sync) and Quantcast – and Quantcast is off by… 20 times!!!! :))) What are these guys smoking? :) Alexa is many times more reliable, sorry.

    Quantcast has some really cool stats some of which are wacky and some are… TRUE! how do they get it beats me…

    Anyway, a curious site, worthwhile looking at but they have long way to go to have any credibility compared to Alexa.

    Cheers
    ig

    Share
  21. Tom – comparing alexa to the wayback machine is just plain madness :)

    it would be like comparing an apple to an oreo.

    Share
  22. Ig,

    Put the Quantcast measurement pixel on your site, and we’ll show you an accurate number for global and US users.

    Our panel estimate – for US alone – is 400,000 users per month (Alexa doesn’t even give you a unique user count, or the option to correct the numbers).

    Paul

    Share
  23. Alexa is a tool, but not “the tool” to rank websites.. Caution when using it..

    Will there ever be a better tool in the market for this task?

    Share
  24. So, this mean if your site ranks well in Alexa, your readers are dumb IE users. Interesting…

    Share
  25. Oh I have seen the so called results of Alexa rankind first hand with my own blogs – and just as with technorati there is a MASSIVE fault towards non english blogs just to start something.

    My german blog has rougly 15-20 times the hits than my english one and still Alexa lists the english one as much more authorative.

    Just as with IE, the plugin has to be installed and to be working – something I doubt for firefox. Just because I do have a google search field in my Firefox does not mean it does provide the same user data to google than the optimized IE toolbar.

    I do not mind if somebody takes those numbers into account additional to other data- but not alone just based on that.

    Share
  26. om,

    i’ve written a number of posts on this

    alexa is unreliable, but i still like to look at their numbers

    for free data, compete, snapshot.compete.com, is the best

    and i think comscore is the gold standard and they are beginning to offer startups their dashboard service at very attractive prices.

    bottom line is i visit all three and rely on comscore when money is on the line.

    Share
  27. Good post. Alexa has issues and ain’t a bible on rankings but it is truly free . . . that’s its renaissance

    Share
  28. We recently launched a public and free version of Compete(dot)com to offer a more intelligent service to the market.

    I want to thank Fred for his earlier comments, but want to add that many of the top web operations in the market use Compete’s Intelligence services and consider Compete the gold standard. Within the U.S. Compete has the largest and most diverse panel and applies a rigorous normalization process to make the data representative of the U.S. internet population.

    As far as an API, it’s coming soon.

    Thanks, TJ

    Share
  29. Om makes a good point here. GYM (Google Yahoo! MSN, hope you understand ;) ) should be more active to launch something jointly. They have all the tools to make it happen, let’s see…

    Share
  30. I was further saddened when I discovered that Text Link Ads used Alexa for their inclusion algorithms… meant some sites had to cool their heels for a bit before being included, regardless of actual traffic. Oh well!

    Share
  31. GNC-2006-11-25 #219…

    I talk in depth about the consumer electronic show and the planned video and audio podcasts. I give one…

    Share
  32. but I still am amazed is the number of people who use Alexa as a fiscal crutch.

    Or for anything! If you make decisions based on data, your decisions are only as good as your data. That Alexa can be gamed has been known for years i.e. Alexa is bad data.

    Solutions like Hitwise are much better for comparing site traffic, as you can see upstream and downstream data, top referers etc. etc.

    Share
  33. Alexa is FREE, Comscore & Hitwise are not, hence it’s easier for most people to use this flawed service than to access the high fee-based services. Not prudent, but accessible and that counts for a lot.

    Share
  34. If You’re Going To Plan Online Doorbusters, Shouldn’t You Plan Ahead For The Traffic?…

    Last week we noted that plenty of online retailers were preparing online “doorbuster” deals for the long…

    Share
  35. First, there is site down time for Alexa, as discussed in the article, but the bigger issue for many people is Alexa being down (or not calculating) for individual sites. Of my approximately 15 sites, Alexa will often have 2 or 3 at any one time in which it will not show rankings for.

    Second, the article is wrong about Alexa not being able to track Firefox users. The SearchStatus Firefox extension shows Google PR and Alexa ranking for sites. As it is built off of the Alexa API, it provides your visit/page information to Alexa the same way that the Alexa toolbar does. http://www.alexa.com/site/help?index=127

    Share
  36. Here’s another article on the same issue – Are you misusing Alexa Numbers? (Probably):
    http://andrewchen.typepad.com/andrewchensblog/2006/11/areyoumisusin.html

    Basically the key points are:
    – Alexa doens’t give pageviews or uniques, just “rank”
    – Alexa’s audience is biased
    – Alexa gives false certainty

    And the finaly question there is, how can we do better? There’s some summary of how Nielsen and comScore approach the problem.

    Share
  37. Here’s yet another blog posting about Alexa.

    It’s basic premise is that Alexa’s numbers are completely out of whack, and they know it.

    http://itsontarget.com/blog/?p=69

    Share
  38. http://www.searchjerk.com has a ranking system that is free and outshines all of the above

    Share
  39. Here’s another thing to think about… I remove LOTS of spyware from customer machines, generally using Spybot S&D. I also install SpywareBlaster, and some other tools. This stuff ALWAYS removes Alexa (along with a lot of other bad stuff). How much do you guys think this might affect the numbers? Do the more common AV/anti-spy solutions do this too?

    • Pat Struthers
    Share
  40. Web Site Rankings Galore…

    There are a plethora of great sites for gathering Internet market data and site usage trends these days. But, who to trust? Who’s got the best data? Who is best at tracking the rise of a young Internet startup company?
    Unfortunately, there is no…

    Share
  41. Wanna be seen on the Internet ?
    Become the New Boss
    How many will you pay to become the new Boss ?
    http://www.whosdaboss.com

    Share
  42. So you need a plugin to make Alexa work on FFX? I mean, I use tons of them, but why bother just to give data to Alexa?

    Share
  43. Alexa Fun: Science Magazine Web Traffic…

    We plotted the daily reach statistics of five popular science magazine websites to see how their traffic compares according to Alexa ratings, which are based on visits by people using the Alexa toolbar (but see this). ScienceBlogs’ traffic ramped…

    Share
  44. Are International Job Websites Useful for Israeli Jobhunting?…

    If you’re a sabra and speak fluent Hebrew, you may think that Israel-based job websites are sufficient. For everyone else living in Israel or not, your Israeli jobhunt may need to involve searching international websites but will you find anythi…

    Share
  45. [...] correlate or if Hitwise even wants to comment on the free service from Alexa. And YES I know Alexa is NOT perfect or even close, but we are comparing sites with audiences that are all interested in podcasts or [...]

    Share
  46. Alexa takes in consideration that most people do not have alexa tool bar installed. Lets say that you have an Alexa Tool Bar installed and vistit a website, Alexa will not only tally a click for you but will throw in a few clicks to offset the people who are not represented by having an Alexa Toolbar Installed. Lets face it, the only people who care about Alexa are those who need it for advertising or bragging rights. The average person does not care to have what is considered “spyware” on their computer.

    Share
  47. [...] their PC sites to register, update their profiles and push data to the mobile. I know Alexa data is considered suspect, but I think it’s actually OK for comparing similar sites. Here’s a graph from Alexa comparing [...]

    Share
  48. [...] Internet traffic: where are the biases? Posted February 21, 2007 There have been quite a few discussions on traffic measurement lately. The general consensus is that all of them have some sort [...]

    Share
  49. [...] information from Fred and Om. Fred Wilson asks Whose Numbers Are Right? Om Malik states that Alexa can be injurious to your wealth. The services we will look at [...]

    Share
  50. I’m surprised to learn that anyone was using Alexa for looking at a site’s page rank! I’ve always used it to check the backlinks for a website and to check who’s really linking to people wanting to link to my site.

    Yes, I know there are other tools for that – but Alexa let’s me see ‘who you’re really sleeping with’ before I accept just any link. Had Google tell me that my natural health site was in the sandbox because a site linking to me (natural health care) was actually linked to ‘get on the xxxxxxpornbandwagon’. My server didn’t catch it, Alexa showed every link that site ever had.

    Thanks folks, but for snooping – Alexa is my pick of the week:)

    Share
  51. Taking you Alexa with a Grain of Salt

    Om Malik writes about the pitfalls of putting too much stake in Alexa. I’ve been meaning to write a post about Alexa for some time, as I’ve been collecting tidbits of data that point to the relative inaccuracies in Alexa rankings. I don&#…

    Share
  52. Even if Alexa ranking is not accurate, it is still used to judge how successful a site is and how much traffic it draws… which is sometimes also used for advertisement pricing purposes.

    Share
  53. Indeed, Alexa’s cock-eyed “ping-data” can hurt a start up website. My back end stats say that we say that we get 2-300,000 request per DAY – which can mean around 15-25K views (per day). If Alexa cannot access my ISP – how can they possibly hope to report this kind of info by pinging my site?? Thats comedy. Just let me now when the CLASS ACTION LAWSUIT happens. They owe me a few bucks.

    Share
  54. [...] Cutts on Alexa 2) Peter Norvig on Alexa 3) Business Blogs on Alexa 4) WebmasterWorld on Alexa 5) Alexa can be injurious to your wealth 6) Can you trust [...]

    Share
  55. [...] Om Malik said in a post today using Alexa as a driving point: Every so often you meet entrepreneurs and venture capital investors [...]

    Share
  56. [...] is just delusive while many others have already posted on the misleading results of alexa here is another example from the trenches [...]

    Share
  57. [...] Om Malik said in a post today using Alexa as a driving point: Every so often you meet entrepreneurs and venture capital investors [...]

    Share
  58. [...] many other folks, I think Alexa data is just plain [...]

    Share
  59. [...] information from Fred and Om. Fred Wilson asks Whose Numbers Are Right? Om Malik states that Alexa can be injurious to your wealth. The services we will look at [...]

    Share
  60. [...] information from Fred and Om. Fred Wilson asks Whose Numbers Are Right? Om Malik states that Alexa can be injurious to your wealth. The services we will look at [...]

    Share

Comments have been disabled for this post