42 Comments

Summary:

In less than 48 hours many of us will be installing Tiger OS-X and with it a brand new Safari browser that can read and display RSS feeds in a simple easy to understand manner. That upgrade while great for the consumers, could come as a […]

In less than 48 hours many of us will be installing Tiger OS-X and with it a brand new Safari browser that can read and display RSS feeds in a simple easy to understand manner. That upgrade while great for the consumers, could come as a big shocker for those blogs whose feeds are included as part of Safari’s default starter package. Infact it could be the biggest stress test for RSS thus far!

Most RSS readers are set to poll for updates every hour, and imagine when half-a-million Tiger Safari users who start hitting a server at the same time, pulling down RSS updates, because they have not changed the default settings. Server meltdown? Or an unintended denial of service? Apple says that most of the default feeds are going to be major news sites like CNN. New York Times, and LA Times. At this time they are not including any personal blogs as part of the default list. Even for them it is not going to be easy.

Lets say if one of these news operations updated their site once an hour and each update results in a nominal 5 kilobytes of RSS generated data, then 500,000 simultaneous Safari users polling at top of the hour would mean a total data transfer of over 2 gigabytes per hour. Times 24, and you have over 48 gigabytes of data transfer every day – just from Safari users alone. What if more than a million Tiger Safaris were on the loose. Oh boy! While an addition 48 gigabytes of traffic a day or 1.4 terabyte a month is not that much for large sites, but it will add up.

Admittedly, since I don’t have Tiger yet, not sure if Safari RSS does time-based check (every hour at :15) or checks related to when the computer/browser is started, which is relatively random and what other feed readers do. Clearly this is an imaginary scenario, but it could happen. So what’s the fix? “I certainly hope that Safari does conditional GET. I can’t imagine it doesn’t but I could be wrong,” says Brent Simmons, founder and the man behind hit feed reader, Net News Wire, “With conditional GET you download the feed only if it’s different from the last time you downloaded it — this cuts way down on bandwidth use.” (More on conditional GET.) “Conditional GET — which NetNewsWire and most other aggregators support — is hugely important,” says Simmons. But even that can go that far, since most of these news operations churn out headlines with monotonous regularity.

Long term, I think RSS is going to become a clear bandwidth hog, unless the RSS people decide and come-up with an intelligent way to fix this problem. I have been tracking my own bandwidth consumption and RSS is just sucking up gigabytes like a parched man on a hot summer day. Some say that randomizing the whole RSS polling process is the answer.

How about randomizing the whole RSS polling process? Instead of pulling down RSS feeds every hour, let the feeds download randomly. Okay that will help distribute the loads on the servers more evenly, but that still doesn’t resolve the issue of inefficient use of network resources, especially for those who pay for those kind of things. Suggestions?

Scott Rafer, CEO of Feedster says, “ISPs can start caching feed URLs but if they do it with cached times of more than 10 min, then people will route around the caches.”

You’re subscribed! If you like, you can update your settings

  1. Media Guerrilla Linkblog Thursday, April 28, 2005

    RSS, Tiger Safari and the Bandwidth Bottleneck

    Link: Om Malik’s Broadband Blog: RSS, Tiger Safari and the Bandwidth Bottleneck.

  2. People have short memory. Pointcast used to clog servers and bandwidth in enterprise data-centers and ISP’s.
    RSS is an open pointcast standard, and whatever algorithms people use the old adage still hold goods , download expands to fill bandwidth..

  3. Yup, in many ways its the same thing all over again. I think the network guys don’t talk to the app guys and that is always going to be a major problem in the future. this is going to only escalate. as more RSS feeds come online, well more clutter.

  4. J. Daniel Smith Thursday, April 28, 2005

    I think the “real” fix is to figure out a way to somehow merge NNTP and RSS. NNTP spreads the load out among several severs, you connect to whichever one is “closest” to you.

    There are some very small steps in that direction, but far too little.

    All of the other solutions are patchwork and don’t really scale well; although they can bring significant improvements.

  5. The Newest Industry Thursday, April 28, 2005

    Apple: Safari Lead DDoS and Web Performance Threat to RSS?

    Om Malik points out a potential threat to blogs: OSX 10.4 “Tiger”. The new Safari that ships with this OS comes with the RSS reader turned on by default!

    That upgrade while great for the consumers, could come as a big shocker for those blogs whose…

  6. Charlie Sierra Thursday, April 28, 2005

    RSS is a scam invented by the web-hosting companies to drive customers to higher rate plans, or drain their pockets with “overage” charges.

    These guys must’ve studied the cellular industry.

    PS. this is all slightly tic.

  7. Om: As long as Safari can use not modified headers, it will only pull the feed down when it has been updated. So it should be no worry.

  8. Coupla things:

    1. Although the RSS capabilities in Tiger/Safari 2.0 look nice from a cosmetic standpoint, they are pretty braindead. There is no useful concept of notification and “feedreading” doesn’t look anything like it does in almost every other reader (Bloglines, NNW, FeedDemon, etc). I don’t expect many people to actually use Tiger’s built-in feed reading functions until they are brought into parity with the leading feed readers. It’s really not even close right now… think Firefox’s Live Bookmarks but even less “live”.

    2. I personally like the server-side solution to solve the bandwidth problems created by RSS. Bloglines pulls one feed on behalf of thousands and thousands of readers… that’s efficient. I’d pay several dollars a month for Bloglines if I had to or alternatively I’d put up with a few ads in order to pay for that bandwidth.

  9. First, does anyone know how Safari RSS actually works?

    The whole idea of time-based updating is kind of silly. I wouldn’t be surprised if it only updates feeds when the user navigates to the feed reading area.

    “I don’t expect many people to actually use Tiger’s built-in feed reading functions until they are brought into parity with the leading feed readers.”

    Uh…I don’t think so. Even if Safari’s feed reading is super-basic, it will gain a large number of users.

  10. The real issue isn’t Safari — that’s just RSS getting more popular — the real issue is RSS’ inefficiency (because it’s based on polling). Ideally, syndication of news should be delivered as “push” updates when there’s new information to deliver. This was solved by the Information & Content Exchange (ICE) protocol years ago — see http://www.icestandard.org . Admittedly push delivery only works for people with routable IP addresses, but with broadband penetration over 40% (and with syndication between web sites) it could go a long way towards saving bandwidth…

Comments have been disabled for this post