11 Comments

Summary:

That was the question asked of the four panelists on Monday’s Scalability Boot Camp Panel at South by Southwest. The panelists, who represented various consumer sites, all said that at some point in their online ventures the answer to that question was no. As a result […]

That was the question asked of the four panelists on Monday’s Scalability Boot Camp Panel at South by Southwest. The panelists, who represented various consumer sites, all said that at some point in their online ventures the answer to that question was no. As a result they’ve ended up learning how to build network architectures that can support a large number of users.

One way (are you listening Yahoo?) was to restrict launches by making them available to a limited amount of people over a fixed period of time, because no one server can handle all the users of a site hopping on all at once. Another was to figure out what your users want before trying to figure out your network architecture. Blaine Cook of Twitter confessed that when they launched their service, no one was sure what people would use it for, calling it the worst idea ever.


All the talk about failure led to a discussion about how scalability problems hurt or enhance a company’s reputation. Cook said the press mentions of Twitter’s downtime as a plus, but the irritated users were obviously a minus. Sandy Jen, co-founder of IM service Meebo, echoed comments made by Jakob Heuser of Gaia Online when she talked about letting users know what’s going on while the site is down.

She also emphasized the importance of getting the business side involved with the technical and operations side. “No feature gets out the door without talking to operations” at Meebo, she said.

The audience members seemed most concerned in how to get the business side talking about scalability issues, while the panelists, all of whom had technical backgrounds, talked about focusing on the user experience and trying to translate down time into money. All good advice, but I imagine that conversation is much easier to have at a startup led by a tech-savvy CEO.

You’re subscribed! If you like, you can update your settings

  1. For the last time: it’s far easier to scale than it is to find (paying) users. Focus on the latter.

  2. @ pwb…. now isn’t that the truth. Amen to that!

  3. Blaine Cook’s comments seem to be making the best out of a bad situation. I don’t blame him for that, but the comments from the panel seems to convey the sense that scale-driven failure is ok.

    Too bad, because it just doesn’t have to be so anymore.

    More in this post: http://www.appistry.com/blogs/node/364

  4. @ pwb – I hear you — I’ve always felt that if you have too many users for your existing scale, then you probably have the $$ to re do your architecture!

  5. Ivan de la Jara Tuesday, March 11, 2008

    Users are NOT money.

  6. @pwb, @Om Malik: You seem to be making the assumption that there must necessarily be a trade-off between building a scalable system and focusing on building the business. As bob Lozano above mentions that doesn’t necessarily have to be the case. There are technologies and products today that allow you to build for scale with REDUCED COSTS and REDUCED COMPLEXITY, and easily scale when necessary.

    Bob’s company makes such products, and so does my company, GigaSpaces. See this little anecdote about our customer TuneWiki: http://gevaperry.typepad.com/main/2008/01/gigaspaces—po.html

  7. For the last time, paying customers don’t want to pay for a bottleneck, (or worse.)

    I don’t even want that experience for free!

    I am a gamer. I am short on digits to count the number of offerings that could not scale to meet demand. They went for appealing, to abysmal failure, as a direct result.

    There is a reason WoW pwns and it isn’t because it is an amazing game. It is mediocre at best. However, I know Blizzard has the goods when it comes to architecture. I know that I will get playtime for my dollar. That fact… ‘THAT FACT’ makes mediocre, pretty damn sweet.

    If I put these demands on what I consider a past-time…

    Scalability needs to be addressed from the start. Otherwise you are in the business of apologetics for your state of ‘perpetual fail.’ Sorry indeed.

    My point of disagreement with pwb’s opening response is, when these companies don’t address this in the early development phases… when they do reach that point where demand is out-pacing capacity, more often than not, to fix the problem they have to throw out a few babies instead of just the bathwater. Adding server resources doesn’t significantly increase capacity to make that viable, because the code is… crap.

    This re-re-deployment is NOT an easy task. Fixing it later, can get time-intensive and expensive. To often limping along like some virtual cripple is what ends up happening. “We’re sorry,” becomes the mantra iterated.

    Focus on being sorry, if that works for you… The minute someone comes along and does it right; you can have something new to focus on: ‘ABANDONMENT.’

  8. Scalability is an easy problem to fix so I am always amazed when this comes up. I think the primary reason it continues to be an issue for unprepared startups is that they just don’t care (I suspect that to be Twitter’s problem).

    The idea of limiting users to manage scale is ridiculous, at best. Why do we play the game? To win.

    Now this is not to say the scalability is not important…quite the contrary. Scalability defines how fast you can grow without the wheels coming off the car: the more scalable business can absorb a faster growth rate better than a non-scalable business. That being said, a scalable business without strong growth will do only one thing fast: disappear into irrelevance.

  9. Yo,

    I’m the one panelist who apparently went under the radar. Oh well.

    @joshtabin -

    The limiting users bit was stated purely in the context of rampup time. If you have a new feature and you release it to 100 users first, you find 80% of the bugs. Once you fix those, you release to 1000 users, who find another 80%. Once you’re ready, you might ramp up the number of users who can see the new feature over course of a day or a week. That prevents a goldrush on the feature and lets you smoke out scalability problems without looking like complete idiots to your userbase.

    Then you usually leave that toggle feature in so, if later something horrific happens, you can fix it and slowly ramp back up if necessary.

    This is a common practice, and a perfectly sensical one that doesn’t at all equate to locking users out for “the sake of scalability” – you’re just letting them in slowly so the bulk of them get the best user experience possible.

    I think the theme was more “nobody’s perfect”…

    I also don’t understand the comparison to WoW. Did you guys use it at all for the first year? They had absolutely terrible scalability issues, even though they had $15 per user per month and each user could only use very limited set of resources (vs a popular webpage on a social networking site which can dwarf other users in cost).

    You can find the slides on slideshare as well, which link to many of the technologies folks should use to scale.

    But I don’t want to start an argument there, I’m educating :) But I also agree with the grow your business thing ;)

  10. I found the use of twitter and blogging interesting, challenging, informative and sometimes addictive. I had being using twitter in the last few weeks or so and equally started blogging not more than six months ago. Whether following Crossing the Chasm theory for adoption of new technology or not, the use of web2.0 tools started when I considered setting up a charity/NGO website. This was partly driven by Google tools. Before then I thought the use of these social networks were for teenagers having fun in their own world. Apart from that, technology was not advanced for the ease use of these tools without having to consult a specialist. Another reason for the wait was the all important question on start ups. Will it scale?

Comments have been disabled for this post