4 Comments

Summary:

Network engineers from Yahoo, Facebook, PayPal and Zynga said that startups and other companies need to think about how they are going to scale their infrastructure as they grow. However, they also said companies need to recognize their predictions will probably turn out to be wrong.

Network engineers from Yahoo, Facebook, PayPal and Zynga told the GigaOM Structure conference that startups and other companies need to think about how they’re going to scale their information infrastructure as they grow. However, the panel said that companies also need to recognize that their predictions will probably be wrong, and look at using cloud computing services to make it easier to adapt that infrastructure later.

Mark Williams, vice-president of network operations for Zynga, said that one of the things cloud computing has done is to provide a relatively manageable number of infrastructure pieces — whether it’s servers, software or a combination of the two — that a company can put together to scale quickly. Zynga, for example, doesn’t know what kind of demand or traffic a new game will produce when it first launches, so the company focuses on having a suite of infrastructure pieces “and then the network team works with the developers and we can swap those pieces out if we need to, and experiment in real time to solve those challenges.”

Facebook’s VP of technical operations, Jonathan Heiliger, said one thing the company has found as it’s grown to more than 400 million users is that whenever it predicts that demand for a new feature or service will be really high, it isn’t, and “when we predict it will be really low, it turns out to be really high.” Jay Parikh, the director of engineering at the social network, said one thing the company does is to try new features with a small group of users “so we can see what adoption rates are like, and then to extrapolate to what it’s going to be like when you turn it on across 400 million users.”

Todd Papaioannou, VP of cloud infrastructure at Yahoo, said that people like to say scaling problems are good problems to have, since it means a company is growing. But companies need to try and think about scaling issues as early as possible when they’re building their systems, “because you’re going to be stuck with them for a long time,” and it’s going to be very difficult and very painful to go back and re-architect later.

Meanwhile, PayPal’s VP of customer quality, Matthew Mengerink, said that companies also need to focus on developing a business model that allows for such incorrect predictions, because they will happen and companies will have to re-architect their infrastructure. “You can’t predict the future or what yo’ure going to need,” he said, and if you don’t spend time on your business model, “you won’t have a scale problem because you won’t have any customers.”

http://cdn.livestream.com/grid/LSPlayer.swf?channel=gigaomtv&clip=pla_12826aa4-2907-4bb1-afa1-b31542150f2f&autoPlay=false&mute=false

gigaomtv on livestream.com. Broadcast Live Free
By Mathew Ingram

Related stories

  1. [...] 2010: The Quest for Exascale Computing Power See All Articles » Structure 2010: How Does a Company Scale in [...]

    Share
  2. There is alot of thrashing in this space and it is hard to determine which Cloud to goto as everyone is doing something a little different – its hard to compare Cloud 2 Cloud. A similar diagnosis is by David Chappell:
    “If I ruled the world”, says David Chappell, “I would make the phrase ‘private cloud’ illegal”. In conversation with David Gristwood, David Chappell, during his recent world tour, discusses the Cloud, its importance and role in the partner ecosystem, and cloud players, such as Google, Amazon, Salesforce.com, VMware and more. You can see his Cloud2Cloud comparison in brief here: http://www.youtube.com/watch?v=e7NHQdh8_uo

    A more recent talk with David Chappell on this topic where he covers others issues such as:
    - IaaS vs PaaS
    - Private vs Public Cloud
    - Applications that are not a great fit for the Cloud and those which are.
    - The threat of Public Cloud to IT departments
    see: http://channel9.msdn.com/posts/David+Gristwood/Conversations-with-David-Chappell-about-Windows-Azure-and-Cloud-Computing/

    thoughts?

    hope that helps,
    -cn

    Share
  3. [...] this isn’t what surprised me the most at Structure. That came in the How Does a Company Scale in Real Time? panel. This panel had representatives from PayPal, Engine Yard, Yahoo!, Facebook, and Zynga. These [...]

    Share
  4. I made some comments about this session in our company blog. Some of the advice offered to startups in this session was very surprising to me. I thought the advice was much more applicable to more established companies.

    Share

Comments have been disabled for this post