Blog Post

eBay shows the world how to measure MPG for data centers

Stay on Top of Enterprise Technology Trends

Get updates impacting your industry from our GigaOm Research Community
Join the Community!

eBay is busy building some of the world’s most-efficient data centers, and its efforts aren’t just show. The company has figured out a way to tie its computing infrastructure to specific business concerns and plans to continuously tweak its operations to meet top-level mandates. On Tuesday, eBay released a whitepaper describing how it accomplished this and laying out a framework for companies that want to do the same.

Dean Nelson, eBay’s vice president of Global Foundation Services, says the effort, called the Digital Service Efficiency report, “is the miles per gallon measure for technical infrastructure for eBay.” Essentially, the company has boiled its business down to a single currency — transactions (specifically URL requests) associated with users’ buying and selling on the site — and created a slew of metrics that measure how efficiently it delivers those transactions in terms of revenue, performance, cost and carbon footprint.

The project has been about 18 months in the making, Nelson told me during a recent phone call, and eBay was finally able to set a baseline measurement of its performance in 2012. Now that it knows what’s in place and how its infrastructure performs over the course of a year, the goal in 2013 is to cut its computing-related carbon usage and costs by 10 percent and increase performance in terms of transactions per kilowatt-hour by 10 percent.

In order to meet these goals, he said, every member of the technical team — from facilities managers to software engineers — has be striving toward them and also be cognizant of how turning their “knobs” will affect the other metrics eBay is measuring. “Think of it like a Rubik’s cube,” Nelson explained. “You can solve one side but screw up the rest of them.”

eBay plans to release quarterly updates on its progress along with its earnings reports, but employees will have access to down-to-the-second visibility into what’s going on. “It makes it personal for them,” Nelson said. “They can see what their efforts mean.”

Digital Service Efficiency

52,075 servers doing a lot of work

Nelson offered some pretty compelling examples of how the Digital Service Efficiency project works in practice. If the goal is to decrease cost per transactions, data center engineers might try to minimize power usage at the facility level while server engineers might look to lower-power gear or better utilization on existing gear. They essentially reduce the denominator in that equation “and the net result is we should make more money from those transactions,” he said.

In one real-world instance, a software engineer tweaked some code that affected how much memory an application requires and the company was able to eliminate 400 servers. That cut energy usage by 1 megawatt and a $2 million savings in capital expense when the time would have come to refresh those servers.

eBay also has created a “list of fame” and a “list of shame” that highlight the 1,000 best- and worst-utilized servers within the company. “We have a hit list,” Nelson said, and it’s going to examine the bottom 20 percent to figure out why they’re as wasteful as they are.

However, he added, it’s important to remember on the server front that improving cost, performance and carbon usage doesn’t always mean buying lower-power gear. If eBay can improve the power density of its racks using technology such as liquid cooling — something its Project Mercury data center in Phoenix is pre-equipped for — it can handle more transactions on less gear. It already has some racks running at a sustained rate of 35 kilowatts and thinks it can push that up to 50 kilowatts, Nelson said.

Clean transactions with solar panels and Bloom boxes

On the carbon front, eBay has nothing but an open field in front of it thanks to some big clean-energy projects set to go live in 2013 in its new Salt Lake City, Utah, data center called Project Topaz. For starters, it’s using Bloom Energy boxes as the primary power source, which mean a slightly higher cost per transaction, but also a 13 percent reduction in carbon emissions and increased reliability (downtime costs eBay a lot of money).

Also, the company has finally cleared some regulatory hurdles to tie an on-site solar array back to the grid. Because of changes to a Utah law that eBay lobbied for, it’s about to start sourcing off-site clean energy for its data centers, as well.

“That is a corporate priority,” Nelson said. “We want to create the cleanest commerce engine on the freakin’ planet.”

Trying to change an industry

Of course, the Digital Service Efficiency methodology isn’t the only attempt by a major data center operator to show the world how efficient it is. Google publishes annual Power Utilization Efficiency (PUE) ratings for its data centers, and Facebook occasionally does as well. On Monday, released a statement underscoring its commitment to sourcing renewable energy.

dse chart

However, Nelson pointed out, what eBay is doing — and encouraging others to do — is more transparent in that it gives a lot more depth about operations, including the company’s server count. Even if companies don’t publish their results, tying operational efficiency to other business objectives should have a positive effect on the bottom line and the environment, regardless. Every company will have its own base currency, Nelson explained, and they’ll have to find their own metrics to measure and figure out what are the knobs that each part of the company can turn to meet goals.

“We all have the same challenges, the same things to solve for, but we have numerous ways to solve it,” Nelson said. …”[Their implementations] may change completely, but the point is the conversation is starting.”

4 Responses to “eBay shows the world how to measure MPG for data centers”

  1. Frank Tiesma

    The DSE white paper is interesting and laudable. I wonder about factors that are outside of the control the data center and software engineers.

    Example: a previous commenter pointed out that a transaction at eBay is not the same as at Ford. From one period to the next, the exact same transaction at eBay might be different. Perhaps a new government regulation means extra processing steps. Or maybe Marketing wants some “next best action” analysis done real time and presented within the transaction experience. Are such changes normalized, or are they perhaps trivial in terms of the goals of DSE? I’m also curious how some controllable factors might be normalized, such as per transaction pricing.

  2. Tony Greenberg

    Wow. This changes everything. Dean Nelson launch a metric in his keynote in todays Green Grid Annual Meeting that’s going to elevate measurement of efficiency and cost from the IT level to the business level. There will be challenges in mass adoption of the metric, and it will continue to evolve. But, in an industry that’s usually a follower, eBay is pushing to have the data center business take the lead in measuring and optimizing to what’s really important.

    Of course, one adopter does not make an industry standard. eBay’s transaction may differ from the Gap’s transaction, which in turn definitely differs from a Ford, Boeing, Fidelity, or GE transaction. But the framework and foundation allows us to start setting standards for various industries and building towards best practices in a way that was much harder without it.

    But even today we can start thinking of how to use DSE in other industries. For instance, we can document that when one of our gaming clients uses 50% of their allocated power on average instead of the industry-standard 75%, they’re not being wasteful – but instead using servers that are put on standby during off-peak hours. Without DSE, there’s no metric that properly rewards that design – PUEs are higher since data centers are designed for peak loads, processor utilization may be lower, and it looks inefficient by standard metrics.

    But if we track, say, energy per gaming hour, that value now shines through. Similarly, maybe a digital media firm’s “transaction” will be a view, while a manufacturer uses a unit shipped. Other industry-specific transaction units may apply for financial services, pharma, etc. The important part is building a platform for a common comparison among things that are alike and then we can tackle the question of who’s more efficient across industries and start putting together dashboards consumable by the business and metrics that financial analysts will probe to predict performance.

    eBay has cracked the door. Now let’s swing it wide open. Let’s start building DSE for every industry, share experiences, and build cross-functional efficiency teams where for once the data center is the hub of the movement as opposed to the low man on the totem pole. Let’s start thinking about overall Service Efficiency and defend the industry from unwarranted challenges while highlighting the laggards that are genuinely wasteful.

  3. Richard D

    This is a great start and something that the industry needs to tie the entire ecosystem together – really looking forward to the ensuing debates and iterative process.