Blog Post

How to add 5.5 petabytes and get banned from Costco during a hard drive crisis

Stay on Top of Enterprise Technology Trends

Get updates impacting your industry from our GigaOm Research Community
Join the Community!

“We buy lots and lots of hard drives . . . . [They] are the single biggest cost in the entire company.”

Those are the words of Backblaze Founder and CEO Gleb Budman, whose company offers unlimited cloud backup for just $5 a month, and fills 50TB worth of new storage a day in its custom-built, open source pod architecture. So one might imagine the cloud storage startup was pretty upset when flooding in Thailand caused a global shortage on internal hard drives last year.

Gleb Budman

“Literally overnight,” Budman told me, “… all the places we would go to get drives said, ‘Sorry, we don’t have any drives.'”

People assumed it was just a blip, and while Backblaze watched cautiously in the beginning, it figured it had enough hard drives stockpiled to make it through. However, when months passed and the situation only got worse — some suppliers were offering 3TB drives that used to cost $129 for around $600 — Backblaze knew it had to act. If the company didn’t want to change its pricing model or throttle users’ capacity, something had to give.

“That’s an absolute, just last, last, last resort,” Budman said.

It’s good to be a startup

Sometimes, it’s nice to have the flexibility of a startup. Rather than jack up prices or lower its revenue guidance, Backblaze just kept going about its business. Well, publicly, at least — behind the scenes, the company was working like crazy farming hard drives from the only places it could still get them at a reasonable price.

Its solution was to eschew the internal hard drives generally put inside servers and buy up the external hard drives sold for consumer backup at stores such as Best Buy, Fry’s and Costco. They fit nicely into Backblaze’s storage pods once removed from their protective enclosures, and the best part was that the 3TB drives Backblaze requires only cost around $169 apiece even during the height of the shortage. The company that builds Backblaze’s pods was even willing to “shuck” the drives for a couple bucks apiece, saving Backblaze a lot of manual labor in order to make its newfound source of capacity production-ready.

And then it happened: The shopping carts Backblaze was initially filling up gave way to two-hard-drives-per-person limits, which meant the company had to scramble. Keep in mind, adding 50TB a day means getting your hands on at least 14 new drives per day. In the end, Budman tells me, its ingenuity meant Backblaze was able to procure about 1,800 3TB hard drives — or about 5.5 petabytes worth of capacity — in the three months it was actively farming them.

Backblaze details much the process in a Tuesday-morning blog post, including the hijinks that followed as the company got creative trying to figure out ways around the new hard drive limits. Maps were drawn, employees were cut off from purchasing hard drives at Costco — both in-person throughout Silicon Valley and online (despite some great efforts to avoid detection, such as paying for hard drives online using gift cards) — and friends and family across the country were conscripted into a hard-drive-buying army.

A chart of local stores and employee purchases

What doesn’t kill you …

Budman said the shortage should have taught everyone a valuable lesson that hard drives aren’t commodities that will always be available at commodity prices, so you need to take advantage of low prices when you can. In fact, Budman said, one of the reasons he resisted venture capital initially was to help stave off the mindset that money from VCs is easy and companies don’t need to spend wisely.

It appears the strategy worked. Backblaze closed a $5 million funding round in July 2012, but it hasn’t abandoned its hard-drive-farming ways. Internal hard drive prices have stabilized, but even today if prices on external drives drop to less than those of internal drives (after factoring in the voided warranty and the cost to shuck them), Backblaze will snatch them up.

But it’s nothing like the glory days in late 2011 and early 2012. Here’s a video of Backblaze employee friend and NextPunch founder Vladik Rikhter telling his story of playing cat-and-mouse with Costco’s e-commerce system and ending up with “a Kia’s worth of drives” in his entryway.

42 Responses to “How to add 5.5 petabytes and get banned from Costco during a hard drive crisis”

  1. LogicBites

    Nice of them to scoop up all the bargains and keep the prices high. Now if people stopped using their, and possibly others, service would the price of hard drives fall?

    You get to pay both for their service and a higher price for hard drive storage.

    Interesting, what?

  2. Fun article. Basically guaranteed that I would NEVER rely on Backblaze for backup!

    What a bunch of amatures and Dropbox wanna be doing hack jobs to be in business. They will 100% fail if they are successful enough to get many customers. You can operate like this when you got a few customers but this high school style of management won’t scale.

    Like what was the morale of the story? How to ripoff retailers? And how to get ppl to drive around source Hard drives?

  3. True Cloud - IMHO

    This type of storage is not true ‘cloud’ backup (IMHO). For true ‘cloud’ backup, for a similar cost, that doesn’t rely on portable hard drives, data centres, or any other costly infrastructure, or

  4. Simon Raban

    Why aren’t they using data center grade storage arrays that are much more reliable? I am surprised that they offer customers storage that can be bought at Costco. I would not sign up for their service.

  5. Yup, I remember around Thanksgiving ’11 when Costco still had 2TB drives available for $99 when any other tech seller who read the news had already jacked up their prices. So Costco is where we should all shop next time this happens. :-P

  6. Matt Cooper

    You’d think he’d go the incredibly economical route of building a REAL storage system. They should be going to a supplier such as Ingram Micro or ASI and just buy in bulk. This sounds almost like the guy sells unlimited storage at $5 a month through a Windows Home Server.

  7. But I wonder how such portable hard drives meant for laptops/desktops work in a server environment? My understanding is that the hard disks used in servers with RAID should have different interfaces for effecting faster access/throughput; in addition they should have higher drive speeds and should be hot-swappable.

    Could someone please wise me up.


  8. I don’t see a linux solution anywhere on the backblaze site. Software install? license key? You may have an open hardware design, but the service isn’t open at all from what i see. maybe i’m wrong. How about just an sftp/ssh solution, drag and drop, so it’ll work with linux as well? it looks like in your paranoia over fanboys or smb businesses backing up nas/san storage arrays (just the upload speeds will prevent most nas/san backups), or not paying for a business “license” for “unlimited” backups, that your solution is the mandatory desktop software. At least put it in your faq or include a search function so we don’t have to waste time on the site trying to find out whether you are linux friendly or not.

  9. I don’t get it neither; with a planned 18,000TB space/year or 6,000 hard drive/year, how can CostCo external drive prices can beat a business wholesaler or a manufacturer negotiated price… ? even here money could have spent more wisely I guess

  10. So Costco was selling the hard drives below the supply/demand equilibrium to the benefit to their customers. Backblaze went to great lengths to conceal what they were doing from Costco, because Costco would never have chosen to sell them the drives because Costco was getting screwed. Backblaze gets through the crisis by avoiding half a million in costs by taking advantage of Costco. How brilliant; Backblaze is really earning their money.

  11. Rob Kyle

    Great article that I found quite inspirational. Kudos to the folks at Backblaze for doing what they had to do to maintain their business growth. The picture of the hard drives at Vladik Rikhter’s front was absolutely priceless.

  12. I don’t get it; why are consumer hard drives cheaper? Especially considering they have extra hardware (the caddy and PSU), packaging, manuals, marketing, retailing store costs etc. that business customers do not need.

    • Yev Pusin

      I’m not in the hard drive industry, but if general business rules apply, it’s because regular consumers are a lot more price sensitive than business’. For example, a business knows they need the hard drives, if the price goes up, they still need them, and will have to get them at a higher price. Consumers are much more inclined to wait for a sale, or just not get the drive. The consumer prices on external drives increased too, but they went from about $110/$120 for a 3TB drive to $160/$170, instead of going from $120 for an internal drive to over $400.

    • Enterprise drives are different. They maintain high throughput I/O rates when subjected to the high vibration environment of many servers and drives densely packed into racks.
      The performance hit is substantial.

      This was an interesting read…

      “Performance Impact of External Vibration on
      Consumer-grade and Enterprise-class Disk Drives”

      (Google the title)

  13. …..time and time again this is why small businesses fail……failing to plan. I know of no storage vendor that did not have a steady supply of HHDs this year. To find out that my accounting files, engineering drawings, contract were sitting on a Costco purchased aka sh$# drives I’d of sued these clowns at Backblaze. Folks checkout EMC’s Mozy, HP, hell I’d go to Dell before this moron……

    • Derrick Harris


      I can see your point to a degree, although I’d note two things: (1) Backblaze hasn’t noticed any reliability difference between the normal HDDs and the external ones, and (2) cloud services are designed with hardware failure in mind.

      Also, I don’t think everyone was loaded up with hard drives. HP and others noted the shortage might affect their bottom lines.

    • Yev Pusin

      Yev here w/ Backblaze -> Hi Mike! We actually use consumer level drives in our regular storage containers. Whether its good or bad, it works for us. This crisis did not force us to go from enterprise grade drives to consumer level drives, instead, we just found different ways of getting the consumer level products. Derrick’s points are great, and you can read about our storage container philosophy here:

    • Sean Patterson


      I was an IT manager at a university during the shortage. I ran into the same difficulties even though I’d done proper capacity planning, and ended up stripping external drives for use in my private cloud for archiving. If my backup services failed, my users stood a good chance of losing critical data and years of research. (For those who wonder, yes, I was using CrashPlan.) I tested the hell out of them before making the decision and I can say for sure (with the exception of the WD Green drives) they were as reliable as the pro versions I was buying. There are many reasons to buy enterprise level HDDs; reliability is not necessarily one of them.

  14. hydrian

    @Not. AWS is the not best priced storage. There are other backup services. I personally use and love CrashPlan. They have multiple levels of scale ability. They support personal, SMB. and enterprise level of support. They support Windows, MacOS, Linux, even Solaris. I use the personal version at home. They have a ‘family’ plan that for $USD12/month you can backup up to 10 computers with unlimited data. This is handy when you have multiple laptops that may not be on your home network all of the time. There client can be used for both local backup also. You can also backup with other CrashPlan friend for free. All data is AES client side encrypted so they don’t see a thing.

    P.S. I know this sounds like an advertisement, but they really do a good job at cloud backup.

    • If only they could handle the amount of files that I want to send them. I can’t even use it to backup one whole drive to another on a local system, let alone try to keep that in sync with their server.

  15. Psyphurr Lock

    I have a small 4 drive 1TB network attached SAN at home thats been running strong for over 7 years. No drive replacements, notta. I keep the data replicated just in case of course. but its still running like it did on day one. So I agree, buying your own disks is certainly less expensive in the long run.

  16. Something does not feel right about Backblaze bragging so much about their “drive farming’ exploits. While I agree that it was an innovative way to keep to their business goals, they did so by exploiting the retailers’ businesses. The retailers still made their money, but you could argue that it hurt their customer relationships. Alls fair and all that, but a company that boasts so much about something like this comes off as a bit tone deaf IMO.

  17. Yes hard drives can fail, but I have a 80GB drive in one of my 8 year old computers that is still going strong. It can be far cheaper to own the drives than to rent space on AWS.

    • Adrian O'Connor

      I presume that’s an essential factor in being able to offer the prices they do? I know my 100GB or so of AWS storage costs me a chunk of change each month, and I guess that add’s up when you’re talking 50TB/day. Don’t forget that data accumulates. 50TB added each day is 1500TB after one month, and 18,000TB after a year. That’d cost over $1million on AWS, versus slightly less for physical disks (but, as time goes on, you’ll get returns on owning the physical disks vs renting space at Amazon).

      • SpecialK777

        You have to keep in mind that hard drives have a finite life so you have to replace dying hard drives as well. Its not a buy once either way. You either rent space on Amazon or you are “renting” your physical hard drive at the cost of $169 for 2-3 years…

      • @ SpecialK777

        If it’s just backups, data is stored most likely for a long time. That 3-5 years you mention could mean writing data just once. I’ll admit that I don’t know exact business model used by Backblaze, but “unlimited cloud backup for just $5 a month”. sounds like something that would make me just store the data and forget about it. Sure HDD not being used for 3-5 years might not function, but hopefully they do some form of duplication anyway.

    • zipperbits

      … Backblaze is a cloud backup service that sells space similar to AWS. Ex: If you were selling ice cream you wouldn’t rent time at DQ to make your customers’ cones. So, not using AWS because they are a separate backup service who manages their own uptime.

    • zipperbits

      Backblaze sells cloud backup service to customers, in my opinion, it would be très weird to purchase infrastructure from a company offering competing services.

    • Derrick Harris

      You should click on the link high up in the post referencing Backblaze’s architecture. It goes into detail on how it builds 135TB “storage pods” on the cheap.

    • Gavin Greenwalt

      AWS is way more expensive.

      Backblaze’s pods cost $7,500 for 135TB or $55 per TB.
      AWS’s cheapest storage is about $35 per *month* per TB.

      Let’s be generous and say one employee needs to maintain a Pod once per month for one hour (way higher than our server management). Let’s assume they consume 1KW of power each. Even if the pod burned out every other year, unlikely but we’ll be generous, to Amazon that works out to ($3750) + (12 months*$100/hr) + ($0.10/KWh * 24hr * 365 days) = $5826/12 = $485 per month / 135 TB = $3.59 per TB per month. That’s assuming a 50% failure rate in HDDs per year.

      Let’s also not forget bandwidth. Backblaze is looking to upgrade to 10gbps for their datacenter (OC-192). Which runs about $200k per month from what limited data I could find. They currently have 40Petabytes which works out to $200k/40kTB = $5 per month per TB.

      So our total is about $9TB per month for Backblaze
      $30TB per month for AWS.

      But that’s just my estimate. You could easily cut almost all of those support time costs. You could very easily get 4 years not 2 years out of a pod and you could probably use green drives that use less than 200watts cutting your power by 80%. In other words $9TB is generous and assuming that they actually need 10gbps right now.

    • Jeff Martens

      Why do so many people assume that AWS is the best/only option? Thank god Larry and Sergey didn’t stop at thinking Yahoo was the best we could ever do.

    • What bit of Backblaze “offers unlimited cloud backup” do you not understand? They’re not a typical start-up, they’re effectively a 100% competitor to AWS in many ways! That would be like telling Oracle to buy their database expertise from Microsoft when faced with a DB consultant hiring shortage. Please!