DSSD was founded in 2010 by Jeff Bonwick and Bill Moore — both part of a select few of engineers with experience building out storage operating systems. With the backing of Bechtolsheim, a Silicon Valley rock star and co-founder of Sun Microsystems, who has backed Google and co-founded switch startup Arista, the company has some of the smartest people in the Valley working there. No one from the company wanted to comment on the story.

My sources tell me the startup is building a new type of chip — they said it’s really a module, not a chip — that combines a small amount of processing power with a lot of densely-packed memory. The module runs a pared-down version of Linux designed for storing information on flash memory, and is aimed at big data and other workloads where reading and writing information to disk bogs down the application.

This fits with the expertise of the team, but this is a problem that others are trying to solve as well with faster and cheaper SSDs and targeted software to to optimize the flow of bits to a database. But the proposal here appears to be about designing an operating system that takes advantage of the difference in Flash memory when compared to hard drives to boost I/O.

For example, on old disk drives you store a group of bits in sequential order, but in reality those bits may get dropped anywhere in the drive. After regular use, when you delete a file, a tombstone marker is placed on the “deleted” file and you have to then find that tombstone and re-write just the amount of data in that space and then find more space for the rest. So the data goes everywhere.

But the DSSD system sounds like it treats files not as a series of bits but as an object that gets a name. That name is the file’s address and it stays the same for the life of the file. The result is there’s no central index that stands between sending the data to storage and storing it, and people can write to it in parallel and not worry abut overwrites. It is both faster and can scale out.

For more details, we can turn to the six patents that DSSD has filed. In mid-March Storage Mojo unearthed patents affiliated with the company that imply it is building a type of faster object-level storage using Flash that’s more durable. From the Storage Mojo article:

So what are they building? They are taking a radically different approach to the problem of high-performance transaction processing storage. The use of flash is a given in TP, and the extra durability, scalability and guaranteed read latency would be very attractive in large TP applications.

The most surprising piece is the object storage-like characteristics suggested by the patents. But handling billions of small objects at high-speed in a flat namespace would make it easy to distribute object indexes among hundreds of users, reducing file system I/O latency. The 3D RAID could eliminate the encoding overhead inherent in advanced erasure codes while providing similar robustness, enabling way-beyond-RAID6 availability.

For those who aren’t storage or computing buffs, the problem here was well explained in a fireside chat that Bechtolsheim had with my colleague Om Malik at our Structure:Data 2011 conference. In it Bechtolsheim outlines the problem that the network causes for access to big data around the 6-minute mark and the need to build new interfaces that can take advantage of the parallelism inside flash chips compared to hard disks. If you do that, you can expand the capabilities of flash beyond just density because you can write data to it faster, meaning the network no longer gums up the works.

Of course, when talking about using flash in more places, there’s always the question of whether this architecture will offer enough of a performance gain to justify the higher price per gigabyte of flash over a hard drive, but for that information we’ll just have to wait.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• AWS Storage Gateway jolts cloud-storage ecosystem
• Dissecting the data: 5 issues for our digital future
• Flash memory: the continuing disruption of enterprise storage

It’s also a huge amount of money, but Skyera is riding a wave of interest in solid state storage, which is both faster and consumes less power than traditional hard drives. But Flash-based SSDs are also more expensive than hard drives on a per gigabyte basis — which has limited their use to places where companies can’t afford not to pay for faster performance.

As my colleague Barb Darrow noted in a profile on Skyera last year, the company brags that it can achieve a $3 per GB cost which is still pricier than a hard drive but less than what Fusion-io recently announced with its latest NAND flash card product. Fusion said it could achieve per GB prices of around $3.89. From Barb’s profile:

To get to cheap flash enterprise storage, Skyera uses inexpensive Multi-Level Cell (MLC) NAND flash, but found a way to do it to prolong the lifespan of the media. It is able to use high-density sub-20-nm MLC flash because its controller dynamically adjusts as the medium ages to reduce damage over time.

The funding will be used to integrate next-generation flash memory chips produced at 20 nanometers into Skyera’s product as well as boost sales.

As for Dell, which is setting itself up to be the provider of hardware for webscale and cloud vendors with its DCS division, investing in SSDs makes absolute sense, especially if there are ways it can integrate Skyera’s technology more deeply in its gear. For Skyera, having Dell as an investor might help it build out a lucrative sales channel.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A near-term outlook for big data
• Infrastructure Q3: OpenStack and flash step into the spotlight
• Flash memory: the continuing disruption of enterprise storage

ScaleIO joins other startups that have launched in the latter half of the year hoping to create pools of storage that can be used by any application in a virtualized environment. Those startups include Nutanix and Convergent.io. Fusion-io has a software product in this space as well.

The idea behind this growing number of products is to help address the problems that virtualization has caused for the storage market by making a storage area network (SAN) look more like direct attached storage. In traditional networks someone would configure a server to point to a SAN but these products take advantage of the ever-growing use of SSDs in the data center and overlay software on top to make a more flexible pool of storage. Most of these products, including ScaleIO’s, also let organizations tie in their existing hard disks and SANs into this pool of storage as well.

The result is that IT organizations can take advantage of fast RAM or nearby flash-based SSDs for applications that need a lot of speed. They can also stitch together several nearby SSDs or RAM to scale out as needed while still using existing storage infrastructure comprised of hard drives as well. ScaleIO also touts that its software can move storage resources around even if a node in its pool goes dark like for example, if a systems administrator shuts down the server. That’s a good indication that the team here recognizes that while the virtual servers in a cloud may be ephemeral but underneath it all are physical machines that must be taken into account.

That’s a good understanding to have when you’re building out software that is designed to create an abstraction layer to manage physical devices. ScaleIO’s team has worked at a huge number of scaled-out storage companies including IBM, Topio (acquired by NetApp), NetApp, LSI, and Xtremio (acquired by EMC). The Palo Alto, Calif.-based startup already has 24 pilot customers and five customers in production including SAP and Colt (a Fidelity company).

Feature photo courtesy of Flickr user getinet

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Flash memory: the continuing disruption of enterprise storage
• Report: How Mobile Cloud Computing Will Change Tech
• A near-term outlook for big data

If you’re a tech company trying to speed up applications and provider faster access to more memory in your servers, the technology could be your holy grail. Everspin, the Chandler, Ariz,-based startup pioneering MRAM, said today that its spin-torque MRAM that it hopes to use to replace DRAM, is sampling and would be available for use in products by 2013. MRAM is one of several new evolutions in memory technology designed to make applications perform faster by removing bottlenecks in getting the data from storage to the processor. Today, even flash-based systems can still cause a delay in application performance.

Steffen Hellmold, VP of marketing of Everspin, says that MRAM in its current version is about 100 times faster than flash, with the caveat being that he’s comparing MRAM to basic flash, while most companies tweak their flash arrays to make them faster and more reliable. Another caveat is that MRAM costs about 10 times more than flash-based solutions, although for certain applications trading 100 times the performance at 10 times the price is an easy call to make. The final caveat is that the current samples are for 64 Mb memory sizes which is less memory than your home PC had in the mid 90s. Other types of memory in this emerging niche include phase-change memory and memristors, which are pretty far away from commercialization.

For more on MRAM check out this article from early this year, or a background[ER] on Everspin written when it was spun out from Freescale Technology back in 2008.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Report: How Mobile Cloud Computing Will Change Tech
• The importance of putting the U and I in visualization
• AWS Storage Gateway jolts cloud-storage ecosystem

In a conversation after our discussion onstage at Structure:Europe on Wednesday, Parikh said that instead of just using flash in places where you need performance, “like adding a Ferrari engine to your server,” the social network was thinking about other places it might be useful. He declined to go into more details but implied there were places one could replace hard drives with flash. Flash is common in data centers where performance matters, such as for database storage and for speeding up access to images. Facebook uses flash from Fusion-io is some areas of its operations where performance is essential.

But flash is expensive when compared to the traditional hard drives that most companies use for storage. Part of that expense comes from the work that goes into making flash memory — which was designed for consumers — dependable enough to handle enterprise data. Flash can be unreliable, and as its storage capacity grows, the number of times it saves or accesses data shrinks. And while the cost of flash is dropping, hard drives are still cheaper on a per byte basis.

But flash memory is also more energy-efficient when compared to hard drives. And several sources have told me that Facebook is building software that will allow it to take advantage of flash that has been rejected for high-performance use cases like memcached servers and instead use it for storing data that might otherwise go on hard drives. The software would map the unreliable areas of the flash memory and then avoid putting data in those spots. Other sources have mentioned that Facebook is building “flash appliances” which could be arrays of the rejected flash drives and the software to manage them.

And I think that kind of appliance or custom-flash array would be perfect for Facebook’s cold storage facility it has been talking up recently. The idea behind the cold-storage facility is that Facebook wants a place to efficiently and cheaply store those 220 billion photos it has — it gets 7 petabytes of new photos each month, Parikh said during our on-stage conversation at Structure.

Since after a few days the desire to access those photos diminishes, Facebook needs a place to put them that’s cheap but still accessible for those times users are feeling nostalgic. Cheap flash-based arrays would be perfect. Because the data in cold storage is infrequently accessed, the diminished read-write capability of the drives is less of a problem. Because flash has no moving hardware it’s not only greener, but its something that can be turned on instantly. In an interview with Wired Tom Furlong of Facebook said the equipment in the cold storage array would be designed to turn on and off when not in use.

So, Facebook officially says it wants to use flash in more places, while sources have been sharing details of Facebook’s flash appliances. I think that adds up to flash being a key component of Facebook’s cold-storage boxes. I reached out to Facebook for this story, but have not heard back.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• A near-term outlook for big data
• NewNet Q2: Google closes the quarter with a bang
• Finding the Value in Social Media Data

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Who and what to watch in the new era of the living room
• 2012: Data, spectrum and the race to LTE
• Connected world: the consumer technology revolution

Network agnosticism is becoming a selling point for flash storage. Earlier this week, Nimbus Data said its new Gemini flash arrays can be software-configured to run on Ethernet, Infiniband or Fibre Channel networks. That’s important for companies that don’t want to pay for lots of new switching hardware to handle network upgrades.

Pure Storage’s FlashArray initially supported Fibre Channel networks only. That limited the product’s appeal for the many businesses that run all Ethernet in their shops. The company is also adding snapshotting — technology which lets users restore data quickly in the event of a problem. The new features will come as part of a free software upgrade to the company’s Purity operating environment, now in beta and slated for release this fall.

Pure Storage CTO John Colgrove (left) and CEO Scott Dietzen.

The Mountain View, Calif. company also announced a new VMware web client plug in so that admins that manage and monitor  virtual infrastructure  will be able to manage FlashArrays from within their familiar vCenter console.

Flash storage in general is taking off if judged by the sheer number of press releases generated in the past month. The technology remains more expensive than disk storage but that price delta is coming down and vendors are also addressing longevity concerns users have about the use of flash or solid-state storage.

Last week, IBM got into the act, announcing plans to  buy  Texas Memory Systems and its flash expertise, for an undisclosed amount. That acquisition should make IBM more competitive with EMC, which earlier bought EXtremeIO for its flash wherewithal.

All of that action means that smaller, but well-funded startups like Pure Storage, Nimbus Data, and Skyera, will compete not only with each other but with established IT giants as well.

It’s going to be an interesting ride.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Cloud and data third-quarter 2012
• A near-term outlook for big data
• Dissecting the data: 5 issues for our digital future

There is no doubt that flash or solid-state memory is a hot commodity with industry giants like IBM and EMC buying up expertise. EMC purchased XTremeIO last spring for a reported $430 million.

At the same time, startups are attacking the opportunity without legacy strings attached. One of those young companies, Pure Storage, just netted an impressive $40 million in venture capital to pursue its all-flash vision. Skyera, another new entrant espousing all-flash-all-the-time, came out of stealth this week.

These companies are pushing the cost of flash storage down — getting it closer to that of slower but cheaper hard drives. However, as many GigaOM commenters have pointed out, price parity is not there yet. Gleb Budman, CEO of Backblaze, which offers inexpensive cloud-based data backup, would love to go all-flash but the economics are not there yet, he said but added  they’re getting closer.

IBM said it will integrate Texas Memory flash into its new PureSystems line of converged data center hardware.

Terms of the deal, due to close later this year, were not disclosed.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Cloud and data third-quarter 2012
• A near-term outlook for big data
• Dissecting the data: 5 issues for our digital future

Mountain View, CA.-based Pure Storage is one of a handful of companies evangelizing use of solid state storage — which is faster but more expensive than disk storage — nearly everywhere. Many companies, including storage leader EMC, preach a tiered approach, using flash where it makes sense but disks and even tape where that is the more appropriate and cost-effective choice. (EMC itself may be wobbling on this stance, given that it spent a reported $400 million on Israeli flash storage player EXtremeIO.)

Earlier this week, another flash storage startup, Skyera, emerged from stealth with aggressive plans to push the cost of its flash storage systems to an impressive $3 per GB.

As GigaOM’s Stacey Higginbotham wrote last August, Pure Storage fields impressive management – with founding team members coming from Zimbra, Veritas and Sun Microsystems. Its CEO, Scott Dietzen, is a respected industry veteran who co-founded Zimbra after years at BEA Systems, now part of Oracle.

Pure Storage says it’s shipped more than 100 of its FlashArrays since launching last year. As part of this funding round, Index Ventures senior partner Mike Volpi will join the board.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Cloud and data third-quarter 2012
• AWS Storage Gateway jolts cloud-storage ecosystem
• A near-term outlook for big data

Solid-state, or “flash”, storage is hot, hot, hot, hot. A few proof points: Pure Storage just netted $40 million in Series D funding and storage kingpin EMC bought XtremeIO for $430 million last May.

But Skyera claims it’s out-doing Pure Storage, Violin Memory or Nimbus Data and others in making solid-state storage more price competitive with as inexpensive as hard drives and by resolving longevity issues that make IT buyers nervous about adopting the technology. The company claims it can cram 44 TB of storage in a 1u rack that comprises a ground-up-engineered system including the flash controller, RAID controller, storage blades and network interface.

Taking flash mainstream, really

“Competitors got their [native] cost to $7 or $8 per gig and that puts you at the high-end niche level of enterprise applications where IT has to put such a premium on performance they’re willing to pay ten times more than a comparable hard disk system,” Skyera sales VP Tony Barbagallo told me in an interview. “Solid-state storage will not go mainstream at those prices and our goal is mainstream.”

Skyera’s founder and CEO is Rado Danilak, who founded SandForce, a storage technology company LSI bought last year for $400 million. He and his engineering team has a ton of credibility in this arena. SandForce controllers own about 80 percent of the market, according to some estimates.

To get to cheap flash enterprise storage, Skyera uses inexpensive Multi-Level Cell (MLC) NAND flash, but found a way to do it to prolong the lifespan of the media.  It is able to use high-density sub-20-nm MLC flash because its controller dynamically adjusts as the medium ages to reduce damage over time.

Putting flash everywhere

The use of consumer-grade flash in enterprise storage is not new.   ”What’s cool here is how much capacity [Skyera] gets in a very small footprint and that they’ve integrated the switching capability right into their array,” said Jim Bagley, senior analyst with Storage Strategies Now, a storage consultancy.

The Skyera team appears to have come up with the next generation of that popular SandForce controller and crafted a huge performance improvement in the process.

Skyera will demonstrate its Skyhawk systems at next week’s Flash Memory Summit. Expect a spate of other flash-related news to come at that event and at VMworld, the following week.

Most storage vendors still talk about using flash judiciously in the enterprise, reserving it for jobs that demand the fastest response. Flash often acts as a “turbo” to an existing storage engine, said Mark Peters, senior analyst with Enterprise Strategy Group.  “Skyera is saying:  ’forget that, use solid state for everything.’ The only reason we haven’t done this already is money. It all comes down to price. And, if Skyera has done what it says it’s done on price, this truly is disruptive,” he said.

Fusion-io, a maker of in-server flash, stands at about $11 per GB, Nimbus Data, a Skyera competitor, at about $10 per GB s0 if Skyera can deliver its promised $3 per GB target, it’s a pretty significant cut.

George Crump, lead analyst for Storage Switzerland agreed. “I get asked all the time when flash will take over the data center and the answer is: when it’s the same cost as hard drives.”

If Skyera Skyhawk  lives up to its billing, this is a big step toward that goal. “The next question is if or when they’ll get leapfrogged? We’ll see,” Crump said.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Cloud and data third-quarter 2012
• Flash memory: the continuing disruption of enterprise storage
• Takeaways from the second quarter in cloud and data