8 Comments

Summary:

Executives at NoSQL startups are keeping a brave face in response to Amazon Web Services’ new DynamoDB offering. They cite the new product as a validation, while generally dismissing the competitive ramifications of having Amazon now playing in the same pool. But is that confidence justified?

dark clouds

Top executives at NoSQL startups are putting on a brave face in response to Amazon Web Services’ new DynamoDB offering. They roundly cite the new product (as well as Oracle’s October entrance into the space) as validation for the technology NoSQL companies have been pushing for years, while generally dismissing the competitive ramifications of having major vendors now playing in the same pool. But is that confidence justified?

Validation is good

Dwight Merriman, CEO of MongoDB proprietor 10gen, summed up the general sentiment of his peers in an email response to my request for a comment:

The Amazon Dynamo DB announcement is further validation that NoSQL is a big deal, and we are excited to see large players like Oracle and Amazon recognizing the need for alternatives to the relational database. Their entry into the field makes it clear to all large enterprises that this is an important trend – as we have seen that traditional databases do not fit well with cloud computing. New database technologies will be needed in the cloud, and also in the enterprise private cloud.

DataStax CEO Billy Bosworth makes a similar argument on his blog, as did new Cloudant CEO Derek Schoettle during a Friday-morning phone call. He said DynamoDB is “awesome” and Cloudant is “excited about it.” “[AWS] will be a competitor by default,” he said “but their success will be our success.” As the saying goes, and as GigaOM Pro’s Jo Maitland explains in research note on DynamoDB (subscription req’d), a rising tide floats all boats.

But is competition really good?

However, there are plenty of reasons for NoSQL-based startups to fear these new big-name competitors. When competing against Oracle, the challenge will be to convince large enterprises that third-party NoSQL databases are a better fit with existing Oracle ecosystems than is Oracle’s custom-built offering. Nobody ever got fired for buying Oracle, and if it’s offering NoSQL as part of an integrated data environment that also includes a relational database, data warehouse and Hadoop, there might be a natural inclination to just go with Oracle.

With AWS and DynamoDB, however, NoSQL companies find themselves fighting for the websites and other web-based customers that are now their bread and butter. Sid Anand, who helped transition Netflix from Oracle to AWS’s SimpleDB to Cassandra and who now is on the LinkedIn infrastructure team, wrote on his blog earlier this week that “[i]f [your NoSQL database] is not hosted (e.g. by AWS), be prepared to hire a fleet of ops folks to support it yourself. If you don’t have the manpower, I recommend AWS’[s] DynamoDB.”

It appears some are following his advice. One commenter on a blog post by Apache Cassandra chairman (and DataStax co-founder) Jonathan Ellis detailing the technical differences between Cassandra and DynamoDB wrote, “Cassandra’s tech is superior, as far as I can tell. But we’ll probably be using DynamoDB until there is an equivalent managed host service for Cassandra. Moving to Cassandra is simply too expensive right now.”

And AWS’s DynamoDB is built atop a solid-state-drive infrastructure, which helps ensure predictable performance that isn’t always available if you’re running a NoSQL database on cloud computing instances unless data is stored in-memory. In August, 10gen’s Merriman wrote a brief blog post simply asking “where are the SSDs in the cloud?”. Now we know: AWS has them, and, as of now, no one else can use them.

It depends whom you ask

As with most cloud services, at least in their initial incarnations, DynamoDB definitely favors simplicity over lots of features and fine-grained control. Amazon CTO Werner Vogels explains as much in his post announcing the service. If those things are important, users are almost certainly better off choosing a full-featured database.

Ellis’ aforementioned post lays out the reasons one might choose Cassandra. A spokesperson for Basho, which develops the Riak database, sent me a list of three questions everyone should ask when choosing a NoSQL option:

  • Is this solution proprietary or open-source?
  • Is my data secure? Is the solution fault tolerant?
  • What are the querying capabilities for search and indexing?

Basho thinks might very well argue that Riak is superior to DynamoDB on all counts, and CTO Justin Sheehy said via email that Riak runs on any infrastructure and very likely will cost less to run over time. Assuming that’s true, it’s really just an extension of the discussion of tradeoffs of choosing cloud-based servers or relational databases, now applied to a NoSQL database.

Cloudant CEO Schoettle acknowledges there’s “about 60 percent overlap” between DynamoDB and Cloudant, but companies dealing with large data sets and trying to solve complex problems would be better off choosing his company’s hosted CouchDB-based service. While DynamoDB is “essentially a key-value store with a hash methodology,” Cloudant offers integrated search, replication and advanced data analysis capabilities. It also offers SSDs if customers need them.

There also are a handful of hosted MongoDB options available, including MongoHQ and MongoLab, and MongoDB instances are available through a number of IaaS and PaaS providers. DataStax’s Cassandra database is currently in private beta on the Heroku platform.

So perhaps NoSQL vendors really are right to welcome Amazon’s DynamoDB with open arms. “You can perhaps get a little weak in the legs [when you hear you're competing with Amazon],” Schoettle said, but Amazon will go a long way toward educating potential customers on NoSQL, generally. When they realize they need something more, the existing camp of NoSQL will be there to help.

Image courtesy of Flickr user Jo Naylor.

  1. The more the merrier!!

    Share
  2. The scary thing for all of the other vendors including Oracle is that Amazon will be able to offer continually decreasing prices as more people use dynamo and hardware prices continue to drop. This will be a continuing advantage that neither pure play software vendors or companies that rely on services revenue will be able to match. If Amazon is going to have competition in NOSQL in the long term it will be from another IaaS or PaaS vendor of similar scale. http://blog.patternbuilders.com

    Share
  3. I believe many are missing a key component to the discussion, the interface. AWS is clearly a leader in many areas of providing cloud services, but they make no attempt to ensure their interface is interoperable or standardized. This is consistent with DynamoDB. While true that their interface is based on open standards, this is a call to arms to the other NoSQL players to join together and standardize interfaces making it easier for customers to select the right NoSQL product without being locked in due to the interface. Playing the lock-in card is both fair and valid as an approach to competing with AWS.

    Share
    1. Per JP’s point above, AWS API is a de facto standard for cloud interoperability; see for example Zynga’s use of the AWS API to shift from AWS to on-prem Zynga when the demand characteristics of a new online game are well understand. OpenStack is a worthwhile non-proprietary open source option but take-up has been slow to date, and standard JDBC/ODBC drivers can be slow and not well designed for migration to/from distributed cloud DB’s unless optimized.

      Best regards,

      Brett

      Brett Sheppard
      Twitter @zettaforce

      Share
  4. Openess will be a huge barrier to entry. Many folks who are deploying on AWS want the option to migrate/move their operations if need be. As I recall SimpleDB wasn’t too popular largely for this reason. Perhaps the weight of vendor lock in will have an influence. The fact that AWS is holding back on SSDs for other platform developers is a sign that they are okay playing a game on uneven turf to their advantage. Beware.

    Share
  5. Not sure if they should be afraid off but yes should definitely feel the pinch as and when the user base increase for Dynamo. Time and again we have seen that no “One” is bigger and there is always a competition and so does Amazon ;). Give it few more months and probably there ll be one of big 5′s competing against DynamoDB with their own launch.

    Share
  6. Moreover MongoDB fulltext search with MongoLantern is there, which is a big advantage for using MongoDB.

    Share
  7. Everyone’s conveniently forgetting how often hackers bring down Amazon instances. Who really has access to yuor data if it’s in the Amazon cloud? They’ll never tell you.

    Share

Comments have been disabled for this post