18 Comments

Summary:

I ran across Terracotta Inc. a few months ago while looking at database companies, and was impressed by the potential of its eponymously named open source software, which can make web applications scale faster and more cheaply than they do when information is stored in a database. […]

I ran across Terracotta Inc. a few months ago while looking at database companies, and was impressed by the potential of its eponymously named open source software, which can make web applications scale faster and more cheaply than they do when information is stored in a database. Instead the software from Terracotta, which was formed in 2003 and has raised $29 million from Accel Partners, DAG Ventures, Benchmark Capital and Goldman Sachs, takes information and writes it to a shared cluster of memory.

That makes the data available for quick access without the need for the arduous and time-consuming processes of structuring it for a database, storing it there and retrieving it later. It’s not appropriate for information such as sales records and other data that either fits well in a table or needs to be accessible for a long period of time. And it only works in Java.

If you think of data as clothing, a database is like a closet, where you can hang and store items in an orderly fashion — knowing they will be left in the same place you hung them. Terracotta is like a dresser drawer where you put your clothes that need to be grabbed on the fly — like a pair of socks. The data is still there, but it’s a lot faster to toss your socks in a drawer than to take the time to hang them on hangers. In a web application with lots of data coming in, it turns out drawers work just fine.

This may be bad news for Oracle, which makes money convincing customers they need more closets (it does have Coherence, a similar product). However, companies from Adobe, that uses Terracotta for its ConnectNow web conferencing service, to startup Gnip, which uses it to deliver a variety of personal tweets, emails and other messages, have decided that they need sock drawers.

You’re subscribed! If you like, you can update your settings

  1. Stacey,

    Oracle has Coherence, which is a distributed cache and designed to act as the drawer to their closets as well. So, Oracle covered their bases here pretty well. Coherence was originally offered by Tangosol, which was acquired by Oracle in 2Q 2007.

  2. silicon valley dropout Thursday, December 18, 2008

    database is big business for oracle. i dont see anyone eating their lunches anytime soon.

  3. There seem to be different claims of that sort in the media in the past few weeks.
    I actually replied to one of them: Kill Your Database with Terracotta by Tushar Khairnar’s. I believe that the same comment applies here: (Quoting from this reply)

    There are few points that one should consider with this pattern:

    1. What do you do in case you already have an existing database, how do you integrate existing system with this model? You would find that this is probably one of the more common scenarios. In this case you will also find that mapping data from its existing format to the In-Memory representation is required. Note the use of external datasource as a plug-in allows you to plug-in different database implementation and mapping styles. The common one would be hibernate plug-in for existing database and black-box plug-in in case your just looking for durability and you don’t care how the data is being stored. We found that this level of flexibility is required as in many cases the database serves also as an integration point with other applications and not just as a persistent storage.

    2. What happens in case of a failure i.e. when the database server is down for 10 hours? Can the application continue to run?

    3. How do you manage the transaction scope? If transaction is blocked until the database is synched you may loose large part of the performance benefit of working in-memory for read/write scenarios.

    This has been some of the assumptions behind the design of our reliable async replication and Mirror service which runs today and already being used in most mission critical application in the trading area.

    You can see more details on this pattern in some of my earlier posts here and here

  4. Stacey Higginbotham Thursday, December 18, 2008

    Zubin, you’re right, I should have mentioned Coherence. Thanks for pointing that out.

  5. This seems to be just re-inventing the old object database technology (ODBMS) pioneered by Versant, ObjectStore, et al in the early 1990s. Look up ‘pointer swizzling’ etc for how they managed to enable this without any application code changes. Similar ideas are also implemented in various forms by Java persistence libraries. Congratulations on them for managing to make it sound new there – triumph of marketing rather than technology though.

    The basic issue with ODBMSs of all types is that they work great for single applications, hierarchical data, OO data, etc, but aren’t so good for more business-oriented flat table data that must be shared between multiple apps. In some cases, a broken app can even corrupt the DB because the DB is really just a persistent version of in memory data strucutures. I’m generalising a lot here, but there’s a reason why people buy object-relational mapping tools – once the data is in boring old relational format it’s a lot easier to ensure its integrity and share it among multiple apps.

    I really think GigaOM needs some savvy technical people to review some of these articles – the other one recently where Google’s Native Client was described as a “scripting language” was so technically inaccurate that it’s really a bit embarrassing. I like GigaOM a lot, but the lack of technical background / insight is worrying sometimes.

  6. well, the team is interesting (ari and amit in particular), but i don’t see any real dbase genius creating something new here – more of the same old same old, in a field in which the technology itself is around 30 years old…what i find much more interesting is what cloudera is doing (worth a look) and also what greenplum and vertica are doing, that feels a bit more fresh and new – but this kind of approach to data store is not novel, not the way terracotta seems to be doing it…

  7. @richard

    I agree.

    I volunteer myself to be giga’s elite geek tech advisor, I’ll do it for free! Email me, no seriously.

    @dave

    You should read vertica’s database blog, called the ‘database column’ it is a GREAT DB read, very down to the details.

  8. 451 CAOS Theory » 451 CAOS Links 2008.12.19 Friday, December 19, 2008

    [...] Terracotta Doesn’t Want to Kill Your Database, Just Maim It Stacey Higginbotham , GigaOM [...]

  9. Just wanted to comment to a few folks posts here:

    1. Nati – weird that you have feedback and concerns on the “design pattern” espoused in the article as none is actually put forward by the author. Are you actually trying to assert that your product does something better than a design pattern that has not been specified? That’s crazy talk…you don’t even know what it is you are doing better than. If I treat your concerns as generic to “write behind to a database” type approach, then you miss the entire point of the Terracotta approach. We represent durable, transactional network attached memory–with Terracotta your objects are on disk at local memory speeds and thus it suffers none of the risks you question. 10 hours db downtime is not an issue because we have the app mem on disk. transactions to the DB are not the same in the presence of transactional memory like ours because your app won’t lose state even if it loses power, so again not an issue. In general, I would recommend studying up on Terracotta some more.

    2. Zubin – Oracle coherence does not compare with Terracotta. The 2 are complementary. Coherence is for large in-memory data distribution and crunching problems. It puts no data on disk anywhere. Terracotta is for enterprise app db offload and simplification. Oracle does not in fact have a solution with Coherence for lowering app costs–Coherence will offload the db but makes the app more cumbersome and you pay Oracle for yet even more software in addition to the DB. Also note for those who will likely want to point out to me that one can pay for Terracotta just as one would pay for Coherence, there’s a difference. With Coherence, our customers who have switched to Terracotta all say they shrunk their DB installation. But when they were using Coherence, since it has no disk-based persistence the same sized Oracle DB one would need to handle all your DB transactions is still needed.

    3. Richard – Terracotta is not 30 year old technology. We are network attached memory. We are fine-grained meaning we push only memory-level deltas. And we are runtime optimized meaning we reorder and optimize application locking and transactions so as to eliminate or reduce network I/O to a bare minimum. Show me any technology in the past 30 years that gets into application memory, reorders the app memory operations at runtime like hotspot would do to code, and most importantly produces software transactional memory across threads running across processes on separate machines. You won’t find all those pieces and parts put together ever in the past. In fact Sun published a paper in 1994 saying they wished such a thing existed because it would save lots of developers loads of work. If you assert, instead, that Terracotta uses all the computer science theory invented by the 70’s and 80’s and very little new then I would agree. But that statement is true of almost _all_ algorithms and ideas anyone uses today. Most things developers do were created at Xerox Parc or elsewhere in the 70’s (and earlier). Not sure this means Terracotta is any less interesting or helpful to developers of Java apps struggling to live with ORM and SQL and, most importantly, Oracle.

  10. Some differences between OODB and Terracotta
    OODB’s tend to be:
    1) An application talking to a database
    2) For long lived data
    3) Has query
    4) Has it’s own usually optimistic transaction model

    In Terracotta:
    1) It is extending parts of your app to the cluster
    2) For data that is short to medium lived
    3) extends the java memory model/ normal java locking model to the cluster
    4) All behavior is executed in client memory.
    5) Pluggable modules for integrating with your favorite frameworks

    I could see a scenerio where someone would use an OODB and Terracotta. Their is
    very little overlap between what one would “Want” to put in a db and what one wants to put
    in Terracotta.

    Alex miller wrote an excellent few blogs on this topic.
    http://tech.puredanger.com/2008/12/07/terracotta-replacing-the-database/

Comments have been disabled for this post