Terracotta Doesn't Want to Kill Your Database, Just Maim It


I ran across Terracotta Inc. a few months ago while looking at database companies, and was impressed by the potential of its eponymously named open source software, which can make web applications scale faster and more cheaply than they do when information is stored in a database. Instead the software from Terracotta, which was formed in 2003 and has raised $29 million from Accel Partners, DAG Ventures, Benchmark Capital and Goldman Sachs, takes information and writes it to a shared cluster of memory.

That makes the data available for quick access without the need for the arduous and time-consuming processes of structuring it for a database, storing it there and retrieving it later. It’s not appropriate for information such as sales records and other data that either fits well in a table or needs to be accessible for a long period of time. And it only works in Java.

If you think of data as clothing, a database is like a closet, where you can hang and store items in an orderly fashion — knowing they will be left in the same place you hung them. Terracotta is like a dresser drawer where you put your clothes that need to be grabbed on the fly — like a pair of socks. The data is still there, but it’s a lot faster to toss your socks in a drawer than to take the time to hang them on hangers. In a web application with lots of data coming in, it turns out drawers work just fine.

This may be bad news for Oracle (s orcl), which makes money convincing customers they need more closets (it does have Coherence, a similar product). However, companies from Adobe (s ADBE), that uses Terracotta for its ConnectNow web conferencing service, to startup Gnip, which uses it to deliver a variety of personal tweets, emails and other messages, have decided that they need sock drawers.



Databases are not a bottleneck on any decently designed application. The majority of the data should already be cached before you even get to the database and the rest cached by the database server.

Cameron Purdy

Ari –

You continue to dodge Nati’s question. You claimed “with Terracotta your objects are on disk at local memory speeds”. Nati asked: “How you can claim that you can persist data at in-memory speed?” (Which isn’t necessarily what you said.) You responded: “Persisting data at in-memory speed is a fundamental Terracotta invention. By being transparent, we get to figure out what is changing and push only deltas ..”

So you get a perfect 10 for dodging the question (and digging yourself a hole ;-) ..


Cameron Purdy | Oracle Coherence


Nati: you are missing something. We are not getting rid of the database. You definitely misunderstood that our intent was to detach for a while, and reattach later.

1. Persisting data at in-memory speed is a fundamental Terracotta invention. By being transparent, we get to figure out what is changing and push only deltas, in big batches to multiple disks for redundancy.

2. We don’t care if a node fails. We have all the data on 2 nodes on disk. We transparently handle failover.

3. “bounded to that specific machine?” What does that mean? We have no problem migrating state from one machine to another using built-in software that allows you to introduce a blank node to the grid, have it image off of another node, and then start doing work. Its quite easy actually. Some of our customers take nodes on and offline in production while serving >10 thousand transactions per second!


Nati Shalom


Maybe i’m missing something.

The argument made here and in recent posts hint that you can get rid of a database with Terracotta right?
If that is the claim then the concerns that i was trying to make are valid.

If you claim that each terracotta node persist its state to disk then that brings lots of other questions.

1. How you can claim that you can persist data at in-memory speed?
2. What happen if one of node fails how do you deal with hot fail-over?
3. If my node becomes persistent to local-disk wouldn’t that mean that it will become bounded to that specific machine?

These are only few of the questions that we had to deal with in the past when we introduced similar approach.
Anyway if there is basic information that i’m missing i’ll appreciate if you could point me (and other readers) to the specific location with the relevant information.

Steven Harris

Some differences between OODB and Terracotta
OODB’s tend to be:
1) An application talking to a database
2) For long lived data
3) Has query
4) Has it’s own usually optimistic transaction model

In Terracotta:
1) It is extending parts of your app to the cluster
2) For data that is short to medium lived
3) extends the java memory model/ normal java locking model to the cluster
4) All behavior is executed in client memory.
5) Pluggable modules for integrating with your favorite frameworks

I could see a scenerio where someone would use an OODB and Terracotta. Their is
very little overlap between what one would “Want” to put in a db and what one wants to put
in Terracotta.

Alex miller wrote an excellent few blogs on this topic.


Just wanted to comment to a few folks posts here:

1. Nati – weird that you have feedback and concerns on the “design pattern” espoused in the article as none is actually put forward by the author. Are you actually trying to assert that your product does something better than a design pattern that has not been specified? That’s crazy talk…you don’t even know what it is you are doing better than. If I treat your concerns as generic to “write behind to a database” type approach, then you miss the entire point of the Terracotta approach. We represent durable, transactional network attached memory–with Terracotta your objects are on disk at local memory speeds and thus it suffers none of the risks you question. 10 hours db downtime is not an issue because we have the app mem on disk. transactions to the DB are not the same in the presence of transactional memory like ours because your app won’t lose state even if it loses power, so again not an issue. In general, I would recommend studying up on Terracotta some more.

2. Zubin – Oracle coherence does not compare with Terracotta. The 2 are complementary. Coherence is for large in-memory data distribution and crunching problems. It puts no data on disk anywhere. Terracotta is for enterprise app db offload and simplification. Oracle does not in fact have a solution with Coherence for lowering app costs–Coherence will offload the db but makes the app more cumbersome and you pay Oracle for yet even more software in addition to the DB. Also note for those who will likely want to point out to me that one can pay for Terracotta just as one would pay for Coherence, there’s a difference. With Coherence, our customers who have switched to Terracotta all say they shrunk their DB installation. But when they were using Coherence, since it has no disk-based persistence the same sized Oracle DB one would need to handle all your DB transactions is still needed.

3. Richard – Terracotta is not 30 year old technology. We are network attached memory. We are fine-grained meaning we push only memory-level deltas. And we are runtime optimized meaning we reorder and optimize application locking and transactions so as to eliminate or reduce network I/O to a bare minimum. Show me any technology in the past 30 years that gets into application memory, reorders the app memory operations at runtime like hotspot would do to code, and most importantly produces software transactional memory across threads running across processes on separate machines. You won’t find all those pieces and parts put together ever in the past. In fact Sun published a paper in 1994 saying they wished such a thing existed because it would save lots of developers loads of work. If you assert, instead, that Terracotta uses all the computer science theory invented by the 70’s and 80’s and very little new then I would agree. But that statement is true of almost _all_ algorithms and ideas anyone uses today. Most things developers do were created at Xerox Parc or elsewhere in the 70’s (and earlier). Not sure this means Terracotta is any less interesting or helpful to developers of Java apps struggling to live with ORM and SQL and, most importantly, Oracle.



I agree.

I volunteer myself to be giga’s elite geek tech advisor, I’ll do it for free! Email me, no seriously.


You should read vertica’s database blog, called the ‘database column’ it is a GREAT DB read, very down to the details.


well, the team is interesting (ari and amit in particular), but i don’t see any real dbase genius creating something new here – more of the same old same old, in a field in which the technology itself is around 30 years old…what i find much more interesting is what cloudera is doing (worth a look) and also what greenplum and vertica are doing, that feels a bit more fresh and new – but this kind of approach to data store is not novel, not the way terracotta seems to be doing it…


This seems to be just re-inventing the old object database technology (ODBMS) pioneered by Versant, ObjectStore, et al in the early 1990s. Look up ‘pointer swizzling’ etc for how they managed to enable this without any application code changes. Similar ideas are also implemented in various forms by Java persistence libraries. Congratulations on them for managing to make it sound new there – triumph of marketing rather than technology though.

The basic issue with ODBMSs of all types is that they work great for single applications, hierarchical data, OO data, etc, but aren’t so good for more business-oriented flat table data that must be shared between multiple apps. In some cases, a broken app can even corrupt the DB because the DB is really just a persistent version of in memory data strucutures. I’m generalising a lot here, but there’s a reason why people buy object-relational mapping tools – once the data is in boring old relational format it’s a lot easier to ensure its integrity and share it among multiple apps.

I really think GigaOM needs some savvy technical people to review some of these articles – the other one recently where Google’s Native Client was described as a “scripting language” was so technically inaccurate that it’s really a bit embarrassing. I like GigaOM a lot, but the lack of technical background / insight is worrying sometimes.

Stacey Higginbotham

Zubin, you’re right, I should have mentioned Coherence. Thanks for pointing that out.

Nati Shalom

There seem to be different claims of that sort in the media in the past few weeks.
I actually replied to one of them: Kill Your Database with Terracotta by Tushar Khairnar’s. I believe that the same comment applies here: (Quoting from this reply)

There are few points that one should consider with this pattern:

1. What do you do in case you already have an existing database, how do you integrate existing system with this model? You would find that this is probably one of the more common scenarios. In this case you will also find that mapping data from its existing format to the In-Memory representation is required. Note the use of external datasource as a plug-in allows you to plug-in different database implementation and mapping styles. The common one would be hibernate plug-in for existing database and black-box plug-in in case your just looking for durability and you don’t care how the data is being stored. We found that this level of flexibility is required as in many cases the database serves also as an integration point with other applications and not just as a persistent storage.

2. What happens in case of a failure i.e. when the database server is down for 10 hours? Can the application continue to run?

3. How do you manage the transaction scope? If transaction is blocked until the database is synched you may loose large part of the performance benefit of working in-memory for read/write scenarios.

This has been some of the assumptions behind the design of our reliable async replication and Mirror service which runs today and already being used in most mission critical application in the trading area.

You can see more details on this pattern in some of my earlier posts here and here

silicon valley dropout

database is big business for oracle. i dont see anyone eating their lunches anytime soon.

Zubin Wadia


Oracle has Coherence, which is a distributed cache and designed to act as the drawer to their closets as well. So, Oracle covered their bases here pretty well. Coherence was originally offered by Tangosol, which was acquired by Oracle in 2Q 2007.

Comments are closed.