Most sites will never hit that painpoint in scaling, but also, plenty of sites do. And many sites don’t need the relational stuff. So it’s a trade-off.

JD

Hi SimpleDB users,

I am an oldtimer in databases, who happened to come across the SimpleDB pages, and so out of curiosity started to read about it. It looked good … at first, then I realized some implications of the simplicity, then I looked at some of the threads here and elsewhere dealing with the relationship to relational databases and how to solve more complicated database problems. Then I realized how history repeats itself and how 38 years of accumulated wisdom of database modelling and design have been disregarded or overlooked or misunderstood or neglected.

In the summer of 1975 I read the first edition of Chris Date’s text book on database technology, then I studied E.F. Codd’s papers on the relational model, starting in June 1970, for which he was given the Turing award in 1979. These two guys are behind the tremendous success of relational database technology. Date’s book in many editions, together with other text books on database technology and various forms of The Entity-Relationship Model, originally created by Peter Pin-Shan Chen in 1976, has educated countless computer science students in database design. All computer science students, all developers, all programmers should get familiar with relational database design, it is simple, it is powerful, it can be done in a one semester course, and then you can disregard it with open eyes. It is a scandal, if you were not taught relational database design alongside basic programming skills.

What is it that you do with SimpleDB? You put your application logic into procedural code, where you query each domain and combine the resulting items by coding loops and comparisons and what have you; this is how you implement the equivalent of joins in SQL queries; this is the old procedural versus non-procedural debate, where your procedural version is much, much costlier to maintain, and much, much harder for others to understand, you bury the application logic in tons of hopeless code. If you want to avoid joins, you probably put everything into one domain, or as few domains as possible; if you know a bit of normalization of database design, you know about the anomalies that you have introduced, costly. The claimed advantage of SimpleDB that you may have more than one value in the same field is one aspect of an unnormalized database, therefore harmful. And by the way, it is not true that union and intersect can do joins for you.

Once we had semantic nets and logic databases, they included the attribute-value pairs of SimpleDB, and did not require schemas or predefined fields, and in addition they provided powerful non-procedural queries. But where are they today? Maybe in student projects, maybe in research projects, but not in serious, important applications.

If I were paying your salary, I would never allow you to implement my important applications in SimpleDB or the like. You should instead use MySQL or another free, powerful, yet simple to use, RDBMS. Use it with EC2 and S3, that is fine.

You have other options, though, XML databases in the first place, where you also have non-procedural queries available. As a database developer, you should be able to judge, when and how to use XML databases for a given purpose. If your application needs full-text indexing or integrated storage of all types of files and documents, then take a look at things like the object-based, web service-based Fedora repository system, it has RDF triple storage with a non-procedural query language also.

In conclusion, use SimpleDB only if your application’s database needs amount to one normalized relational table, else invest your time in today’s powerful technologies.

PS! What if SimpleDB implemented joins, that is, “field1″ = “field2″? That would be a great step forward. Then you would need indexing behind the scene, so that performance of joins could be as good as in RDBMS.