The first day of Google I/O seemed like a coming out party for Google App Engine, the company’s competitive threat to Amazon AWS. For one, the registrations were thrown open to everyone, and for another, two new APIs were released: the image manipulation API, and (more interesting to web app hosting in general), the memcache API. Now the memcache API was something I expected from Amazon a long time ago, but perhaps they don’t use it themselves as much so it’s not in AWS.

With Yahoo in limbo and Microsoft missing in action on the Internet, Google is making a huge play for developer mindshare. As Microsoft and Sun both demonstrated very effectively, focusing on getting developers excited and making them happy is the key to the success of a platform. Google I/O appears to be Google’s big play for developers. And so far it seems to be working.

Google App Engine (GAE) comes with a webapp framework that’s derived from Django, but you can host your own, including CherryPy, Pylons and web.py, all of which are Python-based. No other language is planned at this time. C++ had AT&T, Java has Sun, and Python now appears to have Google behind it, so expect a lot more Python development activity in the global coding trenches. And a lot more Python books being sold.

The big difference between GAE and Amazon AWS seems to be that GAE commoditizes the application hosting layer while AWS commoditizes the hardware and network hosting layer. With GAE you don’t get to choose the web application stack. You provide the UI and the logic; Google provides the scalable datastore and the application hosting and analytics.

There are a number of characteristics about GAE that serve to give me flashbacks. First is the fact that all apps are hosted as CGI apps. I’m sure Google has a reason for this, but it seems so “early Internet.” Then there’s the fact that Google has created its own query language, GQL (pronounced JeeQuel). Facebook has FQL (how is that pronounced, anyway?), so what’s next, YQL? It’s like we’ve reverted back to the late 80s, when all the database companies mangled the SQL standard just enough so that data was bound to their databases in strategic lock-in. This story doesn’t end well for users.

Finally, the GAE Datastore appears to have a native hierarchical structure with parent-child relationships between entities exposed to the programmer in GQL. This harkens back to the hierarchical databases that preceded SQL and relational databases. The power of SQL was supposed to be that it was declarative, that you didn’t have to know how the data structures were implemented. But hierarchical database application code was viewed as impossible to maintain because your data model leaked into your application. The current iteration of the GAE Datastore seems to require a lot of premeditated syntax design on the part of the developer. It reminds me of how queries performed differently in Oracle depending on the order in which columns appeared in the query. I hope, however, that this is a passing phase and that we soon see a better abstraction in GAE.

In contrast, SimpleDB and CouchDB focus on tuples and dispense with the SQL baggage; they’re surprisingly forward-looking compared to the data models of Facebook and Google. While I’m not questioning whether or not the Google Datastore will scale as promised, some of the choices in how these facilities have been exposed to programmers are curious and have rough edges.

A comment about Ruby: While Steve Yegge’s recent article seemed to suggest that it was hard to promote new languages within Google, I spotted some signs at I/O that Ruby might not be shut out of the picture. OK, just two signs. For one, there is a talk scheduled that mentions Ruby in the title. Second, at one of the talks the speaker mentioned a device called “Radish,” described as a 20-percent time project yielding a device that operates on indoor solar energy and is used to update/monitor the meeting room schedules wirelessly. Apparently the data pushed through this device is managed by a Ruby app — yes, there’s a Ruby app running inside Google. He didn’t say “Rails,” just “Ruby,” so please don’t scream all at once, OK?

Other than that, the conference was Google Gears, HTML5, lots of Javascript/AJAX and of course, Android, Android, Android. There’s even a company selling a 12-hour crash course in Android to prepare developers for the October release. Today, who knows? How Android Google Gears app downloads the Internet and beams it to your desktop via Wi-Fu tubes?

Twitter This Article   Facebook This Article   Email This Article

Europe Struggles With Three Strikes, Fox Exec Wants It Anyway NewTeeVee – by Janko Roettgers "; Desert Bus For Hope Rides Again for the Third Year NewTeeVee – by Liz Shannon Miller "; Smartbook No More — What Should We Call Them? jkOnTheRun – by James Kendrick "; Dealing With the Avalanches In Life and Business WebWorkerDaily – by Amber Riviere ";

Posted by Nitin Borwankar on GigaOM  |  Permalink  |  3 Comments
Tags: google app engine, google i/o

The recent flood of activity around data portability has stirred in me two reactions: keen interest and a distinct feeling of being underwhelmed. Having been in the data business since the early 90s, it’s clear to me that data portability is a non-solution to a non-problem, a storm in a teacup, an emperor with no clothes.

The real problem — and the elephant in the room – is not whether web app vendors “allow” me to take my data and go play elsewhere, but whether they “play fair” with my data when it’s in the web app. There are basically two choices for a web app user here — one involves having a “voice,” the other, making an “exit.” Data portability focuses only on the “exit” part, which is not only incomplete but for users of the web app, massively disempowering.

Consider a scenario in which a single web app dominates a market segment, such as YouTube in video sharing, or Flickr in photo sharing. In those cases, data portability enables me to take my data and go to another non-dominant web app with a much smaller playground, far away, where my friends don’t play. I most emphatically do not want to do that and I certainly don’t want that to be my only recourse if I find the web app vendors’ policies to be coercive. Data portability actively enables the dominant player to say “If you don’t like our policies, then leave.” But I want web app vendors to behave, and to guarantee certain data property rights as terms-of-service guarantees. Because if data portability is about “exit,” data property rights are about “voice.”

Call me a loudmouth, but I’d rather have a “voice” than an “exit.”

Current discussions about data portability tend to be rather narrowly focused on a small subset of the problem — namely, the ability to transport a social graph from one social networking site to another. But a large amount of personal data on the Net is not social graph data and a large number of people on the Net may be concerned about data issues not directly connected to social graphs. Additionally, taking a social graph from one social networking site to another may be neither practical nor desirable.

I’d like to suggest an alternate approach, one that supports strong TOS guarantees from web app vendors and includes terms that guarantee four atomic data property rights. (I call these rights “dimensions” due to their orthogonality and fundamental-ness.) And I posit that with these four as TOS guarantees, data portability pretty much becomes a non-issue:

1. Data Accessability: The ability to address each element of my data with a URI when it is in the web app. Zero black-box behavior, data addressable in place. This allows me to upload or create data in a web app and have full, in-place programmatic access to each element of it; it also allows me to mashup at the data level and not have to hack the UI via Javascript shenanigans such as Greasemonkey. Property analog: If I own property I may not be denied access to it.

2. Data Visibility: The ability of the user to control the visibility of the data in their account, so that the web app vendor does not set the default visibility policy, the user does. The default, if at all set by the vendor, must be “private” –- only the user can change it, and user settings always dominate. This prevents web app vendors from exercising intrusive and coercive control over user-generated data. Property analog: If I own property I and only I control who has access to it (i.e. modulo lawful search).

3. Data Removal: On account termination by the user, or by the vendor, all data in the user account must be deleted (possibly after a waiting period). If data is ever restored from backups, accounts terminated must not be reactivated and their data must remain unavailable to business processes, such as targeted advertising, etc. This provides a degree of data privacy and non-intrusiveness. Property analog: If I lease property and the lease terminates then the lessee has no residual rights on my property.

4. Data Ownership: Data created/uploaded by the user belongs to the user, and only to the user; it is not automatically “co- owned by the web app vendor with all rights given in perpetuity,” as is so prevalent today. Property analog: If I own property then no one else, as a side effect of some other contract, can gain automatic co-ownership of my property without my explicit consent.

I’d rather have strong TOS guarantees of the above four dimensions than data portability.

It may be argued that with these four dimensions supported in a web app, a vendor will find it impossible to monetize the available user data via intrusive advertising, reporting and other mechanisms. This is a reasonable argument. However, a web app with the above service guarantees the vendor is providing a valuable service and it is perfectly reasonable to expect the vendor to charge for such a service, so that major revenue comes from a user-in-charge, non-intrusive, for-fee model rather than the usual “intrusive data policies + advertising” model.

Incorporating data property rights into the current conversation completes the picture by adding the web app user’s “voice.” This empowers web apps users and it also seeds new, viable business models. For-fee services providing strong user rights without a coercive advertising model will emerge and form a new “data infrastructure” layer of the Internet operating system. If the dominant players do not want to satisfy this need, market forces amplified by user emotion will disrupt them and we will see once again how the Net routes around damage –- in this case badly damaged data property rights.

Nitin Borwankar blogs on tagschema.

Twitter This Article   Facebook This Article   Email This Article

Europe Struggles With Three Strikes, Fox Exec Wants It Anyway NewTeeVee – by Janko Roettgers "; Desert Bus For Hope Rides Again for the Third Year NewTeeVee – by Liz Shannon Miller "; Smartbook No More — What Should We Call Them? jkOnTheRun – by James Kendrick "; Dealing With the Avalanches In Life and Business WebWorkerDaily – by Amber Riviere ";

Posted by Nitin Borwankar on GigaOM  |  Permalink  |  31 Comments

Amazon continues to amaze us with its Amazon Web Services series of offerings. The latest is SimpleDB, which will be available in limited beta in a few weeks. And it is bound to have a major impact on web infrastructure. As Amazon says in its email to existing developers:

This service works in close conjunction with Amazon Simple Storage Service (Amazon S3) and Amazon Elastic Compute Cloud (Amazon EC2), collectively providing the ability to store, process and query data sets in the cloud.

As we’ve already noted,

…the center of gravity is shifting away from monolithic centralized data management to massively parallel distributed data management.

If you are in the business of managing massive amounts of distributed data, you cannot gloss over the Amazon WS trifecta — data-in-the-cloud is the future and with WS, Amazon is way ahead of the pack.What about the offerings of other vendors? Google, for example, has BigTable, and truth be told, SimpleDB has a distinctly BigTable-ish feel to it. But a side-by-side comparison makes it clear that Amazon WS in general – and SimpleDB in particular — is superior, for the following reasons:

• Google’s offerings – not only BigTable but GoogleBase, Gdisk, etc. — all have an ad hoc, grab-bag-of-tools feeling to them, devoid of any integrated strategy. Or if there is one, it is well-hidden.
• Amazon WS clearly involves a well-designed master plan aimed at changing the face of software as a service, each new offering akin to a chess piece in a game focused on creating strategic long-term value. And with SimpleDB, the queen has moved to the center.
• Amazon WS is based on the YOYODA principle — You Own Your Own Data, Always. Along with Amazon S3, SimpleDB is a sharp arrow in the quiver of open data proponents.
• Amazon WS includes a built-in, flexible payment system so users are neither forced to offer their app for free nor have an “ad-supported” model forced upon them. Now you can build a data-based web app on SimpleDB and seamlessly charge for it.

Tersely put, SimpleDB is hugely disruptive. It will take some time to evolve the new thinking patterns and new design disciplines that this technology forces us to consider. To do so, consider this breakdown of the similarities and differences between SimpleDB and conventional relational databases.

Very, very simplistically speaking, domains are like tables, with items like rows and attributes like columns. A query cannot cross domains, so in this analogy you can’t “join” domains. But that sort of thinking is a holdover from the relational database normalized model.In reality a domain is much more like a database, so we have to stop thinking in terms of tables and joins.

Say we had an SQL database, with tables for “Company,” “Departments” and “Employees.” In SimpleDB, the items (rows) for all three could all go in one domain (database), with it you can run queries on this domain and using operators like UNION and INTERSECT, you can do the equivalent of joins.Existing web technologies such as Ruby on Rails, Django and Hibernate all have an Object Relational Mapper (ORM), which maps language objects to relational database tables.

If designers of these ORMs want to stay in the scalable apps game, they should take a serious look at using SimpleDB as a data store. Better yet, they should build ORMs from the ground up to integrate with SimpleDB.More than two years ago I wrote that Web 2.0 needs Data 2.0. The combination of EC2, S3 and SimpleDB is a toolkit for assembling massively scalable REST addressable web databases. Data 2.0 is now officially here. May the fun and games begin.

Nitin Borwankar is a database guru based in San Francisco Bay Area. You can find his writings on hisblog, TagSchema.

Twitter This Article   Facebook This Article   Email This Article

Europe Struggles With Three Strikes, Fox Exec Wants It Anyway NewTeeVee – by Janko Roettgers "; Desert Bus For Hope Rides Again for the Third Year NewTeeVee – by Liz Shannon Miller "; Smartbook No More — What Should We Call Them? jkOnTheRun – by James Kendrick "; Dealing With the Avalanches In Life and Business WebWorkerDaily – by Amber Riviere ";

Posted by Nitin Borwankar on GigaOM  |  Permalink  |  55 Comments
Tags: Amazon, AWS, google, SimpleDB