While Xoopit’s flagship email product is being touted as the reason why Yahoo agreed to buy the company, I think the underlying technology used by Xoopit is far more interesting. Xoopit has even exposed this infrastructure technology as an on-demand service, which it’s dubbed CloudQuery, with the tagline “Search as a Service.” But I suspect that, once the acquisition closes, CloudQuery will get buried deep in the bowels of Yahoo.
Essentially Xoopit’s CloudQuery allows you to search-enable any application. It combines the functionality of full text search with more structured database-like queries, which have the advantage of being aware of the underlying data structure of the application and thus provide more accurate search results and in near real-time. It is very similar to the open-source project Apache Solr; both are based on the popular Apache Lucene.
Suppose I start a dating site that I want to equip with a sophisticated search engine. I have certain well-defined data fields, which I want to be searched as such (and not with a Google-style free text search); in other words I want people to be able to search for “gender is female,” “age is greater than 30 and less than 40” and “location is less than 5 miles from Zip 94107.” But I also want to enable a full text search that says “long walks on the beach.” With the Xoopit CloudQuery service I can quickly index my application and enable these sophisticated search capabilities, something I can’t do with Google ( s goog) Custom Search, for example.
Because of their near-instant indexing capabilities, technologies such as CloudQuery and Solr will be key in implementing the notion of the real-time web that has been popularized by services such as Twitter. Not only will I be able to perform the above query, but I will be able to set a real-time alert for it. To see a very simple but interesting use of this kind of technology, check out Dave Rosenberg’s “A-Z List of Underrated Bands,” which he created using Twitter Search. If you click on the link you’ll see that the search query at the top says: “underrated bands” from:daveofdoom. This query essentially says “find all tweets that contain the words ‘underrated bands’ posted by the user ‘daveofdoom’.” if you were to leave that page open while Dave was adding to the list, you would have seen it updated on-the-fly — hence, the real-time web.
While Twitter has developed its own solution with significant development resources, Xoopit’s CloudQuery enables any application owner to very easily index and search-enable their application using its hosted cloud service and APIs. In addition to CloudQuery and Solr, other projects that are attempting to provide similar capabilities include the Ferret project for Ruby apps and Sphinx Search. As another indication of the level of interest in this area, Lucid Imagination, the company behind Solr (and Lucene support provider) recently received more than $6 million in funding from Granite Ventures, Walden International and In-Q-Tel, the CIA’s venture fund. But Xoopit remains the only player that has implemented this type of technology as a cloud service.
Yet it appears Yahoo is acquiring Xoopit as part of its strategy of focusing on a handful of core apps — among them email — and turn its email application into a platform, around which it hopes developers will create an ecosystem of complementary apps and plug-ins, utilizing the APIs that Yahoo has published.
But as happens so often in this business, a lost opportunity for one company could prove to be a new opportunity for another.