Blog Post

Will Yahoo Use Xoopit's CloudQuery to Help Usher in the Real-time Web?

Stay on Top of Enterprise Technology Trends

Get updates impacting your industry from our GigaOm Research Community
Join the Community!

xoopit_logo_c1While Xoopit’s flagship email product is being touted as the reason why Yahoo (s yhoo) agreed to buy the company, I think the underlying technology used by Xoopit is far more interesting. Xoopit has even exposed this infrastructure technology as an on-demand service, which it’s dubbed CloudQuery, with the tagline “Search as a Service.” But I suspect that, once the acquisition closes, CloudQuery will get buried deep in the bowels of Yahoo.

Essentially Xoopit’s CloudQuery allows you to search-enable any application. It combines the functionality of full text search with more structured database-like queries, which have the advantage of being aware of the underlying data structure of the application and thus provide more accurate search results and in near real-time. It is very similar to the open-source project Apache Solr; both are based on the popular Apache Lucene.

Suppose I start a dating site that I want to equip with a sophisticated search engine. I have certain well-defined data fields, which I want to be searched as such (and not with a Google-style free text search); in other words I want people to be able to search for “gender is female,” “age is greater than 30 and less than 40” and “location is less than 5 miles from Zip 94107.” But I also want to enable a full text search that says “long walks on the beach.” With the Xoopit CloudQuery service I can quickly index my application and enable these sophisticated search capabilities, something I can’t do with Google ( s goog) Custom Search, for example.

Because of their near-instant indexing capabilities, technologies such as CloudQuery and Solr will be key in implementing the notion of the real-time web that has been popularized by services such as Twitter. Not only will I be able to perform the above query, but I will be able to set a real-time alert for it. To see a very simple but interesting use of this kind of technology, check out Dave Rosenberg’s “A-Z List of Underrated Bands,” which he created using Twitter Search. If you click on the link you’ll see that the search query at the top says: “underrated bands” from:daveofdoom. This query essentially says “find all tweets that contain the words ‘underrated bands’ posted by the user ‘daveofdoom’.” if you were to leave that page open while Dave was adding to the list, you would have seen it updated on-the-fly — hence, the real-time web.

While Twitter has developed its own solution with significant development resources, Xoopit’s CloudQuery enables any application owner to very easily index and search-enable their application using its hosted cloud service and APIs. In addition to CloudQuery and Solr, other projects that are attempting to provide similar capabilities include the Ferret project for Ruby apps and Sphinx Search. As another indication of the level of interest in this area, Lucid Imagination, the company behind Solr (and Lucene support provider) recently received more than $6 million in funding from Granite Ventures, Walden International and In-Q-Tel, the CIA’s venture fund. But Xoopit remains the only player that has implemented this type of technology as a cloud service.

Yet it appears Yahoo is acquiring Xoopit as part of its strategy of focusing on a handful of core apps — among them email — and turn its email application into a platform, around which it hopes developers will create an ecosystem of complementary apps and plug-ins, utilizing the APIs that Yahoo has published.

But as happens so often in this business, a lost opportunity for one company could prove to be a new opportunity for another.

Geva Perry, who writes the Thinking Out Cloud blog, is an adviser to startups and enterprises on cloud computing strategy and marketing. Follow him on Twitter: @gevaperry.

18 Responses to “Will Yahoo Use Xoopit's CloudQuery to Help Usher in the Real-time Web?”

  1. esconsult1

    Xoopit is so far from real-time its ridiculous.

    This article is written on the premise that the author knows about Xoopit, I’m sure they’re trying to be as real-time as possible, but I remember setting up Xoopit and it taking days and days to index my emails.

    Solr is also NOT a real-time search engine. Its a very nice piece of technology, but content has to be indexed just like anything else.

    “…if you were to leave that page open while Dave was adding to the list, you would have seen it updated on-the-fly…”

    Again, that indicates a fundamental misunderstanding of whats going on under the scenes… As Dave adds something to his timeline, it gets added to Twitters search index in a pretty short time. — AJAX requeries the database on the search page every few seconds for new searches. Thus making it “seem” as if its real time.

    My point still stands from yesterday. At its core, this is just a way for Yahoo to be lazy. This should be just an exercise in building a database to capture the meta-information from each email (attachment filenames and types, addresses, keywords). All this information Yahoo already has sitting doing nothing with. Instead they buy a product that does it the long and hard way.

    I’m all for startups getting bought out, but this one takes the cake. This one is a waste of money.

    • Real time means the minute it is indexed the results are accessible in the very next search query. might help you undestand it a bit more.

      I think you can appreciate there is a lot in indexing an inbox that makes the initial indexing slow like fetching huge amounts of data and extracting the unstructured info.

    • twitter is a real time search engine only because their data is with them
      so for that matter the search engine on gigaom is realtime too since the content that is published is searchable at that very instant

      i do agree that yahoo could have built this
      but sometimes companies are bought because
      of teams , product roadmaps , IP or just to keep them out of competitors hands which is actually petty but makes biz sense

      if yahoo had built this then what ? do they sit around and wait for xoopit to release their next release so that they can copy it ? building everything in house might look cheaper but in the long run sometimes tend to be expensive

      xoopit was backed by VC’s so they must be having a decent roadmaps since the VC’s opened up their wallets to them so i guess yahoo also liked what they saw or think this team can deliver so they picked them up

  2. its stuff like this that makes me come to gigaom daily :)
    keep it up y cant other blogs not do analysis of news and give its readers information tht they can
    consume and make sense of but are merely interested in turning their blogs into fancy PR site and breaking
    news site with no details of the deals :(
    like every other major blog has covered this news but no one has even bothered to understand why yahoo
    might be interested in xoopit they only covered the speculated price and product details as if we didnt know what xoopit was :(
    I just hope and hope that gigaom does not turn into one of this other blogs but remain true to the art of
    making sense of what is happening in the tech scene :)

    • Fritz

      Thanks for the nice comments and we will try to live up to your expectations. I deeply appreciate that you are keeping us honest.

      The kudos on this post however go to Geva, who penned the article.


      • Hi om

        thnx for the reply :)

        yep kudos to Geva for the post
        but ur entire team is gd I have never had any issues with the quality of the stuff i read at gigaom daily but have a big complaint abt the quantity :( it might be too much to ask considering a great deal of research that goes into coming up with something substantial tht fans of gigaom will like :)

        oh and since we are being honest here the other thing i like abt gigaom is their design which is more focused towards their users :)

      • Hi Geva

        yep loved the post looking forward to lots more good posts
        its stuff like this that keeps me awake reading blogs till around 4 am in india ;)

    • Oh yes you absolutely right, every day try to find some time to read GigaOM :)
      About this yahoo step, looks like this company is rising from the dusts, starting to be more compepetive to google.
      All main IT companies started to develop something to compete with google, nice to see that. Because for ens users (like me) coming more and more easier to do everyday and future tasks.
      What new thing are going to show GigaOM ?

      • Hey Tomas

        atleast from the past acquisitions of both yahoo and some of the other bigger companies what i have learnt by reading and heard from other sources is that companies do still like yahoo
        not only because of their culture but also because their acquisitions are never to destroy great products like how google does dodgeball ,jaiku etc but they do tend to absorb their new targets into the yahoo core DNA somehow and that is what makes the team behind the product happy believe me no one wants to see their product die altogether but they do like to see it live on in one way or another

        yahoo mail is still big and xoopit is good product that can be assimilated into their mail product DNA yahoo does not have to innovate but if only they just stay relevant for their users they will do well but they do have some innovative stuff happening recently which is gd but i personally feel that yahoo microsoft marriage is good in terms of building a worthy competitor to challenge the mountainview giant who stands tall unchallenged so far and slowly spreading its tentacles in every online aspect of its users life. yahoo with a cash rich partner is the only way anyone can seriously hamper googles march to dominance.