Blog Post

Roll your own Google Search

Stay on Top of Enterprise Technology Trends

Get updates impacting your industry from our GigaOm Research Community
Join the Community!

Not a day passes by when someone or the other bemoans the fact that they cannot find anything on Google anymore. Well, they can stop complaining, because Google is doing something about it. The company has announced Google Custom Search tools, which allow anyone to simply roll their own vertical search engine. Now think of Google Custom Search as Rollyo, on a Google scale.
The new tool allows you to pre-define sources whose content you want to search for, and it also gives you tools to keep adding sites in the future. Since the search can be hosted on either Google servers or your own, you can actually customize the look and feel of the search page, and embed Google Adsense and other advertisements. (Google has been offering something similar as part of its Google Mini effort, but that is more enterprise focused.)

It is a clever idea on Google’s part. By getting folks to build their own vertical search engines, the company is trying to blunt the efforts of some of the VC funded vertical search engines. It is also using “people’s power” to fine tune their own search index. My inner cynic thinks this is – distributed search optimization effort.

However, the problem is that they are not giving any real incentive for people to do that. The share of Adsense bounty is just the same as on a plain vanilla site. It should increase the payouts to the search builders. They are getting more focused search results (hence higher click throughs for their ad), so why not share the profits with folks who are doing the heavy lifting.

The emphasis here should be on getting more and more people to build these specialized micro search engines. Mike Arrington is not going to waste his time for peanuts, but make it a good deal, and I bet he can build what could be the most accurate Web 2.0 only search engine.

Hey if they can buy YouTube for $1.65 billion, how about spending some moolah for the ‘search builders.’

Update: Search Engine Watch has an interesting take on this.

32 Responses to “Roll your own Google Search”

  1. Hi,

    well creating a cse by coop is easy.
    Finding these cse is the hard part.

    We created the first directory for cse created by google co-op or websites that have a google coop cse intergrated.

    Its free of course, and it would really help if you could submit your cse there.

    You can find it at

  2. A bit off context but any particular reason why there is a sudden or rather a gradual increase in the number of comments on this particular blog from people of Indian origin compared to other tech blogs? Any major coverage of late in Indian media about

  3. What will happen to those vertical search engines having invested millions in building their own solutions!!! I think some of them are doomed unless they have value added service in addition to the vertical search!!

    For me now the real Vertical Search Engine war has begun. Also I think with this, Google is combining social search with vertical flavour and allowing community to mushroom around it.

    Google has played the check step and launching the community soldier to play checkmate step. :)

    More comments in my blog :

  4. Om…you said that “It is a clever idea on Google’s part”, what you shouldh ave rather said that this is a clever “copied” idea.

    Microsoft’s Live search had this feature called ‘Macros’ much earlier and the UI look and feel is much better than Google’s

    Seems that these days Google doesnt seem to do much innovation after all. Asa respected blogger you should mention the fact that competing search engines have this feature and this is not something Google thought of for the benefit of its users.

  5. Hey not sure if that is true webtrucker. remember my google finance post – or many others. i am just saying that money (lots of it) with custom search is a good idea. custom search with piddly little adsense money is a big yawn. i guess i need to be more direct when i say that ;-)

  6. WebTrucker

    ah, another smart google move: let our users do the work and we enjoy the nice stock price ride right up to the split level :)

    true that it’s a smart move, but it shows how outdated the companies search technology as they’ve invested in building the google empire … and not so smart for the users.

    Om, you seem to think whatever google does is smart … you need to take your googles off every once in a while ;)

  7. Sangeeta

    Alexa Web Search Vs Google Custom Search Engine

    With Alexa you pay $.30 for 1000 request, Google is free
    Google has this “community” twist, its missing from Alexa’s offering.
    Alexa’s offering require more programming talent.

    How hard it would be to put together a Custom search using Google, nicely integrated with Alexa’s website thumbnail service.

  8. Search is still the main revenue source for google and this feature definitely makes their position solid in the search market.

    More appropriate name for this feature would be ‘google everywhere’!

  9. Om, thanks for the coverage. I’ve given Google Custom Search a whirl and am using it to set up a new and hopefully spam free index at

    You’re on the list in the index and I’m accepting volunteers who want to use Co-op to contribute to my index and/or submit their own blog.

    It took about 20 minutes to set up the new Google Custom powered search at BSE and given the power I have over the index and its contributors, I’m quite happy.

    Now just need to build it :)

  10. This seems like an obvious development to me and I’m not sure why they’ve just now come out with it. I’d think someone would have dropped this little app out in a single day using their 20% time for personal projects. I still think what’s missing in search is a viable social search function that integrates authority and expertise to someone’s rating ( Why should a 12-year old’s opinion of a website on stem cell research carry the same weight as a senior geneticist at the Mayo Clinic?

  11. Upon checking this out, three features stood out:
    1) You can exclude sites that you do not want in the results
    2) You can easily do so using the Google Marker
    3) Anyone can volunteer to help

    So we decided to throw up an experiment to encourage everyone to mark spam sites to be excluded from search results.

    Working together as a community we may be able to radically improve the quality of the search results (or perhaps just get in a blacklisting war?)

    The result is Putch –

  12. For a ‘naive’ Google user, I don’t really see the new customn search engine being a great pull. It’s one more thing for the user to do, after all. Personal Search/search over your search history is a different matter, where your search history inform your future search results. (After all, if i have to visit a site to bookmark it, why not let google just remember that I clicked through to it previously.)

    searchfeedr (website link above) offers a different approach, whereby you can reuse other link collections to provide either a page or domain limited search. FOr example, you can limit your search to domains listed in an RSS feed, or tagged in a particular way on a delicious user’s account; or even use the links contained within – or that link to – a particular page.

    Like Rollyo, the number of domains you can link to are limited – but how many people look past the first page of results anyway?

    Another advantage fo searchfeedr is that it is a search engine intermediary – page/domain limited searches can be made on one of any number of search engines (at the moment, you can choose to pull results from Google, Yahoo, Windows live or altavista).

    Add to this the abilty to save search profiles to a delicious account, from where saved searches can be pulled back in to you searchfeedr page, and the new Google offering seems – well – a bit dull really ;-)


  13. Om,

    This is just letting the the folk get at the metadata build (in the old days – c 2004 – it was called a Folksonomy ;), or “McTagging”

    I agree with your analysis re Google though…this service raises a “what value do they add” question, compared to using something like say Dogpile, and then configuring a relevance vertical on top of that…and then who gets to show the ads?

  14. Michael, isn’t that exactly what I said. They need to sweeten the pot in order to make this work, otherwise not going to happen. Of course, the issue is bigger than that – the search is getting clouded and that’s not good for them.

  15. Michael Griffiths

    Ok, so it’s news.

    I’d prefer to see an analysis comparing the tool – or its goals – to identical offerings from Yahoo! (Searchbuilder) or Microsoft (Search Macros). What does the fact that Google is introducing what sounds like a less developed feature set after its two main competitors say about Google? etc and so on.

    Otherwise, you’re just copying and pasting.