However, the event left me with more question than answers. While Danny Sullivan’s overview answers many of those questions, I still wasn’t clear as to how it worked.

So, how does this Unicorn based Graph Search work? From what I understand, there are two parts to the search, both built in-house. The first part is natural language processing based, which is essentially driven by the users’ questions. The second part is the one that brings back the answers.

The second part is built on an internal retrieval tool called Unicorn that has been in use at Facebook for a while. Facebook has created an index of all objects on the social platform. On the Facebook Engineering blog, Lars Rasmussen (who along with Tom Stocky) was responsible for building Graph Search writes:

Using traditional information-retrieval systems to mix keyword and structured queries is fairly well understood. But we needed the system also to find answers more than a single connection away, such as “restaurants liked by my friends from India.” Here we were in luck: one of our three existing systems, Unicorn, was designed exactly with this in mind.

The search infra team decided on a two-stage approach: first build out Unicorn to manage all the existing search experiences on the site and then, build out Unicorn further to meet all the requirements of Graph Search. Today, we are far enough along now to launch Graph Search as a beta, but we’re still missing is the ability to index all of the posts and comments people have shared on Facebook–they make up by far the biggest dataset we have for Graph Search and Unicorn.

At a very rudimentary level, Rasmussen in a call later explained that when we type a query, say, “Restaurants liked by my friends from India in San Francisco,”  an aggregator tool using natural language processing takes the query and parses it into keywords against which searches are conducted. So “Restaurants,” “San Francisco,” and “Friends from India” become queries whose results are fetched and further sorted to give us the final answer. It is not new, except the scale of data on which such queries are being run is possibly a new (and rising) high watermark.

Rasmussen acknowledged that there are a lot of unknowns for the company and it is still not clear what kind of computing resources are going to be required for Graph Search at scale. Facebook plans to learn from phased beta rollout and do compute-resource planning based on that, he said.

My take: This is going to be a big infrastructure challenge, considering Lars and his cohorts want to keep the latency to below two seconds. Rasmussen pointed out that the “ranking algorithm” for results is going to keep improving as more and more people use the search. With as many as a billion searches on Facebook every day, even few million queries are going to be enough to help fine tune this ranking algorithm.

As for rest of my questions — I guess I will have to wait for the company to share details “about this challenge on the engineering blog soon,” as Facebook said in its blog.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• How search can unlock the power of big data
• Controversy, courtrooms and the cloud in Q1
• A near-term outlook for big data

Google has made it a habit of making news for the wrong reasons! Days after it reported a blockbuster quarter, the company’s chief executive made some childish remarks about privacy.

And, in past week or so, the company saw three well-known executives leave the company. First it was Chad Hurley, co-founder of YouTube, who decided it was time to hang up his CEO spurs. (Hurley is staying on as a special adviser.) Then came news that AdMob co-founder Omar Hamoui was leaving. Then, over the weekend, news spread that Lars Rasmussen, who was one of the co-creators of Google Maps and Google Wave, is leaving.

Rasmussen is trading Google for Facebook. He apparently likes the new gig so much he would give up Sydney and move to the San Francisco Bay Area. In an interview with the Sydney Morning Herald, Rasmussen said:

It feels to me that Facebook may be a sort of once-in-a-decade type of company. The energy there is just amazing, whereas it can be very challenging to be working in a company the size of Google. (SMH)

According to some estimates, one in five Facebook employees have ties back to Google, including COO Sheryl Sandberg and CTO Bret Taylor. Google has over 23,331 employees according to company’s recent filings with the Securities and Exchange Commission. Facebook has about 2,000 employees.

Rasmussen explained in his interview that getting things done (GTD) was a big problem at Google, and shutting down his project, Google Wave, a year after it was made available as a beta is a sign it was becoming difficult to get things done at the search giant.

We were not quite the success that Google was hoping for, and trying to persuade them not to pull the plug and ultimately failing was obviously a little stressful. It takes a while for something new and different to find its footing and I think Google was just not patient. (SMH)

He’s not alone. Many former members of AdMob have privately expressed frustration at their inability to get things done at Google. Others who have quit the search giant have expressed similar sentiments. Sometimes, money and perks aren’t enough to retain talent. If today it’s Facebook, then tomorrow it will be yet another hot startup that will keep fishing in Google’s talent pool and find eager biters.

I don’t think rivals (including upstarts) have the ability to stop Google’s financial steamroller. It will continue to be a dominant force in search and online advertising for years to come. But if it doesn’t reign in its talent problem, the company will have a long-term crisis on its hands. It was exodus of talent and inability to get things done that has brought giants of the past — Yahoo for example — to their knees.

For the longest time, Google has been the beacon for the smartest and most talented people in the world, especially from an engineering perspective. If these super-smart people start getting frustrated by their inability to get anything done, they’re going to follow Rasmussen to somewhere they can find a more receptive and nurturing environment.

In the end, that’s the single biggest problem for Google.

Related content from GigaOM Pro (subscription req’d) about Google:

• Why Google Should Fear the Social Web
• Report: Google’s Voice Possibilities.
  
• How Mobile Cloud Computing Will Change Tech