The people behind the Ancestry.com service have realized this. Now they’re making the most of their 4PB storehouse of official personal records, user-submitted information and other data with a new feature delivering sleek computer-generated but customizable summaries of information available on users’ ancestors.

Ancestry started rolling out the feature, known as Story View, earlier this quarter to a tiny share of its customers, and now it’s active for 10 percent of them. The plan is to analyze the use of Ancestry with and without Story View and round out the feature before making it generally available, probably later this year, said Eric Shoup, the company’s executive vice president of product, in a recent interview. Already Ancestry has made the feature more interactive by letting users move around a single page the images of documents and edit the associated bodies of text derived from the documents.

How it works

Story View builds on top of Ancestry’s already highly evolved tools for mining data about relatives, including some handwritten records. But sometimes only critical fields, such as name and place of residence, have been processed for inclusion in Story View. A customer can access a handwritten record, scroll down to the row in which a relative is described and toggle across columns to see data that hasn’t been processed, such as that person’s occupation.

Ancestry is working on getting more out of its handwritten records by gradually directing armies of “keyers” to decrypt handwriting and turn it into searchable text. Street addresses have been added in this way, and other fields will come later. And since Ancestry continues to add records to its repository, life stories will gain more sources to draw from as well.

The Story View life summary for one of Shoup’s relatives

To generate one-paragraph summaries drawing on information from multiple documents (check out the top paragraph in the picture above), Ancestry looked to Narrative Science, a company founded in 2010 to make machines turn out readable copy. Early use cases came in the production of coverage of sports events and public companies’ earnings reports, but now Narrative Science technology is handling much more personal information.

When Ancestry first got involved with Narrative Science, it was only possible to produce data in big batches, said Reed McGrew, lead developer on Ancestry’s narrative and context services team. “They’ll produce huge numbers of financial reports, and that’s not really the experience we’re trying to deliver,” McGrew said. “Because it was meant for batches, it was pretty slow.”

Within a few months, Narrative Science came out with a new API that could work on a more granular level. “On kind of a user-by-user basis, they generate our life stories,” McGrew said.

Ancestry knows a thing or two about serving up genealogy information. The company’s editors provided editorial standards, or “rules,” for how the data should inform the narratives and how the narratives should sound, McGrew said. One Ancestry standard? “We don’t talk about births that happen to mothers less than 10 years old,” he said. “They’re more likely keystroke errors. They do happen in reality sometimes, for sure, but more often than not, when we find them, they’re errors.”

One of several records containing information on a relative of Shoup’s

Underneath a picture and life summary of an ancestor in Story View are zoomed-out pictures of documents, instead of discrete fields of structured text. Next to the images, Ancestry can plug in blurbs generated from information in the document. Those draw from a system that engineers drew up in house. Once Ancestry has found all the records associated with a person, it selects specific facts to pull out of them based on Ancestry editors’ rules, and assembles them into full sentences. Once the document-based blurbs are displayed in the browser, customers can edit and save them before sharing.

Sharing ain’t easy

The challenge is not the creation and storage of new data and websites that users create, said Scott Sorensen, Ancestry’s chief technology officer. Storage has gotten cheaper and cheaper, and that trend should continue. Accurate processing of handwritten records generally is not an issue, either. Often the keyers are in China, Sorensen said. “The Chinese character set is much larger than our alphabet,” he said. “They’re actually very skilled at keying these records.”

The real hard part is to make sure the service is highly available, to serve up all the right document and text for millions of users and keep the site from crashing when traffic peaks. But since one goal of Story View is to get more people checking out content on the site and eventually signing up, that would be a good problem to have.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Sector RoadMap: Social customer service in 2013
• Continuous delivery and the world of devops
• Listening platforms: finding the value in social media data

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Infrastructure Q1: IaaS Comes Down to Earth; Big Data Takes Flight
• Infrastructure Q1: Cloud and big data woo enterprises
• Migrating media applications to the private cloud: best practices for businesses

Derrick covered some of the techniques being used by Ancestry.com back in June, and today we can reveal the latest weapon in MyHeritage’s arsenal: automated record matching.

Both platforms lean heavily on records as a way of augmenting the drier names and dates that make up family trees, but the Israel-based MyHeritage – which already has its own angle by explicitly treating the service like a social network – reckons it now has the edge.

According to CEO Gilad Japhet, MyHeritage has had its Record Matching tech ready for some time, but needed to set up a server farm, then clear a backlog of four billion historical records (including the world’s largest historical newspaper collection, acquired through the company’s FamilyLink buy last year), before launching it today.

“They come from original documents, birth records, marriage certificates, passenger lists going through Ellis Island, tombstones – in a few cases user contributed, as some people take snapshots of gravestones and upload them – public information, census records, newspaper articles and books. Record Matching covers both text-based and structured records, those that can be filled into a regular database,” he told me.

As an example, let’s say you don’t know the date of birth or death for your grandfather, but you do know his name. MyHeritage has a big database of wills, but again, you’re lacking dates. So the service would use its already-existing Smart Matching technology to compare the known information with that on other family trees, perhaps pinning down dates through other relatives’ connections.

Then, armed with that, it would find what it can in those historical records, using semantic analysis to deal with the free-text newspaper cuttings for example.

The smart thing, and one that Japhet hopes will pull in more subscribers and pay-as-you-go credit users, is that Record Matching works automatically and provides snippets of information for free. If you’re a user, you’ll just get an email telling you what’s been found. If you want to see the full record, you pay, but it doesn’t require that step to prove its worth.

So why did MyHeritage decide to shun the cloud for all this?

“We found it wasn’t very efficient to run this in the cloud because the CPU power you get is typically smaller, as a lot of these servers are virtual,” Japhet said. “We wanted serious number-crunching capabilities, and found it more efficient for us to purchase high-end servers, put together a large farm, run it all and accumulate the matches. It’s an ongoing real-time system.”

Japhet also claims other advantages over Ancestry.com, an older and larger service (38 million family trees to MyHeritage’s 23 million). For one thing, he points out that MyHeritage is available in 38 languages and its rival in just half a dozen – that makes a difference when you consider the international aspect of genealogical research.

What’s more, MyHeritage intends to “launch a massive crowdsourcing based transcription system” for its users within the next year, he added. And so the battle for family history continues.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Sector RoadMap: Social customer service in 2013
• The evolution of consumer-media cloud storage
• Controversy, courtrooms and the cloud in Q1

It you’ve used Ancestry.com recently, you’re probably thankful for its efforts. According to Head of Engineering Scott Sorenson, Ancestry.com has more than 10 billion records that are part of a 4-petabyte (or 4-million gigabyte) data store. If you’re searching for “John Smith,” he explained, it probably has about 60 million for “Smith” and about 4 million for “John Smith,” but you’re only interested in the relative handful that are relevant to your John Smith.

Making models smarter

That’s why Ancestry.com is using machine learning to make sorting through those records a lot less like finding a needle in a haystack and a lot more like having that needle — and any others made from the same batch of steel — delivered right to your door. Here’s how the process works, in a nutshell:

1. Crawl digital records (e.g., newspapers, birth records, death records, census data, ship manifests, etc.) online and extract relevant data
2. (Or 1(a)) Scan, upload and index physical records (via a partner in China)
3. Stitch together new records with user data to add more context
4. And this is key, constantly analyze user behavior in order to make its algorithms smarter

As users make judgments about the records they’re presented, Sorenson said, Ancestry.com’s algorithms get better at performing their particular tasks. So, a system for extracting data from newspaper pages might be able to better recognize the various sections of the page (so as to ignore the ads, for example) and then be able to adjust for mistakes in the section it is analyzing. And as with Google’s search algorithms, the more that users interact with records, the better Ancestry.com’s sorting algorithms are able to determine those records relevance to any given user.

Spit in a tube, pay $99, learn your past

Oh, but Ancestry.com has decided that merely storing and analyzing historical records is just the beginning with regard to providing accurate genealogy information. It also will sequence your DNA, focusing on 700,000 markers important to determining one’s race, lineage and other factors. That service, which simply requires users to swab their cheek or spit in a tube and send it to the lab, costs only $99 (a full genome sequence would cost at least 10 times that, by the way), but could revolutionize the accuracy of Ancestry.com’s models.

Right now, Sorenson said, the DNA service can tell users their race and what country they’re from, and also connect them with other relatives who share a DNA profile. (If your privacy red flag has gone up reading this, Sorenson did note the following: all communications with relatives are optional and initially anonymous; all DNA information is disassociated from personal information; and users get their sequence results via an encrypted key “that we treat with a higher level of security than we’d store your credit card information.”)

Connecting with distant relatives can be valuable, though. A third cousin, for example, might have ancestral information that you don’t, which will help make your family tree that much more accurate. But Sorenson said when it really gets interesting is when Ancestry.com can combine DNA data with record data in family trees. Someone’s DNA might indicate he’s from France, Sorenson explained, but cross-checking that against that person’s family data will let the service discover he’s actually from the Normandy region.

Going forward, Sorenson said Ancestry.com expects its DNA service to take off like a rocket. The company is investing between $10 million and $15 million into that service over the next couple years, and has bioinformatic scientists on staff trying to scale algorithms designed to handle hundreds of samples to work with hundreds of thousands or even millions of samples. In that regard, though, Ancestry.com isn’t alone — the steady drop in the price of genome sequencing has everyone in the sector anticipating skyrocketing data volumes.

What’s next: Telling stories and making genealogy real-time

OK, so it has billions of records and our DNA, what more can Ancestry.com possibly want or need to provide us information on our ancestors? Nothing, actually.

It just needs to make better use of what it does have and the new technologies available for working with that information. Genealogy has traditionally been “dusty,” Sorenson explained, but Ancestry.com is trying to tell the stories behind those dusty records. If you’ve seen the NBC program “Who Do You Think You Are?”, on which Ancestry.com traces celebrities’ ancestral roots, you have an idea of what Sorenson is talking about.

For example, by improving its image-processing capabilities, Ancestry.com could extract more information than just name, data and location from old records that it already knows how to process. It could tell someone that his grandfather was the only person on the block to own a radio, or whether he owned his home. Combined with socioeconomic and other external data, Sorenson said, Ancestry.com could “create a really vivid picture” of what it was like to live during a specific time.

By using location data from cell phones, Sorenson said Ancestry.com could deliver a mobile experience that’s far more than a translation of the web on a smaller screen by making genealogy a geospatial pursuit. For example, Sorenson, explained, if a user takes a picture of a gravestone, Ancestry.com would like to provide him with relevant historical data related to that place, and maybe even some nearby points of interest.

Some might think Ancestry.com’s practices and plans toe the privacy line, but if someone has to toe that line, this might be the company to do it. In a fast-paced world it’s easy to get tied up in the moment and in our own little worlds — especially with big data being used elsewhere on the web to keep our attention firmly on one site or another. Using personal data to let users dig into decades into their family histories ends up looking very refreshing.

Feature image courtesy of Shutterstock user tovovan.

Related research and analysis from GigaOM Pro:
Subscriber content. Sign up for a free trial.

• Connected world: the consumer technology revolution
• How to use big data to make better business decisions
• Takeaways from the second quarter in cloud and data