Summary:

Predicting risk is key to the way insurance companies figure out your monthly rates and premiums, and it needs data and time to do so. Allstate said at Structure:Data that the best use of this data was to give it to the 30,000 scientists competing on Kaggle.

Eric Huls of Allstate Insurance, Jeremy Howard of Kaggle, and Ryan Kim of GigaOM at Structure:Data 2012

If you want to accurately price insurance rates for bodily injury – how much of an impact is the make of the vehicle going to have? Insurance companies have been dealing with these kinds of questions for a long time. Allstate recently decided to get some outside help to leverage all of its existing data and improve its own prediction models, explained Allstate VP Eric Huls at GigaOM’s Structure:Data conference in New York Wednesday.

Eric Huls of Allstate Insurance, Jeremy Howard of Kaggle, and Ryan Kim of GigaOM at Structure:Data 2012

(c) 2012 Pinar Ozger. pinar@pinarozger.com

That’s why Huls and his team worked with Kaggle, a marketplace for data science competitions. Huls told the audience that the cooperation with Kaggle was the result of an interesting discovery: Some of Allstate’s own engineers participated in Kaggle competitions in their spare time. “My first reaction was: Clearly, we are not giving these people enough work,” he joked.

But then he realized that outside expertise could help Allstate build better predictions, which can directly impact Allstate’s bottom line. “How well you predict risk is the difference between being a successful and an unsuccessful insurance company,” explained Kaggle’s president and chief scientist Jeremy Howard.

However, there are also some risks in outsourcing this kind of data science. One of the biggest ones: Data scientists are smart folks, and they might figure out some things that you didn’t really want them to know, or include in their models. That can lead to data leakage and privacy issues, as Allstate experienced first-hand. The company’s data spanned across a number of years, and participants were able to figure out which customers had been driving which cars over a period of time. If you’re an insurance company, you might want to contain that data, admitted Huls: “In our competition, we didn’t do it very well, quite honestly.”

Watch the livestream of Structure:Data here.

Watch live streaming video from gigaombigdata at livestream.com

Comments have been disabled for this post