<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:go='http://ns.gigaom.com/'
xmlns:georss="http://www.georss.org/georss" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" xmlns:media="http://search.yahoo.com/mrss/"
		>
<channel>
	<title>Comments on: Predicting the Unpredictable</title>
	<atom:link href="http://gigaom.com/2009/08/09/predicting-the-unpredictable/feed/" rel="self" type="application/rss+xml" />
	<link>http://gigaom.com/2009/08/09/predicting-the-unpredictable/</link>
	<description></description>
	<lastBuildDate>Sat, 11 Feb 2012 12:44:45 +0000</lastBuildDate>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.com/</generator>
	<item>
		<title>By: Dev</title>
		<link>http://gigaom.com/2009/08/09/predicting-the-unpredictable/#comment-220375</link>
		<dc:creator><![CDATA[Dev]]></dc:creator>
		<pubDate>Fri, 23 Oct 2009 22:11:02 +0000</pubDate>
		<guid isPermaLink="false">http://gigaom.com/?p=60991#comment-220375</guid>
		<description><![CDATA[Without a sense of how well the wine data fit that regression equation, it&#039;s pretty useless. You can multivariate regress almost any data and get a precise-looking equation (to three sig figs, as above, or more) that still has an error to, um, one sig fig. You can regress randomly generated data, and compute the best _linear_ fit--but you still need to know how good that fit is (which may not be that good--try to linearly regress a higher-order polynomial, or exponential, or anything else, really). That&#039;s one problem.

Another is stationarity--the system may or may not have stationary behavior in terms of the specific parameters here. Maybe the equation holds up over time, or maybe it doesn&#039;t (as it didn&#039;t with, for example, mortgage foreclosure rates up to last year, or on bond spreads up to the LCTM debacle, etc). Even decades of stable data don&#039;t &#039;prove&#039; stationarity, as we all found out.

And finally, we don&#039;t know which input data are necessarily significant, and which are noisy (and will actually make the model *worse* as a result of their inclusion, given spurious correlations in finite samples). The three input variables for the wine equation above may be the most meaningful, or they might not be (there may be other, more significant variables at play)--but the equation alone won&#039;t warn you against that. See factor analysis or PCA.

Given all that, the equation&#039;s significance really has to be qualified in a few specific ways--it&#039;s not the best quantitative model of wine quality---it&#039;s merely the best _linear_, _historical_sample_based_, _limited_input_ model of wine quality. You can do polynomial (or fourier, or wavelet, etc) regression instead of linear; you can try to model the internal generators of wine quality rather than only using the historical top-line numbers; and you can try to look at a broader set of candidate inputs.

Oh, and another good book in the &#039;Fooled By Randomness&#039; tradition (and with a foreward by Nassim Taleb)--Pablo Triana&#039;s recent &quot;Lecturing Birds on Flying&quot;.]]></description>
		<content:encoded><![CDATA[<p>Without a sense of how well the wine data fit that regression equation, it&#8217;s pretty useless. You can multivariate regress almost any data and get a precise-looking equation (to three sig figs, as above, or more) that still has an error to, um, one sig fig. You can regress randomly generated data, and compute the best _linear_ fit&#8211;but you still need to know how good that fit is (which may not be that good&#8211;try to linearly regress a higher-order polynomial, or exponential, or anything else, really). That&#8217;s one problem.</p>
<p>Another is stationarity&#8211;the system may or may not have stationary behavior in terms of the specific parameters here. Maybe the equation holds up over time, or maybe it doesn&#8217;t (as it didn&#8217;t with, for example, mortgage foreclosure rates up to last year, or on bond spreads up to the LCTM debacle, etc). Even decades of stable data don&#8217;t &#8216;prove&#8217; stationarity, as we all found out.</p>
<p>And finally, we don&#8217;t know which input data are necessarily significant, and which are noisy (and will actually make the model *worse* as a result of their inclusion, given spurious correlations in finite samples). The three input variables for the wine equation above may be the most meaningful, or they might not be (there may be other, more significant variables at play)&#8211;but the equation alone won&#8217;t warn you against that. See factor analysis or PCA.</p>
<p>Given all that, the equation&#8217;s significance really has to be qualified in a few specific ways&#8211;it&#8217;s not the best quantitative model of wine quality&#8212;it&#8217;s merely the best _linear_, _historical_sample_based_, _limited_input_ model of wine quality. You can do polynomial (or fourier, or wavelet, etc) regression instead of linear; you can try to model the internal generators of wine quality rather than only using the historical top-line numbers; and you can try to look at a broader set of candidate inputs.</p>
<p>Oh, and another good book in the &#8216;Fooled By Randomness&#8217; tradition (and with a foreward by Nassim Taleb)&#8211;Pablo Triana&#8217;s recent &#8220;Lecturing Birds on Flying&#8221;.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: dispatches from TJICistan &#187; Blog Archive &#187; Guerillas either discover this formula, or they get killed off</title>
		<link>http://gigaom.com/2009/08/09/predicting-the-unpredictable/#comment-220374</link>
		<dc:creator><![CDATA[dispatches from TJICistan &#187; Blog Archive &#187; Guerillas either discover this formula, or they get killed off]]></dc:creator>
		<pubDate>Wed, 26 Aug 2009 00:23:46 +0000</pubDate>
		<guid isPermaLink="false">http://gigaom.com/?p=60991#comment-220374</guid>
		<description><![CDATA[[...] http://gigaom.com/2009/08/09/predicting-&#8230; [...]]]></description>
		<content:encoded><![CDATA[<p>[...] <a href="http://gigaom.com/2009/08/09/predicting-&#038;#8230" rel="nofollow">http://gigaom.com/2009/08/09/predicting-&#038;#8230</a>; [...]</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Predicting the BCS Champion : ESPN Dev Blog</title>
		<link>http://gigaom.com/2009/08/09/predicting-the-unpredictable/#comment-220373</link>
		<dc:creator><![CDATA[Predicting the BCS Champion : ESPN Dev Blog]]></dc:creator>
		<pubDate>Thu, 20 Aug 2009 23:14:47 +0000</pubDate>
		<guid isPermaLink="false">http://gigaom.com/?p=60991#comment-220373</guid>
		<description><![CDATA[[...] football.    It was during one of these runs through my required reader that I mistakenly took a GigaOm blog post about prediction functions for a college football piece (most blog posts about college [...]]]></description>
		<content:encoded><![CDATA[<p>[...] football.    It was during one of these runs through my required reader that I mistakenly took a GigaOm blog post about prediction functions for a college football piece (most blog posts about college [...]</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: R</title>
		<link>http://gigaom.com/2009/08/09/predicting-the-unpredictable/#comment-220372</link>
		<dc:creator><![CDATA[R]]></dc:creator>
		<pubDate>Mon, 10 Aug 2009 20:50:12 +0000</pubDate>
		<guid isPermaLink="false">http://gigaom.com/?p=60991#comment-220372</guid>
		<description><![CDATA[Sorry, I didn&#039;t notice you wrote this article.  I just want to say, this is the first time I commented on a story in a long while, so I guess it tells you how much interesting I found it, thanks for putting it together.]]></description>
		<content:encoded><![CDATA[<p>Sorry, I didn&#8217;t notice you wrote this article.  I just want to say, this is the first time I commented on a story in a long while, so I guess it tells you how much interesting I found it, thanks for putting it together.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: R</title>
		<link>http://gigaom.com/2009/08/09/predicting-the-unpredictable/#comment-220371</link>
		<dc:creator><![CDATA[R]]></dc:creator>
		<pubDate>Mon, 10 Aug 2009 19:01:16 +0000</pubDate>
		<guid isPermaLink="false">http://gigaom.com/?p=60991#comment-220371</guid>
		<description><![CDATA[I don&#039;t think anyone would argue that the wrong stats are being used, you just have to look at the top ten draft picks over the years for many of these sports   It might sound strange but if I was picking players for a football team, off the top of my head I would just look at two or three things, a) injury history, b) performance in pressure situations (4th downs, 4th quarters, overtime etc.) but i also think that something like the players relationship with his mother can tell you a lot about his character, and in some cases his character might be strong enough to outweigh some of your other measurements.]]></description>
		<content:encoded><![CDATA[<p>I don&#8217;t think anyone would argue that the wrong stats are being used, you just have to look at the top ten draft picks over the years for many of these sports   It might sound strange but if I was picking players for a football team, off the top of my head I would just look at two or three things, a) injury history, b) performance in pressure situations (4th downs, 4th quarters, overtime etc.) but i also think that something like the players relationship with his mother can tell you a lot about his character, and in some cases his character might be strong enough to outweigh some of your other measurements.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: R</title>
		<link>http://gigaom.com/2009/08/09/predicting-the-unpredictable/#comment-220370</link>
		<dc:creator><![CDATA[R]]></dc:creator>
		<pubDate>Mon, 10 Aug 2009 18:50:09 +0000</pubDate>
		<guid isPermaLink="false">http://gigaom.com/?p=60991#comment-220370</guid>
		<description><![CDATA[I&#039;m not too familiar with what the pats have done, ever since the niner&#039;s dynasty collapsed (i&#039;m a niners fan) i mostly just watch playoffs and the superbowl.  But from what i understand, Bill Belichick like many recent super bowl coaches was also a student of Bill Walsh, i&#039;m sure stats can be put to better use in other sports and people should certainly try, i&#039;m just saying if I was a betting man, I would say that statistics will not be as succesful in other sports (as far as picking the best players for a team goes).  But if your goal is to say pick the winner of this year&#039;s superbowl, in the eighth week of the regular season then i think statistics can definitely be put to good use, as I have personally done, its just the particular case of picking players for a team, where i&#039;m not sure statistics will be as successful as they appear to be in baseball.  Although like Taylor says below, its possible we just haven&#039;t found the right stats.]]></description>
		<content:encoded><![CDATA[<p>I&#8217;m not too familiar with what the pats have done, ever since the niner&#8217;s dynasty collapsed (i&#8217;m a niners fan) i mostly just watch playoffs and the superbowl.  But from what i understand, Bill Belichick like many recent super bowl coaches was also a student of Bill Walsh, i&#8217;m sure stats can be put to better use in other sports and people should certainly try, i&#8217;m just saying if I was a betting man, I would say that statistics will not be as succesful in other sports (as far as picking the best players for a team goes).  But if your goal is to say pick the winner of this year&#8217;s superbowl, in the eighth week of the regular season then i think statistics can definitely be put to good use, as I have personally done, its just the particular case of picking players for a team, where i&#8217;m not sure statistics will be as successful as they appear to be in baseball.  Although like Taylor says below, its possible we just haven&#8217;t found the right stats.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: R</title>
		<link>http://gigaom.com/2009/08/09/predicting-the-unpredictable/#comment-220369</link>
		<dc:creator><![CDATA[R]]></dc:creator>
		<pubDate>Mon, 10 Aug 2009 18:05:37 +0000</pubDate>
		<guid isPermaLink="false">http://gigaom.com/?p=60991#comment-220369</guid>
		<description><![CDATA[i wasn&#039;t aware of this till you pointed it out. i guess it is just a coincidence.]]></description>
		<content:encoded><![CDATA[<p>i wasn&#8217;t aware of this till you pointed it out. i guess it is just a coincidence.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Taylor Davidson</title>
		<link>http://gigaom.com/2009/08/09/predicting-the-unpredictable/#comment-220368</link>
		<dc:creator><![CDATA[Taylor Davidson]]></dc:creator>
		<pubDate>Mon, 10 Aug 2009 12:09:44 +0000</pubDate>
		<guid isPermaLink="false">http://gigaom.com/?p=60991#comment-220368</guid>
		<description><![CDATA[Actually, the whole point of many of the new stats in baseball, football and basketball (check out 82games.com) is to capture the &quot;social aspect&quot; of how individual players contribute to the overall team&#039;s play.

As a &quot;team sport played by individuals&quot;, baseball is an easier sport to isolate the impact of individual performance on team performance, but football and basketball are catching up by developing stats that meaningfully capture the impact of individuals on team performance.  The inability of past stats to capture the impact of &quot;personality&quot; or &quot;team character&quot; is a signal that analysts are using the wrong stats, not that stats can&#039;t be used to understand what&#039;s going on.]]></description>
		<content:encoded><![CDATA[<p>Actually, the whole point of many of the new stats in baseball, football and basketball (check out 82games.com) is to capture the &#8220;social aspect&#8221; of how individual players contribute to the overall team&#8217;s play.</p>
<p>As a &#8220;team sport played by individuals&#8221;, baseball is an easier sport to isolate the impact of individual performance on team performance, but football and basketball are catching up by developing stats that meaningfully capture the impact of individuals on team performance.  The inability of past stats to capture the impact of &#8220;personality&#8221; or &#8220;team character&#8221; is a signal that analysts are using the wrong stats, not that stats can&#8217;t be used to understand what&#8217;s going on.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Mike Speiser</title>
		<link>http://gigaom.com/2009/08/09/predicting-the-unpredictable/#comment-220367</link>
		<dc:creator><![CDATA[Mike Speiser]]></dc:creator>
		<pubDate>Sun, 09 Aug 2009 23:29:42 +0000</pubDate>
		<guid isPermaLink="false">http://gigaom.com/?p=60991#comment-220367</guid>
		<description><![CDATA[You clearly know your stuff.  I&#039;m not sure if your assertion is right or wrong, but I don&#039;t think stats must meet the bar of &quot;better than they are in baseball&quot; in order to be put to good use.  You&#039;re likely right, but my understanding is that the Patriots have used stats to do a good job in the recent past, no?]]></description>
		<content:encoded><![CDATA[<p>You clearly know your stuff.  I&#8217;m not sure if your assertion is right or wrong, but I don&#8217;t think stats must meet the bar of &#8220;better than they are in baseball&#8221; in order to be put to good use.  You&#8217;re likely right, but my understanding is that the Patriots have used stats to do a good job in the recent past, no?</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Mike Speiser</title>
		<link>http://gigaom.com/2009/08/09/predicting-the-unpredictable/#comment-220366</link>
		<dc:creator><![CDATA[Mike Speiser]]></dc:creator>
		<pubDate>Sun, 09 Aug 2009 23:27:27 +0000</pubDate>
		<guid isPermaLink="false">http://gigaom.com/?p=60991#comment-220366</guid>
		<description><![CDATA[Agreed.  Awesome book.  Haven&#039;t made it through, but serves as a superb reference guide.]]></description>
		<content:encoded><![CDATA[<p>Agreed.  Awesome book.  Haven&#8217;t made it through, but serves as a superb reference guide.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Mike Speiser</title>
		<link>http://gigaom.com/2009/08/09/predicting-the-unpredictable/#comment-220365</link>
		<dc:creator><![CDATA[Mike Speiser]]></dc:creator>
		<pubDate>Sun, 09 Aug 2009 23:26:26 +0000</pubDate>
		<guid isPermaLink="false">http://gigaom.com/?p=60991#comment-220365</guid>
		<description><![CDATA[Good point from the guy named after a really cool statistics project.]]></description>
		<content:encoded><![CDATA[<p>Good point from the guy named after a really cool statistics project.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Mike Speiser</title>
		<link>http://gigaom.com/2009/08/09/predicting-the-unpredictable/#comment-220364</link>
		<dc:creator><![CDATA[Mike Speiser]]></dc:creator>
		<pubDate>Sun, 09 Aug 2009 23:25:42 +0000</pubDate>
		<guid isPermaLink="false">http://gigaom.com/?p=60991#comment-220364</guid>
		<description><![CDATA[&quot;much as you might find with a good Cabernet or Merlot.&quot;  Ha.  Thanks for the comment.]]></description>
		<content:encoded><![CDATA[<p>&#8220;much as you might find with a good Cabernet or Merlot.&#8221;  Ha.  Thanks for the comment.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Mike Speiser</title>
		<link>http://gigaom.com/2009/08/09/predicting-the-unpredictable/#comment-220363</link>
		<dc:creator><![CDATA[Mike Speiser]]></dc:creator>
		<pubDate>Sun, 09 Aug 2009 23:24:51 +0000</pubDate>
		<guid isPermaLink="false">http://gigaom.com/?p=60991#comment-220363</guid>
		<description><![CDATA[Thank you Banu.]]></description>
		<content:encoded><![CDATA[<p>Thank you Banu.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Mike Speiser</title>
		<link>http://gigaom.com/2009/08/09/predicting-the-unpredictable/#comment-220362</link>
		<dc:creator><![CDATA[Mike Speiser]]></dc:creator>
		<pubDate>Sun, 09 Aug 2009 23:24:05 +0000</pubDate>
		<guid isPermaLink="false">http://gigaom.com/?p=60991#comment-220362</guid>
		<description><![CDATA[Thank you Chetan.]]></description>
		<content:encoded><![CDATA[<p>Thank you Chetan.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Mike Speiser</title>
		<link>http://gigaom.com/2009/08/09/predicting-the-unpredictable/#comment-220361</link>
		<dc:creator><![CDATA[Mike Speiser]]></dc:creator>
		<pubDate>Sun, 09 Aug 2009 23:23:35 +0000</pubDate>
		<guid isPermaLink="false">http://gigaom.com/?p=60991#comment-220361</guid>
		<description><![CDATA[Thanks Wilf.]]></description>
		<content:encoded><![CDATA[<p>Thanks Wilf.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Mike Speiser</title>
		<link>http://gigaom.com/2009/08/09/predicting-the-unpredictable/#comment-220360</link>
		<dc:creator><![CDATA[Mike Speiser]]></dc:creator>
		<pubDate>Sun, 09 Aug 2009 23:23:01 +0000</pubDate>
		<guid isPermaLink="false">http://gigaom.com/?p=60991#comment-220360</guid>
		<description><![CDATA[Great comment, thanks JS.  Will try your experiment out with friends, but I prefer Napa Valley these days ;-)]]></description>
		<content:encoded><![CDATA[<p>Great comment, thanks JS.  Will try your experiment out with friends, but I prefer Napa Valley these days ;-)</p>
]]></content:encoded>
	</item>
</channel>
</rss>

