Tap The Data Brakes
A batter steps up to the plate and a coach signals to the pitcher to throw an inside curve ball. Why? Because the coach knows that the hitter swings and misses 75% of the time when the first pitch is an inside curve ball. The pitcher nods, tucks his chin, and throws the pitch. The batter hits a home run. Two innings later, the same coach has a decision to make. Does he stick to the statistics or consider the other variables at play tonight?
Data has different degrees of utility; it can be predictive or misleading. The degree to which data can be predictive or misleading is correlated with the control that one has over the experiment that is producing the data, i.e. control over the variables in the experiment. As control of the experiment decreases, the value of the expert increases.