Even with a time machine, Marty McFly couldn’t predict that the movie event of 2015 would be Star Wars and not Jaws. (Photo by Stéfan )

SS asks:

What makes statistical modeling different from, say, new age methods of trying to predict the future?

I suppose it might be somewhat unorthodox to answer any way but “Everything!”, but the more I thought about this question, the more intriguing I found it. I’ll get into the answer in the coming weeks. First I’d like to explore why there’s more to this question than might be immediately apparent.

For starters, it hits rather close to home. My day job involves a fair amount of statistical modeling and prediction. My company aggregates healthcare data and notifies public health agencies whenever we detect anything “unusual.” In order to decide what is unusual, we have to predict what is expected, and we rely heavily on statistical models. Because of the multitude of different ways we slice up the data (30-40 health categories by ~100 counties, every hour for a typical state agency), we see “unusual” things pretty regularly. Not all of these statistical anomalies correspond to the kinds of health events our users need to find. So we are very concerned with our users’ perceptions of our analysis. Do they think we are scientists or snake oil salesmen?

In my spare time, I also have some concerns about the wider public perception of science. Science stakes its reputation on predictions, and its track record is not too shabby. This comet will be visible at that time. This particular antibiotic will clear up that infection. This amount of rocket fuel will get those astronauts to the moon. And so on. Once, the empirical confirmation of gravitational lensing, a prediction of Einstein’s general relativity, made headlines around the world. Now, we make jokes about how often the weather forecast is wrong. We feel like we can’t get straight answers about which foods will prolong our lives and which will shorten them. Our flying cars are perpetually 10 years away. Scientific predictions don’t always feel like such a sure thing.

Finally, what I thought was most fun about the question is the fact that some kinds of new age predictions actually boil down to statistical modeling of a sort. Some folks who offer psychic readings and the like are just playing the odds, relying on commonalities and trends in human experience and behavior to be right–or right enough with sufficiently broad assertions. That’s not to say that all psychic predictions are statistically valid, or that all statistical modeling is hokum. I simply wish to acknowledge that teasing the two practices apart will be a little tricky.

What are your perceptions of statistical modeling? What kinds of predictions do you encounter regularly?