Wednesday, February 15, 2012

Eyeball Statistics

What is “Eyeball Statistics”? It is phrase I haven given to how a majority of people analyze data. In other words, people analyze data merely by what they can deduce by looking at it. Thus, they are not performing the necessary “due diligence” to mathematically analyze the data to find correlation or standard deviations. We are all guilty of using eyeball statistics because it is fast and convenient. Eyeball statistics is certainly acceptable when evaluating a few data points for one or two variables. From this we can ascertain trends. But the human brain is not very good at deciphering trends from a plethora of variables and data points. To do this properly, we must do the math and create complex models.

Let’s evaluate how statistical data can be misleading. One common statistic that most people use is statistical averages. And many people have the ability to compute averages merely by eyeballing the data. Finding the statistical average of a set of data points is easy to compute, but this data can be misleading. In many cases, averages can mean very little without knowing the variance, standard deviation, or even the skewness of the data. For instance, a student who scored 5 points below the class average on a test means very little without understanding the statistical variance of how the class performed as a whole. If the student’s score was within one standard deviation of the average, then they are performing as well as a majority of the class. If their score, on the other hand, was two or three standard deviations below the class average, than they are performing well below the rest of the class. Hence, statistics can be misleading if they are not defined properly. And we cannot define statistics properly merely by eyeballing data.

Most of the economic and scientific problems that face Americans are not easy problems to solve. The effects of carbon emissions on global warming; the effects of Obamacare on doctor availability; the effects of raising taxes on the wealthiest earners on consumer spending; the effects of cap and trade on corporate profits; and so forth. Each of these problems may contain dozens of variables with thousands of data points and no one can eyeball these results. These are the problems and issues that face Americans on a daily basis and we all have an opinion about them. There is nothing wrong with having an opinion. But we must remember that an opinion is neither right nor wrong. The problem is that all Americans feel their opinions are scientifically proven facts that are unequivocally 100% correct. How many people have actually seen and understood a model created by an economist showing the effects of taxing the wealthy on consumer spending or a model by a climatologist revealing the effects of carbon emissions on temperature changes? The answer is pretty darn close to zero. Without seeing these models I cannot agree or disagree with the conclusions. For this reason, I create my own models to better understand these issues, but models are not perfect. If models were 100% accurate to predict the future, we would all be prophets. We could wipe out cancer and solve the world’s problems. But trends change over time, which render models obsolete. And like all statistical models, there is an error associated with them. When I posed this question to one liberal friend about global warming models, he forwarded me a bunch of weather prediction models he found on the web. First, the data within the models was not known and therefore, we could not dispute the results. Secondly, there is a big difference between weather and climate models and for someone arguing that global warming is manmade - he should have understood this fact. Face it; people do not like to show their models because someone can poke holes in them.

Although models are not perfect, they are the only means we have to evaluate complex problems that cannot be solved by eyeball statistics. And anyone who claims to understand the effects of carbon emissions on climate change or the effects of raising taxes on the wealthy has on unemployment are formulating an opinion if they have not done the “due diligence” to understand the math behind the conclusions. This is why the so called experts are wrong over 50% of the time when trying to predict the future – they are guilty of eyeball statistics. I will post this study at a later date (it compiled data over 30 years). For instance, economists were right less than 40% of the time when predicting the future outcome of two possible scenarios. For example, economists where asked will unemployment go up or down next year, or will inflation go up or down next year, and so forth. The average American could guess and get 50% of these questions / predictions correct. Maybe this is why everyone thinks they are an expert - they can guess better than the experts!

2 comments:

  1. Back when I started in qualitative marketing research I worked for a company which utilized computer aided sorting of data (words respondents used during interviews using psychiatric interviewing techniques). Nifty, huh? Back then sample sizes were such that there was a high level of confidence in being able to predict behavior through language use.

    Well to shorten this saga, here are two links from two different companies who reference Dr. Charles E. Cleveland (for whom I worked for years). Both these companies reference Dr Cleveland, who died in 2005.

    http://mentalinsights.com/wp-content/uploads/2011/June24_2011WhitePaperOnlineMR.pdf

    http://www.quester.com/

    Happy reading!

    ReplyDelete
  2. Interesting Mrs. AL, I will check it out. Thanks a bunch.

    ReplyDelete