Lessons Learned About Election Polls

Throughout the 2012 presidential election process both Republicans and Democrats cried foul over election polls. The most common complaint – the poll was either over-sampling or under-sampling political ideology. This complaint was mostly heard from the conservative side since they were regularly down in both national and battleground state polls. Occasionally, we would hear someone complain about a poll’s demographic makeup, but it was rare. In a few instances, the Democrats complained that some polls underestimated the minority turn out – and they were right.

Truth be told, the hardest thing for election polls to model is political ideology. Political ideology changes routinely amongst some people. However, a person’s demographics never change! For this reason, most polling companies do not consider the political ideology of the electorate when conducting polls (most polls provide the data for political ideology makeup of the poll participants, but they do not use this information to model actual results). Instead, polling companies focus solely on the demographics – they want the demographics of the poll sample to resemble the electorate demographics (national or on the state level). Thus, they make sure that gender, ethnicity, age, and other demographics in the poll resemble the electorate. Once the poll is complete, the pollsters let the political ideology fall as it may.

In 2008, an independent or unaffiliated voter may consider their political ideology as lean Democratic or Democrat if they voted for Obama. However, that same person in the 2012 election may consider their political ideology as lean Republican or Republican if they voted for Romney. For this reason political ideology is hard to sample. However, the same person’s age, gender, and ethnicity are not going change (age changes, but it will be the same during the calendar year of the election). To make modeling political ideology more troubling is many states do not track party affiliation. And states that track political ideology data, a person may change their political affiliation to vote in a primary and not change it back before Election Day. For example, a liberal may change their affiliation to vote in the 2012 Republican primary. After all, most liberals already knew that Obama was going to win the Democratic nomination. For these reasons, it is much more accurate for polling companies to focus on the demographic makeup of the electorate.

The average of 17 national polls had Obama winning by 0.7% while the political ideology of the electorate favored Democrats by 5.2%. In the end, Obama won the election by nearly 3% and the political ideology of the electorate favored Democrats by 6%. Using the same data I predicted the outcome of the election using 3 methods. I modeled the political ideology makeup of the electorate to favor Democrats by 4%. Under this model Romney held a slight lead in the popular vote. However, if I modeled the election based on the demographic makeup of the electorate at 53% (women) - 47% (men), and 75% (white) - 25% (non-white), Obama won using both models in a fairly close election. In the end, the actual ethnicity makeup was 72% (white) – 28% (non-white) and obviously Obama won easily with that demographic makeup of the electorate. Hence, while modeling the national election poll data, I was able to correctly predict the outcome of the election using demographics (even though I still did not predict the ethnicity makeup of the electorate correctly), but was wrong when I used political ideology.

Therefore, in the future, pollsters and people critiquing polls would be more accurate and prudent to concentrate on the demographic makeup and not political ideology of the poll participants. Still, it is difficult to predict the outcomes of elections because pollsters must predict if the youth vote will show up, will the minority vote show up, will single women show up, and so forth. For example, in 2004, exit polls showed that John Kerry was going to win easily over George Bush. However, the exit polls consisted of a gender makeup that was 57% women and only 43% male (this should have obviously been seen as a poor sample). Hence, when the actual gender makeup of the electorate was 53% women and 47% male, the exit polls were grossly wrong. Women tend to be more liberal than men.