Engaging the Election: How to Read the Polls

Author’s note: During the US election season, we will be bombarded with a chorus if not cacophony of political coverage, punditry, and even misrepresentation of the presidential campaign. To help us navigate this noisy time, I will be presenting a series of empirical snapshots on how evangelical Christians perceive the two presumptive nominees for president. By “letting the data speak,” I will present a nonpartisan analysis of the social and political attitudes of our fellow Christians. While the findings should challenge our preconceptions and convictions, it is not an endorsement or critique of either candidate. Instead, my hope is that it spurs greater conversation and discussion about how our faith can and should translate into our perceptions and participation in the presidential election. This is the first post in the series,  on how to read the polls. For the second post, on what predicts Evangelical approval or disapproval of Trump, click,  here. For the third post, on what predicts Evangelical approval or disapproval of Clinton, click here. For the fourth post, on  Engaging the Election: The Lesser of Two Evils for Evangelical Voters?, click here.

---

Without a doubt, political polls are increasingly central to our political discourse. It is critical we consider how to best understand the growing abundance and availability of polling data. Just as we are called to be mature Christians not easily swayed by ever changing tides of the world (Ephesians 4:14), so we should be wise citizens who are responsible in how we consume, share, and interpret polling data. Here are four ways we can do that.

First, we should not be swayed by the results of any single poll. For example, a new poll may show a big jump in support for one candidate. Based on how our news outlet of choice (over)reports on this seemingly significant result and how our friends on Facebook enthusiastically share the news, it would be easy to conclude that we have evidence of a significant change in support for this candidate. As one political scientist puts it, “Jumping polls can make for frenzied reporting, but much of this is just noise.”

There are many reasons why a particular poll may reveal unexpected results. Polling data can be affected by differences in how respondents are sampled across polling firms and media companies, inconsistent question wording, respondent misunderstanding of questions, or errors in calculation of results. Or it could be measuring a real change in public opinions. However, it is difficult to determine if seemingly divergent results are measuring polling inconsistencies or a real shift in attitudes from a single poll.

Instead, we should use the moving average of similar comparable polls as a better measure of public opinion. By aggregating results from different sources, all of which have their particular biases if not shortcomings, a statistical moving average better reveals underlying trends in public attitudes than a single poll. In the past, it was difficult to collect and calculate the moving average of presidential polls. However, there are reputable media organizations, most notably the New York Times, Real Clear Politics, and the Huffington Post, that now report daily updated averages of all publicly reported polls.

Third, when reading reporting of polling data, read past the headline. This is especially true in coverage about the political attitudes of evangelical Christians. There are different ways to identify who is an evangelical. Most polls use the white evangelical definition. While a majority of evangelicals are white, this definition excludes non-whites who would otherwise identify as an evangelical (like me) from being counted. Thus, we should not extrapolate polling data about white evangelicals to be necessarily representative of all evangelicals.

Take for example a recent poll described with the headline “Evangelicals Rally to Trump, Religious ‘Nones’ Back Clinton”. From the title alone, it would be easy to miss that their analysis focused on the attitudes of white evangelicals. This lack of differentiation between white evangelicals and all evangelicals is repeated in some headlines (here, here, here), while other headlines more accurately emphasizes the focus on white evangelicals (here, here, here). Given these inconsistencies in media headlines, we should not adopt a “tl;dr” attitude to avoid drawing incomplete if not incorrect conclusions from a recent poll.

Finally, we should remember that polling data is best used as a measure of current public opinion. The majority of polls covers the election as a “horse-race”—it reports and informs the public which candidate is winning today. But it does not reveal the factors, motivations, and reasons why particular groups of people support which candidate. Nor is polling data sufficient to predict the winner of the election. It is a critical component of models such as those from Fivethirtyeight, PredictWise, and PollyVote, but must be considered with other factors in order to make sound predictions about the outcome of the election in November.

Polling data is informative and exciting to follow, but we should be careful about conclusions we make about present attitudes (from a single poll), about conclusions based on just reading the headline (which could imprecisely describe if not misrepresent polling results), or conclusions about the outcome of the election (as polling data alone is not sufficient to predict the future).

Most significantly, horserace polling data also cannot provide insights into how or why groups of people are more likely to support the major party candidates. To try and fill in some of these gaps, my next few posts will examine how and why evangelicals and non-evangelicals (dis)approve of Hilary Clinton and Donald Trump.