Systemic errors, group behaviour and uncertainty: Why the polls did not predict Trump's victory
Estimated reading time: 4 Minutes
Many people predicted that Hillary Clinton would win the US presidential elections – academics, pollsters, newspapers. I was assuming so, too. She was leading with a significant edge in national polls over the last weeks of the electoral campaign. But now Donald Trump is the President-elect. Why did the polls fail to predict his victory?
[caption id="attachment_8545" align="alignnone" width="650"]
US general election polls 2016. Source: Wikimedia[/caption]
Actually, the polls were not so much off-target compared to previous elections. Hillary Clinton will win the popular vote by 1 or 2 per cent, two per cent below the predicted outcome – more accurate than in 2012 election, when the error accounted for 2.7 per cent. Clinton lost the election due to the Electoral College, namely because of Donald Trump’s victory in the Midwest. In all Midwestern states, Clinton underperformed the polling forecasts significantly – effectively, in all these states, the same polling error got replicated. Let me explain this.
Some of the major challenges in modelling an election outcome are voter turnout and the representative pool of people to reflect the demographics. Foreseeing voter turnout is incredibly difficult, even more so in exceptional circumstances such as this election. Pollsters ask the respondents whether they are registered voters, whether they voted in the past, and whether they will vote in the upcoming elections. With the corresponding results, they create a sample of likely voters to simulate an election demographic. What we have seen this year is a significant number of people without a previous voting history who have cast their ballot for the first time. For obvious reasons, these cases could not have been included in the forecasts. In fact, whites without college degree were clearly and somewhat surprisingly underrepresented in most state polls in the Midwest – and that specific group played a central role in determining the winner.
Besides sampling issues, another major issue for pollsters is how to reach likely voters in surveys, thus having an accurate representation of the demographics. Over the last decade, the response rate in surveys conducted by phone has dropped to single-digit numbers. Furthermore, the so-called “social desirability” factor plays a much bigger role in surveys conducted via phone compared to online surveys (which have other weaknesses, though). In a polarised election, many respondents may choose to not reveal their real choice on a call.
We still do not know who exactly voted until the voter file data, which includes information on the people who vote in each district and their voting history, gets updated. But what lessons can be learned so far from this election? Whilst it is actually difficult to solve these systemic polling errors, polling agencies have to be more critical regarding errors that other polling companies make and replicate them to align the forecast. Most of all, it is necessary to understand that uncertainty is a central part of the game. In elections, many things can happen and many factors may influence the final result – voter turnout in general or by subgroup, high numbers of late-deciders which swing overwhelmingly to one side (as it happened this year), and/or systemic polling errors.
What does this mean for the future? The worst is still to come – namely the French presidential election in 2017. Approached after the election of Donald Trump, French pollsters assured that their forecasts are much more reliable, for two reasons. First, because they allegedly have learnt from the surprising success of Jean-Marie Le Pen in 2002 when he entered the second round of the presidential elections. Second, because the voter turnout is much higher in France and therefore, the risk of hidden voters is much lower. This is quite a weak defence. The current leader of the Front National, Marine Le Pen, is much better than her father in appealing to the political centre and campaigns with a less extreme agenda, making it much harder to predict her performance. Besides, voter turnout may vary largely from the forecast even in France and social desirability could also be a bigger issue. In some predictions, Le Pen reaches 40% in the second round of the elections, and it’s not certain that such percentage won’t increase.
Furthermore, recent research (to be published by Purpose in January 2017) on the general attitudes of the public in France shows a devastating picture of French society, where pessimism, fatalism, and a deep sense of national decline prevail.
Donald Trump’s victory shows us that there is no glass ceiling for authoritarians. Jonathan Haidt of the University of New York explains how authoritarian attitudes are developed and triggered – largely through predisposition and by special circumstances. Both is incredibly accurate in the French context, due to a common feeling of decline as well as terrible economic and social circumstances. In half a year, we will now how much.
Vincent Wolff is an alumnus of the Blavatnik School of Government (Class of 2015) and is currently working at Purpose. He is working in the Refugee and Migration Hub and is leading the research on Germany.
[caption id="attachment_8545" align="alignnone" width="650"]
US general election polls 2016. Source: Wikimedia[/caption]Actually, the polls were not so much off-target compared to previous elections. Hillary Clinton will win the popular vote by 1 or 2 per cent, two per cent below the predicted outcome – more accurate than in 2012 election, when the error accounted for 2.7 per cent. Clinton lost the election due to the Electoral College, namely because of Donald Trump’s victory in the Midwest. In all Midwestern states, Clinton underperformed the polling forecasts significantly – effectively, in all these states, the same polling error got replicated. Let me explain this.
Some of the major challenges in modelling an election outcome are voter turnout and the representative pool of people to reflect the demographics. Foreseeing voter turnout is incredibly difficult, even more so in exceptional circumstances such as this election. Pollsters ask the respondents whether they are registered voters, whether they voted in the past, and whether they will vote in the upcoming elections. With the corresponding results, they create a sample of likely voters to simulate an election demographic. What we have seen this year is a significant number of people without a previous voting history who have cast their ballot for the first time. For obvious reasons, these cases could not have been included in the forecasts. In fact, whites without college degree were clearly and somewhat surprisingly underrepresented in most state polls in the Midwest – and that specific group played a central role in determining the winner.
Besides sampling issues, another major issue for pollsters is how to reach likely voters in surveys, thus having an accurate representation of the demographics. Over the last decade, the response rate in surveys conducted by phone has dropped to single-digit numbers. Furthermore, the so-called “social desirability” factor plays a much bigger role in surveys conducted via phone compared to online surveys (which have other weaknesses, though). In a polarised election, many respondents may choose to not reveal their real choice on a call.
We still do not know who exactly voted until the voter file data, which includes information on the people who vote in each district and their voting history, gets updated. But what lessons can be learned so far from this election? Whilst it is actually difficult to solve these systemic polling errors, polling agencies have to be more critical regarding errors that other polling companies make and replicate them to align the forecast. Most of all, it is necessary to understand that uncertainty is a central part of the game. In elections, many things can happen and many factors may influence the final result – voter turnout in general or by subgroup, high numbers of late-deciders which swing overwhelmingly to one side (as it happened this year), and/or systemic polling errors.
What does this mean for the future? The worst is still to come – namely the French presidential election in 2017. Approached after the election of Donald Trump, French pollsters assured that their forecasts are much more reliable, for two reasons. First, because they allegedly have learnt from the surprising success of Jean-Marie Le Pen in 2002 when he entered the second round of the presidential elections. Second, because the voter turnout is much higher in France and therefore, the risk of hidden voters is much lower. This is quite a weak defence. The current leader of the Front National, Marine Le Pen, is much better than her father in appealing to the political centre and campaigns with a less extreme agenda, making it much harder to predict her performance. Besides, voter turnout may vary largely from the forecast even in France and social desirability could also be a bigger issue. In some predictions, Le Pen reaches 40% in the second round of the elections, and it’s not certain that such percentage won’t increase.
Furthermore, recent research (to be published by Purpose in January 2017) on the general attitudes of the public in France shows a devastating picture of French society, where pessimism, fatalism, and a deep sense of national decline prevail.
Donald Trump’s victory shows us that there is no glass ceiling for authoritarians. Jonathan Haidt of the University of New York explains how authoritarian attitudes are developed and triggered – largely through predisposition and by special circumstances. Both is incredibly accurate in the French context, due to a common feeling of decline as well as terrible economic and social circumstances. In half a year, we will now how much.
Vincent Wolff is an alumnus of the Blavatnik School of Government (Class of 2015) and is currently working at Purpose. He is working in the Refugee and Migration Hub and is leading the research on Germany.