The woes of election pollsters
Another election, another polling failure. Why are pollsters so bad at predicting elections? In an age of political upset, why does the media keep treating the polls as gospel?
Pollsters have not covered themselves in glory in recent years. Whether it’s the abject failure of state polls in the 2016 US election or the underestimation of Labour support in June, pollsters have failed to accurately predict voting intentions both now and in the past.
The final polls the day before the June general election were as follows:
- Ipsos MORI – Conservative 44%, Labour 36%, Liberal Democrats 7%
- ICM – Conservative 46%, Labour 34%, Liberal Democrats 7%
- ComRes – Conservative 44%, Labour 34% Liberal Democrats 9%
- Kantar – Conservative 43%, Labour 38%, Liberal Democrats 7%
- YouGov – Conservative 42%, Labour 35%, Liberal Democrats 10%
- Survation – Conservative 41%, Labour 40%, Liberal Democrats 8%
- Panelbase – Conservative 44%, Labour 36%, Liberal Democrats 7%
- BMG – Conservative 46%, Labour 33%, Liberal Democrats 8%
The final result was Conservative 42%, Labour 40%, Liberal Democrats 8%, giving the Conservatives a lead of 2 percent. Of the pollsters, only Survation was close; no-one else predicted Labour would gain such a high percentage of the vote. For all the sophistication of modern pollsters, and decades of experience, there was an almost across the board failure to accurately predict the shares of the vote in the 2017 UK general election.
The problems are various and complex, and will no doubt be raked over in the coming months. But to me there are two major problems when using these polls to try and gauge how an election is progressing. The pollsters are commonly using samples of 1000-2000 people to try and gauge a result where 30,000,000 plus people vote. That is not impossible, as long as your sample is representative of the people who end up voting on the day, but the pollsters continue to struggle to get representative samples of voters.
The second major problem, which is definitely not the pollsters’ fault, is that the electoral system does not map average vote shares very accurately to numbers of MPs. The first-past-the-post system rewards those parties who win votes in the right places, and is particularly harsh to smaller parties, unless their vote is highly concentrated. A striking example of this is UKIP winning one MP for 3.8 million votes in 2015, while the SNP won 56 MPs for 1.5 million votes. This means even if the national vote shares shown by the pollsters were spot on the result would still not be clear, as it is the number of MPs that matter in parliament, not the number of votes.
All this being said, the accuracy of polls would not be such a problem if they weren’t so influential in the shaping of the election. There is an understandable thirst from the media and the public, to know the state of the race as it progresses; day in, day out. The problem is not only that the polls might be incorrect, but this is compounded by the media repeatedly misreporting them. Polls are stated as fact, methodology and margin of errors are not mentioned, and every minor change in the polls makes the front page, and is portrayed as if it is a major event, even when the changes are not mathematically significant. There are literally hundreds of lines of print attributing the latest twitch in the polls to a particular misstep or fantastic speech in the campaign. Political punditry is predicated on interpreting why the polls are showing what they are. Predicting the future is difficult, but demonstrating causality is nigh on impossible. Media coverage ends up being a statement of where we are based on misreported polls, informing a multitude of analysis of why that is the case and what each party should do to improve their position. A house of cards doesn’t adequately express the fragility of the conclusions the media lays out.
This ‘analysis’ in turn drives public opinion. People turn to the perceived leader in the race, seeking answers for what they see is happening, and this turns into narratives that are repeated time and again. These narratives begin to stick to the parties and candidates who can’t shake them off as they harden and become quasi-factual. Thus the, “Corbyn is unappealing to the public because he’s too far left” narrative, for example, becomes entrenched in political discussion, despite it being clearly incorrect.
Perceptions of the strength of candidates and parties matter massively in politics. While they will always play a part, as we thirst to know what is happening, the media has to stop giving polls such an outsized role in the political landscape, and use them with the caveats and caution that the pollsters themselves urge.