May 7, 2015, was a banner day for Britain’s Conservative Party. After five years of uneasy coalition government, the Conservatives easily gained an overall majority in the UK parliament — defying the pundits’ expectations.
For the pollsters, it was an unmitigated disaster. They had predicted a dead heat, with polling averages suggesting that the Conservatives and their rival party, Labour, would each win 34% of the popular vote. Not one firm had put the Conservatives more than a single percentage point ahead. Yet when the votes were tallied, the Conservatives won 38% to Labour’s 31%.
Nine months later, a panel of political scientists and survey experts delivered a damning post-mortem on the pollsters’ performance. They considered several possible explanations: Were “shy” Conservatives lying about their voting intentions? Was Labour undermined by “lazy” supporters who couldn’t be bothered to vote? No, the report concluded, there was a more fundamental problem with the pollsters’ methods: They had polled too many Labour supporters. In other words, their polling samples simply hadn’t been representative of the real electorate.
It was an astonishing conclusion. The cardinal rule of survey research is that the sample has to be representative of the population you are interested in. How could the pollsters have screwed up so monumentally?
Quite easily, if you consider the perfect storm that has buffeted the polling industry in recent years. As the way in which we use our phones and the internet has shifted, it has become harder and more expensive to recruit representative samples — just as the media companies that commission most election polls have been hit by declining revenues.
“There’s nothing in the US that makes us immune.”
ID: 9676414
US pollsters face exactly the same pressures as their British counterparts. And while they’ve so far avoided major embarrassment, the next few years will provide a serious test of their ability to keep calling elections correctly.
“There’s nothing in the US that makes us immune,” Courtney Kennedy, director of survey research with the Pew Research Center in Washington DC, told BuzzFeed News.
Random sampling is how it used to be done.