What good are polls when they get you wrong?

In all the months leading to the US elections, we were awash with talk about the hazards of letting Donald Trump - President "bad-hair-day" - become Commander-in-Chief. "He's a sexist," people would say, "a racist, a moron, a blah-blah… and he'll start World War III just to settle some petty squabble." No doubt we'll find out now that he has won the White House.

But to think that his elevation would amount to a little war in itself; not even the pollsters saw that coming.

"We're talking in the aftermath of a little nuclear bomb having gone off, so the survivors are still staggering around, wondering what went wrong," says Ben Page, chief executive of British pollsters Ipsos MORI.

Clearly, a full, considered post-match analysis will have to wait.

But what went wrong exactly? The US voted in a democratic election and got what the majority - at least in terms of electoral votes - wanted. True, it was close. We knew it would be. Or so we were told by the pollsters. But no one, not even they, wanted to believe Trump would win. And that's what went wrong, horribly wrong.

"It's similar to Brexit in that there was a surge in turnout," says Page. "In both Brexit and the Trump victory, you see older working class voters turning out, who haven't turned out in recent elections."

This is especially problematic "where politics is unstable." And for pollsters, that's when their predictions become a judgement call about which factors and people to include or leave out.

"When somebody tells me they are certain to vote tomorrow but that they don't usually vote," says Page, "to have included them in previous general elections would have made my poll wrong. The difficulty in both Brexit and in this [US] election, is that by not including them we've made the poll wrong."

Hidden factors

We often overlook this fact that hidden in election predictions are a number of other nano-predictions. Pollsters don't only base their predictions on traditional telephone surveys or online questionnaires, but they are constantly having to make decisions about what and who is relevant.

"They have to make decisions about the demographic groups who will vote, so they come up with 'likely voter screens' which are based on past voting behavior," says Mark Kayser of the Hertie School of Governance. "And what I see this year as being really anomalous is the emergence of this white, rural group identity."

It's a group, says Kayser, which was always such as large majority in American politics that it didn't have to vote like a minority. But this time it apparently did.

"There wasn't this strong sense of a white identity before," says Kayser. "And that's probably what Trump fed off, and that was probably a problem for a lot of the likely voter filters that went into polling models in Michigan, Wisconsin and Pennsylvania."

Even if you include the right groups in your samples, though, there is a risk they will "shy away" from telling you the truth about their voting intentions. 

"That's the 'shy-Tory' phenomenon from the UK, that when people go behind the curtain and cast their ballots, they vote for the Conservatives but in public they wouldn't admit it," says Kayser, "and people have wondered whether there is such a thing as a 'shy-Trump' phenomenon, but it wasn't in the data."

It was rather, says Kayser, that the pollsters thought more Hispanics and women would come out to vote for Hillary Clinton, but they didn't, and so the pollsters got their models wrong.

What does it matter?

You could argue it doesn't matter what the polls say. What matters is the result. And I'd agree - up to a point. But there's also a case to suggest that polls can prejudice elections.

"Polls play a big role in the media," says Nico Siegel, a German pollster at Infratest dimap. "But the price you pay in a liberal democracy, in a market economy, is that you have suppliers on the polling side and people in the media who don't see polls so scientifically."

Say there are strong indications for one candidate, or an outcome in a referendum. If a media organization pushes it hard enough, even if only because it makes the better story, it may mobilize people on the other side of the divide. For good or bad.

Plus, ours being market economies, as Siegel points out, you get what you pay for, just as you do at the shops. Polls are more accurate the larger the sample of people you include. But including one hundred people is cheaper than one hundred thousand.

"Polls of a thousand or two thousand people will not predict the outcome of an election," says Siegel, "they reflect a general mood. Predictions can only really be made on the day based on exit polls."

Confirmation bias

So considering all this while, reading how Clinton would weather the FBI's new email investigation due to her lead in early voting, or about the damage done to Trump by the locker-room tape, were we wasting our time? When some of the final polls kept Clinton ahead by 7 percent, was that just a collective heads in the sand to protect us against the oncoming desert storm?

"I wonder whether we're all in this bubble of reinforcement, particularly with social media, and journalists interacting with their readers in a constant fashion, and interacting with each other," says Page.

If he is right, which he probably is, this will lead us further and further into a world of "confirmation bias."

"It's a challenge for pollsters because we can only learn by past events and when the whole structure of politics changes," says Page, "polls find it hard to pick that up."

Both Brexit and Trump are seen as lurches to the right, possibly even the far right. There are a good number of people who wish they had seen that coming… in the polls. And it's got some worried about two of the next big elections in Europe - France and Germany in 2017. France, for one, could get its first female president where the US just missed out - Marine Le Pen of the National Front.

So how do we improve the polls if social media is part of the problem? Will big data help? Probably not.

"In countries where classical polls - on the phone or online - don't work well anymore, there should be a combination of polls and some kind of big data or social media analysis," says Siegel. "But even with big data, you're right, it would be wrong to say you could predict an election 10 or even three days before the ballot."

You can't predict the unpredictable. Not where humans are concerned.