We’ve been reminded this week that an 8-point lead in the dying days of an election campaign is about as safe as a 2-goal lead in the final 90 seconds of a playoff hockey game. Never take anything for granted.
Despite leading by between 2 to 9 points in every poll fielded over the past week, Adrian Dix managed to snatch defeat from the jaws of victory. It was a stunning result that no one saw coming – even though the exact same thing happened just one year ago in Alberta. In that campaign, Allison Redford trailled by 2 to 10 points in every poll, but still crushed Danielle Smith’s Wildrose Alliance on election night.
This has, of course, set off another round of polling post-mortems. I blogged about six possible polling error after the Alberta Surprise, and the issues are largely the same in British Columbia. So rather than rehash each point I want to look at the big picture.
We can quibble about things like question wording and ordering, but the largest problem cuts to the very core of the science of sampling – simply, polls are not truly drawn from a random sample of voters. I have no doubt if everyone was forced to vote and everyone was forced to answer the phone when pollsters came a calling, we’d see results within the margin of error. But that’s simply not the case, even though we pretend it is.
Indeed, only half of British Columbians bothered to vote yesterday. Admitedly, it’s difficult to figure out who is really going to vote in a world where 80% of people intend to…but then don’t bother showing up because they get distracted…or tied up a work…or because the weather sucks…or because the weather’s too nice to spend voting. There are ways to minimize this source of error, but it doesn’t appear polling companies made any effort to screen out unlikely voters or to gauge how solid support levels were. If they did, it wasn’t reported in the methodology, which is another problem in and of itself.
Moreover, there were warning signs the NDP was destined to lose the turnout game. Both Ipsos and Angus Reid showed the NDP and Liberals neck-and-neck among older voters, with the NDP up by 20-30 points among the under 35 crowd – a group notorious for their loud music, baggy pants, and low voter turnout rates.
The other side of the equation is that, sadly, not everyone is forced to respond to pollsters when the phone rings during Survivor. If you’re willing to spend the money, you can get a respectable response rate via traditional phone surveys, but all polls published during the BC campaign used either robocalls or online pannels.
Both of those methodologies have inherent problems. You often need to make 50 to 100 robo calls to find one sap willing to complete the survey. So we know Adrian Dix is popular with shut-ins, but extrapolating beyond that is risky. Moreover, since robocalls can only ask 5 simple questions before respondents drop off, you rarely have the opportunity to collect enough demographic information to judge how representative the sample is.
You can get those demographics using online panels, but while a national panel will have hundreds of thousands of Canadians on it, you’re fishing from a much smaller pool when you get down to the provincial level. You can always try to correct for demographic biases via weighting, but this can lead to a whole new set of problems. And it’s almost impossible to correct for attitudinal biases. The bottom line is that if you don’t have a large enough sample from Vancouver Island on your panel, you’re not going to get good data from Vancouver Island. It’ll be the same hundred people answering every survey.
Still, when different methodologies in different provinces keep missing the mark in the same direction, it feels like there’s something larger at play here. While the Clark and Redford miracles stand out, Jean Charest exceeded public polling numbers in 2012, as did Stephen Harper in 2011.
In all cases, voters had the opportunity to turf long-time and largely unpopular governments – then chickened out on change at the last minute. If an increasingly disengaged electorate truly is making up its mind more and more in the dying days (or hours) of the campaign, then a horse race poll is never going to predict the outcome spot on.
But maybe that’s not the end of the world. After all, superficial media polls are not designed to provide anything deeper than cheap entertainment. And where’s the fun in cheap entertainment, if the chance for a last-minute comeback doesn’t exist?
18 responses to “Chickening Out On Change”
The federal Liberals can’t count on youth or social media to actually deliver votes. As one pollster remarked about Trudeau’s lead, the Conservatives still lead among people who actually vote. BC does show that incumbancy and the economy are strengths for the Conservatives. The NDP can find solace in taking 39 percent of the vote, which would be a lot of seats in BC in a federal election.
Agreed. There are some lessons to be learned in this election for anyone who thinks the federal Liberals are going to coast to power. Given how incumbents have been performing of late, Harper is still very much the favourite in 2015.
National polling has tended to be better than provincial polling, as I recall, for whatever reason (larger samples?).
Not that I think winning in 2015 will be easy, either.
Scrutineering in False Creek (where the NDP’s star media CEO Matt Toner went down) the ‘shut in’ factor was obvious… all of the EBC workers were, quietly, rooting for us, and while disabled folks and seniors needing assistance came and went with a sunny, expectant air about them.
Seems fair to assume those were disproportionately our voters; then you had the Busy Angry Yuppy crowd, incredibly eagre to rant about how long they were made to wait, etc. because 10min while a poll clerk used the mens’ room was way too long “Are these people being PAID?!!”. Having myself experienced some actual inconvience for demcracy (i.e. a year in Kandahar), I could barely restrain myself from bellowing: “Lady, If you want customer service, try Sears!”
These were obviously BCLs… and there was way, way more of them…
This attitude is a major reason why the BC NDP lost. New Democrats like to demonize their opposition. They claim that everyone who opposes them is a huge asshole and that they are on the side of the angels. They imagine themselves in the sunny centre of the political spectrum, and from the relativistic vantage point, that everyone to their right must be a robber baron or crony capitalist.
To me, this smacks of a complete failure to imagine other people complexly. You can’t get people to vote for you if you don’t make some kind of effort to understand them, empathize with their concerns and needs, and work to meet them. You’ve basically dehumanized all the people who don’t agree with you.
Ah, but who cares. They’re just a bunch of Angry Yuppies, and they shouldn’t count.
Matthew hits the nail on the head. I live in Vancouver-False Creek, and despite having a pretty good track record of supporting the NDP, I have frequently found myself and my neighbours scorned by NDP members as “rich assholes” living in “Yuppieville”. The irony is the votes are there for the NDP in Vancouver-False Creek, but no effort has ever been made to seriously go after them. Seems like it’s much easier to tar everyone living there as an “angry yuppie”.
Which is a strange way for a people involved in a positive campaign to behave, I must say. (Also from an NDP supporter, provincially.)
I never understand the interest in polls; still, this was very interesting to read over. Thnx.
There are now media reports of polls which proved to be very accurate – the internal polls for the BCLiberals, which apparently projected a 48-seat win.
Also used stronger methodologies than the “loss-leader” polls published during the election period.
i just read about that in the National Post. apparently the BC Liberals were “certain” they were going to win a full 4 days before the end of the campaign.
I’m skeptical of that. The campaign certainly did not behave like a campaign that was expecting to win, right down to the tiny-as-hell room they’d booked on election night.
Other reports had the Liberals sharing their internal poll numbers only within the smallest circle of campaign strategists – maybe six people, according to some reports.
The folks arranging the election day party would likely not have seen the internal numbers.
There’s some data here: http://www.theprovince.com/news/Liberals+internal+polling+pointed+election+victory+courtesy+those/8403620/story.html
Although it only shows the age 55+ and female demographics in the graphs. Looks like maybe they did have some idea internally that things were going their way.
Booking a “tiny-as-hell room” is an old political trick. You always want the place to look like it’s jammed. There is nothing worse than the look of a big empty room, even if there are lots of people actually in it, viz. NDP HQ at the Convention Centre last Tuesday.
You do that to make a crowd look bigger. That’s not the same thing as booking a place for a victory celebration, which you would expect to have plentiful attendance (as the NDP did).
some of the numbers from internal Liberal polling released today – http://www.theprovince.com/news/Liberals+internal+polling+pointed+election+victory+courtesy+those/8403620/story.html
[…] CalgaryGrit (quite a polling fan and number-crunching enthusiast) has written a bit about flaws in some polling methodologies (in certain polls) which lead to woefully mistaken e…. Despite my total lack of interest in polls, I found this interesting […]
[…] CalgaryGrit (quite a polling fan and number-crunching enthusiast) has written a bit about flaws in some polling methodologies (in certain polls) which lead to woefully mistaken e…. Despite my total lack of interest in polls, I found this interesting […]