With four by elections tonight, the party spin machines are in overdrive, playing the expectations game. The best example comes from this article where the parties trip over each other to lower expectations – “we expect to lose all four” says the Tory spokesman with a smile, “if we beat the Marijuana Party it’s a win” proclaim the NDP, “we don’t expect a single voter in any of the four ridings to vote for us – they’d be insane to” the Liberals proudly exclaim.

So, regardless of what happens tonight, the results will be spun, hyped, and downplayed by each and every party. Half the newspaper columns and blogs will try to read something into them (Green Party heading for Maritime breakthrough!), with the other half saying that by elections are absolutely meaningless, the modern day equivalent of examining entrails to predict the future.

So which is it?

Let’s take a look at what the numbers say.

There have been 10 by elections since the 2004 election – that’s when the ridings were redrawn, the Liberal monolith crumbled, and the conservatives merged, so it’s probably a good place to start, even if it makes for a small data set.

What I did was use the by election and previous election vote percentages as predictor variables and the ensuing election results as the response. Doing this for the four major parties gave 34 data points.

So what does the data say?

For starters, the entrails are actually telling us something. Election results for a riding are more closely correlated to the most recent by election (r = 0.926) than to the most recent general election (r = 0.870). The best model to predict election outcomes is based about two-thirds on the by election and one-third on the previous election. So, at least in recent years, by elections have been able to give us an idea of the way the winds are shifting in a given riding, but they’re certainly not the end all.

Here’s another way to think about it. The following graph shows the relationship between the by election change (i.e. Liberals get 30% in a 2007 by election after getting 35% in 2006 – a 5% drop) and the general election change (i.e. Liberals then get 37% in the 2008 election – a 2% gain). You can definitely see there’s a relationship.

If you’re curious, the top right hand corner points are Outremont (NDP up 30% in 2007 by election, up 22% in the general), Roberval Lac St. Jean (CPC up 23% in 2007 by election, 6% in general), and Labrador (CPC up 17% in by election, 24% in general). So of those “take notice” by elections that really surprised people, in at least 2 of the 3 cases the results somewhat transferred through to the next election.

Along with Roberval, the two other by elections that were most out to lunch were also in Quebec. In Repentigny (2006), the Bloc gained 6% in the by election, but dropped 10% in the general. In Saint Hyacinthe (2007), the Conservatives picked up 12% in the by election, but the breakthrough cooled come election time and they finished behind their 2006 total. But in all three of these cases, the by election may not have been a bad thermometer, since it’s generally agreed that Harper crashed and burned in Quebec during the 2008 campaign.

So there may be some use in using by elections to get a sense of how a riding will adjust to the loss of an incumbent or to shifting political winds locally. But using them to create a national or provincial narrative? That’s where it becomes fool hardy to read too much into things. The Bloc was heading for a wipe out in 2008 if you only looked at by elections. Even the now-mythic Outremont by election turned into a bit of a dud – the Liberals lost the seat, but gained both votes and seats across Quebec in an otherwise dismal 2008 election.

So, I guess the short of this would be that there’s nothing wrong with celebrating by election wins – doing well tonight is a sign the parties will do well in these ridings next election. But as far as reading something into the national results? I have doubts about how useful that will actually be.

Which is why, of course, I will be doing just that starting at 10 eastern tonight once the results roll in…