A few years ago, I had a brief discussion on Twitter with the pollster Keiran Pedley (then with GfK NOP, now with Ipsos-Mori) about what I believed to be the unfair practice of polling firms failing to include one particular political party in the main menu of options that respondents are provided with, even though other parties of similar size are included. He made the point that it wasn’t about ‘fairness’ as such, but instead about what experience had shown to be the most accurate approach – if including that party in the main menu consistently led to an overestimate of their support, it was absolutely justified to exclude them. I’m not convinced it’s quite as simple as that, because polls (absurdly) have a quasi-constitutional role these days – they’re factored in to decisions about the airtime each party is entitled to, and they supposedly will determine whether or not Northern Ireland is allowed a referendum on its constitutional future. When poll results are an integral part of the democratic process, it’s arguable that poll methodology needs to be fair and even-handed as much as it needs to be accurate.
However, at least in the example I discussed with Mr Pedley, the pollsters were actually attempting to estimate support for the party in question. If respondents indicated that they were planning to vote for “some other party”, they were taken to a second menu of options in which the party was included. A much greater problem occurs if respondents have no means at all of indicating their preference for a party – when all they can do is say “some other party” and it goes no further than that. If decisions about airtime are made on the basis of such a poll, there’s a gross unfairness, because no effort was made to measure the party’s support. And that, unfortunately, is the point we’ve reached with Panelbase polls of Holyrood voting intentions – the two that have been conducted since the election in May have not allowed respondents to express support for Alba in any form.
I can’t understand the rationale for that. Although Alba didn’t meet its own targets in May, it did secure 2% of the list vote, and for as long as any party is “troubling the scorer”, so to speak, you’d think it’s important to continue to know how well or badly it’s doing. Alba also of course has two Members of Parliament and a significant number of local councillors, which makes it of greater interest than most parties that receive 2% of the vote on their first outing.
All of this presents me with a bit of a dilemma, because I’m hoping to commission another Scot Goes Pop poll reasonably soon (funding permitting) and Panelbase would usually be my first choice – but I have a feeling they would want to maintain consistency by using the same question/answer format for Holyrood voting intentions in every poll they conduct, regardless of client. However, I’ll cross that bridge if and when I come to it.
On a more positive note, the Savanta ComRes poll published last Friday did include Alba as an option on the Holyrood list ballot, and 2% of respondents said they would vote for the party. That will be a great disappointment to the Alba-haters who gloated at considerable length about the Opinium poll published the previous day which showed Alba on zero for the very first time – a result that was taken to mean that “the monster had been slayed” and that Alba could expect to receive negligible support from that point on. The difference between the two polls is actually quite striking and hard to explain – in absolute terms, Opinium found only two Alba voters among their sample, while Savanta ComRes found seventeen. Here are the full ComRes results…
Scottish Parliament constituency ballot:
