As I’ve spent a fair bit of the last 48 hours having exchanges (sometimes downright surreal exchanges) about this subject on Twitter, I thought I might as well make the point here as well. It’s a very simple one. Here is how the Britain Elects account reported the results of the new YouGov poll a couple of days ago…
I’m not having a go at Britain Elects specifically, because the above is absolutely typical of how most news/political outlets summarise such polls – ie. with no sign of the SNP (or indeed of Plaid Cymru).
How do you think any reasonable person would be most likely to interpret the absence of the SNP? I’d suggest they’d reach one of two conclusions. Either: a) respondents in the poll were not given the option of expressing a voting intention for the SNP, or b) the SNP were on less than the 2% of the vote enjoyed by the Greens, the lowest-placed of the five parties that were deemed worthy of a mention in the summary. But both of those conclusions would be completely incorrect, meaning that by either accident or design people are being very seriously misled. In reality, the SNP and Plaid Cymru received 5% of the vote in this poll, putting them in a clear fifth place ahead of the Greens, and only just behind UKIP in fourth place. (Because YouGov lump the SNP and Plaid together as a single option for GB-wide polls, it’s impossible to separate out the support for each of the two parties, but given what we know about their respective levels of support it’s inconceivable that the SNP would have received less than 4% if offered as an option in their own right, and would still have been well clear of the Greens.) Why, then, is the sixth most popular party reported as if it was the fifth most popular? Why is the fifth most popular not even mentioned at all?
A mistake?
An oversight?
Nope, it’s the intentional withholding of information, and it’s done as a matter of routine. Over the last two days, apologists for this downright weird practice have put forward a number of speculative justifications for it, and not one of them makes any sense. I’ll go through them individually.
“Not editing out the SNP’s vote would give a misleading impression of the trend in Scotland, because trivial changes that might barely register at Britain-wide level would be enough to make a big difference in terms of seats.” This doesn’t stack up, because essentially the same is true of both the Greens and UKIP – any seats that they might win depend on very localised contests, meaning that their national share of the vote is hardly even relevant. In 2015, UKIP took 13% of the vote but won just a single seat. If the media can ‘take the risk’ of revealing information about the popularity of the Greens and UKIP that has little or no relevance in terms of seats, it’s murderously hard to understand why the public must be ‘protected’ from similar information about the SNP. The bottom line is that in a first-past-the-post election, the number of seats won by each party is only very weakly correlated to the share of the vote. The winner of the popular vote may or may not be the largest party in terms of seats. A third party with 17% of the vote may win more than twice as many seats as it did a decade earlier with 23%. The purpose of polls is not first and foremost to predict the number of seats for each party, but rather to estimate each party’s absolute popularity in terms of votes. In that respect, the fact that the SNP is on 5% of the vote in this YouGov poll is no more or less important than the fact that the Greens are on 2% or that UKIP are on 6%.
“The estimated vote for the SNP is less reliable than the vote for Britain-wide parties, because it is drawn from a tiny subsample, not the full-scale GB sample.” Not true. YouGov allow respondents across Britain to select the SNP/Plaid as a voting intention option, as can be seen from the fact that the two parties between them have 1% support in London in this particular poll.
“Nevertheless, in practice the vast bulk of support for the SNP and Plaid comes from Scotland and Wales, so effectively is based on a subsample that is too small to be statistically reliable.” That’s really an argument for not taking individual subsamples too seriously, which indeed they shouldn’t be. But the SNP’s GB-wide vote is not a subsample figure – it’s rounded to the nearest percentage point and therefore normally falls in a range between 3% and 5%. If anything, the SNP’s reported vote is more stable than the reported vote for the Britain-wide parties and isn’t subject to random variations outside the standard margin of error – which is what you’d expect if the charge of an unusual level of statistical unreliability had any truth to it.
“The SNP’s support is not only effectively drawn from a small Scottish subsample, but one that might be incorrectly structured – for example, it might have far too many pensioners, or too many women.” Not so. YouGov indicated a couple of years ago that they had decided to start structuring and weighting their Scottish subsamples separately to improve the accuracy of their polls. It seems highly unlikely that they reversed that decision at any point, because their subsample figures have become (relatively) more stable since then.
“The SNP should be edited out of poll results because not everyone in Britain can vote for them.” That’s a British nationalist argument rather than a statistical one, but it doesn’t even make sense on its own terms, because not everyone in Britain can vote for UKIP or the Greens either. In the 2017 general election, the Greens stood in only 467 of the 650 constituencies, and UKIP stood in only 378 of 650. Both figures were sharply down on the candidates for each party in the 2015 election. Nobody has a clue how many candidates UKIP and the Greens will put up at the next election, which means that in all probability many respondents will have told YouGov in good faith that they plan to vote for one party or another even though they will not be able to do so. If reporting the SNP’s Britain-wide vote “lacks context”, reporting the Green or UKIP vote must inevitably lack a great deal more context. And yet nobody would dream of withholding that information (unless of course the numbers fell to a statistically insignificant level).
There is no possible logic to the exclusion of the SNP from poll summaries. It’s an arbitrary decision rooted in Anglocentricity.
* * *
Fundraiser: If you find Scot Goes Pop’s polling coverage useful and would like to help it continue, donations can be made via the 2017 fundraiser page. The initial £7000 target was reached last summer, but one year on that money has all been used up. I know there are always lots of very worthy pro-independence causes looking for support, so I’ve held off for as long as I possibly could before actively seeking donations again.
Related
