Skip to main content

Polling problems

A few days ago we saw that rare thing in Welsh politics: the results of an opinion poll on Welsh voting intentions, taken in September by Beaufort for Plaid Cymru.

Asked how they would vote in both a general and assembly elections, the results for those certain or very likely to vote were thus:

Plaid Cymru described the poll results as showing "...that it is Plaid as opposed to the Conservatives who are benefiting most from the decline in Labour support in Wales."

However, at this point we should heed the words of Welsh poll expert Professor Richard Wyn Jones:
“Opinion polls in Wales are strange things. We don’t have many of them, but what we know is that for whatever reason they tend to overestimate support for Labour and Plaid Cymru, and underestimate support for the Conservatives in particular. We don’t know why that is – perhaps it’s something to do with the way samples are taken, or perhaps because of the traditional anti-Conservative hostility in Wales there is an unwillingness on the part of some to disclose they are Conservative voters."

Indeed, my good friend Cai Larsen elaborates on the unreliability of Welsh polls, particularly Beaufort ones, on his blog.
"Beaufort's record is far from perfect when it comes to predicting election results in Wales.

For example, during the weeks before the 2007 assembly election there were two polls by the company.."

And here they are, put side by side with the actual result, demonstrating perfectly what Richard Wyn Jones had to say on the subject:

Cai continues:
"Now, when a polling company is consistently incorrect there is a reason - and that reason is defective methodology.

"I know nothing about Beaufort's methodology - and to be fair it's quite possible they have thought carefully about their methodology by using British polling methods - but it's impossible at the moment for the methodolgy to be effective. The reason for that is that there isn't enough polling here in Wales, and that polling isn't tested against real elections often enough. It isn't possible for all-UK methodology to work in Wales - the profile of voters is different here.

"An effective methodology evolves over time. For example MORI made a mess of predicting the result of the London mayoral election. Their interpretation of how this had happened is that people working in the public sector are more likely to respond to an opinion poll than somebody working in the private sector - and those people are more likely to vote Labour of course. Perhaps MORI's interpretation is correct, perhaps it isn't - but the point is that it will be proved or disproved in due course. If this additional consideration makes the poll more correct next time, it will remain as part of the methodology. If it does not, it won't be used again.

"I welcome the fact that Beaufort is polling in Wales, and I welcome the fact that Plaid Cymru are commissioning polls - but before Wales can have a system of credible polls there needs to be much more polls in order for a culture of effective polling to develop. The reason why that is not happening is because the Welsh media isn't greatly interested in commissioning polls. Until this changes I won't be able to put much faith in any Welsh poll I'm afraid."

In the comments thread to Cai's post, Alwyn ap Huw adds some additional details concerning Beaufort's methodology:
"Beaufort's problem is that the company doesn't do surveys of political opinion. The company does markeing surveys; the questions concerning voting intentions are a supplement to a questionaire on shopping and the profile of a likely shopper is a very different one to the profile of a likely voter.

"Plaid Cymru used the Beaufort company to hold surveys before the 2005 general election - surveys which showed that Plaid would easily regain Anglesey and which failed to predict the danger of losing Ceredigion. As the company had given Plaid such disastrously incorrect information on that occasion it has to be asked why Plaid continues to use the company. Money, it seems is the answer; a supplementary question on a Beaufort marketing survey costs a lot less than asking MORI to hold a real opinion poll."

To close I should also mention that the team behind YouGov this year set up its own online gatherer of Welsh opinion called WalesView. It has yet to publish any results. This post should give you some idea of the difficulty of the task facing YouGov in this field, but they should be congratulated for making the effort.

You can find more details on the results of the Beaufort poll here


mister_choos said…
I read somewhere about a similar problem in the US with black candidates. when asked, people don't want to appear racist, so say they will vote for the black candidate, but end up voting for the other guy. It cited a Californian governor election I think where the black candidate was well ahead in the polls, but got humped on polling day. This could, of course, be the McCain camp clutching at straws.

Popular posts from this blog


Your humble Devil apologises for his lack of posting: it has become increasingly difficult to actually put quill to vellum, as it were.

It's not purely that the political situation is rather uninspiring, it is also that I have become very much out of the habit of writing (about politics, at least). As such, every time that I fire up the blogging screen, I feel an incredible weariness.

I asked Pete to blog here because I thought that contemplating the actual mechanics of leaving the EU was important: I wanted to know, as much as anything. My reasons for voting Leave are actually very similar to Pete's, i.e. the rebooting of democracy and power structures in this country: however, he has a knowledge of the intricacies of the technical aspects that is beyond mine and I thought these worth setting down, here, for the record.

I shall try to post a little more frequently going forward. But, please, be warned that the reasons for eschewing this format haven't really gone away. My…

Gove's legacy?

Michael Gove has, quite honourably, said that it was right for Theresa may to sack him as a minister...
"I had six years when I was a government minister. I had a chance to make a difference - I hope that I did."The reforms that Michael Gove made in his time as Education Secretary will come to be seen as the most significant improvements to the British education system since the late 1800s—particularly in the introduction of Free Schools.

Gove made a difference—and his contribution should never be forgotten.