So, there’s this from Mike Smithson and IPSOS-Mori:
https://twitter.com/MSmithsonPB/status/279216069661564929/
There’s this, from the Indy and ComRes.
There’s also this from Patrick “Unseasonably Mild” Wintour:
tp://http://t.co/70Ce8csl seriously what is the point of the Conservative Party asking questions as loaded as these – to check messaging ?
— Patrick Wintour (@patrickwintour) December 20, 2012
What’s going on? Well, I think this tells us something about how the coalition forms strategy. You’ll have noticed that they keep repeating certain visual images – alarm clock, curtains closed – as if they thought they were enormously compelling in and of themselves. I’ve not heard of anyone who heard this and said “Wow!”, but y’know, Pauline Kael effect and all that.
When Clegg started talking about “alarm clock Britain” I assumed someone had just translated Nicolas Sarkozy’s line about “la France qui se leve tot” freely. But then, just lately, I saw a quote briefed out by “Tory sources” saying that the image their focus groups associated with Labour was a run-down house with the curtains shut and full of black people…I made the last bit up. Not the rest, though. And I had a eureka moment.
What’s going on here is that they run focus groups to decide on presentation strategy, and then use that as policy. But the thing here is that it’s much, much easier to stick your thumb on the scales with qualitative data than with quantitative data. You can do it – click through to the private poll the Tories leaked to Wintour – but it’s harder and it’s much more obvious.
The appeal to private polls is rightly mocked – it’s the political equivalent of “the lurkers support me in e-mail!” – because you usually don’t get to see the question, the methodology, the sample size, or the crossbreaks, and therefore you can’t tell if the results are trustworthy. That is, of course, why politicians commission them and brief them to journalists, especially ones like Wintour. On this occasion, oddly, they did release more information.
But I have never, ever heard of a focus group that let even a word slip about how they actually picked the participants and what happened. Because they are environments in which the facilitator controls all the inputs, they are by definition very easy for the facilitator to influence. There are reasons why people who run them get described as “gurus” – it’s the sense of the word Peter Drucker used when he said that “guru” is spelt C-H-A-R-L-A-T-A-N.
If your aim is to convince the rest of your party that you are right, this is useful. It’s also a good way to test different marketing options against each other, because you can isolate their effect up to a point. If you use them as survey research, though, I suspect you will mostly come away convinced of whatever you believed going in. You will, in a nutshell, believe your own propaganda.
Now, Tony Blair was an expert in using policy-based evidence to sell his policies within the Labour Party. And we know that the coalition’s culture is a kind of Blairite Cosplay fetish. But you’ll note that although he certainly did end up believing his own propaganda, Blair usually decided on the policy and then created the policy-based evidence (rather, got Philip “not the Rugby League coach” Gould to create it) in order to legitimise it.
Narrative and imagery are tools of persuasion. They are not methods of inquiry.
I disagree strongly! narrative most certainly is a form of inquiry, and focus groups can and do give good results – to questions that they’re suited to answer, I should add.
Of course, I practice and/or teach (on any given day) qualitative as well as quantitative methods, so I would say that – but qualitative methods cna give insights that quantitative methods simply can’t. However… it’s insights: I always drum into students that focus groups can only be used for exploratory research; you can’t draw conclusions from them, only frame problems better for further, deeper research (interviews, observations or quantative methods, depending on circumstances). Yes, people who use focus groups as confirmatory (and good god, I’ve been asked to do enough of those) are just collecting ‘evidence’ to persuade someone, not doing research – but that’s on par with surveys such as the one you link to.
The problem is not that it’s easier to put your thumb on the dial of focus groups or of qualitative research methods, because I’ve also seen enough statistical analyses performed that managed to produce outcomes that were the opposite of what the data said (and all good focus groups publish both their selection criteria and processes *and* make available the primary data (videos and transcripts)).
The problem is that the research fetish in politics is about confirming prejudices; about appealing to fake ‘objectivity’ via use of some sort of misused research tool to justify what you were going to do anyway. Doesn’t matter if that tool is focus groups, or A/B testing – it’s just about creating a veneer of ‘evidence’ to justify what you were going to do all along. And yes, sadly there are market research firms how make it their business to provide this sort of ‘research’. In my experience, you can spot them by how close to Westminster/ The City their offices are.
Most of what are called “focus groups” in politics are not “focus groups” as social scientists would recognise them. The objective of social scientists in using this tool (getting a deeper understanding of a social phenomenon by getting a group to discuss it) is not what it is used for in politics. There is quite a lot of evidence of bias in the use of all techniques in politics and not all “focus groups” are biased (though many probably are): but often they are used for asking questions that really should be done through a large sample quantitative survey.
I have technical/philosophical problems with focus groups in just about every setting, but that’s a discussion for another day.
I agree with the other posters, the crucial problem here is the culture in politics of taking tools with known biases and declaring their output “evidence” or (see the Rob Marchant/Peter Kellner faction) “scientific evidence” that everything matches their preferred beliefs, perspective and strategy…
Permalink
Permalink