Behind the smoke and mirrors of market research
In this guest post, an anonymous researcher questions the motives of the market research industry.
In the surprisingly litigious wake of the McCrindle debacle, I feel it necessary as a long-time researcher to voice my anonymous opinion regarding the legitimacy of market research.
To preface this article with a story: I once saw the MD of a research agency get up in a strategy meeting and doodle out a mathematical formula on the whiteboard, all of it weird Greek letters and unfathomable symbols.
He explained something about an analysis of a variance statistical model and strangely, I didn’t get any of it.
Why? Because he didn’t either; he was making it all up on the spot.
I’m pretty sure the clients didn’t understand it either, but unfailingly nodded with cow-blank expressions and busily took notes – and lo, the not-insubstantial contract was won.
This is particular case is extreme and an obvious outlier, however, having been around for a while, and having witnessed varying degrees of swindle, this type of thing goes on all the time. To be clear, I should stress that the issue raised by Media Watch about small sample sizes in the McCrindle Research in question, is not something I would describe as one of those swindles, although the publicity has been unhelpful for perceptions of the industry.
The odd thing is that except for a few times, such as when Media Watch happens to look up the research industry’s skirt – no-one cares.
And this is my probably wholly unwelcome defence of market research; that there is an unspoken B2B contract between the agencies and their clients.
Clients know surveys and focus groups aren’t hard science. If they were, the agencies would issue white lab-coats and clipboards as a standard uniform. You know who does wear white lab-coats? Actual researchers, do, while carrying out actual research, in an actual lab. They’re looking for things like, the cure for cancer, or less prestigiously, a better way for aging conservative males to maintain an erection, there are probably more than a few on the books of BP looking for a more efficient way to murder doe-eyed baby seals.
They’re not asking a group of fat middle aged women what they’re looking for in a chocolate bar, discussing which ridiculous car ad, most ‘empowered the consumer’, and they’re certainly not concerned with what Generation Y thinks of Kyle or Jackie O’s latest carefully structured PR ‘disaster’.
No, because all that is ‘marketing measurement’ maybe ‘consumer insights’ even, but not research. Yes, actual research does measure things, and will sometimes employ surveys, but to elevate market measurement to the status of research for this reason would be akin to giving a monkey a driving licence simply because it could beep the horn.
I’m pretty certain most (if not all) the clients out there understand that generally, few market ‘researchers’ have PhDs, or even care to understand statistics beyond the basics. I do distinctly remember a client who didn’t understand this, someone in charge of thousands of jobs and literally hundreds of millions of dollars, once briefing me, and I quote:
I read about choice modelling. What I need is for you to do that and just make a thing that will predict market movements next month, quarter and year, but it has to be correct within 1% of what actually happens – your company will be liable for any costs incurred by the failure of the model.
Now if I could do that; predict the future with a mathematical model, I’m fairly certain I wouldn’t be sitting in a briefing meeting with a man in a pink shirt and an intimidating tie. I’m pretty sure I’d be applying it to the stock market, living on my yacht and spending the rest of my brief, but thoroughly enjoyable life, up to my armpits in tequila and Tahitian princesses.
And yet, often, ‘researchers’ are trotted out, time and time again (usually when there’s budget to be spent or something has gone very, very wrong) and lauded as ‘insights gurus’ or messianic business-prophets. Regarding McCrindle, I would be very surprised if the actual count of respondents would have mattered at all to the public’s perception of the inferences made from the data.
But what it did do is attack this counterfeit-scientific legitimacy that agencies are desperate to maintain – which is why, I suspect AMSRS were investigating Mr. McCrindle, and he in turn looked to the courts.
Clients get that agencies probably can’t predict the future, but that’s not what they’re paying them to do. They’re paying them to be generally better with numbers and PowerPoint, and lets face it, carrying out surveys or focus groups is a massive pain in the arse.
The sooner agencies drop the charade of imagining they’re scientists or psychics, the sooner everyone can get on with their jobs and stop taking each other to court on ridiculous pretences.
- The person who wrote this still works in research