To help us understand this better we can look at results based on those who had previously heard of or experienced any of the campaigns (regardless of their level of engagement – which is a more difficult beast to measure). Basically we saw that previous knowledge of the campaign did significantly improve perceptions of it. Some (Blunty) more so than others (Facebook).

Cheers,
Scott

It would be really interesting if you could feature a story where all agencies nominate a spokesperson to talk about the campaigns – the learnings, the wins, the loses.

I also think your intro is quite misleading: “YouTube blogger Blunty was the favourite among consumers, new research suggests.”

When in actual fact it was a favourite amongst people who may or may not have even know or seen any of the campaigns. Rather the sample were sent links to investigate each and then asked a series of questions.

Honestly – this is a bit strange. It may work for traditional advertising but this is a different beast completely. Clearly the researchers were looking for a way to measure these campaigns but it misses the critical element the experience.

As mentioned above by someone, the competition was about engagement. So how on earth are these people in the sample group going to comment? They are viewing camaigns – one would assume (because we don’t know the date the research was completed) that are near complete. They haven’t experienced these campaigns as they have developed – which is half the fun and one of the main functions of social media.

They wouldn’t have received the tweets, Facebook status updates and so on that bring these campaigns to life or even the opportunity of engaging with the promotions.

For example: The Saatchi & Saatchi campaign dropped funny film clips to fans on Facebook (a Star Wars one springs to mind) to spur people on for ideas for their own short films, and One Green Bean’s effort of picking people up in a real Yaris with a Werewolf on the streets of Sydney was a real highlight! A physical interaction that brought the whole thing to life and allowed for a true real world engagement with the product itself.

So giving people a few links to look at when its all over, is a little bit like giving people pictures of amazing food to look at after everyone has eaten…..a bit two dimensional.

I understand that perhaps this was one of the only options available for research but lets not frame it as more than it is. How about a chance for each team to respond?

Cheers,

Saxon

None of means a thing if there’s no continuance of the strategy.

To call the cited research qualitative is simply ludicrous (though not nearly as ludicrous as calling a sample of 1,042 too small – I have seen several PhD academic papers published on less than 3% of that!).

A representative sample of a population is representative of the population from which it is sampled, nothing more, nothing less. Scott, and the Soup folks, from what I can see, do not make any claims further than this. Granted, the article may not have been clear as it could be on this point, but anyone with sufficient knowledge about MR would know to interpret the results within the necessary context.

Perhaps the best lesson is that the proper use of surveys in journalism is a skill that continues to need developing. Of course, it’s also a tender balance between an accurate piece (with all the tedious explanations of samples and methods) and an interesting one

We did not do this research to find a ‘winner’. We did this research to learn more about what works for social media campaigns: the chance to put 5 different social media ideas with the same objective side by side and get people to tell you what they think is very rare. The post campaign method reflects this research objective.

Tim’s article really just shows the tip of the iceberg in terms of insights we gained from this research.

We agree that a sample of opinion leaders is not a nationally representative view but a quantitative sample of over 1,000 does give you a valid and robust result for what opinion leaders think.

‘Winners’ are determined by the brief (which we were not privy to) but this research was about moving beyond ‘reach’ and asking people their opinions about a number of attributes related to the campaign.

So does it mean that Blunty won? Not necessarily.

To ensure impartiality we gave Tim the full results and Tim wrote a ‘who was the winner’ article and that’s totally up to Tim. But we’re happy to share the results with anyone who wants to get in touch to make their own assessment – like most things the devil is in the detail.

Just drop us a line. Scott@thesoup.com.au

You wouldn’t usually expect to pay bloggers – many would be offended at the suggestion. You pitch them an idea that helps them create interesting content. Then they’ll do it for free.
It’s often closer to PR than it is an advertising model.

And John,

Please do consider yourself our honorary Mumbrella watchdog (although I always liked to think of you having that role anyhow…)

Cheers,

Tim – Mumbrella

same thing happened with nielsen’s facebook numbers blunder of last year.

not enough seem to question data – which is weird, for an industry that supposedly prides itself on a robust data driven decisions.

Total budget for campaign $15K.

Toyota Yaris: $12,605 (for base line model not shown on ’09 model discount)
Blunty’s payyoff, Production, Angency fee: less than $2,400.
Total: Bullshiet.

Did I miss something? How is that the winner?

To explain. As a member of AMSRS and being a QPMR, I have a sworn responsibility and duty to ensure that research is not only conducted but reported to the highest standards. . Clearly, I take that responsibility seriously.

Maybe “Mumbrella Watchdog” as a new moniker?

But I do like the idea of there being a crime against reality… If we can organise a show trial, would you be willing to pass sentence?

Cheers,

Tim – Mumbrella

Must be a combination of boredom and jealousy as I don’t get any press anymore

Thanks for not “outing me” the comment was meant more tongue in cheek than an actual jibe, hope you saw it that way.

Cheers,

Somebody

However, to report QUANTITATIVE data from qualitative research is, in the world of serious qualified market researchers, seen as a heinous crime that does violence against the reality of the situation, irrespective of the sample size.

As a Qualified Practicing Market Researcher (QPMR) of the Australian Market Social Research Society (AMSRS) I will not be putting any credence to the numbers reported.

I’d answer that in two ways.

First, no, Scott Rhodie didn’t put me up to it – the first he’ll have known of this story was when he read it.

And secondly, your IP address is a familiar one. I’m not going to out you here, but it’s fair to say you’re not exactly a disinterested observer, are you?

Cheers,

Tim – Mumbrella

OF COURSE a cute lego film will appeal the most, but that isn’t the point.