Why it’s ‘impossible’ to solve the measurement issue

Marketers and publishers seeking a single source of truth in their analytics are ‘doomed’, Nunn Media’s managing director Chris Walton has argued.

Speaking at the final session of Mumbrella’s Publish conference, Walton was trying to answer the question of whether it’s time to start again with metrics. 

Mal Dale, Chris Walton, Annabel Hodges, Michelle Levine and moderator Zoe Samios at the Publish conference

Walton said his view was the existing metrics, however flawed, are fine as practitioners understand their limitations.

“Our role as an agency is to be healthy sceptics about what’s shown to us,” Walton said. “If it’s truth or not truth, I think it’s more important to be aware of the shortcomings of any one approach. With all the best will in the world, just about every approach to measurement has a flaw somewhere.

“It’s knowing what those flaws are and what to do with it,” he said.

“When I was at Quantium I had access to the NAB transactional database, 3m people and their spending habits. I could see from that the claimed ‘Have you shopped at David Jones in the next 12 weeks?’ results from Roy Morgan were overstated by 150%. Did it make Roy Morgan useful to us? It was still useful, but it was understanding that difference.

“When I look at EMMA’s figures I’m meant to believe each copy of The AFR is read on average by seven people. Do I believe that? No. But I take that and it helps me understand the relationship between the data and what is really going on and then, as an agent, make an informed decision.

“I remain healthily sceptical about all forms of measurement. I believe you simply cannot have, as an agency doing integrated campaigns, a metric, a single source of truth that can deliver what you want. It’s impossible, it’s an impossible goal to aim for.”

Walton added the current problems with Nielsen’s digital surveys are frustrating, but he’s still prepared to use them, saying:  “I’m not happy with some of the stuff I’ve seen out of Neilsen, I’m not shy in telling them that’s my opinion and I expect it to get fixed.

“The reason why we use Nielsen is that they are the accredited IAB currency and our goal is to work with currency data. As long as Nielsen is IAB currency data we will use them.

“I’m comfortable in the fact it’s imperfect knowledge. We use readership extensively, we use Roy Morgan data, we use all sorts of data, contingent on the fact you can game them. It’s very important to have your eyes open and understand what questions you want to answer.”

Roy Morgan CEO Michelle Levine added a broader view of the risks in relying on narrow sets of metrics and benchmarks, saying: “Australia is actually suffering from an explosion of mistrust – media, the banking industry, sport, health insurance, almost any industry we look at is suffering an explosion of distrust and a lot of it has to do with metrics. A lot of it has to do with the gaming of metrics.”

Summing up the challenge facing both advertisers and publishers with relying on metrics, Walton warned the quest for a perfect set of data is fruitless: “We’re never going to get to this utopian position in my view, it’s therefore making the best of we’ve got and being open in our view of shortcomings and opportunities and the where improvement and then moving ahead as it’s not going to get any easier.”


Get the latest media and marketing industry news (and views) direct to your inbox.

Sign up to the free Mumbrella newsletter now.



Sign up to our free daily update to get the latest in media and marketing.