The Guardian questions ‘inconsistencies’ in Nielsen audience numbers
The editor of The Guardian’s new Australian website has questioned the accuracy of audience measurement company Nielsen’s Online Ratings, after the June results reported that its Australian audience had fallen in its first full month of operation.
The Nielsen results, released last Friday, reported that The Guardian had an Australian audience in June of 1.02m, down from 1.09m in the previous month. However, launch editor Katharine Viner said the reported decline does not match with their internal traffic numbers.
“It’s interesting to see Nielsen reporting a small decline in our traffic when our internal metrics have seen a substantial increase over the past month,” Katharine Viner told Mumbrella.
“We’ve had a fantastic launch on Guardian Australia, with our traffic levels rising substantially over our first month.”
The Guardian is citing internal data across its websites, by the Adobe-owned Omniture, shows a seven per cent rise in the unique Australian audience between May and June this year, plus a substantial surge in comments on the site.
“Our internal Omniture traffic figures confirm that Guardian Australia’s page views in June 2013 were up 10 per cent month on month and 57 per cent year on year,” said Viner.
“Monthly unique browsers were up by seven per cent month on month, 55 per cent year on year, reaching well over three million.”
“We also saw 33,700 comments on the Guardian Australia site, a 51 per cent month on month increase, clearly demonstrating the appetite for our ground-breaking open journalism approach in Australia.”
Each month Nielsen issues its Online Ratings which measures the size of Australia’s online audiences and is endorsed by the industry body the Interactive Advertising Bureau.
Matt Bruce, Nielsen’s managing director for media audience measurement, said that while Australian engagement on the Guardian’s site was up there data showed a drop in audience.
Nielsen’s hybrid data system involves both a consumer panel and tags on websites within the rankings.
“Online Ratings hybrid data shows increases in page views and sessions for the entire Guardian Media Group between May and June and a small decrease in unique audience. This suggests the Guardian has an audience engaging in the site more in June versus May as they are visiting the site more often and looking at more pages,” said Bruce.
Bruce also argued Nielsen Online Ratings metric and Omniture’s measurement system are not be easily compared.
“Omniture and Nielsen Online Ratings are used for different purposes – internal analytics in the case of Omniture and Industry endorsed online audience measurement currency in the case of Nielsen Online Ratings,” he said.
“The services differ significantly in methodologies which leads to differences in metrics.”
“Key differences in methodology include, necessarily, different implementations of tags and tagged pages amongst different clients for Omniture, as it is an internal tool, versus Nielsen Online Ratings where tags are implemented consistently across the market. Omniture may also rely on cookies for some metrics versus measuring people as is the case in Nielsen Online Ratings hybrid data.”
The Guardian’s concerns over their traffic numbers come only days after a number of publishers raised concerns about Nielsen’s reporting of News Corp Australia’s video traffic which surged between November and February before suddenly collapsing more 50 per cent last month, after the company instituted a metered paywall on many of its news websites.
Viner said the UK newspaper would be speaking to Nielsen about the methodology of Nielsen Online Ratings and watching for what it called “further inconsistencies”.
“We talk regularly to Nielsen about their methodology, and will be watching closely over the coming months for further inconsistencies,” said Viner.
Nic Christensen
Aha, the mysterious “panel” strikes again.
User ID not verified.
Hybrid has serious problems. Even if fed with NetRatings tag data, it doesn’t actually use this data. I’ve seen examples where Hybrid page impressions are greatly less than the tag data – so one of the systems is wrong in this scenario. And it doesn’t consider mobile usage *at all*.
No question Guardian are right here and Nielsen is wrong. The problem is that it impacts businesses significantly.
User ID not verified.
I guess there might be a less technical explanation for this?
I’m sure The Guardian’s traffic in AU will rise over several months but was their audience in May higher than normal because of the Lee Rigby murder on 22 May?
In the following days there was wall-to-wall coverage in AU and I’d guess a spike in unique audience for them.
User ID not verified.
Funnily enough I completed a Nielsen Radio Ratings diary last week. Apart from noting down my listening behaviour for a week there was also a quiz to be completed about other matters including which newspapers I had read in their paper and online versions. There was an “other” option but the Guardian wasn’t listed at all and there was no space to indicate which “other” you had read or visited.
User ID not verified.
Of course readership will go up, SMH started to charge a few minutes after The Guardian launched there site, smart bastards at the SMH, “I know, we have competition so why not start charging our online customers a fee”. No wonder they lose money, I won’t be going back.
User ID not verified.
Sounds a tad like yet another publisher not fully understanding the differences between traffic and audience. Stunning really.
User ID not verified.
Baz – go into your internet settings. Delete cookies and history, etc, and bob’s yer mother’s brother – no more “you’ve read your limit this month” on SMH.com.au . slight flaw in having to relogin in to various sites, but worth it for the pecuniary savings!
User ID not verified.
@Sharples, an easier way is to use incognito mode (Available on Chrome and Firefox) to browse news sites. You can use a regular window to stay logged into fb/twitter etc.
User ID not verified.
@John Sharples – or even better. Don’t log in at all!
Makes it hard for them to track your every move and sell it. Why give away your privacy?
User ID not verified.
Did it last week, also to all my friends, mobiles, tablets and laptops. But thanks anyway.
You can actually just delete the SMH site in the details menu while keeping all the others, just have to remember to do it each time you get the message.
User ID not verified.
I noted a change in geo-location code around the same time. I was previously reading UK content. I stopped reading when the UK content was replaced by AU content . I never bothered to work out how to turn the geo-location stuff off.
User ID not verified.
@Baz & John Sharples
No need to delete cookies – just open Chrome in incognito mode – The Age and SMH for free.
User ID not verified.
@Arthur – not really
Guardian report 7% rise in audience and 10% rise in traffic by citing a $2B+ analytics product that also segments audience.
Yet the industry endorsed Nielsen mark them down in front of their own customers on the back of a few surveys.
Hmm.. Fossil?
User ID not verified.
Arthur is exactly right.
Omniture measures the number of browsers (or cookies) that come to the site.
Nielsen use the Netview panel and tagged data to report an audience of people who got to the site.
First rule of online measurement: cookies do not equal people.
User ID not verified.
Oooooh…. Incognito mode!!! Thanks for the tips chapsters. Thought it was just for Pr0n while at work….. hehehehehhe…
User ID not verified.
@JB – not “exactly”
I agree cookies are not people. But neither is a panel comprehensive.
A digital publisher spends all day measuring performance data for each piece of content. They probably use multiple systems. And yes cookies can be deleted by users and yes different devices set different cookies so guesswork is involved. But over time, their site centric tools will allow statistical analysis against the entire data range. And the results of these server logs and statistically paired devices will give imperfect but better informed insights than a representational panel.
And of course that data is not public or standardised or even easy. And absolutely industry measurement by cookies is inherently flawed. But none of this is the point.
The point is that a publisher has the right to dispute representation bias when Nielsen sell down their audience in public
And if you’re going to brush off their dispute by asserting base misunderstandings between traffic and audience then you’d better have something more comprehensive than Nielsen panel data up your sleeve.
User ID not verified.
And you can force certain sites to always go to incognito mode – https://chrome.google.com/webstore/detail/ghost-incognito/gedeaafllmnkkgbinfnleblcglamgebg
That, or read your 30 articles on SMH, then 30 on The Age, then 30 on… well you get the idea.
Or even, shock horror, pay for it if you like it.
FWIW I find myself reading the Guardian less now it’s got an Australian edition. I used to read it for UK news and the international coverage, I find that the Aussie stuff isn’t as good and not as regularly updated. As the site defaults to Australia, then it being less regularly updated means I tend to visit less.
User ID not verified.
Digital, you make good points, but you don’t seem to understand the idea behind hybridisation (which is being used in the measurement of many media now – and will increase and become the de-facto standard).
Of course a panel will not project to the total universe. Even a statistically representative panel – i.e. balanced by geography, age, gender, OS etc. – will not because there are sections of the online universe that you simply can’t sample – e.g. schools, universities, public places, government departments, financial institutions, many large businesses etc.
This is why the hybrid system uses as its starting point the gross traffic data (as measured by tags), and then uses the panel to ‘convert’ this traffic to people.
It does this by looking at things like duplication, reach, frequency, and propensities. The good thing is that most of these metrics are a percentage i.e. somewhere between 0% and 100%. For example, you can look at the incidence of duplication of accessing a news site both at work and at home. Let’s say the panel finds it is 37%. We can then use that to adjust the traffic data downwards to account for the same person accessing the same site in two places. Also, the panel gives robust data on the incidence and frequency of cookie deletion to again adjust the traffic data.
In essence, if you rely only on tagged data you will get accurate traffic data (if done correctly – ever asked why different systems give different results?) but very poor audience data. If you rely only on panel data you will get estimates that are low and biased (the sites excluded referred to above).
As a rough guide, the tagged data will over a month inflate the market data (as expressed by monthly UBs)by around 500%. A large site may be inflated by 250% and a small site by 150+%. A panel will underreport the market by around 25% and sites at a similar rate.
So using hybrid techniques, you get the advantages of census traffic data and valuable duplication and usage data to produce the best available audience estimates we have to date. I stress that these are estimates (as are all audience data), and that trying to measure audience exactly is both a folly and a vanity. We’re also in the process of reviewing the whole hybrid procedure to fine tune things that will increase the accuracy.
Cheers.
User ID not verified.
The brand ‘Omniure’ doesn’t exist anymore. Please say Adobe SiteCatalyst instead
User ID not verified.