I wrote the Facebook My Screen report – this is the story behind it
Last week, amid claims of incorrect data, breached terms of service, and everything in between, Facebook and PwC pulled their My Screen report - which had been commissioned to provide a "balanced and independent view of the video market". Here, the report's author Ben Shepherd comes out swinging in defence of the report, and says if the My Screen report is "dodgy", then the entire industry is influenced by dodgy data.
For context, I was formerly a director at PwC and was the person responsible for the My Screen work that was commissioned. I am now the chief media officer at CHE Proximity and work across its large advertiser clients in Australia and New Zealand.
Ten months ago, I met with Facebook and posed to them a question. “Do you think marketers in Australia are clear on how video is consumed by Australian audiences – across all channels and modes of delivery?”
Our general feeling was – they weren’t. Research would either focus on pockets of consumption, or ad only, or others.
I put to them that I had an idea on how a single document could be distributed that sought to provide a wider view. I had heard so many incorrect claims around TV and video usage and they were all extreme in their views. “No one watches TV” or “no one watches digital video”, “people don’t pay attention to ads”, “everyone skips pre-rolls.” It was exhausting and old.
It would be a challenge, I told them, because there’s no single source of measurement and even disparity in time and ways things are measured. My view was, let’s use what we feel is the best available and spark a discussion. Let’s not position it as a single source of truth or a definitive piece. It was a jump off for discussion around how marketers navigate all different kinds, as well as understand and adapt to, the rapid rise of ad-free video.
My proposition to him is this is the sort of research Facebook should finance as a large participant in the industry. It is what marketers expect of them.
It needed to be independently produced and free from any influence. This was mandated in the contract and scope of work.
Facebook agreed. And PwC commenced work to produce this independent report.
The team at PwC spent hundreds of hours looking at Nielsen info, Roy Morgan, desktop research, OzTAM, overseas trends, PwC local and global pieces. PwC built a brand new panel of 3,050 people matched to the Australian population across metro and regional.
My team at PwC spent the better part of five months collating and analysing. What the PwC team wanted to understand was clear:
- The impact of ad-free video mediums such as Netflix and Stan across demographics
- The motivations behind the migration from linear TV to subscription ad free services
- Users’ content preferences
- Users’ device preferences
- Overlap and usage of BVOD, SVOD, online video and feed/story-based video
- Differences by demographic in terms of device, duration, content type, platforms etc
- Understanding of usage occasion by demographics
- Attitudes to services
- Drivers of preference and usage
Would anyone argue these are not areas marketers and advertisers would find helpful? And could anyone argue this information exists now? I would say no.
The report set out to explain these areas and my belief is it does. There are 50-plus pages of analysis and data across all these areas and independent analysis of implications for marketers.
My team at PwC also had to use varied data sets and suppliers, and the team knew this would be contentious. So, the approach was to “run towards the fire” and openly acknowledge this.
- For digital/web consumption, Roy Morgan is what has been used. Nielsen Digital Panel was initially used. Roy Morgan can provide web consumption data that provides de-duplication and demographic information, as well as user overlap. Nielsen also provides time spent and this we felt was important. The team knew Nielsen couldn’t measure Connected TV – and the report openly stated this within the first few pages, but there is no source of data that provides the above that does. Nielsen does however include desktop, tablet, mobile and app. And Nielsen has stated digital panel can include video. Nielsen refused to allow permission to use their data.
- For TV and SVOD consumption, Roy Morgan was also used. Why? Roy Morgan provides the only independently verified measurement of SVOD services.
- For TV, the team used the last seven-day figure on Roy Morgan. It was correct as we used Roy Morgan on all non-digital device consumption. This was used this only to show the sheer scale of TV. The figure used is as extracted from Roy Morgan
- For programs ‘love to watch’, the team used Roy Morgan – again, the only source of this and commonly used and validated.
- For top 20 2018 the team used OzTAM as printed in the Australian Government Screen Australia 2018 review.
- All other data was taken from the PwC-built panel.
Our team at PwC debated the idea of taking all data from the PwC-built panel, but we wanted to use existing and widely used sources to ensure the data was consistent with that which marketers and agencies use.
Every year, billions of Australian ad spend is influenced by both Roy Morgan, OzTAM and Nielsen Digital Panel. This is a fact. Using it is not contentious – it is in line with how the industry functions. If this is “dodgy data” then the entire industry is influenced by dodgy data. Media companies spend hundreds of thousands of dollars – if not millions – ensuring they are parts of these tools and included, as well as using them to convince ad buyers to spend with them.
In particular, it was interesting that Nielsen will sell access to use a service, but they don’t stand behind when it comes to usage. They weren’t confident enough in it – despite their own information stating it does measure video consumption on desktop, mobile, tablet and app, excluding Connected TV as per the below Nielsen graphic available online.
The limitations of Nielsen Digital Panel in my opinion deserve greater scrutiny, and as an industry there needs to be questions around how relevant and valid this panel – which is still actively sold and maintained – is for the future.
In the report to be released, all Nielsen numbers have been removed and Roy Morgan has been used across multiple dimensions instead. The reasons for Roy Morgan usage is consistent – it’s a panel and it allows the analysis required at the depth we believe marketers need.
Then there is OzTAM – the TV networks’ owned and operated measurement service. Good luck using these numbers too – they control all usage. You may “copy, reproduce, republish, or broadcast the VPM Content (or part thereof) to the extent required and as relevant to your ordinary business operation” says their website, but then they define what is or isn’t relevant use. I personally have received takedown requests from OzTAM for using their information previously – but selectively and only when the story wasn’t determined as positive to TV. This happened when I wrote a post on LinkedIn showing significant consumption declines in linear TV viewing across various demographics year on year (which were accurate). I’ve also been asked to take down VPM info taken from its public website. When I wrote a positive piece about Married At First Sight and its high VPM ratings the reaction was the opposite. Confusing terms of usage to be sure.
The report was not given permission by OzTAM to use OzTAM numbers either. Think TV has stated that “At Think TV, we’d rather spend time helping the industry better understand how to be as effective as possible when advertising on TV and broadcaster video on demand (BVOD)”, but this appetite for clarity and education doesn’t extend to providing access to data unless the content is to their liking.
The allegations that the data contained within the My Screen report is “dodgy” or inflated or made up is absolutely incorrect. As someone who has spent the better part of two decades advising clients on how to invest their money wisely and worked hard to push debate around transparency of fees, inventory, contracts and governance, I find the assertion disrespectful. The tools used were correct and the issue was reproduction rights not incorrect usage.
So much for wanting an informed market.
The report delivers on its intent, but the debate has ensured this is smothered. The report as read end to end paints a picture that shouldn’t be surprising to those in the industry from observation, but supports it with new data. When the report is released next week this will be evident.
The pieces that ran in the AFR and AdNews were also ones that clouded the debate. These deserve to be challenged. I am a huge reader of the AFR and have been since 1996 when I subscribed as a uni student. I read it daily and will continue to. I am convinced under their new owners the Nine Entertainment Company it will go from strength to strength. However they got it wrong here.
In response to the AFR 25 July, article written by Myriam Robin.
“It tried to argue that 17 million people watch free-to-air TV, with almost the same number using Facebook. But the TV reach figures, according to the footnotes, are calculated in the “last 7 days”. The social media figures were monthly.”
Yes, this is what the report released said and clearly acknowledged. This wasn’t argued, it was stated and sourced. The credit of the data was not in the footnotes – it was in the table, and then it was stated again in the surrounding paragraph.
“Facebook’s “single source of truth” used all sorts of non-standard metrics.”
This isn’t true. The metrics used are and continue to be industry standard. Nielsen and Roy Morgan, plus OzTAM data were used in the initial report. The report never positions as a single source of truth, so this quote has been taken out of context.
“And then there’s the fact that when considering the broadcast video on demand figures (that is, content aired by broadcasters live but available on their catch-up TV services as well), the measurement used by Facebook’s report excluded smart TVs.”
The report excluded Smart TV – and acknowledged this – as there is no way to measure it like for like with other digital properties. This is a legitimate marketer frustration – Connected TV remains cloudy around measurement at a granular level.
“Smart TV owners account for well over half of BVOD viewing.”
This number is printed without any verification. Free TV CEO Bridget Fair said in late 2018 this figure was 35%. This 50% ‘fact’ comes with no data to back it up and comments from the TV body CEO that contradict it. Confused? I am.
Think TV’s Steve Weaver also ran a piece in AdNews, dated 24 July. I appreciated the Mitchell & Webb references, but not the inaccuracies. (Note – the original of this piece referred to ‘Numberwang’ as a Peep Show reference. Peep Show is a TV show, but was the wrong reference point for this term. This usage of this TV show was incorrect and for transparency this is being acknowledged.)
“It’s clear some of the numbers have been used to inflate, some to hide and others to completely bury the real story about video consumption.”
“When numbers are used this way, it’s called numberwanging and if you numberwang too much you’ll be blind to real growth opportunities for your brand. You’ve got to dig below the surface look at all the facts before you go making any rash decisions about your media spend.”
This is just opinion isn’t it?
“According to Facebook, users spend 14 hours 42 minutes a month browsing on the platform while TV racks up 73 hours 51 minutes for the same time period. In terms of demographics, consider this: kids watch 40 hours of broadcast TV a month, teens 21 hours and 18-to-24 year-olds more than 25 hours, peaking with the often scoffed high-disposable income over 55 demo who watch in excess of 100 hours. That’s a sizeable chunk of video viewing time to consider when planning and buying media.”
This is what the report states data wise. No argument here.
“But to further dispel this notion, simply consider your own viewing patterns. Even the most avid fan is unlikely to re-watch their favourite show on BVOD after turning in for it the night before. But if you missed the broadcast, you’d absolutely be on your mobile the next day to catch up.”
Weaver is completely missing the point here. We agree at the individual show level BVOD is incremental to that particular show – the analysis was does BVOD as a platform (inclusive of all shows) bring new eyeballs to TV? New eyeballs who aren’t watching it or who watch it at very light volumes?
“Facebook would have you believe that only people over the age of 55 are watching TV. This argument has become de rigueur but it’s simply not accurate.”
This was NEVER claimed. This statement is grossly inaccurate and completely misleading. The report clearly states TV consumption for all audiences over 18 and clearly acknowledges the wider reach of TV at all demos. The variable is volume of consumption by age.
The reality here is there is a difference between data being incorrect or wrong, and some parties not liking what it says.
I stand by the report being accurate in the former and really hope that the latter isn’t what is driving the intense and aggressive take down of what I stand by as a very strong piece of work. If the industry wants robust and transparent debate then it needs to hear all sides and not throttle the distribution of information that provides clarity.
My view and PwC’s view was, and remains that, marketers needed a broad view and a broad collection of data to be more informed. This is a view I 100% stand by. TV absolutely is a channel marketers need to be invested in. The report backs this 100% and encourages advertisers and marketers to be prudent, diligent and do their research before they invest.
It’s a shame this whole piece of work turned into a bloody one-sided boxing match before it even had a chance to be read properly. Ultimately, the report once it is re released deserves to be read and the readers craft their own views. And I am confident it will be.
If we all want balanced, informed debate then surely we can allow this.
Ben Shepherd is the chief media officer at CHE Proximity. This piece was originally published on LinkedIn, and has been republished here with permission.
Nice response Shep. The industry is so quick with knee jerk reactions that suit whichever agenda they agree with (or get paid by). It’s about time people realise the digital vs traditional era is over. Let’s just focus on advertisers getting what they paid for and dodgy or imperfect data being called out. It’s impossible to get a single source of truth with so many agendas, methodologies and tech being used but let’s try get as close as we can, together, and make sure the caveats are clear.
User ID not verified.
So not even Nielsen believe the Nielsen numbers??
User ID not verified.
This gets even more strange e.g. “For top 20 2018 the team used OzTAM as printed in the Australian Government Screen Australia 2018 review.” There is not a single reference to Screen Australia in the finished report, and yet here they are presented as an impartial data source?
And I cant find any 2018 printed document from Screen Australia. Only thing I could find was their 2018 Year in Review which only talks about the top 10 TV dramas:
https://www.screenaustralia.gov.au/sa/media-centre/news/2019/01-22-2018-year-in-review-and-2019-preview
On their website they have top 50 TV drama EPISODES of 2018…again just narrative content:
https://www.screenaustralia.gov.au/fact-finders/television/australian-content/top-drama-titles
And why would PwC use OzTAM ratings via Screen Australia instead of just going to the source?!
So odd.
User ID not verified.
Shep
You just used the wrong data probably best to be upfront about that rather than blaming everyone else. What a mess for PwC.
User ID not verified.
Wow
User ID not verified.
Tea sipping gif… this is easily the best media fight of 2019.
User ID not verified.
Opinions are not facts. This wouldn’t have happened under Megan Brownlow. PwC advisory brand has really been tarnished.
User ID not verified.
This defence is rather lacklustre and if anything it has further deepened my skepticism of the entire exercise
I’m also surprised PwC doesn’t have a restraint clause on former employees…
User ID not verified.
Let’s get as close as we can (and exclude half the digital audience of television while we’re at it)… that’s not a caveat it’s a huge chasm and one that potentially invalidates key findings of research which claimed (according to Facebook) a complete view of the video landscape.
Also worth noting at no point does Ben actually explain what the issue was which lead to the report being retracted by his client. If it’s Nielsen (as he seems to allude) why weren’t the numbers checked with them beforehand?
User ID not verified.
The problem is not the numbers themselves – though they are cobbled together from different sources.
The problem is the way that they’ve been conflated and mis-represented so that PWC can sell a puff piece for Facebook.
User ID not verified.
Classic case of my effect here, Ben.
You weren’t associated with this report until now. Should’ve kept your head down.
User ID not verified.
To Ben for putting his hand up.
Didn’t need to and it shows serious character.
User ID not verified.
“at PwC our purpose is to build trust in society and solve important problems.”
The report fails miserably on both accounts.
How many Shred-X trucks can PwC fit in the loading dock at Barangaroo?
User ID not verified.
What is with these comments about excluding half the viewership of BVOD? The article points out that there is no data to back it up. And in fact the number that has been put out by a TV body was 35?
And yet Ben is the one accused of making up numbers?
Honestly, if you come at this with your mind already made up, what’s the point? As an industry we’re screwed if we can’t be evidence based.
To paraphrase Cuba “SHOW ME THE EVIDENCE”…. We’ve got a study here showing some evidence. If we disagree with it, surely we have to have some evidence to counter it, rather than just your opinion? Doesn’t that just make you a huge hypocrite?
User ID not verified.
If you commission a report via Facebook, you have to expect them to spin it and it seems like the issue stems from Facebook’s response in AdNews.
Their takeout that BVOD doesn’t add incremental reach to linear TV was quite naive – as ThinkTV said, why would someone watch the same show/episode on both? You would expect overlap in audiences. Does the original report discuss any of this?
User ID not verified.
Actually we don’t have some evidence here, as they retracted their report…
And yes TV industry should make their numbers available but are we really saying that we don’t think video viewing on Smart TVs and the large screens is a big part of BVOD consumption? Surely some of that onus the authors of the review of the “complete” video landscape would seek to look at and accurately quantify not just exclude.
User ID not verified.
Agree that many people are applying their own confirmation bias to this study. Good on you Ben for standing up for your work, there is no silver bullet and I think your ambitious study made that quite clear.
That said, there were some things that didn’t sit right with me:
1. FB claiming the biggest reach when it is not a 100% AV platform. If I was Google, I’d be really salty. I highly doubt video minutes consumed on FB’s family of apps outweigh minutes consumed on YouTube. Hoping the report did factor that in rather than taking that reach number out of context. Happy to be proven wrong here.
2. There was no delineation between personal and shared screens that naturally influence reach and attention quality. SVOD was the winner in that report, but there wasn’t much discussion around co-viewing and shared screens – are those numbers potentially even bigger? Given FB does not play in TV at all, I do think this is important.
3. No clear view on fast growing screens like CTV. There is existing research on this area but it would have been good to understand from this report – how many consumers in Australia have pulled the plug on linear TV? What % of Aussies are live streaming? With YouTube ramping up in CTV, it isn’t just BVOD/SVOD vs linear, it’s AVOD too.
At the end of the day, a big tech giant is paying good money for reputable research and PWC can’t exactly serve their client an unpolished turd when it comes to stats about their brand. Rather than complaining, everyone should disregard the FB is #1 slides and take the valid points from the research. We’ve never had a silver bullet solution so why is it such a big surprise that we need to apply critical thinking to what we are reading?
User ID not verified.
Many in this debate have conducted themselves rather shabbily in my view that it is little wonder those in the industry are as trustworthy as car dealers.
The report is not without flaws and shows such callowness from Pwc to believe it could be done with the data available. Or that any report using data that may be uncomplimentary to TV would not attract a rabid defense from the cosy media selling and buying groups
User ID not verified.
Another case of the accountants trying their hand at advertising!
User ID not verified.
I read, and still have, the original version. It’s flawed in way more ways than just the Neilson data. I guarantee that anyone with more than 5 years in media would identify 5 major defects.
User ID not verified.
This has all the ingredients of the EMMA vs Roy Morgan debate on newspaper readership.
Wind forward 6 years and it’s all about SVOD.
Although it looks like this time around Roy Morgan is the winner.
User ID not verified.
You are right. This report is a hack job, which shouldn’t see the light of day, even with the latest revisions. If they couldn’t be bothered contacting the relevant data owners directly to source the required information, then they shouldn’t try and position the report as “a more informed view of the video marketplace”.
User ID not verified.
No disrespect to the author but this is a brave article. I read the article multiple times to be completely subjective about the topic but despite of the topic being about the report, be it “dodgy” or not, all I could really grasp was “I wrote this article to tell you I sold a report while employed from another company, no one likes it, so I am selling it again but I don’t work for that company anymore but here I am anyway, I’m still selling it”.
I get PwC’s business but if we really want to drive a step forward, which was what the report meant to be, should we not just rally the key stakeholders without having to sell something to a company? The bottom line, IMHO, is the report was commissioned (i.e. sold) as the employment’s KPI, nothing more, nothing less. Arguing the belief – re: the value and credibility of the report – is kind of diluted from that perspective.
User ID not verified.
The defense of the report was about as long as the report itself – which is a pretty telling sign of a cock up by PWC, the author and FB.
Wouldn’t it be better to just own the mistake and clearly flawed logic in its methodology, as well as indicating it was always going to heavily bias / overstate FB and PWC’s impact?
Agree with “Fail Fast” – Shep, admit the stuff up and move on – rather than egotistically defend what is clearly a gaping hole of a report and ultimately shitty PR endeavor for PWC. Hopefully a lesson in not getting drunk off the PWC coolade and remaining continuously objective and critical in the formulation published work… before publishing.
User ID not verified.
For the last few years , media agencies have been putting together video plans combining FB, YT, TV & VOD. What is alarming is some of them are guessing the right mix of investment as they don’t have a viewpoint on combined reach and frequency. One thing is clear despite some of the facts not being clear – media agencies have over invested in channels like paid social without understanding its contribution to the media mix / weight of campaign. This push and overemphasis for the newer non linear TV. Channels has been driven by much bigger mark ups for media agency profit – as well as creating an overly complicated martech ecosystem – that make it easier to hide very high profit margins. In fact this is only ever rally spotted when the business moves. I saw a newly won client was paying 8x the rate they should be paying to FB. To name only one of many. Clients need to ask tougher questions of their media agencies. Media agencies need to own the research / tool to work out the right levels of spend and the mix. It shouldn’t be left to anyone with an interest in selling airtime to a client.
User ID not verified.
Bring back Megan Brownlow. This current pwc team doesn’t care about good work just $$$$
User ID not verified.
Why is the Pwc partner Justin Papps not saying anything? Shouldn’t everything Ben has been saying be commercially confidential? Is this how Pwc will treat my data??
User ID not verified.
The YouTube numbers shared by Nielsen in a rival website showed YouTube at 17M viewers per month and 20h+ viewing on average.
The report had them pegged at significantly lower, probably enough to get legal folks excited.
User ID not verified.
Megan certainly managed to not rock the boat, but what did she really achieve for the industry versus good branding PR for PwC.
I always found their reports too polite and politically correct.
User ID not verified.
I haven’t seen this report yet, but . . .
Surely this work deserves a place in the sun. Ben clearly sets out its intent and potential defects. This should be a breath of fresh air in a stagnant pool of regressive heavily media owner controlled, so called currency data from OzTam, Nielsen, ThinkTV, IAB etc. Lets face it FreeTV and media agencies are doing it tough and any independent research that seeks to provide new insights that reveal declines in their popularity will be heavily criticized by them. It directly impacts their main traditional revenue stream of agencies who are the lifeblood of FreeTV.
This pattern of ‘kill it off’ criticism reflects the head in the sand, ‘we control the world’ attitude that has brought them to this unhappy place. (They do the same thing trying to kill of the ABC, the countries most trusted news source)
With all this hoo haa i reckon it deserves a hard second look!
User ID not verified.
Pathetic! Someone tries to shine new light on video consumption, albeit imperfect, and all the old guard vested interests do their best to kill it off. Nothing has changed!
User ID not verified.
…he is protecting his own brand, not PwC.
This would be eating away at him.
User ID not verified.
Nothing to do with ‘the old guard’, it’s to do with a clearly flawed approach creating a skewed report that never should have seen the light of day, and hubris it takes to then try and defend it despite the clear and glaring problems.
User ID not verified.
PWC: Pricey, Worthless, Clowns
User ID not verified.
The author exposes an open secret – Nielsen’s data is crap and they know it’s crap. Plus the point about heavy handed terms of use of the data is one I’ve personally experienced with Nielsen too. It led to my company terminating our contract with Nielsen a number of years ago, and their behaviour in that period was execrable – so much so that I’ve made it a kind of sport to torpedo the Nielsen subscription at every organisation I move to. So far I’m 4 for 4.
User ID not verified.
Wow, you are so powerful Fundy. We are in awe of you
User ID not verified.
Much authority, such power!
User ID not verified.
@Fundy – ‘Most virile, I am Morgan made death. Behold my power and kneel before my notice to terminate’
User ID not verified.
Hey thanks for the feedback, much appreciated.
User ID not verified.
Groan O wins the thread with this one.
User ID not verified.
This is why I support the continued anonymity of Mumbrella comments…. top notch copy like this.
User ID not verified.
so, an imperfect (i.e. incorrect) report is released and you think that we should give it more support because it is ‘new’?
what strange times we live in…
User ID not verified.