Online monitoring tool suggests public think Guardian and Fairfax headlines are more biased than News Corp and ABC

Shot of the headline worm to   date.

Headline Worm: Vertical axis shows percentage of people voting a particular headline as biased. Above zero is a headline seen to lean towards Labor; below zero towards the Liberals

Headlines on news stories from the Fairfax Media stable are more biased than those of News Corp, a new online tool measuring public perception suggests.

The Headline Worm has been created by technologist Nic Hodges, who stresses the tool is experimental and only shows the perceptions of the readers of the headlines.

Hodges, who is also head of innovation at media agency Mediacom, created the metric on his personal blog earlier in the month. So far more than 2000 people have used the headline worm ranking more than 7,000 headlines, with The Guardian and Fairfax’s headlines assessed as pro-Labor while the ABC and News Corp trending toward neutral.

In each case, users are shown a headline and introduction to a news story and asked to vote whether it leans to Liberal, Labor or is neutral.

Example of Headline Worm voting mechanism

Example of Headline Worm voting mechanism

Hodges told Mumbrella that the project was inspired by news coverage of the return of News Corp’s NY Post editorial boss Col Allan to Australia to lead the company’s coverage. It was widely reported that he was to lead a campaign for News Corp titles such as The Daily Telegraph, Courier Mail and Herald Sun against Labor in  the run up to the election.

“It’s one of the strange coding experiments that I have had in the back of my head for a while,” Hodges told Mumbrella. “The whole Col Allan arrival in Australia happened and it was interesting as an observer watching what was going on around the election.”

“I was interested in question of bias. I think it is clear that News Corp has a bit of bias in their print product but I was interested in whether that comes through as much in digital,” he said.

“So News (Corp) is at zero on the metric, which is surprising. That’s what I wanted to find out and to be honest it’s not what I expected.”

It does not tell them who published the article. So far News Corp is the closed to the neutral ranking. However, Hodges – noting its recent campaign against Kevin Rudd and Labor – says he believes the quantity of stories may be the reason for this ranking. “Part of the reason for that is the sheer volume of content News has produced,” said Hodges.

“The Guardian has not published the volume of content that News Corp has but then again an outlet like Fairfax has.”

The Headline Worm also implies that those who have voted see more bias across the board in favour of Labor.

“Fairfax and News are essentially on par with the amount of content but Fairfax’s content has been ranked quite left while News and the ABC are being ranked quite neutral.”

Hodge stressed that the Headline Worm is not a scientific assessment of bias in the election but rather  anecdotal assessment of how readers consume and assess online content.

“When you rank the headline you see the header and the slug which gives a bit of context, however, this was never meant to be a scientific poll – it was purely out of interest,” he said.

“When you look at the average time spend according to Nielsen, people aren’t reading full stories and often only read the headline and a couple of lines.”

The finding comes days after Fairfax Media launched a marketing campaign celebrating its independence.

Nic Christensen

Comments


  1. Will
    26 Aug 13
    2:33 pm

  2. What about an assessment of bias against? The News Corp coverage has been demonstrably anti-Rudd, but semantically that doesn’t automatically fit into “Pro-Liberal”.

  3. Anonymous
    26 Aug 13
    2:36 pm

  4. I love The Guardian but I go off them at election time as they become almost as screamy as News but on the other side.

  5. TheFacts
    26 Aug 13
    3:36 pm

  6. I was about to say this tool is a total failure if it finds Fairfax biased and News Corp not. Total failure. BUT if it is just taking digital into account that may explain it, to some extent at least. The print version of the Tele always seems to be far more in-your-face partisan than the articles on their website.

  7. Useless
    27 Aug 13
    11:10 am

  8. Data for the sake of data. It’s a bit like the “debate” that asserts News is merely expression the view of its readers. Or that it’s merely a bit of “satire”.
    Anyone who has worked anywhere near news media knows [perfectly well what is going on and it’s blatant.
    The only thing that saves News in this is that the ALP is so badly served by its representation. Not just now, but harking back – to Keating, Richo, etc. All aligned with News.And the ALPO is on the nose.
    As usual, Rupert has picjked the likely winner and is backing to the hilt to maximise the impression that “It’s The (Tele) Wot Won it”.

  9. KA
    27 Aug 13
    12:59 pm

  10. How does it feel to be a pawn of News now?

    Why bother with this stuff unless you make it with some scientific rigour?

  11. KA
    27 Aug 13
    1:02 pm

  12. TheFacts
    27 Aug 13
    2:02 pm

  13. ^ I reckon you are spot on Useless. It seems that by picking a target (preferably one they know is vulnerable) and bullying the heck out of it, they can then claim a ‘win’ and make themselves (and their readers) feel empowered. It seems like that is the business model.

    I also note that both The Australian and The Tele jumped on this story today. But as Crikey pointed out, any suggestion that these papers are neutral is laughable. Everybody is laughing.

  14. Nic Christensen
    27 Aug 13
    2:18 pm

  15. Hi KA,

    The Australian and The Daily Telegraph have both run stories on Nic Hodges’s tool however, it is worth noting that both of these stories were run after ours. That’s their right.

    I can’t speak for their coverage but our story makes clear that it is a limited sample and that the process was not scientific. That said as a blind test and a limited view of public perception is interesting and in our view newsworthy.

    Cheers

    Nic – Mumbrella

  16. Andrew Bolt & Gina Rineharts Lovechild
    27 Aug 13
    3:47 pm

  17. Dads pretty pumped that Mumbrella is endorsing the data. He was saying something to Mum about how this sort of rigorous scientific research proves that he (nor any one else) at News Ltd is biased.

    Mum thinks it’s all bullshit and all she cares about is how Tony promised her that he will gut the ABC. I think Tony is going to send in a crack team of Dad, Col Allen and Miranda Devine to “sort out those fucking communists”.

  18. KA
    27 Aug 13
    8:01 pm

  19. Nic, I’d be careful about chest beating. You don’t get points for being the first to promote an unscientific experiment. It doesn’t matter that Nic admits it’s unscientific or that it’s some clever application of technology. It just shows how un-newsworthy it is if it doesn’t add any value.

    Please, don’t spread this crap.

  20. David Roth
    28 Aug 13
    9:39 am

  21. Not statistically rigorous at all. Ever heard of selection bias? The people who respond to this sort of survey are self-selected. The results can’t be valid unless the selection is proper sample.

  22. Warren Bley
    28 Aug 13
    3:25 pm

  23. Just displays clearly the blind bias of those who used your tool!
    Fairfax biased and Murdoch’s rat pack neutral! – what planet are your participant on?!

  24. Collette Snowden
    2 Sep 13
    11:55 am

  25. The main problems with this ‘experiment’ are:-

    As noted in previous posts the sample is unknown, and may itself be biased

    What were the stories posted for assessment, and what was the context of the reporting?

    How ‘bias’ is defined and assigned is subjective – what exactly is meant by left and right?

    I take issue with Mumbrella’s defence of publishing this flimsy exercise in research, as newsworthy, especially when admitting to realisation that it is flawed. Do you normally publish inaccurate material? If so then the credibility of your approach to journalism is questionable. Poorly designed research yields poor results, but many readers take the reporting of it seriously, just sk any Doctor who struggles to communicate against the self-diagnosis geniuses with access to the Internet!

    If you can spot a serious flaw in research design and methodology then the results are invalidated. They may be interesting, but should not be promoted – that’s how confusion is spread.

  26. Klem
    11 Sep 13
    7:40 am

  27. 2000 people responded to a personal blog, I think the writer of this column is the monitoring ‘tool’ in this instance.