News Corp blocks search engine

Tim Burrowes

January 11, 2010 7:26

News Corp has for what appears to be the first time prevented a search engine from crawling a publicly accessible site in a move that was threatened by boss Rupert Murdoch late last year.

The company has targeted a specialised search player – the UK-based news aggregation site NewsNow – for the move, which has seen it institute the robots.txt protocol on its British Times Online site. The piece of code tells search engines not to crawl a site. In this case, it specifies that the block is for NewsNow’s web crawler.

The move was revealed in a statement from NewsNow.

In November, Murdoch threatened to use robots.txt to make News Ltd sites invisible to search giants such as Google.

However, as Duncan Riley argues in The Inquisitr, NewsNow makes an easier target than Google.

Tim Burrowes

Tim Burrowes is the publisher of Mumbrella and Unmade. He co-founded Mumbrella in 2008 and sister title Unmade in 2021. He's worked in the UK, Middle East, Asia & Australia and has specialised in writing about the communications world for the last 25 years. He wrote the book Media Unmade, published in 2021, and became the presenter of MediaLand on ABC Radio National in 2025.

Topics

Duncan Riley Google online paid content Rupert Murdoch The Inquisitr Times online

1 COMMENTS

Senior, junior or perhaps midweight 12 Jan 10

Meh. So what. Who is NewsNow? Noone.

Blocking aggregators is hardly unusual. Now, if it was Google, well that would be something to talk about. Will the Dirty Digger make it happen though? Or is he just all talk.

User ID not verified.