Blog Post

Times Online Blocks Media Monitor Meltwater

Stay on Top of Enterprise Technology Trends

Get updates impacting your industry from our GigaOm Research Community
Join the Community!

Public relations news monitor Meltwater, which is still refusing to pay UK newspapers for crawling their websites, has now been blocked from indexing Times Online, the most serious of Rupert Murdoch’s UK newspapers.

The news site, which is due to go behind a paywall this spring and which had already blocked the NewsNow news monitor in January, enacted the block via the standard robots.txt protocol on Tuesday. It means thousands of Meltwater customers around the world won’t get to inform clients when their company is mentioned in The Times.

The move stems from an increasing desire, from both Murdoch’s News International specifically and the rest of the UK press, that commercial crawlers pay them to crawl their sites.

In January, the Newspaper Licensing Agency (NLA) – which is owned by eight leading UK national news publishers and, for years, had charged clippings agencies to photocopy their pages – introduced two new licenses requiring (1) that agencies pay between £5,000 and £10,000 per year to crawl online papers and (2) that agencies’ clients pay £58 a year to receive that intelligence.

Most such companies complied. News aggregator NewsNow tried to make the issue a matter of broad web principle with its Right2Link campaign. But it complied by removing NLA members’ stories from its pay-for service, leaving Meltwater – formerly called Magenta News – the only non-compliant agency.

The way the NLA sees it, news monitors are effectively making a copy of articles when they process articles to provide the categorisation and alerts that are so valuable to PR clients.

Meltwater, seeing differently, in December went to the UK’s Copyright Tribunal to challenge the fairness of the NLA’s new licenses – a ruling is still awaited, though the tribunal on Monday, a day before Times Online’s block ruled the Meltwater can go ahead with its challenge. The NLA is only charging crawlers that charge for a service, so Google (NSDQ: GOOG) is off the hook.

Murdoch’s News International is not exercising the NLA’s new licenses despite being an NLA member. But News Corp (NYSE: NWS) has, of course, been very keen of late to stress that users of its content, like search sites, should be paying it. Unclear how much News International is asking for, compared with the NLA’s £5,000-to-£10,000-a-year demand.

4 Responses to “Times Online Blocks Media Monitor Meltwater”

  1. Having read your item with interest, I’d like to clarify a few points we think may be a little misleading.

    You stated that Meltwater “is still refusing to pay UK newspapers for crawling their websites.”

    This isn’t entirely accurate. We have not refused to pay the NLA for crawling their members’ sites. The fee the NLA has asked for in this regard is actually relatively small, so it’s not the amount of money which is the point of contention. While we disagree in principal that such fee is needed, what we have actually referred to the Copyright Tribunal is the fee NLA wants to enforce on our clients.

    By asking our clients to accept the NLA’s request for them to have a licence to receive the links we send them, our clients – as you have pointed out – will face increased costs. However the figures you quote don’t give the full picture.

    The £58 annual fee you mention above is the absolute minimum. It actually ranges from £58 to £22,316 per client, and the cumulative annual fees facing Meltwater clients is more than £1m.

    You also say it is “unclear how much News International is asking for.”

    However, News International has not stated any claims for payment. They have simply taken the decision to block our access to the Times Online.

    We have, based on status quo, tried to get into negotiations to seek a mutual beneficial agreement, but they have so far refused to meet us.

    I, or one of my colleagues, would be more than happy to explain any of the above in more detail should you wish. In the meantime, here is a link to Meltwater´s response to the Times Online block.

    Best regards,

    Jens-Petter H. Glittenberg, Director of Business Development, Meltwater Group

  2. Anonymous

    “It means thousands of Meltwater customers around the world won’t get to inform clients when their company is mentioned in The Times.” – Robert Andrews – it seems you find it hard to hide your bias here. This sounds like you are almost crowing.

    “The move stems from an increasing desire, from both Murdoch’s News International specifically and the rest of the UK press, that commercial crawlers pay them to crawl their sites.”

    Well who says? You, or some anonymous source? If that were the case, how does News International justify blocking Meltwater and NewsNow, while not blocking or charging any other aggregator, whether free or paid-for? Since News International is not part of the NLA Web Licensing scheme, what on earth does it say that News International has blocked only the two publicly dissenters and not any other paid-for aggregators?

    Wouldn’t it be more correct to say that Dominic Young – the point man at News International – and co-incidentally chair of the board of the NLA – is pissed off that NewsInt’s share of the NLA’s profits is going to be hit by the Copyright Tribunal ruling that the NLA has to pay Meltwater’s costs, and so he’s throwing his weight around – because he can?

  3. charliejackson

    Call me a cynic but a typical knee jerk reaction by an injured dinosaur to the announcement yesterday that the Copyright Tribunal has ordered the NLA to pay the PRCA and Meltwater’s costs in their case against the NLA (
    As the major competitor to Meltwater in the UK, I should be euphoric that their business is being hampered in such a way and the fact that we are now profiting from an exodus of Meltwater customers enquiring about our services. Unfortunately this is not the case. The NLA, the modern day Dick Turpin, which the major 8 are using to rape and pillage the market are giving wildly inaccurate figures . The penal ‘tax’ to the end user is far in excess of the £58 sum mentioned in your article, in reality some of our larger customers (worth while charities and cash stretched health authorities included) are facing additional bills of many thousands of from the NLA. Meltwater are not disagreeing to the Aggregators fees and being licensed but are fighting for the right to maintain the privacy of their customers and for end users to search the internet and receive links to freely available news form the worldwide web without a link tax from the NLA. (Right2link campaign). The only reason that the NLA and Murdoch are attacking the media market place is that they see business as an easy way to fill their coffers and support an old fashioned business model that is failing to deliver the revenues required to support the lifestyles of the powerful media moguls. I challenge the Newspapers to have the Balls to put up pay walls to all and disband the much hated NLA.

  4. Yeah, you show em Rupert! That robots.txt file got teeth! That will stop content aggregators cold in their tracks!!!!

    Most content aggregators used the same technology that has been used to simulate “pay-per-clicking” where you automate the computer to act like a normal subscriber to pull in content. Most of these paywall news sites are not as sophisticated as Google recent algorithm or TicketMaster Captcha system to stop this type of automated content scraping.