Chris Elliot from The Guardian’s “Open Door” column contacted me last week with a few questions about News Sniffer. I answered in detail and the article was published today. It quoted a tiny proportion of what was said, so I thought I’d publish my whole response here:

Hi Chris,

thanks for your email.

When I started News Sniffer several years ago, I wanted it to be a
resource for those discussing and analysing on-line news articles. As
News Sniffer makes clear, those articles evolve rapidly and that makes
it difficult to discuss them without having the virtual rug pulled from
under you. Some organisations try indicate when they change their
articles, but there is nothing more succinct than just highlighting the
exact differences as News Sniffer does. And we’re also independent, so
there are less concerns about changes not being declared, which is very
common.

The vast majority of changes are of course mundane, but it’s common to
catch things that were published before proper editing which can tell
you a lot about the journalists own ideas and biases.

Just recently the organisation Medialens reported on an article by The
Guardian’s political editor Patrick Wintour. The article was about
media bias towards George Galloway and featured some remarks they
describe as “elitist disdain for majority British views”, which were
quickly removed.

http://tinyurl.com/chltkrc

http://www.newssniffer.co.uk/articles/509152/diff/0/1

News Sniffer clearly shows the remarks disappearing about four hours
later but with no official note about their removal. Three hours after
their removal, an official notice is finally added. I like to think that
News Sniffer helped make that possible.

There are lots of cases like this that News Sniffer has picked up, and
I’m very happy to see it being used as a tool to demand more
transparency from our media.

If you have any further questions, feel free to drop me a line.

Feel free to quote this email. I’m hoping you’ll cover my comments of
your own company’s changed article, in particular how Medialens is using
News Sniffer – this would be a good signal that you’re committed to the
honesty and transparency that is so badly needed!

Kind regards,

John.

Chris replied letting me know he’d quote what he could and seeking clarification on why News Sniffer focuses primarily on the BBC and The Guardian:

Thanks, I am afraid your email is a bit too long to quote verbatim but I am happy to quote as much as I can. What you haven’t answered is why just the BBC and the Guardian, especially as the Guardian does largely footnote and explain as many others don’t?

I explained succinctly as I could, but none of it made it into article. As always, I’m sure it was for “lack of space”:

sure, understood, I didn’t expect verbatim. Glad you’ll try to get what
you can in though.

Monitoring all those articles (250k articles, 668k versions currently!)
takes up scarce resources, so I had to choose what to monitor carefully.
I’m particularly interested in pro-establishment bias and nobody really
expects The Telegraph, The Times or tabloids to seriously challenge the
establishment. The BBC and The Guardian are generally seen as part of
the “liberal media”, which many expect would be less biased in this way,
so they’re much more interesting to look at in these terms.

I do plan to expand it’s scope though and the source code is open so
anyone can contribute code to monitor any type of news article – they
could even run their own News Sniffers.

News Sniffer is now on the popular microblogging site, Twitter.

News Sniffer now tracks Guardian news articles for changes. You can limit searches to The Guardian (or BBC) by using the keyword source:guardian or source:bbc in your searches. By default, both sources are searched at once.

If you’re a Ruby programmer, you can add support for other news sources to News Sniffer by extending the Web Page Parser library, the code is available over there on Github. Happy hacking!

News Sniffer has just been updated to fix a bug where many BBC news articles were being missed (especially any sub category of world news). This started happening when the BBC updated their url structure a few months ago. Sorry that it’s not been caught until now.

I’ve given News Sniffer a huge overhaul – the culmination of over a year’s work (on and off part time). The biggest change is that we’re no longer checking for censored comments on Have Your Say. This was an intensive system to run and maintain and I just can’t dedicate the time and resources to it any more. The archive is currently removed but will be restored soon.

News Sniffer’s focus is now on monitoring changes to news articles on the web.

The whole news article version checking system has been overhauled making it much more efficient and easier to maintain. This makes it much easier to add features, particularly additional parsers for other news websites (we’re currently only watching the BBC). Watch this space over the coming months.

The check scheduling system is now much more reliable too, allowing more regular checks of more articles with less resources.

Due to a recent change at the BBC, Revisionista can no longer properly detect changes in articles. To fix this, I’ve brought forward plans for a new version of Revisionista. This should be up and running within a couple of weeks.

The new version will fix these problems, but also make it much easier to fix if things change again in the future. It will be much easier to add additional news sources too – so stay tuned for a great new Revisionista.

In the mean time, sorry for the inconvenience.

UPDATE: The BBC seem to have made more changes and now Revisionista is working again, thanks BBC!

The Register used News Sniffer’s Revisionista to track the changes of a BBC article on the ID cards.

The report’s revision history, documented by News Sniffer, takes us on an impressive Odyssey from “Smith to unveil airport ID scheme”, through “Shops may take ID Card biometrics” to the final sales-pitch version, cunningly deniabled with quote marks.

http://www.theregister.co.uk/2008/11/14/bbc_stories_cant_wait/

Both Revisionista and Watch Your Mouth have not been updating correctly for about a week due. Working again now. Sorry for the outage.

The BBC upgraded their forum software on the Have Your Say site and it broke News Sniffer. If you’re interested, they’ve changed from using the “Thread ID” in their urls to using a “Forum ID”. I’ve done a quick patch up to fix things, but any comments censored in the last week or so are lost to News Sniffer. It’sl going to take a while whilst things get back up to speed, but it should be done by tomorrow morning.

Thanks to the News Sniffer reader who spotted the lack of updates and got in touch with me. It was a relatively quick fix once I knew about it.

I’ve just added the BBC health feed to Revisionista so we’ll start monitoring more health related articles. So far, the only health articles included were from the World or UK feeds.