A new website called the Trust Tracker aims to provide ratings on the reliability of climate change media.

Tracking Trust in Climate Journalism

The history of climate change coverage is littered with problematic articles from writers who’ve paired weak science with canonical work, strived for a false balance, or relied on over-hyped single studies.

Climate Feedback, a website that allows climate scientists to annotate and provide commentary on the work of journalists covering the issue, was founded to address this very problem. The idea, in a nutshell, is to subject climate change reporting by the mainstream press to the scrutiny of peer review.

The team at Climate Feedback, which works to fact-check coverage of climate science in the mainstream media, plans to up the ante by evaluating not just individual articles, but whole publications.

The team at Climate Feedback, which works to fact-check coverage of climate science in the mainstream media, plans to up the ante by evaluating not just individual articles, but whole publications.

In practice, the website’s staff targets articles culled from old- and new-media publications around the globe — from The New York Times and The Wall Street Journal to Mashable, Rolling Stone, Forbes and Vox. These are then sent to a group of at least five invited climate scientists who frequently publish on the issues in question. “Scientific papers usually get two or three reviewers,” explained Climate Feedback founder Emmanuel Vincent, “so we’re significantly more than that.”

The scientists evaluate and annotate the article paragraph by paragraph, identifying what they find accurate and, more importantly, what they find misleading. Users of a special browser plugin can then view this commentary directly alongside the original article, and all of the annotations — as well as an overall “credibility” score ranging from -2 to +2 — are also published at the Climate Feedback website.

A recent piece on sea level rise by Justin Gillis in The New York Times, for example, received a rating of 1.7 — considered “high to very high” in scientific accuracy. A Wall Street Journal commentary that called global warming into question, meanwhile, received a “low to very low” score of -1.8.  A group of at least five climate scientists are needed to review each article. “Scientific papers usually get two or three reviewers, so we’re significantly more than that,” explains Vincent.

It’s a small bit of pushback against inaccuracy in climate reporting, but given the right audience, it has the potential to make a difference.

Still, Climate Feedback’s progress has been curbed by its small staff. The site chooses the articles they annotate carefully, based on the prominence of the outlet and the traction the subject is receiving on social media. But they can only fact-check a few pieces at a time, in a process that often takes several days. “Scientists just respond to one article at a time,” Vincent said “But at the speed that you publish, even in a day it’s almost forgotten.”

That’s why, following a successful crowdfunding campaign, Vincent and the Climate Feedback staff are gearing up to introduce the Trust Tracker, a new way of fact-checking climate change media. Unlike their piecemeal rating system, the new system will assign publications a composite score, based on all the reviews of articles from a single outlet.

It would look something like this:

After Climate Feedback has evaluated at least three articles from a given publication, the outlet will receive an overall evaluation, using icons and keywords like “insightful,” “accurate,” and “unbiased” at one end of the spectrum, and “misleading,” “flawed reading,” and “cherry-picking” at the other.

The idea is for conscious consumers of science news to check in with Climate Feedback first and see the rating for a publication before they digest the latest news. That way, the reader can approach an article in, say, Mashable — which receives a high rating on the Trust Tracker – with less skepticism than an article published by the Telegraph, which is labeled “misleading” on the site.

Vincent hopes to have the Trust Tracker running shortly. Meanwhile he’s been reaching out to science journalists for feedback, and has received mostly positive comments. (He was surprised, for instance, to receive praise for the Trust Tracker from an editor at The Wall Street Journal, which receives a low rating on the list.)

The Trust Tracker isn’t without risks, however. First, there’s the question of whether it can develop a large enough audience to have an impact on how readers view the accuracy of a publication. There’s also the risk that a few pieces of poor journalism could pigeonhole a publication permanently with an inaccurate rating.

But Vincent emphasized that the Trust Tracker is only meant to be a tool for readers, not a permanent sentence for publications.

“It doesn’t mean that everything that they publish in the future might be flawed,” says Vincent. “But it will hopefully raise a flag for the readers, so that they are extra skeptical when they read an article from a publication.”

This post has been updated to more precisely characterize the amount of time it takes for scientists to develop and deliver comments and annotations for articles targeted by Climate Feedback. It typically takes days, not months.