Tracking Trust in Climate Journalism
The history of climate change coverage is littered with problematic articles from writers who’ve paired weak science with canonical work, strived for a false balance, or relied on over-hyped single studies.
Climate Feedback, a website that allows climate scientists to annotate and provide commentary on the work of journalists covering the issue, was founded to address this very problem. The idea, in a nutshell, is to subject climate change reporting by the mainstream press to the scrutiny of peer review.
The team at Climate Feedback, which works to fact-check coverage of climate science in the mainstream media, plans to up the ante by evaluating not just individual articles, but whole publications.
In practice, the website’s staff targets articles culled from old- and new-media publications around the globe — from The New York Times and The Wall Street Journal to Mashable, Rolling Stone, Forbes and Vox. These are then sent to a group of at least five invited climate scientists who frequently publish on the issues in question. “Scientific papers usually get two or three reviewers,” explained Climate Feedback founder Emmanuel Vincent, “so we’re significantly more than that.”
The scientists evaluate and annotate the article paragraph by paragraph, identifying what they find accurate and, more importantly, what they find misleading. Users of a special browser plugin can then view this commentary directly alongside the original article, and all of the annotations — as well as an overall “credibility” score ranging from -2 to +2 — are also published at the Climate Feedback website.
A recent piece on sea level rise by Justin Gillis in The New York Times, for example, received a rating of 1.7 — considered “high to very high” in scientific accuracy. A Wall Street Journal commentary that called global warming into question, meanwhile, received a “low to very low” score of -1.8. A group of at least five climate scientists are needed to review each article. “Scientific papers usually get two or three reviewers, so we’re significantly more than that,” explains Vincent.
It’s a small bit of pushback against inaccuracy in climate reporting, but given the right audience, it has the potential to make a difference.
Still, Climate Feedback’s progress has been curbed by its small staff. The site chooses the articles they annotate carefully, based on the prominence of the outlet and the traction the subject is receiving on social media. But they can only fact-check a few pieces at a time, in a process that often takes several days. “Scientists just respond to one article at a time,” Vincent said “But at the speed that you publish, even in a day it’s almost forgotten.”
That’s why, following a successful crowdfunding campaign, Vincent and the Climate Feedback staff are gearing up to introduce the Trust Tracker, a new way of fact-checking climate change media. Unlike their piecemeal rating system, the new system will assign publications a composite score, based on all the reviews of articles from a single outlet.
It would look something like this:
After Climate Feedback has evaluated at least three articles from a given publication, the outlet will receive an overall evaluation, using icons and keywords like “insightful,” “accurate,” and “unbiased” at one end of the spectrum, and “misleading,” “flawed reading,” and “cherry-picking” at the other.
The idea is for conscious consumers of science news to check in with Climate Feedback first and see the rating for a publication before they digest the latest news. That way, the reader can approach an article in, say, Mashable — which receives a high rating on the Trust Tracker – with less skepticism than an article published by the Telegraph, which is labeled “misleading” on the site.
Vincent hopes to have the Trust Tracker running shortly. Meanwhile he’s been reaching out to science journalists for feedback, and has received mostly positive comments. (He was surprised, for instance, to receive praise for the Trust Tracker from an editor at The Wall Street Journal, which receives a low rating on the list.)
The Trust Tracker isn’t without risks, however. First, there’s the question of whether it can develop a large enough audience to have an impact on how readers view the accuracy of a publication. There’s also the risk that a few pieces of poor journalism could pigeonhole a publication permanently with an inaccurate rating.
But Vincent emphasized that the Trust Tracker is only meant to be a tool for readers, not a permanent sentence for publications.
“It doesn’t mean that everything that they publish in the future might be flawed,” says Vincent. “But it will hopefully raise a flag for the readers, so that they are extra skeptical when they read an article from a publication.”
This post has been updated to more precisely characterize the amount of time it takes for scientists to develop and deliver comments and annotations for articles targeted by Climate Feedback. It typically takes days, not months.
Comments are automatically closed one year after article publication. Archived comments are below.
Brilliant idea – and a great way to expand upon the idea of something like aFactCheck.org while providing solid backup for evaluations. I can only imagine that the anti-science hordes will resort to questioning the objectivity of the Climate Feedback staff itself, since this is what they tend to do with the scientific community at large. Defending themselves from accusations of bias will be an annoying side-job, I’m sure. But, science often has a decidedly left-wing tilt, and false equivalencies are a trap that media often falls into.
I kinda think these things are too ambitious. Worthwhile but hard to do and with limited impact. I’ve been focusing on the climate deniers’ canon, which includes things like “satellite data shows no overall planetary temperature rise in 18 years” (it harks back to a specific outlier year in the older data), cold winters, low recent hurricane activity, and methane vs CO2.
All that said, I’ve seen some bizarre stuff in the global warming canon as well — my favorite is that polar bears will disappear from Spitsbergen because the ice pack there is almost gone. Never was pack ice there — it is warmed by Gulf stream offshoot. An eco-tourism promo brochure had made the claim (come to Spitsbergen before the bears are gone) and it ranked high in Google for years. I know (and knew to track it down) because I did a TV shoot there in 2009. Who on a five-expert panel would know that specific fact? Al Gore used to quote a Princeton study on slices of energy technology to replace fossil fuels over the next 5 decades from 2000. He omitted a very significant slice, out of the seven in the study — nuclear power. It would have pissed off his base.