Of Climate Science and Peer Review
A few weeks ago, torrential rains caused heavy flooding in Germany and France, forcing the evacuation of thousands and killing at least nine. Within days of the natural disaster, a group of climate researchers published a scientific paper concluding that the flooding, at least in France, had been made far more likely by climate change.
The study used state-of-the-art modeling and the work was performed by a group of highly regarded climate researchers.
It was not, however, peer reviewed.
Scientists were quick to publish research — prior to peer review — linking flooding in France to climate change. Is it an effective strategy? (Visual by iStock.com)
Heidi Cullen, one of the study’s co-authors and leader of the World Weather Attribution program, noted that while the un-reviewed analysis was released to the public, it had also been submitted to the journal Hydrology and Earth Systems Sciences, which is peer reviewed. The problem, Cullen said, is that this sort of work needs to become part of the conversation more quickly than the formal peer review process typically allows.
“It’s generally in the immediate aftermath of these events that important decisions get made,” she said. “If we can inject a rigorous objective analysis into that conversation, we’re hoping to further that conversation and make sure it’s informed by the best available science.”
It’s not unusual, of course, for scientists to share results ahead of formal peer review and journal publication, whether as working papers circulated among colleagues for feedback, or as preprints uploaded to popular repositories like the arXiv.org, which focuses on physics and mathematics research, and the newer, life sciences research archive bioRxiv.org.
But what Cullen and her colleagues are doing is, in at least one important way, very different. The World Weather Attribution program, in addition to making studies available before publication, is writing up non-technical summaries of their analyses that specifically target a popular audience.
The feeling seems to be that on an issue of such importance — and one where disinformation and denialism still percolate within the public discourse — climate researchers simply cannot afford to wait the weeks, months or even years that the scientific peer review and editing process often requires.
It’s a feeling that has animated other scientists, including the prominent climatologist James Hansen. Last year, his team published — ahead of peer review — a troubling new analysis on potential sea level increases associated with rising global temperatures. The goal was to influence world leaders ahead of the looming climate summit in Paris, and the study drew both headlines and some criticism from fellow scientists who called it speculative and irresponsible. That paper was eventually peer reviewed and formally published, albeit with some of its initial findings tempered — and a lingering air of activism that made some scientists and journalists uncomfortable.
The question for Cullen and her colleagues, then, is whether this sort of scientific activism is truly effective. After all, other research has suggested that lingering skepticism and indifference toward climate science has little to do with a lack of scientific understanding among the general public, and is rather a product of social tribalism, where competing worldviews and value systems shape public attitudes far more than scientific literacy.
In this light, suggests Andrew Hoffman, a professor at the University of Michigan’s Ross School of Business who has researched the role of academics in public and political discourse, the tactic might actually prove counterproductive.
“What if [the study] is rejected by peer review? What then?” Hoffman asked. “It could be very embarrassing and damaging. As tempting as it is and as urgent and passionate as the researchers are, it opens up another line of attack for those who want to discredit the science.”
The Seine River, which runs through the heart of Paris, recently hit a 34-year high. Is climate change to blame? (Visual by iStock.com)
Cullen acknowledged those risks, though she said her team wasn’t circumventing peer review so much as participating in a crucial scientific discussion as it’s happening — and making that discussion available to the public. “I think we’re in uncharted territory, a new place that the scientific community finds ourselves in,” Cullen said. “There are going to be risks associated with doing this, but we feel like we’re not bypassing the stage of writing up the paper and submitting it for peer review.”
Cullen also noted that the recent analysis of European flooding — which included work by a team of scientists from the University of Oxford’s Environmental Change Institute, Princeton University’s Climate Central, the Royal Netherlands Meteorological Institute, the University of Melbourne, and the Red Cross Red Crescent Climate Centre — is based on time-tested methodologies that have been peer-reviewed in past studies.
“We are all actually still very much wearing the hat of scientist,” Cullen said, “not just individuals speaking about our feelings on climate change and what should or should not be done about it. We’re actually sharing results. The message, if you will, is going to be dictated by the analysis.”
Just who that message will ultimately reach and how it will be received, however, remains an open question. Hoffman, for example, pointed to the “Six Americas” study out of Yale University, which identifies several distinct categories of American attitudes toward the climate issue. They range from the alarmed, concerned or cautious at one end of the spectrum, to the disengaged, doubtful and dismissive at the other.
“One of the first rules of effective communication is to ‘know thy audience,’” the research team at Yale’s Center for Climate Communication note. “Climate change public engagement efforts must start with the fundamental recognition that people are different and have different psychological, cultural, and political reasons for acting – or not acting – to reduce greenhouse gas emissions.”
Given this, Hoffman says it’s important to target scientific communications strategically. “That undecided middle is where we have to focus,” he said, adding that he does believe that scientists have an obligation to make their work known and understood by the public. Given the prevailing stratification in attitudes, however, publicizing results ahead of peer review can have unpredictable results.
“There’s an idea that creators of knowledge should not cross into advocacy,” he said, “that if we cross that fuzzy line, we lose the objectivity of the academia and we start getting into very tricky territory. We’re still trying to figure out where that line is.
“I appreciate the tension of the people that want to get material out before peer review,” he added, “but it’s a slippery slope.”
Comments are automatically closed one year after article publication. Archived comments are below.
Remember the cold fusion claim by Fleischmann and Pons in 1989? Remember how there was a press conference announcing their discovery before peer review? The same justification was applied: This work was so important and had such enormous implications for the economy, global politics, energy policy, and of course, climate change, that the world needed to know NOW. Of course, their work was flawed and debunked. Careers were ruined. Institutions were embarrassed. And in the aftermath of that event there was a lot of talk about the importance of peer review. Sound familiar? Climate science infused with activism and politics is, in my opinion, junk science. Our knowledge will advance much more quickly if we remain scientifically cautious and keep the politics far far away–which means embracing high quality peer reviewed work and allowing for a culture of questioning results without being branded a “denier”.
Hey, Jack – The cold fusion example is great to bring up in this context since it was pre-Internet. Say that happened today. Instead of calling a presser and conveying their (wrong) info in a one-way fashion to press, those researchers would have had to publish everything online. It would have gotten scrutinized like crazy because it was such an out-there claim. It probably would have gotten debunked more quickly, too.
Even look at something more recent like the cell phone / rat brain cancer study. The initial reporting, some of it sensational, and the debunkings of that sensational reporting, were all published on the same day. The communications environment is just totally different now.
If this team is wrong, we’ll learn soon enough and everyone will get to be even more skeptical next go around. But they’re strong scientists, they’ve developed robust models for analyzing these phenomena and they’re sharing those model results with the public and other stakeholders at a time when they are interested in the results.
“Climate science infused with activism and politics” is very much in the eye of the beholder, in my experience. When researchers are working on a topic that is the subject of public controversy, they should be more open, not less so.
Hey Jack,
Good example, but extremely rare case. Since Fleischmann and Pons story, there were hundreds of “peer reviewed” papers retracted as a result of post publication scrutiny. Further, research has shown that almost 80% of “peer reviewed”, published science is not reproducible. So on average, the level of confidence in “peer reviewed” publication is not much better than a story published in tabloid press. Competent scientist will distinguish good science from bad science, regardless whether the paper was peer reviewed or not.
This is an excellent discussion. Personally, I’d rather see scientists inform public conversations like this with some analysis rather than no analysis. As long as scientists are being clear about the process they are using – pre-publication release w/ standard peer review, open review, etc. – they’re doing fine in my book.
I also thought the Retraction Watch team made a pretty compelling case here for more open peer review: http://www.slate.com/articles/health_and_science/science/2015/08/peer_review_in_public_james_hansen_s_climate_predictions_released_as_a_draft.html.
It’s not going to be a fit for every paper or project – scientists still deserve the chance to fail quietly among their peers! – but it’s another tool scientists can and should use to communicate their work, especially when it has bearing on societally relevant and publicly contentious issues.
Big picture, all publishing is becoming more open and subject to review and feedback throughout the process. In that sense, journalism and science face some similar challenges, though the scientific community has arguably been slower to adopt new tools. Scientists and their institutions need to be able to trust their audiences, including journalists and the public, to be savvy consumers of scientific information. Journalists are certainly going to be more skeptical of papers that haven’t gone through full review yet, but that’s a skepticism they can pass on to their readers.