On June 17 of this year, a research paper about T-cell mediated immunity and the novel coronavirus was posted on the academic publishing website Research Square. Though it had not undergone peer review, the study had direct implications for the development of Covid-19 vaccines. In just three months, it was downloaded more than 100,000 times, and reached perhaps 10 million people on Twitter, according to figures provided to me by the website’s editor-in-chief Michele Avissar-Whiting. Not until September 30, however, was it published in a traditional journal, Nature Immunology. At a moment when every day counts in the race to develop a coronavirus vaccine, the wait for peer review would have kept this crucial information hidden for three and a half months.
The flourishing of preprints, scholarly papers published prior to being assessed by peer reviewers, has been one of the few bright spots of this year’s devastating pandemic. Preprints have been standard for decades in fields like physics and math, but they are relative newcomers in medicine and the life sciences. Better late than never.
As of this week, roughly 10,000 preprints about the novel coronavirus were available on the preprint servers bioRxiv and medRxiv alone, a remarkable feat given that this virus has existed for less than a year. Collectively, these preprints have put vital research information into circulation much faster than would have been possible under the traditional academic publishing model, in which emerging knowledge is sequestered until it clears peer review. Although peer review has long been held up as the gold standard of academic publication, the flowering of preprints during the pandemic gives the lie to the fiction that pre-publication peer review is essential to ensuring scholarly rigor. In a fast-moving era of digital information, preprints should become the new normal.
This is not to say that review itself is dispensable; everything from this modest op-ed to the most complex scientific paper benefits from scrutiny. But the pandemic has shown that post-publication review, conducted swiftly and transparently via online platforms, can provide adequate safeguards against misinformation while still allowing the fast, unimpeded flow of scientific knowledge.
Nothing gets published on a preprint server without being scrutinized; the sites typically deploy volunteers who put in work up front to ensure a preprint is worth sharing. At bioRxiv and medRxiv, for example, if a preprint submission reeks of pseudoscience or plagiarism, it’s not posted. Anyone who might be concerned that preprint servers will become wastebaskets for crackpot or stolen ideas should rest easy.
Moreover, the pandemic has inspired the emergence of several third-party services that review or curate published preprints. PreLights, a preprint review site supported by the not-for-profit publisher The Company of Biologists, maintains a running timeline of what they deem to be “landmark” preprints about the biology and transmission of the novel coronavirus. Each entry gives a brief description of why the preprint is important, along with a direct link to the paper. As of this writing, they have highlighted approximately 125 preprints, providing a useful filtration system for the thousands of preprints that have been published about the coronavirus.
The pandemic has also accelerated the rise of “overlay journals,” digital publications that compile and review previously published, open-access scholarly work — often preprints. In August, Rapid Reviews: Covid-19, published by the MIT Press, began offering free access to comprehensive reviews of preprints about the coronavirus and related public health interventions. As of this week, they have already published more than 50 reviews.
And then, of course, there’s social media, which provides less formal, but perhaps more nimble, platforms on which researchers can scrutinize the work of their peers. Often, these informal reviews can reveal flaws that traditional peer review misses. Earlier this year, for instance, both The New England Journal of Medicine and The Lancet published papers based on research from the medical analytics company Surgisphere. The two papers purportedly used a vast database of Covid-19 patient medical records to glean mortality risks associated with treating the disease. Both papers, however, rested on a flawed foundation. Within days after The Lancet article was published, for instance, many researchers pointed out that the article was almost certainly fraudulent. This campaign took root on Twitter and culminated in an open letter signed by dozens of researchers. The critics charged that the Lancet study was based on extraordinarily detailed global hospitalization records about the coronavirus that would have been impossible to assemble in just a few months. In addition, Surgisphere did not share its computer code or the underlying data so that others could review it. Here, post-publication review, the very technique that so many preprint skeptics worry is fraught with risk, saved the day. The Lancet and The New England Journal of Medicine ultimately retracted the papers on June 4.
Anyone who might be concerned that preprint servers will become wastebaskets for crackpot or stolen ideas should rest easy.
The fact is, although there’s always the chance that dangerous or fraudulent manuscripts will slip past the guardrails that have been put in place to vet preprints, the same is true of traditional pre-publication peer review. Many parents still wrongly believe that vaccinations will increase their children’s risk of developing autism, and they believe this in large part because a peer-reviewed paper published by The Lancet in 1998 told them so. In that since-retracted paper, discredited physician Andrew Wakefield concocted spurious associations between vaccination rates and autism rates, fueling a disinformation campaign that will probably never die.
Another flaw in the traditional peer review model is that it discourages researchers from publishing studies that did not work out as planned, even when that information could be useful to the science community — a form of publication bias. Researchers have great incentives to publish only their most unique and counterintuitive results, due to an editorial and peer review process that prioritizes novelty. This can lead to frequent publication of high-profile results that other researchers have a hard time replicating or building on. A shift to embrace preprints as a legitimate form of scholarly publication would help to mitigate that publication bias.
At some leading institutions, that shift is already underway. For decades, the U.S. National Library of Medicine (NLM), which serves as the world’s leading repository of biomedical literature, has only catalogued formally peer-reviewed literature. This spring, however, it launched a pilot project to begin to also catalog preprints about Covid-19 research funded by the National Institutes of Health. I worked at NLM for two years and can attest that the library is conservative, cautious, and bureaucratic in its approach to disseminating the world’s health research. If the NLM is recognizing preprints, they are truly here to stay.
Certainly, there was once a time when pre-publication peer review was the only system that made any sense — a time when scholarly journals were printed and mailed, and when it was essential to do all the intellectual work up front so that the final product on library shelves was as accurate as possible. Today, though, that’s all changed. We publish online with ease and have every tool necessary to manage scholarly review online too. No longer is there any reason to give just two or three experts, often cloaked in anonymity, the full power to confer legitimacy on an academic paper. No longer is there a reason to sequester scientific knowledge from the public for months as it slogs its way through an antiquated publishing pipeline. It’s time to build a scholarly review system for the internet age by embracing preprints as the new standard in academic publishing.
Marcus Banks was a health sciences librarian for more than a decade and is a recent graduate of the Science, Health and Environmental Reporting Program at New York University. His writing has appeared in Medscape, Cancer Today, and Smithsonian.
Mr. Banks hasn’t been paying attention. One preprint server currently hosts misinformation from a Steve Bannon-funded Chinese conspiracy scientist claiming Covid is a bioweapon (soon amplified on Fox News and social media); bioRxiv and medRxiv have hosted papers about hydroxychloroquine, oleandrin, and other bogus therapies (amplified similarly, and by the White House).
Preprints receive hardly any scrutiny before posting. That’s a fact.
That scare about cell phones and brain cancer? Thank a preprint. The American Cancer Society had to rebut it for years, yet PubMed still indexes it.
Even studies of plausible therapies of Covid-19 — antibodies, remdesivir — have seen hopeful preprints toned down to equivocal results after review (and better studies never preprinted).
More scientists are becoming disenchanted with preprints as a result, seeing that it’s better to wait for robust, reviewed results reports.
Preprints have been used to justify “science by press release,” and opportunists have used them to temporarily goose sales and stock prices.
The FBI has warned scholarly publishers that their outlets are being used for misinformation, and preprints are front and center here.
White nationalists use them to disguise racist ideas as science.
As realized, preprint servers are poorly designed, easily exploited, easily amplified/abused by misinformation peddlers, and mislead scientists and the public with false information and false hopes.