When a doctoral student at the Australian National University was preparing a review on systems for harvesting water from the air and purifying water through desalination, she ran into something strange. The student, who chemist Brett Pollard was co-supervising, had been looking into a bunch of academic papers that cited concentrations of lithium, potassium, sodium, calcium, and magnesium that indicated the purity and safety of drinking water. The World Health Organization and U.S. Environmental Protection Agency were cited as the sources for those values.
But there was one problem: The WHO and EPA had never produced any standards for these metals, according to Pollard and colleagues. So how had such guidelines and standards been circulating in literature for several years, unchecked and being cited again and again? “In a lot of instances, they’ve just made up values as far as I can tell,” Pollard told me. (Neither the EPA nor the WHO responded to specific questions about where such numbers may be coming from by press time.)
Pollard and his colleagues are not dedicated research integrity experts who spend hours and hours combing through scholarly literature searching for odd trends and patterns that may indicate wrongdoing. But they still managed to dig up around 20 papers citing non-existent WHO and EPA standards. Pollard thinks many more likely exist. “This problem seemed to be really propagating,” he said.
“It just seems like something easy to get right,” he added, “so struck us as quite strange that everyone seems to be getting it wrong.”
It’s not difficult to see what information does or does not exist on the WHO or EPA websites, but it wouldn’t have been easy for research integrity experts to spot such a pattern in the literature without expertise in Pollard’s field. In fact, Pollard and his colleagues only checked the primary sources after realizing that multiple studies on the topic all cited different numbers. But that leads to the question: Where are all these different figures coming from?
“In a lot of instances, they’ve just made up values as far as I can tell.”
In today’s world, the immediate culprit that comes to mind is generative AI, which is widely known to make up citations when it hallucinates. Some researchers are using AI to draft, edit, or academic papers, which could lead to corruption in the scientific publishing process.
But in Pollard’s case, something different seems to be happening because some of the citations predate November 2022, when ChatGPT was released. It’s also possible that the papers in question are products of paper mills, which are shady entities known for churning out nonsensical or low-quality papers as well as selling authorship slots and citations.
While Pollard says the studies didn’t seem to be fraudulent when he inspected them, some had text written in a style that he doesn’t expect in scientific papers. To some extent, researchers may have also used the same values without checking primary sources — a practice sometimes referred to as “cold citing.”
When they initially came across these papers, Pollard and his colleagues had several discussions on whether they should report these issues to the journals. “It just doesn’t feel like my job to have to police these to that extent,” he recalled thinking. “But we also felt that it wasn’t right to just leave it unaddressed and let this propagate because it can just become part of the accepted literature.”
Fact-checking seems even more vital given that policy decisions are often made based on scientific documents, which are ultimately built upon and cited by other studies.
Pollard and his team reported this issue in a separate academic paper in an attempt to alert their colleagues of the problem. But it’s unlikely that all the journals in question will retract the affected papers or that the non-existent standards will stop appearing in new papers. Even if research integrity sleuths or keen insiders with a knack for noticing such problems stand guard, the problems would only be solved after false claims in the literature are removed.
So what can be done to prevent false or inaccurate claims being mentioned in the first place? Scholarly journals should hire dedicated paid fact-checkers whose remit it is to rigorously check all claims — including references — made in academic papers before they go live. That would be an additional round of quality control in addition to peer review.
An array of newsrooms and media outlets (including Undark) have dedicated in-house or external contractors who, among other tasks, make sure all news, feature, and comment pieces contain factual information that is attributed properly to primary sources. (The same happened for this piece before it was published.) Why can’t we do the same with academic papers?
Support Undark Magazine
Undark is a non-profit, editorially independent magazine covering the complicated and often fractious intersection of science and society. If you would like to help support our journalism, please consider making a donation. All proceeds go directly to Undark’s editorial fund.
Fact-checking seems even more vital given that policy decisions are often made based on scientific documents, which are ultimately built upon and cited by other studies. While tools like generative AI are making it easier to increase research output, it’s prudent for academic publishers — especially ones that have profit margins rivaling Google and Microsoft — to police the literature more proactively.
To do that, fact-checkers could play an important role alongside publishers’ research integrity teams — which keep an eye out for issues like paper-mill activity and illicit use of generative AI — that need to be expanded much further. In a world marred with disinformation, misinformation, and information overload, science could and should lead the way. Science should pride itself in providing rigorously checked factual information that has been manually scrutinized by a human, who uses automated tools at their disposal. This approach is especially crucial amid fears that fake science could be becoming harder to spot.
Fact-checking is how journals could add real value at a time when the academic community — which is also carrying out peer review, usually as a labor of love — are questioning what publishers bring to the table apart from typesetting and copy-editing articles and making them look pretty.
Dalmeet Singh Chawla is a freelance science journalist based in London.