https://soundcloud.com/undark-magazine/test
Join our podcast host Kasha Patel as she talks with Sara Van Note about the resilience of desert bees, and with Undark’s Matters of Fact and The Tracker columnist Michael Schulson about a recent study in transferring memory. Also, former NYT editor David Corcoran talks with Kathleen Hall Jamieson, director of the Annenberg Public Policy Center, about rethinking media narratives about the well-being of science.
Below are the individual segments and a full transcript of the podcast, lightly edited for clarity. You can also subscribe to the Undark podcast at iTunes.
Kasha Patel: This is Undark. We’re a magazine devoted to exploring the intersections of science and society, and we’re this podcast.
I am not David Corcoran, who is your usual podcast guide. I am a tall, skinny, Indian girl who answers to Kasha Patel, because that’s my name. During the daytime I work as a science writer for NASA in the Earth science department where I get to talk with scientists about air pollution, climate change, hurricanes — really uplifting topics. Then I balance all that out by producing science comedy shows where only science jokes are allowed. And I take the stage myself.
Speaker 1: Your next comedian is, Kasha Patel.
Kasha Patel: Hooray.
Yep, that is stand-up comedy.
Some people think that the world of particle accelerators, lab coats, and prestigious journals can only be serious, but in a way, comedy and science are set up in a very similar way. They’re both about looking deeper into something to find some hidden truth — whether that’s in a moment of humor or a discovery. And that’s what this podcast is about too.
Plus scientists make an interesting audience. One time a scientist came up to me after the show and said “I really enjoyed your…um…presentation.” Cool. Every month, I’ll bring you stories about discoveries, mistakes, disasters, triumphs, and anything else that happens when science meets society. Plus a bit of humor along the way.
Like, neuroscientists are really good spellers because they have to spell all these long complicated words like “hippocampus,” “amygdala,” “insufficient funding.”
This month is all about unexpected results. From bee survival to slug memory to the state of science itself. It’s just like a comedic twist. Right when you think you know where it’s going … I have to read a lot of studies and surveys in my job. I recently read that the Pew Research Center said that 80 percent of Americans said that science has actually improved their eating habits, well-being, just overall life.
It goes somewhere else … The other 20 percent are Ph.D. students.
Freelance journalist Sara Van Note brings us a story on bees that live in unexpected places — a much hotter, drier world than we thought. Understanding how the bees adapt in this environment can help scientists anticipate how a changing climate will affect these critters. Sara, welcome to the show!
Sara Van Note: Thank you.
Kasha Patel: My friend said, you are more likely to die from a bee sting than a shark attack. I said, not if I’m in the water.
Sara Van Note: Although I’d imagine it actually is possible, to get stung in the water.
Kasha Patel: Probably, but those aren’t the bees we’re talking about today, right?
Sara Van Note: No, no. In fact, native bees, which are the bees that the story talks about, are very unlikely to sting you, which is another reason to be a fan of native bees as I am.
Kasha Patel: Ooh, then I’m a fan too. Sara, what’s the story about?
Sara Van Note: The story is about desert bees and their amazing adaptations to survive in pretty harsh conditions here in the desert.
Kasha Patel: Yeah, the environment of the desert is so different from where I picture bees, you know, buzzing around the garden.
Sara Van Note: Right, so they only emerge when there are plants blooming, basically.
Kasha Patel: It’s like me, except I only emerge when I’m done binge watching Westworld. Well I’m excited to hear more, lets listen.
Sara Van Note: Awesome.
Olivia Carril: Remember how I talked about Penstemon? There’s penstemon.
Sara Van Note: I got it. These dark colored fuzzy insects don’t look like stereotypical yellow and black striped bees. But scientist, Olivia Carril is quick to scoop up a handful in her net.
Olivia Carril: Little tiny [inaudible 00:04:17]visiting these gorgeous purple penstemon flowers.
Sara Van Note: They’re leafcutter bees, she says, out foraging on penstemon plants on an unseasonably hot, dry May morning. Carril quickly captures the bees in glass vials. Each another data point for her survey of the bees of New Mexico’s Bandelier National Monument.
We cross the mesa through wide barren patches, where in normal years, winter snows bring spring blooms. But this year, 80 percent of New Mexico is in severe to exceptional drought. And Bandelier is in extreme drought. Despite the critical role bees play as pollinators in diverse ecosystems, huge swathes of the U.S. remain un-surveyed. Scientists say if they don’t know what’s out there, they can’t accurately assess how native bees are doing. Carril’s study aims to help close that gap.
Carril crouches to look at that penstemon flower.
Olivia Carril: See, you can see how when they crawl in, they end up with the pollen dusting their backs and then the next time they visit, that little brush right there, gathers it all.
Sara Van Note: She’s collecting native bees in several representative plots around the Monument, which had only two documented bee species in one part of the Monument before she began. Carril has added another 46 species over the last year.
Olivia Carril: See it?
Sara Van Note: Oh, yeah.
Olivia Carril: She’s got pollen on her legs. It’s a little sweat bee.
Sara Van Note: Bee scientists have lots of questions about bee biology, diet, nesting, survival strategies. And few answers. Carril works to reveal a more complete picture by documenting the plants they visit. She points out a bright yellow aster.
Olivia Carril: This is Senecio. A cute little yellow composite, looks like buttons.
Sara Van Note: She says she’s collected a lot from the same bee genus today. Not much diversity, but they are visiting different flowers.
Olivia Carril: You want to talk about the plant pollinator network. The plant part is as important as the bee part.
Sara Van Note: Other details like location and time of day help Carril see patterns when she analyzes her data.
The Bandelier Monument ranges from lush riparian zones, to piñon juniper foothills, to alpine forests. Most habitats have been touched by wildfire, with two extreme blazes charring the park in less than 20 years. At the high elevation survey site, the landscape changes to scattered ponderosa pines, their bark singed black. And thick hummocks of grass and large shrubs.
Olivia Carril: Most of what I’m seeing blooming is the dandelion, just getting started. And behind you, this beautiful ribes, which I am about to go collect a whole bunch of bees off of.
Sara Van Note: Since it’s so dry this spring, there’s fewer plants in bloom and not as many bees to catch. Still, Carril counts over 30 in her vials.
Olivia Carril: If this were peak bee season and this is how many bees I got, I think I’d be pretty disappointed, but for not seeing many flowers in bloom today, I think we’ve done remarkably well.
Sara Van Note: The long days in the field are challenging, but really the work has just begun. All the bees Carril collects are taken back to her house to be pinned and identified.
Olivia Carril: I think the easiest part is going out and collecting them. And the other nine months is spent getting them ready and analyzing them. That’s a big process.
Sara Van Note: It requires a good microscope to see all the details on tiny body parts. A key for identification and a lot of patience. Examining body features gets pretty intricate.
Olivia Carril: It’s kind of ridiculous. It depends on the bee; it could be whether it has ridges or curves on the different segments of the legs. Mouth parts, how many teeth, how rounded or pointed the teeth are.
Sara Van Note: Since Carril works independently, she has to be resourceful. Her [preserved] bees live in her laundry room and she’s been known to use her kid’s Play-Doh as a stand for pinned bees.
Later this year, she’ll take the bees to a big lab to confirm her IDs with its collection.
Olivia Carril: These are gorgeous. I don’t even know what this is, yet. It’s got white fur all over its back. And you can see the sting sticking out of her abdomen.
Sara Van Note: Given her past experience with bee surveys, Carril thinks it’s possible she could discover a new species. And she’s documented unique bee communities in each microcosm sampled, hinting at a variety of survival strategies.
The American Southwest is one of the world’s hotspots of bee diversity. This diversity may have to do with the abundance of specialists — bees that rely on just one group of plants.
Olivia Carril: The plants that you find specialist bees on are prickly pear. It’s everywhere. It’s all across the West. Globemallow, it’s everywhere. Sunflowers. These things that are really widespread, so again, the chances of them blinking out are much smaller. I think that specialists have picked plants that thrive, even in a dry desert environment. That’s how they managed to make it. That’s their adaptation.
Sara Van Note: Since their adult stage is so short, often just six weeks, specialist bees must time their emergence to coincide with the bloom of their host plants. But those flowers are often also short-lived.
Olivia Carril: Plants up here send a flash. You see, evening primrose comes out in May and then it’s gone. The chollas bloom in the beginning of June and by July it’s done.
Sara Van Note: There’s a higher proportion of specialist bees in arid environments than humid ones, but scientists don’t know why.
Bob Minckley: One interpretation is to survive in the really hot, unpredictable parts of the desert, you need to be able to be synchronized with that plant very well.
Sara Van Note: That’s Bob Minckley, a professor of biology at the University of Rochester, who’s led long-term studies of bees in the desert southwest. One theory on how they synchronize is that specialist bees may respond to the same cues their host plants do, like moisture. When rainfall triggers the bloom of their preferred plant, the bee larvae emerge from their nest, just in time to feed. This specialization may provide an advantage in an era of climate change and increasing drought. While generalist bees seem to emerge every year, specialists may wait out the bad years with low precipitation.
Bob Minckley: If droughts were predictable, then I think the bees would be fine, but they’re not predictable. I think the specialists will probably do better maybe than the generalists, because they’ll come out or not come out in the right years.
Sara Van Note: But Minckley and Carril both stress it’s extremely hard to predict how bees might respond to climate change in the future. The last extended drought occurred in the 1950s in the Southwest. But there’s very little information about bee communities before then.
Bob Minckley: That’s a really big problem is we don’t have really good baseline data to talk about impacts. The variability of bees in deserts is really high. And so to see a downward trend, would take a lot of sampling, a lot of effort.
Sara Van Note: There are just a handful of long-term studies of bees anywhere in the world. Short-term data sets are the norm. Carril’s two-year survey of the bees at Bandelier will end this year, but she says the protocol she established could be continued in the future. Despite the current drought and climate predictions, Carril says she’s optimistic about bees response to climate change.
Olivia Carril: The image that comes to mind is we’re turning the Earth into Mars and there’s not going to be anything left and everything’s going to die. And I’m not sure that’s quite accurate, one thing I know about bees is that they like dry and hot.
Sara Van Note: Her 2017 collecting showed bees were both abundant and diverse in the burn scars of Bandelier. Carril says bees are opportunistic and persistent.
Olivia Carril: One of the things that happens after a burn is the understory can really take off, because the overstory isn’t there shading them out. And so as far as a planned diversity, it was pretty high under there, and flowers and bees go together.
Sara Van Note: Back at Carril’s high-elevation site, another gooseberry bush attracts a varied group of bees. There’s a charred stump next to it and Carril wonders whether it could be a home for a cavity nesting bee.
Olivia Carril: If it’s too burned, it’s probably disintegrating inside, so probably not the best bee habitat. But see, no, she went in right there. Did you see that? This little bee flew in this hole right there.
Sara Van Note: A resourceful bee is making her nest in the blackened stump. Carril sits nearby with her camera poised for the bee’s return. The wait she says is worth it.
For Undark, I’m Sara Van Note.
Kasha Patel: One of the perks of being a science writer is the privilege to learn about a scientist’s life’s work and cram it into a 10 word headline. Each month, I’m going to award kudos to a superb headline. Basically, I just look through Twitter and see what headlines make me laugh.
This month, scientists published a study titled “Organic matter loading by hippopotami causes subsidy overload resulting in downstream hypoxia and fish kills.” The science writer version of that? “Hippos poop so much that sometimes all the fish die.” That was brought to you by Ed Yong, writer at the Atlantic. According to Yong, the dung of a hippopotamus consumes the oxygen around it creating lethal pulses of suffocating water.
There was another study this month that sparked a big debate about the essence of memory. I’m bringing in Michael Schulson, who is a columnist for Undark. He writes the Tracker and the Matters of Fact columns. Michael, welcome to the show.
Michael Schulson: Thanks for having me on.
Kasha Patel: We just went over some of the best headlines, and this one in itself also has a pretty interesting headline. So the scientific study was … oh my gosh, I’m going to butcher this, but we’ll try for it. RNA from trained Aplysia, maybe, can produce an epigenetic engram for long-term sensitization in untrained Aplysia. That is very much a scientist study name. The headline from the New York Times was, “Scientists Made Snails Remember Something That Never Happened to Them.” Michael, tell us first of all, did I pronounce any of those words correctly in the study?
Michael Schulson: That sounded pretty good, but I’m not an expert on how to pronounce, slug and/or snail scientific names so probably we should ask somebody else to make sure we have it right.
David Glanzman: Aplysia.
Michael Schulson: That was professor David Glanzman, who studies integrative biology, physiology, and neurobiology at UCLA and was one of the authors on the study.
Kasha Patel: Tell us about this study. What is it about?
Michael Schulson: So the big question here is about something that’s called the engram, which is basically the physical substrate of memory. So we all have memories. How does this get recorded? What kind of physical correlates in the brain or what kind of physical correlates in the nervous system are actually encoding memories? And for this particular study, they took sea slugs [the researchers use slug and snail interchangeably], and they basically trained them to do something. And then they took RNA from those sea slugs. They put it in different sea slugs that had not been trained to do that, and they waited to see whether the sea slugs that had never gotten this training would somehow, from that RNA, somehow experience some of that memory or had some of that memory transfer over. And what the researchers are arguing is that, in fact, they did. That they found that RNA could be a kind of engram, that RNA, in this very particular case at least, can be the stuff of memory.
Kasha Patel: That’s kind of a big deal because that kind of changes how we think of memory right now, right?
Michael Schulson: Well, maybe. It depends on two things. The first is whether you think that the scientists in this particular study actually found the thing that they’re saying that they found. And then the second is what exactly you’re talking about when you’re talking about memory. And both of those are kind slippery questions, and both of those are pretty hotly-contested.
Kasha Patel: Let’s start with the first point that you said. Why would I not believe the scientist did what he said he did?
Michael Schulson: So there was an interesting kind of gap, especially if you look at the way that this study was covered. Some articles went, they talked to a lot of other scientists who were working in this area. And those researchers would say things like, “This study is really cool, but I think that probably it didn’t actually happen.” Or, “This is just so far outside of what we think is possible that we’re really not sure that this is real.” Or, “Maybe something happened, but we’re not” … There was an enormous amount of skepticism, and there was a really clear divide.
The coverage on this between studies that talked to other researchers who said these skeptical things and the articles that didn’t talk to other researchers often, which were more likely to say, “Oh, wow, this is this unbelievable memory transfer thing. ‘Westworld’ is right around the corner.” And also it’s a dramatic sci-fi innovation. So we actually reached out to Professor Glanzman to see what he thought about the coverage and what he made of the way that the coverage was portrayed in the media.
David Glanzman: (Via audio recording) I think the coverage was mixed. I mean, so some outlets obviously went for the science fiction angle. I mean, I think it’s inevitable. I think as soon as you say memory transfer, people are going to go “Blade Runner” and “Westworld” and that sort of thing. So I thought, on the whole, the coverage was variable. But given the topic, I think that was something to be anticipated.
Kasha Patel: Yeah, I saw one of the headlines saying something about they did this in snails, and the same could happen in humans. I mean, I feel like with any scientific study, when it’s done in animals it’s just such a huge leap to automatically go to humans.
Michael Schulson: Right. I mean, we’re talking about one particular species and one very particular interaction. And that doesn’t mean that we should say this is a bad study. It seems like a really elegant study that’s gotten a lot of kudos from other researchers. But, as with anything, jumping from “this is a single study that says an interesting thing” to “this is a new grand scientific fact about the universe and the nature of memory” is probably a little bit more of a leap than it would be safe to take.
Kasha Patel: So the second part that you brought up is it kind of depends on what we define as memory. So can you explain that?
Michael Schulson: Yeah. There was a great exchange about this on the Neuroskeptic blog. Neuroskeptic is an anonymous blog written by a neuroscientist somewhere in the U.K. that talks a lot about new research in neuroscience and often takes a fairly skeptical take on what new developments might mean.
Kasha Patel: It feels very secretive, very XOXO, Gossip Girl but for science.
Michael Schulson: Neuroskeptic was appropriately kind of skeptical of this particular study. And what happened in the comments section on that was, I think, really interesting, which was that the head of the lab went on the comments section and said, “I’m afraid you have a fundamental misconception of what memory is.” And really kind of hit back. What we’re imagining memory as you remembering a really great day walking down the street in a beautiful city or whatever is going on, right? Those kinds of very rich memories can be very different from what they were talking about in this study, right? Which was basically slugs remembering certain kinds of electric shock that happened to them and changing their response, right? So basically a reflex that changed.
Kasha Patel: Yeah, that’s interesting because when I think of memory, it’s something that happened to me, like a childhood memory or something like that. It’s not if I touch a hot pan, then I let go, and then I learn not to do that again. To me, I mean, I guess in a way … I guess that is memory. I just never thought of it like that.
Michael Schulson: It can get kind of poetic. One thing that the study PI brought up is you can say, okay, when you fold a piece of paper and its creased, that crease is a kind of memory, right? I mean, at that point, you can say almost any trace on the world is a memory. And this idea that we can transfer traces of the world from one body to another body is a little less radical, right? If I’m extremely drunk and somebody draws a bunch of my blood and injects it into someone else’s blood and they have a higher blood alcohol content, that’s not their body. Is that a memory of my drinking, right? These sort of questions of history and trace get a little bit more slippery.
Kasha Patel: You said that there was a lively debate in the comments section. So who won the debate?
Michael Schulson: I don’t know. I don’t know. It’s a great question. I mean, it’s kind of beautiful to see that you could have these public forums where scientists and interested laypeople can duke it out. They can ask each other questions. They can talk about what a study looks like.
Kasha Patel: Did he feel like he got his point across?
David Glanzman: (Via audio recording) I do. I do. Yeah. That’s exactly the kind of exchange that I think scientists like myself should be engaged in.
Kasha Patel: What’s an example of an outlet that did a good job in conveying the study?
Michael Schulson: Yeah, I think STAT really won the day on this one with a really, really in-depth piece that gave a lot of the context and texture that other outlets struggled to find, although there was a lot of really good coverage of this as well. It seemed like covering a study like this in such a contentious, sort of larger debate about brain science and memory really took two things. One of them, which was really the simplest, was just reaching out to a lot of other researchers in this field and saying, “What do you think of this? What does it mean? How skeptical are you?” The other thing was to really get the context. This study is coming into a very, very long-running debate over how memories are stored. I mean, it’s just like the moral of this is that this is a study where it’s, well, what’s going to happen next, right? Follow-up research, will it confirm this? Will it not? And as with so many of these things, one study does not offer definitive proof of some radical change.
David Glanzman: (Via audio recording) What I’m looking forward to is replication. A discovery in science is not a discovery until someone can replicate it.
Michael Schulson: It’s going to take a lot of research. It’s going to take a lot of work to start to see what might actually matter here and what ends up bearing the test of time.
Kasha Patel: Well, thank you, Michael, very much.
Michael Schulson: Thanks for having me on.
Kasha Patel: To take this one step further, though, we’re going to talk to a researcher who has broadly looked at the current state of science and how we talk about it. Conducting this interview is senior editor of “Undark Magazine,” David Corcoran, he will be bringing in a guest each month for an in-depth discussion of an issue affecting science or the media.
Okay, David, so who do we have this time?
David Corcoran: Our guest is Kathleen Hall Jamieson. She’s the director of the Annenberg Public Policy Center at the University of Pennsylvania. And kind of more to the point, she is a really distinguished, longtime commentator on communicating policy and politics. She has taken on a subject that I think is quite new for her, which is this so-called crisis in science.
Kasha Patel: So why were you attracted to this particular subject?
David Corcoran: Well, there’s a lot of criticism out there of scientists. That they can’t reproduce their results, and that the articles that they write in journals are hyped and overblown. And what Professor Jamieson has done is taken kind of a deep dive into this whole issue. And the point she makes is that the crisis is really exaggerated, if it exists at all. I want to start by reading a few of the headlines you cite in your article in PNAS: “Trouble in the Lab,” “Big Science is Broken,” “Science in Crisis.” What’s the crisis these authors are talking about?
Kathleen Hall Jamieson: These authors are variously talking about problems in peer review and problems in replication and an apparent rise in the rate of retractions and overgeneralizing them to a problem that is systemic across science.
David Corcoran: So you say that they are greatly exaggerating the problems, not that they necessarily don’t exist, but that this narrative that big science is broken is a serious exaggeration. So before we get to your arguments, I want to ask why you wrote the article, how you got interested in this subject in the first place.
Kathleen Hall Jamieson: I was on the train reading my issue of The Economist, going to Washington, D.C. and saw an article that you cited, “Trouble in the Lab,” whose subtitle suggested that science thinks it’s self-correcting, but it may no longer be. And said, “Well, this is an interesting claim.” And as I read the article, I realized first, that the evidence in the article was gotten from scientific exploration showing problems in science. Hence, the first stage of self-correction was at work. Science had flagged the problem. And secondly, in the same article, some major initiatives to address the identified problems were featured. And so the headline badly misrepresented the actual content of the article.
I said to myself, “I wonder if this is a one off, if this is just the kind of headline that is designed to get you to read or if there’s anything else out there.” So I came back to my office, had a meeting with my research team, and asked, “How would we determine the extent to which this broken crisis rhetoric is now appearing in headlines?” And that was the genesis of the article.
David Corcoran: It sounds like a productive train ride.
Kathleen Hall Jamieson: Yes. Well, it was an anxious train ride because in a polarized environment in which ideologically inconvenient science is attacked and as a result we have more difficulty addressing real needs with actual solutions in the time that we have available to address them, the possibility that in the line of argument, science is broken might be used to stall actions in some areas that might actually be socially beneficial was part of what was motivating my concern.
David Corcoran: Got it. You said earlier that this is dangerous, not just because it’s spreading a kind of false or overblown picture of the scientific world and the scientific process, but because there are specific examples of problems that can happen when people buy into it. Can you give us a couple of those?
Kathleen Hall Jamieson: Yeah. You see instances, now this is in political discourse so now we’re not in the domain of scientists talking to each other, but political discourse in which someone will stand up and say, “Well, you can’t trust that claim in science.” The scientific process itself is broken. We’re not in that discourse examining the legitimacy of those individual pieces of evidence, which is what we should be doing in that exchange. But rather, simply dismissing everything because we all know science is broken.
Well, because we’ve now made whole areas of science highly controversial in the political domain, and vaccinations of some sort in some communities fall into that category, it becomes convenient for those on one side of an argument who are losing the argument on the merits of the science to say, “You can’t trust the science, and here’s how we know.” See, those scientists are saying science is in crisis, and that’s my concern. When scientists uncritically use the word “crisis” and when journalists uncritically use it or uncritically use words like “broken” or “beyond repair,” they are setting up a premise that can be used for ends that they themselves would not have endorsed because they’re being used in ways that run counter to the best available science in order to discredit it.
David Corcoran: Now what about climate science?
Kathleen Hall Jamieson: Climate science is the most contentious of the scientific areas right now, period. And also the one that one could argue is the most consequential. And where rhetoric comes in that essentially says, “The science is all broken. We can’t trust it,” that is a rhetoric that gives you delay as the policy outcome in an environment in which we can’t afford to delay. It’s more difficult to see that when people are challenging the scientific premises by saying, “That whole area is just broken. They don’t really know what they say they know.”
David Corcoran: Of course science is an ongoing process, and so is public policymaking based on science. So we’ve got legislators and lawmakers making policy based on their best understanding of science right now. If science is a self-correcting process, how are laws to also be self-correcting?
Kathleen Hall Jamieson: One of the questions with an enterprise that is ongoing, as science is, is when do you know enough to say we can reliably act on the knowledge? So the large answer to the question is we grapple with this every day. We’ve set up governmental structures as well as structures in the scientific community that make those kinds of decisions. And their criteria to determine that if the need to make a decision is urgent, we will act with lower levels of certainty than we would in areas in which there is no urgency. And you’ve got this case with vaccinations. So look at the process right now. A scientific community is in the process of testing a Zika vaccine, or they’re in the process of developing a universal flu vaccine. At some point, the regulatory structure is going to be asked the question, “Are we sure enough that this works and does not have unintended consequences and side effects we can accept given the range of benefits that we have to put this on the market?” And when you’re dealing with things such as climate, the scientific consensus is we know that carbon is a problem. We know we need to reduce the amount in the atmosphere. We know the consequences of not doing that are extraordinarily problematic. The need to act is now.
And sometimes replication is used in order to discredit an area of science. So we need to be careful about this. When we’re dealing, for example, with large-scale studies of what’s happening in a real environment, you can’t actually replicate that. Because the thing that you’ve studied isn’t there anymore. And so people who say every single piece of large-scale science needs to be replicated before we act, particularly in the climate area, are in some cases, using the call for replication as a way of delaying needed action.
And when we cannot reasonably replicate, we need now to intensify our scrutiny of the quality of the work, to ensure that if anything there is suspect, we’ve caught it.
David Corcoran: So I just want to ask you to unpack that phrase, “Science is self-correcting.” That’s very important.
Kathleen Hall Jamieson: Yeah, one of the norms of science is that science is self-correcting. The scientific enterprise has, as its underlying set of norms, norms that ensure a culture of critique. The transparency of the scientific enterprise makes it possible for others to examine what we have done. And for us to examine their work as well.
And that culture of critique, which is built on healthy competition, ultimately should uncover error, and increase the likelihood that we move toward reliable knowledge.
So the first question is, are we honoring the norms? But, to the extent that self-correction is a norm, it is a process, not a single act. But, uncovering a problem is a first step in the process of correcting. When people say, “Here’s how we could fix it,” that’s a part of the self-corrective process. And when we ask whether it actually corrected, that is the final stage, as we also ask did it create any unattended consequences.
So one of the purposes of my article was to say, “Self-correction is a process, not a single act. But the act of identifying a problem is a step in the self-corrective process.”
David Corcoran: Of course, we know that scientists work for institutions. Are there institutional barriers to this self-correction process that you describe?
Kathleen Hall Jamieson: It’s difficult to get the funding to replicate research. It’s difficult to publish replications of research. And it’s difficult when you publish a replication to get it to count in the tenure process, as equivalent to research that is the first discovery, as opposed to the replication of the discovery.
And all of those create incentives that are misaligned with the best interest of science. So there are replication projects that are now emerging to ensure that at least in consequential areas of science, funding is available for replication. There are journals now that are opening replication sections, and with the advent of online access to scholarship and knowledge, the possibility that replications can find a home in a legitimate journal that is peer-reviewed, are now much higher than they once were when we were really constrained by the number of pages that one could afford to produce.
So is it a problem? Yes. And I’ll phrase it more strongly, if we really said every piece of research has to be replicated before it is produced, we might, in some areas, cut the amount of research in half. Because, literally, you’re going to re-do the whole study. And so, the question becomes, what are those areas that are sufficiently consequential that one needs to ensure that before it reaches the public it has been replicated? And how do we build elements of replication into the funding process, and rewards for replication into the tenure process?
David Corcoran: And I guess this is a question that journalists should be asking when they write these stories. You say that a large part of the problem that we’re grappling with here is the way that the media covers science. And you identify three types of stories that seem to drive a lot of the coverage. What are they?
Kathleen Hall Jamieson: Well, the first is a quest narrative. And in the quest narrative, you have a classic literary structure that suggests that the scientist is a hero, or the scientific group is a group of heroes collectively engaged in honorable enterprise in which discovery is the goal, and hence the accomplishment. And when we look at the stories that had the most coverage across a one-year period, we found that discovery narrative was the dominant narrative that the public receives about science.
The second narrative is a false quest narrative. In this narrative, something has gone wrong. The person we thought was the hero-scientist, is, in fact, someone who has misused the process. It may be someone who’s actually engaged in fraud, or a finding has proven unreliable because something in the process has not worked as it ought to have. And here you’ve got your retractions in news.
And the third structure is the one that I saw in that headline in The Economist, “The Whole System is Broken.” Now, this is not about individuals, this is about science writ large. And in it, we are suggesting that there’s something inherently wrong, and not featuring the fact that people are trying to address it. This is an odd narrative in news, because it says, problem. Science broken. Crisis. Exclamation point. Be alarmed, be alarmed, be very alarmed.
Now, the flaw in the first narrative is that it doesn’t show you all the times that scientists tried something and it didn’t work. And it also doesn’t tend, to the extent that it ought to, to show that the discovery that’s being featured is a step in the process, and it might be revised in that process.
The problem in the second narrative is that, it often doesn’t feature, to the extent that it should, the hero in the narrative. It features the scientist who’s done wrong, but not those scientists who caught the error, and those scientists who are trying to ensure that it isn’t going to repeat itself.
And the problem of the third narrative is, it’s an over-generalization. Science as a whole is not broken. Even if you took every single piece of indictment about every single area of science that is out there, and you aggregated it all, you’d still have a huge amount of science that has not been indicted.
David Corcoran: As science editor myself, I confess that I have been drawn into this narrative that science is in crisis. But I mean, I have to say, there is a lot to support it, isn’t there? I mean, there’s a whole publication called Retraction Watch that’s identified hundreds of cases of sloppy science, or even fraud. And a famous article by the scholar John Ioannidis with the title, “Why Most Published Research Findings Are False.” I mean, isn’t there a problem?
Kathleen Hall Jamieson: There are problems, not just a problem. But, identifying flaws in the ways in which people are working, or areas in which science could do better, doesn’t indict the entire enterprise.
David Corcoran: You mention a study in the journal Nature that suggests that scientists themselves are kind of getting drawn into this overblown story of science in crisis. Tell us about that.
Kathleen Hall Jamieson: The article that you’re speaking about specifically is a piece that appeared in Nature, which is a major, reputable journal, that was very poorly done, that essentially primed, or made salient, the concept of crisis as it asked questions about whether people perceived that science was in crisis. And as a result, as a social science literature would predict, found people saying that science is in crisis.
And it was poorly specified, its definitions were not careful. It didn’t specify the actual target audience for the study clearly. It looked as if it was a sample of a population of scientists, when it was not. It was a study in which people came into the study if they wanted to do it. It’s a self-selection process by which you guide your subjects. And then, it was amplified by people who came in as part of a web survey.
Now, this is not good science. So my headline on this is, “Piece of Shoddy Science Alleges Science is in Crisis: Bogus Claim Unsupported by Evidence in Major Scholarly Journal.”
David Corcoran: So, what should we in the media do about all this? Is there a way to kind of counter the effects of this narrative you’ve identified?
Kathleen Hall Jamieson: I made a series of recommendations in the article. And they involve a little extra writing from journalists. But, in the discovery narrative, the quest discovery narrative, which is the dominant narrative of scientific reporting right now, if they would, as journalists, simply increase the likelihood that there’s a paragraph that says, “And leading to this process there was this attempt, and it didn’t work. And this attempt and it didn’t work. And this, which found this, but here it’s modified in this way.” They would give people who are readers a better sense when something goes wrong, that no, that’s actually part of what we expect in science.
Secondly, then that discovery narrative, the reporter would indicate what are the unanswered questions? So not simply this is it, we’ve found it. End of story, we can walk away. But rather, scientists are still grappling with these. Those unanswered questions sometimes are the basis for coming back and revising our understanding of the discovery. So make the discovery narrative more complicated.
In the second narrative, the false discovery, the counterfeit discovery. If, in that narrative, we are sure to point out that someone caught this, and that someone was a scientist. Then the journalist is performing his or her accountability function in relationship to the process. But also, accurately explaining how we got to the discovery of the wrongdoing.
And in the third narrative, when the journalist writes the story about problems, as the journalist should, these problems are uncovered by scientists, this is legitimate inquiry, and it’s very important. The journalist and the scientist should both remember that they should tell the public, and they should tell each other, what it is that’s being done to correct the problem.
And so, my objection isn’t to the coverage, it is to the coverage that neglects the fact, and I argue this is in the article, that the very same scientists who’ve been pointing to the problems, have also been the ones offering and implementing remedies. And the stories were shortchanging that in my study, and I worry about that.
David Corcoran: So hopefully journalism can also be a self-correcting process.
Kathleen Hall Jamieson: Yes. Journalists have an accountability function. And if you go back historically and ask, what was the journalist’s role in uncovering fraud in science? You come up with real heroes in the journalistic community. I mean, Brian Deer in Britain, played an important role in unmasking the serious fraudulent problems in the Wakefield bogus study alleging an MMR-autism association. That wasn’t just the scientific community figuring out that something was wrong there.
And you see it in other retractions as well, where the journalist played a role in uncovering the problem. We should cheer journalism on when it does that.
David Corcoran: Well, Kathleen Hall Jamieson, you’ve given us a lot to think about. And I want to thank you for doing the article, and for coming on to talk about it.
Kathleen Hall Jamieson: It’s a pleasure to be with you.
David Corcoran: Kathleen Hall Jamieson is director of the Annenberg Public Policy Center, and a professor of communication at the Annenberg School for Communication at the University of Pennsylvania.
Kasha Patel: Well folks, that’s all for this month. We talked about bees and slugs and how science probably isn’t doomed. And we got in three Westworld references. We’re produced by Lydia Chain, Music is by the Undark team. I’m your host, Kasha Patel. Thanks for joining us.