Some scientists are accusing the media of hyping the announcement last March that a telescope at the South Pole had picked up a subtle signal confirming the leading version of the big bang theory. The signal came in the form of polarized microwaves – a pattern in the radiation left over from the early universe.
But in this case, reporters were doing their jobs and the scientists have only their colleagues to blame. They were the ones telling everyone the findings were revolutionary. They didn’t say much in that first couple of days about how perilous these observations were – how close the signal was to several big sources of noise.
The scientists behind this collaborative project, known as BICEP2, set up the situation in a way that was almost guaranteed to generate hype. There was no paper for the astronomy/cosmology community to look over. There was just a back-to-back announcement and press conference putting all the emphasis on the importance of the finding, if true, and very little on why the researchers were so sure it was true. It’s complicated stuff, even when the findings are straightforward and uncontroversial.
In his Achenblog, Washington Post reporter Joel Achenbach wrote that someone asked him why the media couldn’t have just ignored the Harvard press conference trumpeting the most important finding of our lifetimes:
When I attended a luncheon at the Institute for Advanced Study, one scientist mentioned “cold fusion” as an analog. Another asked if it would have been possible for The Post to have held off on publishing our initial story on the BICEP2 announcement at Harvard on grounds that the associated scientific paper had not yet been peer-reviewed or published.
In cases like this, the story is out anyway, readers want to know what’s going on, and so the best course is usually to remain skeptical and keep asking those pesky critical questions even as the scientists are popping champagne corks.
And of course, follow up, which is exactly what Achenbach was doing there. A number of science writers have been following up as questions and doubts have surfaced. The first round of doubts came out in May, summarized in this Tracker Post. Science, Nature, and New Scientist led the way. That weekend the Washington Post ran a front page follow-up by Achenbach. It’s not our job to make scientists look good. It’s to chronicle the episode from start to finish even if it ends up a debacle.
A second round of skeptical stories came last week, after the scientists finally published an official paper in the journal Physical Review Letters. The claims in the paper included a much greater degree of uncertainty than was expressed in the scientific talk and press conference last March.
At the New York Times, Dennis Overbye followed up with Astronomers Hedge on Big Bang Detection Claim.
It’s a good story. He has to gracefully review some very complex background material on the origin of the universe in order to make sense of the latest developments:
There was one thing that didn’t add up. Somehow it was implied the experiment would be considered a success even if they accidentally mistook background for a signal:
Now, after weeks of wrangling, discussion and debate with peer reviewers and other astrophysicists, the group, which goes by the name Bicep, has published its paper in the journal Physical Review Letters. The authors, led by John Kovac of the Harvard-Smithsonian Center for Astrophysics, write that they stand by their discovery — but they also now acknowledge that it is possible that interstellar dust might have produced much or even all of their signal. New data expected within weeks from the Planck satellite of the European Space Agency and other experiments should help clarify the situation, the authors say.
“The basic takeaway has not changed; we have high confidence in our results,” Dr. Kovac said in a phone call from a conference in Moscow.
If the detection is ALL noise, how does that qualify as a discovery? In what corner of the multiverse can they rightfully claim credit for a discovery if they don’t measure a signal that’s distinguishable from background or foreground noise? If this other experiment, the Planck satellite, sees a signal, shouldn’t that be considered the discovery rather than the confirmation?
A reporter who has been doing a great job of chronicling this affair is Jonathan Amos at BBC. His latest: Cosmic Inflation: Confidence Lowered For Big Bang Signal
As a side note, when I was teaching science writing last March and I showed my students all the different stories, they thought the BBC version by Amos was by far the clearest, no contest. This follow-up story has the same nice clarity.
Still, none of these stories answer a looming question about the prospect of so-called confirmation: What if it turns out this experiment, BICEP2, only measured noise, and their announcement is based on a miscalculation of their error bars, and then another group finds real evidence for gravitational waves. In that case, does it make any sense to credit BICEP2 with the discovery and the other experiment with confirmation? Shouldn’t the first experiment to get a real signal be considered the discovery?
And one point of clarification that probably would have helped all these stories is the background on the connection between the big bang and the theory in question, known as inflation. There’s already abundant evidence that our universe is expanding and that in the past it existed not as stars and galaxies but as a hot soup of particles.
But the big bang theory alone doesn’t work. It can’t produce a universe with the kind of distribution of matter we see. Inflation solves the problem by positing a burst of superfast expansion that sort of smooths out the cosmos. It’s not a radical theory, or a wild theory as some have called it. Inflation is the default theory, the view of the majority. The universe is certainly weird, but inflation is the least weird theory to describe what’s been observed. And so, these signals from space are what everyone expected to see. This isn’t one of those shocking findings – not like faster-than-light neutrinos or microfossils in a meteorite from Mars – neither of which turned out to be real.
What was surprising was that a relatively inexpensive telescope at the South Pole would find it before the much bigger, high budget Planck satellite. The fact that this polarization is what people expected to see may well add to the danger of fooling yourself.
And from the academic side is this intriguing post in a blog called Not Even Wrong by mathematician Peter Woit – Smoking Gun No Longer Smoking. Here he asks some critical questions about the connection between theory and observation. He raises questions about whether finding would really be the incontrovertible evidence it’s claimed to be. It’s worth reading, especially since the connection between the observations and the theory is not obvious and most science reporters are either taking it on faith or asking their readers to do so.