A Major Industry-Funded Alcohol Study Was Compromised. How Many Others Are Out There?

Last year, the National Institute on Alcohol Abuse and Alcoholism, part of the federal National Institutes of Health, laid out plans for what is a rarity in the realm of public health: a high quality clinical trial. The “Moderate Alcohol and Cardiovascular Health Trial,” known as MACH15, was to be randomized so that some subjects would be selected to drink and some would not. It would follow participants “prospectively,” over time, not retrospectively. And in the end, the results were to be adjudicated by evaluators blinded to which subjects had been instructed to drink and which to abstain. The goal was an assessment of the effect of alcohol consumption on cardiovascular health.

VIEWPOINTS
Partner content, op-eds, and Undark editorials.

But last month, the National Institutes of Health took the unusual step of shutting down one of its own clinical trials — a $100 million dollar experiment gone wrong. The announcement followed an internal investigation, prompted by a dogged New York Times report, that uncovered inappropriate interactions between the alcohol industry (Anheuser-Busch InBev, Heineken, and others) and the NIAAA in the execution of MACH15.

By law, federal health agencies can receive funding from for-profit industry. But direct courting of funding, coordination, and collaboration on research design, and excessive communications are not permitted, and according to The Times and the findings of the NIH’s subsequent internal investigation, these violations occurred early and often during the development of the MACH15 trial. The NIH report concluded that the actions uncovered “calls into question the impartiality of the process and thus casts doubt that the scientific knowledge gained from the study would be actionable or believable.”

Those are damning words, but perhaps the most salient thing to take away from the collapse of MACH15 is that the conflicts of interest at its core are probably not all that rare. The researchers here just happened to get caught.

The scrutiny applied to this trial, after all, is wildly unusual — and it only occurred because of The Times’ investigation, which unearthed years’ worth of incriminating emails, travel vouchers, and other indicators of questionable research ethics. This prodded NIH investigators to reassess the trial far more carefully than is standard. Doing so led them to observe what appears to have been an intentional bias in the framing of the study, which would have led to an inevitably positive outcome.

Here’s how it was supposed to have worked: Volunteers aged 50 or older with either substantial cardiac risk factors or existing heart disease were to have been randomly assigned to either abstain from alcohol completely or to consume precisely one alcoholic beverage per day. They were then to be monitored for cardiovascular health. Investigators were interested in whether the patients assigned one drink per day were more, less, or equally likely to experience a heart attack, stroke, or even die in just six years? This was primary goal of the MACH15 trial.

On closer inspection, however, the methodologic problems of the trial’s design are considerable. First, one drink per day may be moderate drinking for women, but even the NIAAA’s own definitions state that for men the upper limit of “moderate” is actually two drinks per day. Second, patients are known to be highly inconsistent when reporting their own alcohol consumption. Answers patients provide depend on many factors, including the doctor-patient relationship. And patients’ spouses often disagree with their partners’ assessment of their drinking. The MACH15 trial had some sophistication built in to it, including the use of random smartphone-based check-ins for patients. But while some evidence suggests that smartphone-based self-reporting on alcohol consumption often contradicts patients hindsight reports, MACH15 had no ability to tease out which patients would adhere to the smartphone check-ins, and which were providing accurate accounts of their consumption. In essence, the NIH was making a $100 million gamble that volunteers would portray their alcohol consumption accurately.

That’s not where the problems ended. In their report, the NIH investigators said they had found that the trial was apparently set up to lean towards “demonstrating a beneficial health effect of moderate alcohol consumption,” and that it was exceedingly unlikely to find that alcohol was harmful, because the trial design ignored some of alcohol’s most notable long-term risks, including cancer and congestive heart failure — the latter of which is linked to alcohol consumption. The eventual study would have under-reported cancer risks simply because, at six years, the trial was too short for those dangers to have caught up to patients who drink. But neglecting to include heart failure as an outcome of interest in a trial assessing cardiovascular health of alcohol is a far more flagrant oversight — akin, perhaps, to a study assessing the health risks of sugar intake in which researchers failed to ask whether patients developed diabetes. How the outcome measures of MACH15 ever passed muster defies logic, but in a statement attributed to Dr. Kenneth Mukamal, the study’s principal investigator, and supplied by his hospital’s public affairs office, he said the protocol was approved by “three external scientific panels and by ten institutional review board.”

The problem is that he’s telling the truth. It isn’t that Mukamal and his co-investigators told the panels they would assess for heart failure and then failed to do so. It’s that the review boards didn’t notice its glaring omission to begin with. Too often, these boards are prone to being bamboozled by experts whose knowledge of a particular field surpasses their own. More ominously, some board members may even be in on the game. Many review boards and peer-reviewers at top journals have their own industry conflicts of interest, for example. Reviewers are usually required to disclose conflicts to their committees, but those disclosures do not necessarily spell disqualification from participation.

Nor is it particularly newsworthy that research funded all or even partially by for-profit industry, even when ostensibly carried out by government and academic institutions, tends to produce data that favor its benefactors. In an email, Adam Dunn, an associate professor in health informatics at Australia’s Macquarie University, wrote that “most trials undertaken today are destined to produce a certain conclusion through designs that are manipulated just enough to almost guarantee a result without losing their integrity, so we can only spot differences retrospectively when looking at lots of trials in aggregate.” Dunn has published on the topic of industry’s influence on research and clinical practice — and the results are a kind of scientific gerrymandering. Like politicians choosing their voters instead of the reverse, modern scientists have perhaps become too adept at letting their beliefs shape and influence the data they collect.

Embedded in this observation is the challenge of detecting problems of trial design before research dollars are spent, or worse, dubious papers are published in highly respected journals. The sheer number of ongoing trials is daunting and searching for them prospectively is neither glamorous nor particularly appreciated by anyone other than scientific purists. Currently, there are 414 active NIH-funded studies that have at least some industry funding. One wonders what might be uncovered if the level of scrutiny MACH15 received were to be visited upon one of those 414 studies (the protocols for all of them can be searched at ClinicalTrials.gov).

Perhaps what sets MACH15 apart is not what it did, but who did it. The alcohol industry, while powerful, is far less experienced in the realm of biomedical research than large pharmaceutical companies who perhaps, by now, know just how to expertly game the research system without breaking the rules outright. Alternatively, it might be that alcohol itself is expected to play by a different set of rules than other would-be medicinal compounds. “Alcohol was never meant to necessarily benefit people, unlike medications,” wrote Florence Bourgeois, an assistant professor of pediatrics and emergency medicine at Harvard Medical School and a faculty member at Boston Children’s Hospital, in an email. “With drugs, pharma can justify their trials and marketing as an attempt to help people in need of medical breakthroughs. But alcohol is generally perceived as harmful to our society, so attempting to promote it on the false grounds of having health benefits doesn’t go over well in the lay media.”

While it is easy to point fingers at the researchers themselves, perhaps their alleged behavior is merely the inescapable consequence of a generation of steadily decreasing public funds available for medical and scientific research in the United States. In such a climate, seeking industry funding seems understandable. Through the lens of “publish or perish,” a pressured researcher bending the rules may suddenly seem almost pitiable, if not pardonable.

The question is, how closely are we willing to look at hundreds of other industry-funded studies that are likely to be vulnerable to these forces? The odds are high that the MACH15 debacle is not an isolated case. The closer we look, I predict, the more problems we will find.  

Jeremy Samuel Faust is an emergency medicine physician at Brigham and Women’s Hospital in Boston and an instructor at Harvard Medical School. He is the co-host of FOAMcast and the board president of the Grammy Award–winning vocal ensemble Roomful of Teeth. The opinions expressed in this article are solely those of the author and do not reflect the views and opinions of Brigham and Women’s Hospital.