Interview: How the U.S. Government Monitors Vaccine Safety

In late November, the Food and Drug Administration’s top vaccine regulator, Vinay Prasad, made headlines with an email to his staff announcing that the agency had linked the Covid-19 shots to the deaths of a small number of U.S. children. A team of career employees, he explained, had reached this conclusion by analyzing deaths that had been reported to a government-run database. “This is a profound revelation,” Prasad wrote.

The database, the Vaccine Adverse Event Reporting System, known as VAERS, serves as a repository to which people can voluntarily report health problems that occur after receiving a vaccine. While the database alone can’t determine causality, the FDA team has been conducting a deeper analysis of individual reports of death, Prasad wrote. His email, which was leaked to the press, didn’t offer many specifics. The agency has since told the press that it will release a complete report by the end of the month.

In the meantime, a dozen former FDA commissioners have written a dissenting response to the memo, which landed at a time of tremendous scrutiny of vaccine safety in the United States. Health and Human Services Secretary Robert F. Kennedy Jr. has assigned government positions to some well-known vaccine critics who appear eager to overhaul the country’s approach and whose views have sparked concerns from inside the public health community.

Against this politically charged backdrop, Jeffrey Morris, director of the Division of Biostatistics at the University of Pennsylvania’s Perelman School of Medicine, has published a new paper assessing the country’s vaccine safety surveillance system. As a biostatistician, he has a unique perspective on how the various components of the system — including VAERS — work together to generate data that can then be used to identify potential safety issues.

In a recent interview, Morris called for moderation. “In today’s public debates, people often gravitate towards extremes, either dismissing the vaccine safety monitoring system as broken or negligent — or treating it as if it’s beyond criticism,” Morris told Undark. “Neither position is helpful,” he added.

Morris’ paper describes an integrated system with four components. The first are the randomized clinical trials that pharmaceutical companies conduct in order to get their vaccines approved by the FDA. These studies are the gold standard for determining safety and efficacy, Morris said, but they have limitations. The studies typically aren’t large enough to detect bad outcomes that occur very rarely — say, 1 in 10,000 people.

To get a handle on such rare events, public health officials often turn to the system’s other components. These include passive monitoring via VAERS, so-called “active” surveillance systems that gather and assess information from electronic health records, and then case-level evaluations. The latter allow for a careful look at individual cases in which patients may have experienced a vaccine-related health problem.

Undark sat down with Morris to discuss how vaccine safety data are sometimes misused or misinterpreted to make vaccines appear less safe than they are. He also discussed the link between Covid-19 vaccines and myocarditis, or heart inflammation, which emerged during the pandemic as a rare but potentially serious vaccine side effect occurring at elevated rates in teenage boys and young men. Morris also shared the criteria he’ll use to assess the FDA’s analysis once it’s available to the public.

Morris’ paper was published by the Annenberg Public Policy Center. Our interview was conducted over Zoom and has been edited for length and clarity.

UD: People sometimes misunderstand how VAERS is intended to be used. Can you talk about this?

Jeffrey Morris: First, some people mistakenly assume that every health problem reported after vaccination was caused by the vaccine. In reality, medical events happen every day for many reasons, and some will occur by coincidence soon after vaccination — especially when something is given to tens to hundreds of millions of people. To evaluate safety properly, it’s necessary to ask whether the event happens more often than expected after vaccination compared with the normal background rate.

A second issue is that others look only at the raw number of reports over time or compare different vaccines without considering how many people received each vaccine or how reporting behavior might have changed. For example, I’ve seen a lot of people plot graphs that show very low reports of an event and [then] a huge spike in 2021, and use this as evidence that that event must have been caused by the Covid vaccines. But during the Covid-19 pandemic, heightened media attention and public awareness led to an enormous surge in VAERS reporting. Virtually any event was reported much more in 2021 than ever before, including such things as sunburn, pregnancy, broken bones.

UD: Are active surveillance data sometimes misinterpreted as well?

JM: Yes, sometimes they are. One naive approach is the use of simple, unadjusted comparisons of the events between vaccinated and unvaccinated groups with the reports interpreted as evidence of harm from vaccines. These analyses fail to account for important differences between the groups that can themselves affect health outcomes.

And also, some published analyses have used blatantly flawed methods. One clear example is a series of nine case control studies published between 2015 and 2018 by HHS data scientist David Geier, using the Vaccine Safety Data Link active reporting system data. These studies reported large increases in autism and other chronic conditions following hepatitis B vaccination. When I examined the methods in the papers, I identified a basic but serious problem in how the study groups were constructed.

These studies are completely invalid — yet they’re still cited by many talking about the risk of harm from hepatitis B vaccines today.

UD: You have some ideas for strengthening the current system. Can you give me an example?

JM: I’d recommend improved methods for identifying potential safety signals in VAERS. This includes studies to better understand how different types of events are underreported, approaches to account for underreporting when evaluating the signals, and for more rigorous modeling of background event rates so that the reports occurring more frequently than expected could be identified more accurately.

UD: Let’s shift to vaccine-induced myocarditis. This is a rare but potentially very serious event that tended to occur in adolescent boys and young men. How was this initially detected?

JM: Israel was the first to detect the myocarditis risk. They first reported it in April 2021.

These reports were then soon confirmed in other regions, including the European Union, the USA, Canada, Nordic countries, and other places. Then in May 2021, the World Health Organization’s Global Advisory Committee on vaccine safety acknowledged the myocarditis risk. Shortly after that, the FDA’s Vaccines and Related Biological Products Advisory Committee discussed the issue, and the risk was added to the mRNA vaccine labeling by the FDA in June 2021.

UD: In the first half of 2021, as health officials were looking into whether [Covid] vaccines had caused myocarditis in a small number of young adults, Pfizer announced results from its clinical trial of 12- to 15-year-olds. The drugmaker found that the vaccines were effective and well tolerated among 1,131 children who received it.

Given the unknowns about myocarditis at that particular point in time, was the study large enough to warrant emergency authorization of the vaccines for adolescent boys who are at low risk from Covid?

JM: As I mentioned previously, randomized clinical trials are typically not large enough to detect rare events. That includes myocarditis in this case.

Pediatricians did not consider the risk of Covid to be negligible. One major concern was [that Covid can cause] multisystem inflammatory syndrome in children, MIS-C, which was observed in roughly one in 3,000 pediatric confirmed infections in 2021, and typically it also involved cardiac inflammation. MIS-C typically occurred three to five weeks after infection, frequently required intensive care, and carried a measurable risk of organ dysfunction and death on the order of 1 to 2 percent. Reducing Covid-19 infections in this age group, therefore, offered a clear clinical benefit of reducing risk of MIS-C, in addition to any other risks.

But both MIS-C and vaccine-associated myocarditis are rare outcomes that would not have been detected in any [clinical trial], even if it had been much longer and larger. Identifying and characterizing these types of risks necessarily relies on large observational studies — and when it comes to safety, post-authorization observational studies within the vaccine safety monitoring system.

UD: Now that we have some large studies that we can look back on, do you think that all age groups experienced a net benefit from an initial round of vaccination against Covid-19? And what are your thoughts on boosters?

JM: Based on the full body of evidence, my overall conclusion is that in 2021 after rollout, they provided a clear net benefit virtually for all adult age and sex groups.

The one group where the balance might have been closer is young adult men, with the actual benefit-risk depending on the actual rate of myocarditis, downstream risks, and how these counteract the reduction of infections, severe disease, and risk of post-infection sequelae that was provided by the vaccines, especially during the delta waves that were extremely damaging and in the U.S. really led to a huge noticeable uptick in deaths, even in young and middle-aged adults.

When we talk about the pediatric populations: For pre-pubescent children, the MIS-C risk in 2021 was greater than any of the risks for the vaccine in that age group, especially considering the myocarditis risk was not high for pre-pubescent children at all. So even without a high risk of death from Covid-19, the vaccine seemed to have provided a positive net benefit for the pediatric population.

But for teens — especially teen boys — it may have been closer depending on how the rates of Covid, MIS-C, and myocarditis compare to each other.

The question of boosters really requires its own analysis. Just my own personal view: I’ve been skeptical about the need for universal annual boosters from the start and wrote about this in late-2021. I see strong evidence in the literature that older, more vulnerable groups likely benefited from it. For the other age groups, it’s less clear.

UD: One international study was a Korean study of 44 million vaccinated individuals. Researchers found that the vaccines caused 21 deaths from myocarditis. Do you think this has implications for the United States? In your view, how likely is it that a tiny fraction of people in the U.S. died from the Covid-19 vaccines?

JM: Given its population-level design, the study provides highly valuable information on post-vaccination myocarditis risk, including who was affected, incidence rates, timing, and the likelihood of severe fatal outcomes. Its findings were also broadly consistent with results from similar studies conducted worldwide.

The available data clearly shows that widely circulating claims that Covid vaccines caused hundreds of thousands or millions of deaths — or were a major driver of excess mortality during or after the pandemic — are clearly false. But at the same time, it remains plausible that in rare instances, post-vaccination myocarditis or other factors may have contributed to some deaths occurring after vaccination.

UD: The FDA says it will release more details of its VAERS analysis by the end of the month. What are your thoughts on this effort, and what kinds of details will you be looking for to determine that the analysis is solid?

JM: I think this type of effort, if rigorously and transparently done, is extremely valuable, as the Korean study demonstrated. It’s an important effort. I’m really happy that they’re going to release the details. It’s critical that the details are going to be released.

To do this type of analysis to adjudicate cause of death, it’s necessary to follow up and get verified medical details and more complete demographic and medical history information. When made public, their report should clarify what information was available, whether it was verified, which experts conducted the analyses — and what their expertise was — and what level of evidence they had for their various conclusions of likely, probable or possible, or unlikely association with the vaccines.

Ideally, they’ll follow the CARE standards for completeness, methodological rigor, and transparent documentation.