Fact Check: Moderna, Pfizer Vaccines Do NOT Show Increased Risk Of Serious Adverse Events Of Special Interest In Randomized Trials

No Excess Risk

Are Pfizer and Moderna mRNA COVID-19 vaccines associated with an increased risk of serious adverse events of special interest? No, that’s not true: The U.S. Food and Drug Administration (FDA) said in an email to Lead Stories that the vaccines meet the agency’s rigorous standards for safety, effectiveness, and manufacturing quality. In addition, the authors of this study created their own subset of serious adverse events of special interest (AESI).

The claim appeared in a preprint article published by SSRN (archived here) on June 23, 2022, titled “Serious Adverse Events of Special Interest Following mRNA Vaccination in Randomized Trials.” It opened:

Introduction: In 2020, prior to COVID-19 vaccine rollout, the Coalition for Epidemic Preparedness Innovations and Brighton Collaboration created a priority list, endorsed by the World Health Organization, of potential adverse events relevant to COVID-19 vaccines. We leveraged the Brighton Collaboration list to evaluate serious adverse events of special interest observed in phase III randomized trials of mRNA COVID-19 vaccines.

This is what the post looked like on SSRN at the time of writing:

(Source: SSRN screenshot taken on Fri Jun 24 21:39:35 2022 UTC)

This article is a preprint, meaning it has not been peer-reviewed. This is how Columbia University describes a preprint:

‘Preprints’ are preliminary versions of scientific manuscripts that researchers share by posting to online platforms known as preprint servers before peer-review and publication in an academic journal. Preprint servers are publicly available online archives that host preprints and their associated data. They typically employ basic screening processes to guard against ‘offensive and/or non-scientific content and for material that might pose a health or biosecurity risk.’ However, these screening processes do not share the rigor of peer-review. …

The peer-reviewed journal publishing process is lengthy, whereas preprints can be available online in a matter of minutes.

Preprint articles don’t hold the same status in the medical community as those that have already cleared the rigors of peer review. Experts at the Mayo Clinic, Johns Hopkins Medicine, Oxford University and the National Health Service in the United Kingdom all declined to provide feedback on the findings of this article. Mayo representative Robert J. Nellis said this in a June 23, 2022, email to Lead Stories:

We are passing on this one. Mayo experts often comment on the findings or science of third party papers that have been accepted by peer reviewed journals, but not pre-prints.

SSRN.com is a site that offers public access to papers that various science journals have identified as being of interest. The SSRN article titled “Serious Adverse Events of Special Interest Following mRNA Vaccination in Randomized Trials” detailed the study’s methods, results and discussion:

Methods: Secondary analysis of serious adverse events reported in the placebo-controlled, phase III randomized clinical trials of Pfizer and Moderna mRNA COVID-19 vaccines (NCT04368728 and NCT04470427), focusing analysis on potential adverse events of special interest identified by the Brighton Collaboration.

Results: Pfizer and Moderna mRNA COVID-19 vaccines were associated with an increased risk of serious adverse events of special interest, with an absolute risk increase of 10.1 and 15.1 per 10,000 vaccinated over placebo baselines of 17.6 and 42.2 (95% CI -0.4 to 20.6 and -3.6 to 33.8), respectively. Combined, the mRNA vaccines were associated with an absolute risk increase of serious adverse events of special interest of 12.5 per 10,000 (95% CI 2.1 to 22.9). The excess risk of serious adverse events of special interest surpassed the risk reduction for COVID-19 hospitalization relative to the placebo group in both Pfizer and Moderna trials (2.3 and 6.4 per 10,000 participants, respectively).

Discussion: The excess risk of serious adverse events found in our study points to the need for formal harm-benefit analyses, particularly those that are stratified according to risk of serious COVID-19 outcomes such as hospitalization or death.

The study looks at data that was included in the FDA clinical review memo, statistical safety review memo and other public documents used in the authorization of the Moderna and Pfizer (Pfizer partnered with BioNTech) vaccines. The “harm-benefit analyses” called for in the study have already been performed by the FDA. This type of analysis can’t provide new information about product safety beyond what is already documented.

New trials would have to be conducted to determine if there’s a cause and effect between the vaccines and the serious AESI, which are pre-specified potential side effects.

Lead Stories asked Abram L. Wagner with the department of epidemiology at the University of Michigan for his interpretation of the study. This was his response in a July 1, 2022, email:

The pre-print compares the number of adverse events in those vaccinated compared to those not vaccinated. Even taking the results at face value, there needs to be a cautious interpretation of them. The data they analyzed included almost 40,000 individuals enrolled in clinical trials, and yet their results include wide confidence (or ‘compatibility’) intervals. This speaks to the very low number of adverse events as well as uncertainty in their analysis.

I have the following issues with the study. For one, they do not have access to individual-level data, and so have to compute statistics based on aggregate data. This limits a comparison of their data with what vaccine companies previously published. So in this pre-print, one individual reporting multiple adverse events over time is counted as multiple events, whereas in previous reports, this would just be one case of an adverse event. Their analysis therefore could result in double counting of events if they derive from a single adverse reaction to a vaccine.

Additionally, they modified the adverse events included in the original Bright Collaboration list. They do include the original list as a sensitivity analysis, and this sensitivity analysis shows a much broader, and therefore higher level of uncertainty in their data.

The authors of the study effectively created their own special list of AESI. Adverse events can be either serious or non-serious. The study only looks at what it calls the serious ones. It’s a category defined by the authors of the study and isn’t used by government agencies when putting vaccines through the authorization process.

In a June 24, 2022, email to Lead Stories, the FDA said it found the preprint article lacking:

FDA disagrees with the conclusions in the paper from [Peter] Doshi et.al. Based on the agency’s thorough evaluation of the safety and effectiveness data for the mRNA COVID-19 vaccines, as well as the ongoing safety surveillance of the vaccines, we continue to find their benefits far outweigh their risks in preventing COVID-19, including its most serious outcomes of hospitalization and death.

In a July 1, 2022, email to Lead Stories, the University of Maryland, Baltimore responded to a request for comment on the study:

The paper you referenced was co-authored by a faculty member in our school of pharmacy, Peter Doshi. Faculty at the University of Maryland, Baltimore are free to research and publish, and to engage in public discourse as they will, but in so doing they do not represent the opinions or views of the university.

VAERS data

The data on adverse events comes from the Vaccine Adverse Event Reporting System (VAERS). It is co-managed by the FDA and the Centers for Disease Control and Prevention as a crude early warning system and not as a database for the quantification of specific outcomes.

Anyone with internet access can add a report to the VAERS list of reports. The public access link to it expressly warns against unwarranted conclusions based on VAERS material because the list only provides a tally of unverified notes about any health event people experience after they are vaccinated.

The list itself cannot be used to prove or quantify, since all it shows is a chronological correlation, not the causal link that would be more difficult to establish. It’s the equivalent of a police precinct’s running “blotter” reports that may serve as a starting point for police work, not an endpoint.

VAERS website warnings include:

When evaluating data from VAERS, it is important to note that for any reported event, no cause-and-effect relationship has been established. Reports of all possible associations between vaccines and adverse events (possible side effects) are filed in VAERS. Therefore, VAERS collects data on any adverse event following vaccination, be it coincidental or truly caused by a vaccine. The report of an adverse event to VAERS is not documentation that a vaccine caused the event.

In its email to Lead Stories, the FDA said it takes adverse events seriously and investigates them:

It’s important to note that reports of adverse events to the Vaccine Adverse Event Reporting System (VAERS) following vaccination, including deaths, do not necessarily mean that a vaccine caused a health problem. In fact, reviews by FDA and CDC have determined that the vast majority of the deaths reported are not directly attributable to the vaccines. FDA requires healthcare providers to report any death after COVID-19 vaccination to VAERS, even if it’s unclear whether the vaccine was the cause. Reports of adverse events to VAERS following vaccination, including deaths, do not necessarily mean that a vaccine caused a health problem.

The FDA continued in its email:

Reports of death after COVID-19 vaccination are rare. As reported on CDC’s website, More than 592 million doses of COVID-19 vaccines were administered in the United States from December 14, 2020, through June 21, 2022. During this time, VAERS received 15,193 preliminary reports of death (0.0026%) among people who received a COVID-19 vaccine. CDC and FDA clinicians review reports of death to VAERS including death certificates, autopsy, and medical records.

To put into perspective the approximate number of doses of vaccine administered during a specific time frame after authorization of COVID-19 vaccines we suggest that you reference CDC’s data at https://covid.cdc.gov/covid-data-tracker/#vaccinations_vacc-total-admin-rate-total. CDC clinicians review reports of death to VAERS including death certificates, autopsy, and medical records.

FDA continues to find the three authorized and approved COVID-19 vaccinations meet the agency’s rigorous standards for safety, effectiveness, and manufacturing quality. These vaccines have proven to be an important tool for fighting the COVID-19 pandemic.