Posts Distort Questionable Study on COVID-19 Vaccination and EMS Calls

“The findings raise concerns regarding vaccine-induced undetected severe cardiovascular side-effects and underscore the already established causal relationship between vaccines and myocarditis, a frequent cause of unexpected cardiac arrest in young individuals,” the authors wrote in the abstract.

Following publication, the study was heavily criticized online and has been accessed 518,000 times, receiving one of the highest attention scores given by a data company that tracks quality and quantity of online attention for published research. 

“Unfortunately, despite the somewhat cautious language used by the authors, this paper clearly fails to adhere to sufficient standards of scientific rigor throughout,” reads an unpublished manuscript asking for the retraction of the study, posted online on May 5 and signed by 10 scientists. “While the authors have described their analysis as ‘not establishing causal relationships’ … they have in fact not even established useful correlations. The paper does not pass basic statistical and epidemiological review, which brings into question whether the findings do indeed ‘raise concerns regarding vaccine-induced undetected severe cardiovascular side-effects’ … or whether the reported weak correlations are simply the result of inadequate and inappropriate methodological choices.” 

The study, which has been referred to as an MIT study published in Nature, was not actually published in the prestigious scientific journal Nature. It was published in Scientific Reports, a peer-reviewed open access journal that is part of Nature Portfolio, a group of journals that includes Nature published by Springer Nature.

On May 5, the journal added a note alerting readers that “the conclusions of this article are subject to criticisms that are being considered by the Editors.”

But that didn’t stop online publications from citing the study to suggest the vaccines were causing harm, without mentioning any of the disputes. New versions of the story have continued to pop up in June. 

Rafal Marszalek, Scientific Reports’ chief editor, told us in an email the investigation into the paper is still ongoing and that he can’t comment on it for confidentiality reasons. “[W]e are considering the matter very carefully, in line with COPE Guidelines and Springer Nature’s policies,” he wrote, referring to the standards set up by the Committee on Publication Ethics. 

Criticism of the Study

As we explained earlier, the Scientific Reports study looked back at data from the Israel National EMS, which manages all emergency medical services calls in Israel, and analyzed all calls related to cardiac arrest and acute coronary syndrome from Jan. 1, 2019, to June 20, 2021. The call dataset was coupled with COVID-19 infection rates and vaccination rates over the same period of time. 

The authors looked at how the call counts changed during the pandemic and vaccination campaign compared with the pre-pandemic period of time, and found increases of over 25% in calls for both events from January to May 2021 compared with 2019 and 2020. They concluded there was a “robust and statistically significant association” between the call counts and the vaccines being administered to people ages 16 to 39. 

But critics have said the correlation is “clinically irrelevant” and there are several problems with the methodology, statistical analyses, and in the data visualization. 

The call database used in the study, for instance, did not include vaccination status, COVID-19 status or any underlying comorbidities of the patients. Therefore, there is no way of knowing if the increase of calls was among vaccinated people, or if they had COVID-19 or any other disease that could have caused the cardiac event. The authors identify this as the “main limitation” of the study.

“Those missing data points make any interpretation invalid and any extrapolation irrelevant,” wrote the authors of the rebuttal manuscript, titled “Should we publish every correlation during the COVID-19 pandemic?”

Critics of the Scientific Reports study have also pointed out that the difference in the number of calls is not really substantial, and that because the incidence of cardiac events varies a lot by year and season and can be caused by multiple variables, the correlation might be coincidental. Since there is no data before 2019, some have said it is hard to understand how these changes compare with earlier years.

A group of vaccine experts called Voices for Vaccines speculated the increase could also be related to resumed physical activity as COVID-19 cases decreased and the weather got warmer, for example. 

Gideon Meyerowitz-Katz, an epidemiologist working on his Ph.D. at the University of Wollongong in Australia and one of the authors of the rebuttal, found the graphs in the paper to be “INCREDIBLY misleading.” 

“The authors have got 3 axes there, one of which starts at 4, and it’s incredibly easy to manipulate the axes to remove this apparent relationship entirely,” he wrote on a Twitter thread about the study. When downloading the data and making a corrected graph, he also found COVID-19 cases appear to be more related to cardiac arrest calls than vaccines. 

17/n This is a classic @callin_bull point – here is the original graph vs my extracted data with one tiny change to the axes (can you spot it?)

Suddenly, it looks like COVID-19 cases are more closely tied to CA calls than vaccines! pic.twitter.com/gjJgksVjnb

— Health Nerd (@GidMK) May 5, 2022

The authors of the rebuttal also noted that the authors of the study made several “vague and confusing statistical choices.” For example, they “approximate[d]” the data of infection counts for people ages 16 to 39, using a COVID-19 infection database that registered cases for the age group 0 to 39. 

“This is just…wrong? You can’t ‘approximate’ the COVID-19 cases for one age group using another, even if they’re overlapping. The risk of getting COVID-19 is highly variable by age, so this could easily confound any correlations you’re looking at,” wrote Meyerowitz-Katz on Twitter. 

Meyerowitz-Katz added that the authors used a post-hoc power analysis, a method that has been criticized for being flawed and misleading. 

Some findings of the study also contradict clinical evidence on myocarditis following COVID-19 vaccination. For instance, as we have reported, most cases of vaccine-induced myocarditis have been observed in young males, following a second dose of mRNA vaccines. But the study does not find much of a difference in calls after doses one and two, and both types of calls increased more in females than in males. The authors also said they opted to use biweekly counts of first and second vaccine doses because studies “suggest myocarditis typically appears within two weeks from vaccination.” But according to CDC data, most myocarditis cases occur within a few days of vaccination. 

One of the authors, Retsef Levi, told us in an email that the study’s finding are “merely correlation and NOT proving causality!” and that the paper suggests more studies should be done to “better understand causal mechanisms that could be any combination of COVID-19 infections, vaccines and potentially other factors, such as changes in patient behaviors.”

He told us the paper was driven “by the known fact that both COVID-19 infections and vaccines could cause adverse cardiac related outcomes, for example, myocarditis.” And added that it “uses unique comprehensive country level EMS data” and “well-accepted statistical methods to study temporal correlations between infections, vaccination level and the respective weekly EMS calls.”

“I do not support any interpretation of the paper as a proof that the vaccines have caused this increase in EMS calls. The paper only shows correlation and calls to check the matter broadly and explore all possible causes,” he wrote.

The Israeli Ministry of Health also assessed Levi’s claims, and found the data did not support them, consistent with other studies from Israel. 

Jeffrey S. Morris, director of the Division of Biostatistics in the Department of Biostatistics, Epidemiology and Informatics at the University of Pennsylvania Perelman School of Medicine, said the study seemed to be part of a trend.

“Much of what I am seeing is people presenting more examples of time series for select places and times, or more scatterplots, considering this validation of a narrative of vaccine harm. Many times I don’t see an acknowledgement of the limitations of these approaches … or acknowledgement that these are hypothesis that need validation,” he wrote on Twitter on May 4. “A few like this Scientific Reports public paper include disclaimers calling for validation, but are written like they believe the fact is demonstrated by their analysis and just needs some confirmation.”

Editor’s note: SciCheck’s COVID-19/Vaccination Project is made possible by a grant from the Robert Wood Johnson Foundation. The foundation has no control over FactCheck.org’s editorial decisions, and the views expressed in our articles do not necessarily reflect the views of the foundation. The goal of the project is to increase exposure to accurate information about COVID-19 and vaccines, while decreasing the impact of misinformation.

Sources

Block, Jason P., et al. “Cardiac Complications After SARS-CoV-2 Infection and mRNA COVID-19 Vaccination – PCORnet, United States, January 2021-January 2022.” Morbidity and Mortality Weekly Report. 1 Apr 2022.

Xie, Yan, et al. “Long-term cardiovascular outcomes of COVID-19.” Nature Medicine. 7 Feb 2022. 

Christopher L. F. Sun, et al. “Increased emergency cardiovascular events among under-40 population in Israel during vaccine rollout and third COVID-19 wave.” Scientific Reports. 28 Apr 2022. 

Voices for Vaccines (@Voices4Vaccines). “Recently, a paper in Scientific Reports began circling that suggests mRNA vaccines may have yet undetected heart risks. The authors used EMS data in Israel looking at calls for cardiovascular events. They also draw some conclusions that go beyond their data. A 🧵 (1),” 1/27. Twitter. 2 May 2022. 

Meyerowitz-Katz, Gideon (@GidMK). “This paper was published recently, and has gone hugely viral because it seems to indicate that COVID-19 vaccines in Israel caused cardiovascular events in young people The paper has some serious flaws. I’m quite surprised that it was published 1/n,” 1/29. Twitter. 4 May 2022.

Morris, Jeffrey S. (@jsm2334). “Much of what I am seeing is people presenting more examples of time series for select places and times, or more scatterplots, considering this validation of a narrative of vaccine harm. Many times I don’t see an acknowledgement of the limitations of these approaches,” 1/2. Twitter. 4 May 2022. 

Florens, Nans, et al. “Should we publish every correlation during the COVID-19 pandemic?” OSF. 5 May 2022. 

Rafal Marszalek. Scientific Reports’ chief editor, email to FactCheck.org via Alice Kay, Springer Nature senior communication manager. 8 Jun 2022. 

Oster, Matthew E., et al. “Myocarditis Cases Reported After mRNA-Based COVID-19 Vaccination in the US From December 2020 to August 2021.” 25 Jan 2022.

Kracalik, Ian. “Myocarditis Outcomes Following mRNA COVID-19 Vaccination.” CDC presentation to Advisory Committee on Immunization Practices. 4 Feb 2022. 

Jaramillo, Catalina. “Benefits of COVID-19 Vaccination Outweigh the Rare Risk of Myocarditis, Even in Young Males.” FactCheck.org. Updated 5 Apr 2022. 

“Fact Check-Study using Israeli emergency services data does not prove COVID-19 vaccines cause heart problems.” Reuters Fact Check. 10 May 2022. 

“Cardiac arrest and acute coronary syndrome diagnosed in MDA.” Information Division National Center for Disease Control Ministry of Health, Israel. Accessed 13 June 2022.