Flawed Analysis of New Zealand Data Doesn’t Show COVID-19 Vaccines Killed Millions

SciCheck Digest

Data from around the world support the general safety of the COVID-19 vaccines. Some people online, however, incorrectly claim that illegally obtained data from New Zealand show the vaccines have killed 13 million people worldwide. Experts say the analysis is bogus.

How safe are the COVID-19 vaccines?

Full Story

Numerous studies have found the COVID-19 vaccines are quite safe, with only a few rare serious side effects. There’s nothing to support the notion that the COVID-19 vaccines are killing large numbers of people, as some people online have claimed for years. Despite the lack of evidence, the claims remain popular, revived periodically by various pseudoscientific analyses.

Most recently, Steve Kirsch, a tech entrepreneur who has become a major source of COVID-19 vaccine misinformation, has claimed that leaked data from the New Zealand government “prove” that the vaccines have killed on average 1 person for every 1,000 doses — or when extrapolated to the entire world, about 13 million people.

“There is no possible way that this data is consistent with a safe vaccine,” Kirsch wrote in his Substack.

Experts say this is wrong. Even if the underlying data are accurate, they can’t be used to make causal claims about the vaccines, Jeffrey S. Morris, director of the division of biostatistics at the University of Pennsylvania’s Perelman School of Medicine, told us. On top of that, he said, Kirsch’s methods are flawed and based on invalid assumptions — and his interpretations run counter to existing evidence.

“His methodology is extremely ad hoc and arbitrary and wrong in very specific ways,” Morris said of Kirsch.

Kirsch debuted his claims in a Nov. 30 presentation at the Massachusetts Institute of Technology, his alma mater, where he had been invited to speak by a student group. He also shared them in a Substack post, which has been shared on social media. Others have picked up his claims or circulated related claims about the New Zealand data online.

New Zealand Data Breach

A database administrator employed by Te Whatu Ora, or Health New Zealand, illegally gave Kirsch the underlying data he uses in his analysis. On Nov. 30, the worker appeared in a video with Liz Gunn, a New Zealander known for spreading conspiracy theories and her opposition to the COVID-19 vaccine, discussing the vaccine database information and making claims similar to Kirsch’s.

Soon after, the employee, a man named Barry Young, was arrested and charged for his role in the data breach, according to the New Zealand Herald. Te Whatu Ora said in a statement that there is “no evidence whatsoever that vaccination is responsible for excess mortality in New Zealand,” adding that Young has “no clinical background or expert vaccine knowledge” and that “[w]hat he is claiming is completely wrong and ill-informed and his comments demonstrate this.”

Te Whatu Ora was also granted an injunction to remove any database information that remains online to protect people’s privacy. In his Substack, Kirsch had posted the data, which he said had been anonymized, and encouraged others to download it to perform their own analyses. In some cases, people who have done so have had their file hosting accounts suspended.

Kirsch and others have tried to spin the arrest of the database administrator and subsequent removal of the data as evidence of government efforts to hide “the truth.” But Morris said Kirsch was downplaying legitimate legal issues.

“It’s a very serious thing to share identified data,” he said, adding that the information given to Kirsch “had the birth dates of the individuals, the dates they got their vaccines, the dates they died, which could uniquely identify every single person.”

Kirsch has argued that his sharing of the data is not a problem because it was anonymized, but it’s not necessarily clear yet whether the data contains any identifying information. Morris said he would have downloaded the data and done his own analyses to refute Kirsch’s claims, but he did not feel comfortable doing so given legal and privacy concerns.

Plausibility Check

There are multiple problems with Kirsch’s analysis that Morris and others have noted, as we’ll explain in more detail below. And tellingly, even many people who have spread misinformation about the COVID-19 vaccines before and believe the vaccines are killing people have pushed back on Kirsch’s claims.

But Morris also suggested looking at excess death graphs for New Zealand and other countries just to see whether Kirsch’s claims pass a basic sniff test — and they do not.

In much of the world, spikes in excess deaths closely correspond to when countries experienced COVID-19 deaths. There is no evidence that millions of people died from the vaccines.

New Zealand is a little different in that during much of the pandemic, the country actually had a deficit of deaths, likely due to the mitigation measures the island took and the success the nation had in keeping the coronavirus out. Those measures may have also reduced flu deaths and other kinds of deaths.

But there, too, the only real period of excess deaths occurs during the two waves of COVID-19 New Zealand experienced in March and August 2022, a year or so after vaccination began. Some excess deaths in 2022 could also be a kind of catch-up, Morris said, from older people who avoided dying in 2020 and 2021. 

Regardless, the pattern is clear, Morris said, and part of what happened in 2022 in New Zealand was a return to a baseline level of death after a couple of years of fewer deaths.

“What he’s saying is implausible,” Morris said of Kirsch.

Flawed Analysis with Incorrect Assumptions

According to his description, Kirsch’s analysis consists of what he calls a “time-series cohort analysis” of “record-level data” from 4 million out of the 12 million COVID-19 vaccine doses given in New Zealand. These doses, he says, were part of a pay-per-dose program in the country, which he claims were randomly administered.

Kirsch uses the data to plot death rates from any cause over time since vaccination, claiming that if the vaccine is safe, the graph should level off and be flat or decline three weeks after vaccination. If it’s not, he says, any increase reflects deaths caused by the vaccine. Using this flawed logic, he then calculates a death rate for all ages of 1 death per 1,000 doses, which he applies globally to arrive at his 13 million estimate of the number of people killed by the vaccines, and an estimate of 675,000 for the U.S.

There are numerous problems with this approach. To start, Morris said many of Kirsch’s assumptions are simply not true. 

“There’s nothing in the literature that says, oh, these plots should be completely flat. And if they’re not flat, the only explanation can be that the vaccine is causing death,” he said. “There’s no such thing.”

In theory, Morris said, the curve for a “safe vaccine” would be flat if there’s a constant death rate that never varies throughout the year, and if the decision to get vaccinated or additional doses is completely random. But that’s not the situation with real data. So in fact, Morris said, there are many reasons why a curve might increase, even if a vaccine is not killing people.

Susan Oliver, an Australian scientist who corrects misinformation, similarly explained in a YouTube video critiquing Kirsch’s claims that such curves are not expected to be flat for seasonal vaccines. “The reason for this is because deaths [due to any reason] don’t occur uniformly throughout the year,” she said. “They follow a seasonal trend with higher deaths in the winter months.”

The same could be true during a pandemic if many people are vaccinated around the same time, and then COVID-19 restrictions are lifted and a COVID-19 wave hits, as occurred in New Zealand. Or, as also happened in New Zealand, an abnormally low mortality rate rebounded to a normal level after a period of intense COVID-19 restrictions.

Kirsch has simply claimed that the vaccination records he has represent a random sample of the 12 million doses given in New Zealand. But he provides no evidence that this is true, instead arguing that others have to prove him wrong, incorrectly reversing the burden of proof. 

There is little public information about the pay-per-dose program, and Te Whatu Ora told us it could not provide further comment given the ongoing investigation and injunctions. However, it appears the pay-per-dose system was for reimbursing providers such as primary care practices, which would have been separate from mass vaccination clinics, for example. It is hardly clear that the populations served by both groups would be identical. And in any case, there remain important differences between the populations that get a different number of doses.

Additionally, while Kirsch is aware of what’s called the healthy vaccinee effect — a phenomenon in which, especially at first, vaccinated people will have an artificially lower rate of death than the overall population because very sick people would not be getting vaccinated — Morris said Kirsch arbitrarily decided that it ends after exactly three weeks. There’s no basis for that, he said.

Kirsch then uses the rate of death at three weeks as the baseline for death and counts all deaths above that as excess deaths caused by the vaccines. Again, Morris said it’s invalid to just assume that all excess deaths would be vaccine-caused — but also, the baseline may be completely incorrect.

Morris suspects that this is the case, noting that while Kirsch’s main analysis misleadingly focuses on all ages and all doses, when the rising death rates only exist for older people, one would really need to break the data down by age and by dose, and then use actuarial data for each age group to get some idea of an accurate baseline. The baseline Kirsch uses is much too low, he said.

The entire “increase” in deaths could simply be a recovery back to the actual baseline. “He hasn’t ruled that out at all,” Morris said of Kirsch.

This gets at another fundamental problem with Kirsch’s analysis: the lack of an unvaccinated group. Kirsch claims this isn’t needed for his type of analysis, but as Morris told us, Kirsch has no idea how the death rates in vaccinated or unvaccinated people compare.

“When you don’t even have the unvaccinated, then you have no calibration point because it’s possible that whatever you’re looking at in the vaccinated, it’s possible that their death rate is lower than the unvaccinated across the board,” Morris said. “So if that’s the case, how can you argue that the vaccine is killing people on the basis of this data?”

Indeed, while Kirsch presents his analysis as iron-clad proof that the vaccines have killed millions of people, it’s incorrect to even say that this data could provide that degree of certainty.

“The data that he got, even if it’s fully legitimate and accurate, cannot be used to answer the question that he wants to answer about causal effects of vaccines on death,” Morris said.

Kirsch also hypes the data, misleadingly claiming that “[n]o State or country has ever released record-level public health data on any vaccine” and casting his analysis as special.

Morris said that in fact, while no one else uses the term “record-level data,” such information has been used in numerous COVID-19 vaccination studies around the world, which have turned up no evidence of mass vaccine-related death. 

But unlike in Kirsch’s case, those researchers have a legal right to use the data, and the raw data are not shared because of privacy reasons. In addition, Morris said, that data usually include information on confounders, or other variables that might be associated with an outcome of interest such as death — precisely the data that would help make a causal connection, but something that is lacking in Kirsch’s dataset.

Editor’s note: SciCheck’s articles providing accurate health information and correcting health misinformation are made possible by a grant from the Robert Wood Johnson Foundation. The foundation has no control over FactCheck.org’s editorial decisions, and the views expressed in our articles do not necessarily reflect the views of the foundation.

Sources

Morris, Jeffrey S. Director, Biostatistics Division and Professor of Public Health and Preventative Medicine at the University of Pennsylvania, Perelman School of Medicine. Zoom interview with FactCheck.org. 8 Dec 2023.

“UPDATE: Unauthorised data breach and attempt to spread misinformation.” Statement. Te Whatu Ora/Health New Zealand. 8 Dec 2023.

Plummer, Benjamin and Lincoln Tan. “Police arrest man in connection with alleged Te Whatu Ora mass privacy breach of Covid vaccination data.” New Zealand Herald. 3 Dec 2023.

Griffiths, Ethan. “Te Whatu Ora employee charged with Covid-19 vaccination data breach granted bail.” New Zealand Herald. 3 Dec 2023.

“Reminder of vaccine safety and effectiveness following release of misinformation.” Statement. Te Whatu Ora/Health New Zealand. 1 Dec 2023.

“Investigation ongoing into release of data and spread of misinformation.” Statement. Te Whatu Ora/Health New Zealand. 3 Dec 2023.

Gorski, David. “Steve Kirsch’s ‘mother of all revelations’ about the ‘deadliness’ of COVID-19 vaccines goes poof.” Science-Based Medicine. 4 Dec 2023.

“Steve Kirsch’s claim that New Zealand data shows COVID-19 vaccines killed millions is based on a flawed analysis.” Health Feedback. 8 Dec 2023.

“First batch of COVID-19 vaccine arrives in NZ.” Press release. New Zealand government. 15 Feb 2021.

Back to the Science. “Record level stupidity – Steve Kirsch and the New Zealand data.” YouTube video. 13 Dec 2023.

Te Whatu Ora/Health New Zealand. Email to FactCheck.org. 12 Dec 2023.

“Briefing on the Government’s response to COVID-19.” New Zealand Parliament. 26 Aug 2021.