Fact Check: ‘Survey’ Does NOT Prove Vaccinated Over ‘2X More Likely To Get COVID Than the Unvaccinated’

Analysis Flaws

Did a “survey” find that “Vaccinated people are over 2X more likely to get COVID than the unvaccinated”? No, that’s not true: The survey in question was not scientifically sound and included methodological flaws, an infectious disease expert told Lead Stories. The claim originated in a blog post by Steve Kirsch, whose false claims regarding COVID-19 vaccines have been debunked by Lead Stories before. Research that is considered scientifically sound is often reviewed by scientific experts in the subject matter or published in a reputable scientific journal. In credible research, the data are combed over by experts, and calculations are checked. Kirsch’s survey is neither of those.

The claim appeared in a newsletter posted by Kirsch, whose claims Lead Stories has previously fact checked, to the self-publishing website Substack on August 27, 2023. Titled “Vaccinated people are over 2X more likely to get COVID than the unvaccinated. Whoops!” (archived here), the article opened:

According to the survey I just did of over 16,000 people, fully vaccinated people are more than 2X as likely to get COVID than the unvaccinated.

What I found was fascinating is that if you did get COVID, the probability of being hospitalized was identical in the two groups.

My survey results are publicly viewable and all the entries can be verified (I have the contact info for each respondent). Happy to provide to any mainstream media fact checker.

Bottom line: The CDC lied, people died. If you disagree, please post your row-level data survey results.

Below is how the post appeared at the time of this publication:

(Source: Substack screenshot taken Mon August 28 01:04:00 UTC 2023}

Kirsch claimed that the survey he “just did of over 16,000 people” found that “fully vaccinated people are more than 2X as likely to get COVID than the unvaccinated.” This was determined, according to Kirsch, by an analysis of a “database” of responses to a survey (archived here) he completed “in <24 hours.”

According to Kirsch, his survey results indicated that:

• The vaccines in general make [a person] 2X as likely to get COVID.

• If [a person does] get COVID (which is half as likely for the unvaccinated), the COVID vaccines do NOTHING to reduce [the] risk of hospitalization.

• The vaccines will increase [a person’s] risk of dying from COVID … by up to 3x.

Dr. Richard Novak, professor and head of the division of infectious disease at the University of Illinois, Chicago College of Medicine, told Lead Stories that the survey is not “a randomized, controlled study,” indicating that it was not scientifically sound and that it included methodological flaws.

“Some of [the people] admit to not testing for COVID because they do not trust the test either, so it is very hard to know who on the list actually had COVID, and how often,” Novak wrote in an email received on August 28, 2023.

“So, there is no way to validate this data. How did he survey people who were dead to determine the risk of dying?”

Below are several such examples of survey respondents who admitted that they did not test for COVID:

• Comment 130: “I never tested for ‘covid’ because the tests are meaningless, but I have not have symptoms attributable to what is called ‘covid’ during the relevant timeframe.”

• Comment 306: Even when I was sick, I would never test for Covid, it’s bullshit. I treat the symptoms.

• Comment 2,632: “I didn’t test for any virus to verify my sickness, to give it a name.”

• Comment 17,324: “Never tested, no jabs.”

• Comment 17,370: “I have never tested but to my knowledge I have never had covid.”

Survey respondents reflect selection bias

The pool of survey respondents also reflects what is known as selection bias, or when the study population is not representative of the target population. In this case, the people who responded to Kirsch’s survey were likely readers of his anti-vaccine blog, skewing the data due to self-selection.

“A quick look at the ‘data’ indicates a lot of anti-vaxxers were on the list, which means there is a group of people who read his blog and are predisposed to disbelieve the efficacy of vaccines, so it is highly biased, to begin with: we call this selection bias,” Novak told Lead Stories.

For example, below is a small example of some of the comments pulled from the survey to illustrate how ideologically driven some of the respondents may have been:

• Comment 1: “Covid is stupid and I hate it.”

• Comment 140: “Covid felt dangerous, strange.”

• Comment 180: “They will have to tie me down to inject me with their poison.”

• Comment 206: “Phuck Pfizer!”

• Comment 1812: “I am convinced covid vaccines cause serious injuries and death.”

• Comment 17,281: “I fear the ‘vax’ WAY more than I fear COVID and I am 76 years old. Thank you for your work!”

Methodological flaws are present in survey response collection and analysis

There are inherent flaws with self-reporting data and Kirsch did not independently vet responses against hospital data or vaccination status. As we have previously reported, research that is considered scientifically sound is often reviewed by scientific experts in the subject matter or published in a reputable scientific journal. In credible research, the data are combed over by experts, and calculations are checked.

Neither of those was done in Kirsch’s survey.

Kirsch generalized his findings to populations beyond the skewed survey respondents. This does not provide any information about other groups of vaccinated individuals and variables that may impact their infection rate, including age, overall health and type of vaccine, among other factors.

Additionally, Kirsch did not differentiate between the types of vaccine analyzed nor did he specify whether such a vaccine was an initial dose or subsequent booster. So potential conclusions are not necessarily relevant to all vaccines available on the market.

Kirsch drew conclusions about hospitalization rates yet such a question was not included in the survey, and Kirsch never explained from where he collected that data.

Lastly, it appears that the survey data was analyzed in less than 24 hours as the questions were posted on August 26, 2023 — just a day prior to the article that is the focus of this fact check.

Survey oversampled people who were unvaccinated against COVID

In addition to a generally biased respondent pool, the survey also oversampled individuals who were unvaccinated. Lead Stories determined this by going into the survey results hosted by Airtable and following the below steps:

This will show a count for this condition in the first row of the table: “Count 15554,” meaning 15,554 survey respondents said they were not vaccinated against COVID out of nearly 16,000 responses. Repeating this for the “# Covid shots” values indicated that roughly 2,000 of the respondents reported getting any number of vaccinations. Together, these findings show that the survey pool is not a representative sample and is heavily skewed toward the unvaccinated.

COVID vaccines are deemed safe and effective by U.S. health officials

As of August 31, 2023, COVID vaccines are considered safe and effective by U.S. health officials.

“The existing well-controlled prospective studies of the vaccines clearly showed they were highly efficacious, so the efficacy of the vaccines has already been proven,” Novak told Lead Stories.

Lead Stories has also reported that it is not true that one in every 35 people who received a Moderna COVID mRNA-1273 booster vaccination suffered myocardial injury, that a July 2023 study did not prove that the Pfizer mRNA COVID vaccine causes “turbo cancer” and that a reanalysis of Pfizer and Moderna mRNA COVID vaccine data did not show an increased risk of serious adverse events of special interest.

Other Lead Stories articles on COVID and vaccines are here and here.