Estimating vaccine effectiveness by comparing case rates is seriously flawed
23 February 2022
What was claimed
Raw data from a recent Vaccine Surveillance report from the UKHSA shows a sharp drop in vaccine effectiveness against infection in most age groups
Our verdict
This is based on comparing Covid-19 case rates between vaccinated and unvaccinated individuals which is inappropriate.
A Facebook post claims that “raw data from a recent Vaccine Surveillance report from the UKHSA shows a sharp drop in vaccine effectiveness against infection in most age groups.”
It doesn’t. There are various reasons why this data should not be used to estimate vaccine effectiveness. Similar assertions have been a common theme of Covid-19 misinformation in recent months.
The data presented in the post compares the Covid-19 case rate among vaccinated people to the case rate among unvaccinated people in different age groups over time in England.
For example, the post claims that vaccine effectiveness in the 18-29 age group is “minus 75%”.
This is calculated by taking the difference between the case rate for vaccinated 18-29 year olds in the four weeks ending 19 December 2021 (2,809.1 per 100,000) and the case rate for unvaccinated 18-29 year olds in the same period (1,603.5 per 100,000) and dividing this difference (1,205.6 per 100,000) by the unvaccinated case rate.
This shows that the case rate among vaccinated 18 to 29-year-olds was 75% higher than among unvaccinated 18 to 29-year-olds.
But the UK Health Security Agency (UKHSA) says in the same report, in bold: “These raw data should not be used to estimate vaccine effectiveness.”
There are various reasons for this.
Perhaps most importantly, the case rate among unvaccinated people is very uncertain.
The UKHSA estimates this by subtracting the number of vaccinated people (which can be accurately counted based on vaccination records) from the total number of people on GP registers.
But GP records are expected to overestimate the total population of England (and therefore the total number of unvaccinated people), as they include people who are still registered with the NHS but moved abroad for example, and may double count people registered with more than one GP.
This means the unvaccinated case rates may be greatly underestimated.
The UKHSA also warns that being vaccinated is not the only difference that could affect case rates between vaccinated and unvaccinated people.
For example, the UKHSA says people who have never been vaccinated are more likely to have already caught Covid-19, giving them some natural immunity
These other factors, rather than vaccination itself, may be driving some of the difference in case rates between the two groups.
In its latest vaccine surveillance report the UKHSA says: “Vaccine effectiveness against symptomatic disease with the Omicron variant is substantially lower than against the Delta variant, with rapid waning. However, protection against hospitalisation remains high, particularly after 3 doses.”
Image courtesy of US Department of State
This article is part of our work fact checking potentially false pictures, videos and stories on Facebook. You can read more about this—and find out how to report Facebook content—here. For the purposes of that scheme, we’ve rated this claim as false because comparing case rates between vaccinated and unvaccinated people is an unreliable and inappropriate method to calculate vaccine effectiveness.
_K.jpg){kind=link}