Pfizer vaccine's real-world effectiveness in protecting against COVID-19 admission and death in the Discovery Health client population

Authors:
Shirley Collie (Chief Healthcare Analytics Actuary, Discovery Health) and Jared Champion (Senior Actuary, Discovery Health)

Investigating the Pfizer COVID-19 vaccine's real-world effectiveness in the Discovery Health administered medical scheme member base

- The difference between vaccine "efficacy" and "effectiveness"

Real-world vaccine effectiveness may differ from vaccine efficacy. That's because vaccine efficacy is established under ideal or controlled conditions (the tightly controlled conditions of randomised clinical trials).

Once a vaccine is rolled out into the real world, we begin to gather data in a less controlled setting and this data refers to real-world vaccine effectiveness i.e., a COVID-19 vaccine's protective capacity under normal conditions.

Clinical trials are required to establish not only efficacy but also safety and are key to a vaccine being authorised under emergency use or approved for public access. However, once public mass vaccination programmes begin, it remains important, to assess the effectiveness levels of vaccines in scaled real-world rollouts, to ensure a vaccine's ongoing relevance.

Vaccine effectiveness may differ from reported efficacy. This is due to variation in vaccination schedules, vaccine-handling logistics, the emergence of viral variants not considered during a clinical trial, and other factors.

Investigating three areas of interest: Real-world effectiveness of the two-dose Pfizer COVID-19 vaccine

We consider:

1) The effectiveness of the Pfizer vaccine over time in reducing the relative risk of COVID-19 hospital admissions and COVID-19 death, in both partially and fully vaccinated people.

2) The effect of individual or underlying risk factors on the two-dose Pfizer vaccine's effectiveness in reducing the relative risk of COVID-19 hospital admissions (across different segments of the Discovery Health population).

3) Super immunity, or the significant protection from COVID-admission enjoyed by people who have recovered from COVID-19 in the past, and who are also fully vaccinated.

• This third section is not covered in this post, but in our related, stand-alone article on this theme.

First, how did we establish real-world effectiveness?

To assess real-world vaccine effectiveness, we applied a test-negative design (TND) study. The TND method is commonly used to assess the effectiveness of annual influenza vaccines. This method compares the vaccination status of test-positive cases (pathology specimens testing positive for the virus in question) and test-negative controls (pathology specimens testing negative for the virus in question) (Jackson ML, 2017).

The literature and research conclude that the TND provides vaccine effectiveness estimates consistent with randomised control studies. In other words, estimates that are equivalent to population-based cohort studies (provided the data is adequately adjusted for confounding variables which influence the risk of the outcome not being measured correctly - such as age, sex, and co-morbidities) (De Serres G, 2013 Sep 12).

(1) Pfizer vaccine effectiveness (by days since vaccination) in reducing relative risk of COVID-19 admission and death

- Our methodology

We analysed 526 516 pathology test results, 14 673 COVID-19 admissions, and 3 441 COVID-19 deaths.

All of these occurred between 17 May 2021 (the start of South Africa's mass vaccination campaign) and 23 September 2021.

We adjusted the data for the following confounding variables:

• Age
• Sex
• Number of risk factors (as per the US CDC's high-risk criteria for serious COVID-19 illness)
• The province within South Africa where scheme member is located
• Calendar week
• Medical Scheme Plan grouping (in tertiles, based on premium ranges and volumes of members)
• Documented prior infection - from PCR pathology test results in categories of: None, 0 to 89 days ago, 90 to 179 days ago and, more than 180 days ago.
  • No more than three test results per patient were included. We randomly selected three of these tests amongst patients with four or more test results meeting the above-mentioned criteria.

The following pathology test results were excluded from the analysis:

• Test results for individuals vaccinated with the J&J COVID-19 vaccine (Ad26.COV2.S)
• Negative test results within 21 days of positive test results
• Negative tests results within seven days of each other

- Our findings

In summary: We find the Pfizer COVID-19 vaccine 78% effective at 28 days post-dose one and 94% effective 14 days post-dose two, in reducing the relative risk of hospital admission.

Figure 1 below provides the relative risk (odds ratio) of vaccinated individuals having a COVID-19 admission after a certain number of days since their Pfizer vaccine dose.

At 28 days post a first vaccine dose, relative risk stabilises to 0.22.

- In other words, at this stage, the first Pfizer COVID-19 dose of vaccination is 78% (1-0.22) effective in reducing the risk of being admitted to hospital for a COVID-19 infection (relative to an unvaccinated individual).

At 14 days after dose two, the relative risk rate (odds ratio) stabilises to 0.06.

- In other words, at this stage, Pfizer vaccination is 94% effective at reducing COVID-19 admission risk (relative to an unvaccinated individual).

Figure 1: Odds of COVID-19 hospital admission, in days since vaccination and relative to an unvaccinated individual.
- What is the effect of "wellbeing bias" in the first three days post a vaccine dose?

It is interesting that between days one and three post-vaccine dose, we note a significant reduction in relative risk of admission to hospital for the Pfizer COVID-19 vaccinated population.

We are seeing a "wellbeing bias" in the vaccinated cohort, which is evident in the ten days post-vaccination - and especially from days one to three. The lower odds of COVID-19 admission seen cannot be attributed to the clinical effectiveness of the vaccine in this early period. Randomised control studies started showing vaccine efficacy from day ten post-vaccination onwards (Polack FP, 2020).

These results likely stem from the fact that individuals with flu-like symptoms are advised against getting vaccinated so it's likely that vaccinated people are healthier than those in the unvaccinated, comparative group. The unvaccinated cohort includes people who have symptoms of illness and are therefore more likely to be hospitalised.

The duration of the observed wellness bias is anticipated to dissipate from four days since vaccination onward, so we do not need to restate our data to take wellness bias into account.

- What is the effect of "unanticipated bias" in days 4 to 9 post vaccine dose 1 and how do we account for it?

In addition to wellness bias, we observe an unanticipated bias. This refers to the observed lower rate of relative risk of COVID-19 admission between days 4 to 9 after dose one (evident in figure 1), the period before COVID-19 vaccination confers protective effects.

To address the effect of any unanticipated bias, we restated the odds of COVID-19 admission relative to the baseline risk of the population between days 4 to 9 since vaccination. Our findings are depicted in in figure 2, below.

In figure 2, we see stabilisation in the odds of COVID-19 admission between days 28 and 41 post-dose one, with odds of 0.27 (i.e., vaccine effectiveness of 73%), and relative stabilisation post day 14 of dose two with odds of 0.08 (i.e., vaccine effectiveness of 92%).

- In summary: Restating the odds to address unanticipated bias shows Pfizer COVID-19 vaccine 73% effective at 28 days post-dose one and 92% effective 14 days post-dose two in reducing the relative risk of hospital admission.

This restating approach was followed in a study conducted the United Kingdom (Lopez Bernal J, 2021), where the baseline risk of the vaccinated population between days four to nine was higher than that of the unvaccinated population studied. This higher baseline risk was a result of the vaccine then being rolled out to patients in nursing homes, considered at high risk of serious COVID-19 illness.

We hypothesise around a contributor to unanticipated bias in the early days post-vaccination: Individuals who were first to be vaccinated at the start of South Africa's mass vaccination campaign (from 17 May) were of higher socioeconomic status and less likely to be essential workers in high-risk environments, less exposed to COVID-19, and more likely to contribute pathology test results in these early exposure periods, when the rate of infection during South Africa's third COVID-19 wave was high.

Figure 2: Restated odds of COVID-19 admission in days since vaccination relative to unvaccinated population
(b) Relative risk of COVID-19 mortality post-vaccination

Figure 3 below provides the relative risk (odds ratio) of COVID-19 related death in vaccinated individuals, again in the days that follow vaccination.

This relative risk stabilises after 14 days post-dose 1 (overlapping confidence intervals with midpoint ranges of odds of COVID-19 death between 0.15 and 0.23).

• This implies that the Pfizer COVID-19 vaccine is 77% to 85% effective at reducing the risk of dying from COVID-19, at 14 days post the first dose.
• Post dose two the relative risk of death rate stabilises after 14 days to 0.04 i.e., the vaccine is 96% effective at reducing COVID-19 mortality risk relative to unvaccinated individuals.

Figure 3: Odds of COVID-19 mortality in days since vaccination relative to unvaccinated

- What is the effect of unanticipated bias on our data?

We explained the unanticipated bias observed in our vaccinated population above (in relation to COVID-19 admission risk). Once again, we see an unanticipated bias pattern in our vaccinated population (relative to our unvaccinated population) when looking at relative mortality risk.

Again, this leads to a need to restate the data.

Figure 4 below restates the odds ratio, between days four to nine, to address this bias.

In figure 4, we see stabilisation in the odds of COVID-19 death between days 28 and 41 post-dose one, with odds of 0.21 (i.e., vaccine effectiveness of 79%), and relative stabilisation post day 14 of dose two with odds of 0.06 (i.e., vaccine effectiveness of 94%).

Figure 4: Restated odds of COVID-19 mortality in days since vaccination relative to an unvaccinated population

(2) What is the effect of individual underlying risk factors on vaccine effectiveness?

Scheme members are considered fully vaccinated 14 days after their second Pfizer vaccine dose.

What is the effect of individual underlying risk factors on vaccine effectiveness in reducing the relative risk of COVID-19 hospital admissions across different segments of the Discovery Health population base - in fully vaccinated scheme members?

Here we consider the effect of individual underlying risk factors on vaccine effectiveness.

- Do males or females experience different admission risk? In short - no.

For fully vaccinated clients, there is no difference in the relative risk of COVID-19 admission (since the 95% confidence interval overlap) as indicated in Figure 5.

Figure 5: Odds of COVID-19 admission do not vary by sex, for fully vaccinated medical scheme members
- Do specific chronic condition/s affect admission risk in fully vaccinated people? In short - no.

As indicated in Figure 6, the odds of COVID-19 admission for fully vaccinated clients do not vary across the type of chronic conditions considered (since the 95% confidence interval overlaps across all categories)

Figure 6: Odds of COVID-19 admission by chronic condition for fully vaccinated Discovery clients

- Does the number of chronic conditions (using the US CDC's high-risk condition categories) that a fully vaccinated individual lives with affect their admission risk? In short - yes.

As indicated in Figure 7, the odds of being admitted to hospital to treat serious COVID-19 illness in a fully vaccinated individual increases with the number of chronic illnesses one has. Risk increases particularly for individuals with three or more risk factors.

Figure 7: Odds of COVID-19 admission by number of CDC risk factors for fully vaccinated Discovery clients
- How does age affect the relative risk of admission to hospital to treat COVID-19 in fully vaccinated individuals?

The odds of COVID-19 admission in fully vaccinated scheme members are highest for those over the age of 80. However, even for this age band, the risk remains relatively low, equating to vaccine effectiveness of 88% against COVID-19 admission (see Figure 8).

Figure 8: Odds of COVID19 admission by number by age group for fully vaccinated Discovery clients
- Is there a difference in relative risk of COVID-19 related hospital admission across South Africa provinces in which fully vaccinated scheme members are located? In short - no.

Figure 9 shows the way how the odds of COVID-19 admission for fully vaccinated clients do not vary across provinces (since the 95% confidence interval overlaps across all provinces).

Figure 9: Odds of COVID-19 admission to hospital by province for fully vaccinated Discovery Health administered medical scheme members

Conclusion

• The Pfizer vaccine's effectiveness in protecting against COVID-19 admission risk is 73% in the 14 to 41 days post-dose one and 92% between 14 and 99 days after dose two
• The Pfizer vaccine's effectiveness in protecting against COVID-19 mortality risk is 79% in the 14 to 41 days post-dose one and 94% between 14 and 99 days after dose two.
• At 28 days after dose one and 14 days after dose two, we observe a stabilisation in the vaccine effectiveness in protecting against COVID-19 admissions and mortality.
• Encouragingly, in considering the effect of sex, type of chronic condition, and province/location, we observe little to no variation in vaccine effectiveness in protecting from COVID-19 admission. However, we note a 5 to 7% decline in vaccine effectiveness in protecting against admission in people over the age of 80, and in individuals who have three or more chronic conditions.
• To date, we have observed no waning in vaccine effectiveness over time (in the three-month data interval considered) for COVID-19 admissions and death. Studies are being conducted to establish how long vaccine effectiveness lasts in this regard.

All in all, this analysis of Discovery Health data shows that being vaccinated with the two-dose Pfizer vaccine confers significant protection against both serious COVID-19 illness (relative risk of admission to hospital) and death.
Finally, those people who have recovered from COVID-19 and become fully vaccinated seem to gain the highest level of protection from future COVID-19 admission risk. We explain this in our related post on the substantial protection (so-called "super immunity") from COVID-19 vaccination available to people who recovered from a past bout of COVID-19 infection.

DID YOU FIND THIS POST USEFUL?

You may also be interested in reading another post explaining how medical scheme members over six years of age are at varying degrees of heightened risk of mortality when infected with COVID-19. There is also additional risk peaking in members aged 56 to 60. This drives home the importance of the preventive behaviours that limit our exposure to COVID-19 disease, and of being vaccinated against the disease.

References:

De Serres G, S. D. (2013 Sep 12). The test-negative design: validity, accuracy and precision of vaccine efficacy estimates compared to the gold standard of randomised placebo-controlled clinical trials. Euro Surveill., 20585.
Diseases, N. I. (2021, 08 24). COVID-19 VACCINATION: REPORTING ADVERSE EFFECTS FAQ. Retrieved from NICD: https://www.nicd.ac.za/covid-19-vaccination-reporting-adverse-effects-faq/
Jackson ML, C. J. (2017). Influenza Vaccine Effectiveness in the United States during the 2015-2016 Season. N Engl J Med, 534-43. doi:doi:10.1056/NEJMoa1700153 pmid:28792867
Lopez Bernal J, A. N. (2021). Effectiveness of the Pfizer-BioNTech and Oxford-AstraZeneca vaccines on covid-19 related symptoms, hospital admissions, and mortality in older adults in England: test negative case-control study. BMJ.
doi:doi:10.1136/bmj.n1088 Polack FP, T. S. (2020). Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. N Engl J Med, 2603-15. doi:doi:10.1056/NEJMoa2034577
Tegally, H. W. (2021). Detection of a SARS-CoV-2 variant of concern in South Africa. . Nature, 438-443. doi:https://doi.org/10.1038/s41586-021-03402-9
World Health Organisation. (2020). Public health criteria to adjust public health and social measures in the context. Retrieved from https://apps.who.int/iris/bitstream/handle/10665/332073/WHO-2019-nCoV-Adjusting_PH_measures-Criteria-2020.1-eng.pdf?sequence=1&isAllowed=y

Interested in knowing more or reporting on these findings?

Please contact us on MEDIA_RELATIONS_TEAM@discovery.co.za to request any updated data available since publication and any further context required.