Surveillance for COVID-19 Vaccine Effectiveness and Serious Adverse Events Following Immunization

Noni MacDonald¹, Shelly A. McNeil², Jeannette Comeau^1, Shawn Harmon^1,3, Eve Dubé⁴, Lucie M. Bucci⁵

1.    Department of Pediatrics, Dalhousie University, IWK Health Centre, Halifax, Nova Scotia
2.    Department of Medicine, Dalhousie University, Nova Scotia Health Authority, Halifax, Nova Scotia
3.    Policy Analyst, Health Law Institute, Schulich School of Law, Dalhousie University, Halifax, Nova Scotia
4.    Quebec National Institute of Public Health, Department of Anthropology, Laval University, Québec, QC
5.    Immunize Canada, Ottawa, Ontario

As the COVID-19 pandemic shows no signs of abating globally, efforts to develop vaccines to prevent infection have become an urgent international priority. Having widespread community immunity across the age span and around the globe may be the best way to bring this pandemic under control. The global increase in COVID-19-related deaths has placed intense pressure upon governments to fast-track pathways in vaccine development (1). Traditional timelines for new vaccine availability are between 10 to 15 years (2). The current aim is to have COVID-19 vaccines available in a startling 12 to 18 months. Over 100 research consortiums and companies have stepped up using a wide range of vaccine development platforms (killed, live attenuated, non-replicating adenovirus vector, protein subunit, replicating virus vector, mRNA and DNA) (3). A report on immune responses of patients who have recovered from COVID-19 infection suggests that vaccines might require strategies that selectively target the most potent neutralizing epitopes (4), indicating that some of these development strategies may lead to a more effective vaccine than others. Strategic collaborations for clinical trials and scale-up of manufacturing pathways are in development to compress the time to vaccine availability (5). The realities of the pandemic and the parallel and diverse efforts to overcome it have thrown up a number of uncertainties and issues, some pre-approval and some post-approval, that are critically important both to Canada’s COVID-19 response and to its ongoing objectives around routine immunization. From a pre-approval perspective, questions remain about how Health Canada’s approval standards and processes will be applied in this dynamic and time-compressed context. In late August 2020, Canada announced agreements to purchase large quantities of vaccines from four different companies if their COVID-19 vaccine trial is a success and Health Canada’s review results in authorization for use in Canada (6). From a post-approval perspective – the focus of this paper – profound questions arise about Canada’s practices for assessing vaccine effectiveness (the ability of the vaccine to prevent infection and serious disease) and vaccine safety (in the real-world setting, the vaccine’s benefits must clearly and definitively outweigh the risks).

COVID-19 Vaccine Effectiveness

Vaccine approvals in Canada, as elsewhere, require preclinical trials to show efficacy and safety (7). Efficacy will likely include assessment of immunologic response (i.e., in different age groups and in different medical conditions), as defined by the presence of antibody titres above a specified level, and most importantly, on the evidence of disease prevention in the Phase 3 clinical trial setting where COVID-19 is widely circulating. The Health Canada regulatory review will also take into account what is known about the immune response to natural COVID-19 infection, and more importantly, the Phase 3 randomized trial results of COVID-19 PCR-confirmed cases of infection in the vaccine and the control to see how well the vaccine prevented infection and disease. However, the effectiveness of the different COVID-19 vaccines – that is, their impact in the real world – will only be known following widespread use of the vaccine with careful tracking for vaccine failures (cases of COVID-19 among vaccinated individuals). Effectiveness will likely vary by different COVID-19 vaccines, by age and level of frailty, by medical conditions, by geography (i.e., the potential for exposure), and by time after immunization. When the data are available from various settings, including from Canada, these data can be used by the National Advisory Committee on Immunization (NACI) to fine-tune Canadian vaccine-specific COVID-19 immunization recommendations, including homing in on which COVID-19 vaccine(s) may be most useful for the Canadian population. Such effectiveness data will also help verify whether a specific antibody titre is indeed reflective of protection, and if so, whether it is effective in only some or all age groups. As with any new vaccine, the length of time that protection lasts will not be known for a number of years, and will likely vary by vaccine, by age, and by pre-existing medical conditions.

In Canada, we appear to have variable but overall good systems for testing and for detecting COVID-19 cases (8). The Canadian Immunization Research Network (CIRN)’s Serious Outcomes Surveillance (SOS) Network has a track record for assessing influenza vaccine effectiveness in preventing influenza-related hospitalizations and serious outcomes in Canadian adults (it detects influenza vaccine failures) (9). This infrastructure has been leveraged with funding from the Canadian Institutes of Health Research (CIHR) and the Public Health Agency of Canada (PHAC) to conduct active surveillance for COVID-19 in hospitalized Canadian adults at eight (8) specified sites in three (3) provinces.

Overall, for vaccine effectiveness to be ascertained in terms of disease prevention in both hospitalized and non-hospitalized patients, all detected positive cases will need to be queried about the receipt of a COVID-19 vaccine – and if the person has received a COVID-19 vaccine, about the specifics (i.e., what vaccine, when, where, how, etc.). The details are important for comparing the effectiveness of one vaccine to another:

• Is the failure directly due to the specific vaccine, and/or was it a storage/handling, diluent or other program error?  
• Are failures also being seen amongst other people who received that specific vaccine during that time period, in the same population, in the same subgroup, etc.?
• Are there underlying clinical reasons that may have altered the response to the specific vaccine?

This means that electronic health information systems will need to be searched for cases, the specific vaccine received, the diagnosis of COVID-19 infection, and having the clinical course verified.

It will not be easily done, given the myriad of discontinuous electronic health information systems across Canada (i.e., no link between hospital data, community clinic office data, and immunization registry data). None of the systems are easily searchable and they may not all contain the information needed to make reliable assessments on whether the COVID-19 infection was due to vaccine failure or program error, etc. This situation is unacceptable, given the importance of such data for crucial program decision-making. Although relatively low-cost, patient-centred, fully integrated health information systems that can be searched in real time do exist and have been developed in Canada (10), multiple competing interests (political, financial, system control, union, bureaucratic, etc. ) have precluded their introduction (11). To control the pandemic here in Canada and elsewhere, we need to optimize vaccine use – that is, pick the best COVID-19 vaccine for the setting in terms of effectiveness and safety data (see section below: COVID-19 Vaccine Safety) – and this requires quality ‘fit for purpose’ data. This gap in our ability to sort through real-time health data quickly and efficiently to detect mild to severe potential cases of vaccine failure should be rectified now, including ensuring that there are no legal or regulatory impediments to making it happen. In the interim, each province and territory needs to ensure that it has a robust system in place to detect COVID-19 vaccine failures by asking all who test positive for COVID-19 if they have received a COVID-19 vaccine. However, this method will miss those who may have COVID-19 symptoms who are seen in the health care system but are not tested. COVID-19 vaccine failure is critical information for health and economic decision-making. While no vaccine is expected to be 100% effective – some vaccine failures are to be expected – knowing each COVID-19 vaccine’s effectiveness in our population and subgroups is important for clinical decision-making, as well as for maintaining public trust (see section below: Communications).  

COVID-19 Vaccine Safety

An adverse event following immunization (AEFI) is “any untoward medical occurrence that follows immunization and that does not necessarily have a causal relationship with the usage of the vaccine”  (12). Pre-approval trials (trials that are done before the vaccine receives approval from Health Canada for use in Canada) are able to detect only common AEFIs – those that occur in 1% to 10% of recipients. Such common events might be a sore arm, poor appetite, local site redness, fever, etc. As they are usually mild, self-limited problems, they do not preclude vaccine approval by the regulator if they are not excessive in the rate of occurrence. If serious AEFIs are detected in pre-approval trials – events that are life-threatening or that lead to death, hospitalization, significant disability or congenital anomaly (12) – the manufacturer usually does not submit the vaccine for consideration of approval (i.e., the vaccine is not considered to be safe enough). The problem with pre-approval trials is that they are not large enough to detect rare (> 0.01% and < 0.1%) and very rare (< 0.01%) serious AEFIs, nor those where the onset is greatly delayed. These rare and serious AEFIs are detected only when a very large number of individuals have been immunized (i.e., after the vaccine has been approved for use and is used widely in the country). Detection of these serious and very rare events is critical for COVID-19 vaccines, as many of the types of vaccines in development involve new platforms (i.e., no previous widespread experience in use in humans).

In Canada, we have a passive AEFI surveillance system for vaccines given to children or adults, and an active surveillance program for the detection of serious AEFIs for childhood vaccines (8). In the passive system, health care workers (physicians, nurses, pharmacists, health administrators, and others) are “required” to report serious AEFIs, but there is no incentive to report, nor is there any penalty if they neglect to report. Standardized AEFI reporting forms from either the province/territory or from Health Canada (13) are used.

The active surveillance for childhood vaccines is the Canadian Immunization Monitoring Program ACTive (IMPACT), which actively monitors admissions to 12 tertiary pediatric hospitals to detect vaccine-preventable diseases and vaccine failures, as well as scrutinizing admissions for specific clinical diagnoses that might be an AEFI (14). This system is well placed to detect COVID-19 vaccine failures in children if the illness is severe enough to require admission, but it will not detect those who do not require hospitalization. IMPACT will be helpful in the detection of rare and very rare AEFIs that are COVID-19-vaccine related, although the list of clinical diagnoses to prompt questioning about whether the child had been immunized might need to be expanded. We have no such similar active system in adults – which is a serious gap. The SOS Network described above could be further retooled to conduct active surveillance for serious AEFIs associated with COVID-19 vaccines, but this would require further funding. Sentinel surveillance systems in the United States of America use large healthcare administrative databases to check for serious AEFIs; however, this can only be done if they know what symptom or diagnosis they want to check (i.e., check on an AEFI signal detected through the passive system through analytic studies such as case control).

In Canada, we do not have large patient databases that can be checked in real time because of our discontinuous health information systems.  As noted above, this gap needs to be rectified – not just to better address COVID-19 but to improve our health outcomes and decrease healthcare costs. In the interim, we must put in place a robust system to detect serious AEFIs actively amongst COVID-19 adult vaccine recipients.

Whether serious AEFI cases are detected through the active or passive surveillance systems, each case requires a causality assessment (7; 12) to determine if the case is vaccine related, vaccine manufacturing related, arising from a vaccine program error, an immunization stress-related response or a coincidental event (15). As part of the causality assessment, knowledge of the background rate of the diagnosis is required. In Canada, this usually comes from the literature, as again, we do not have large patient databases in which to check.

Communications

A crisis communication plan must be in place before the first COVID-19 vaccines are distributed. Vaccine failures will occur, and serious safety issues may arise. Both may precipitate distrust in the COVID-19 vaccine program, undermine COVID-19 vaccine acceptance, and limit our ability to control this pandemic. These factors could also undermine confidence in routine immunization and other health programs more generally. One of the most important aspects of crisis communication is that the information on vaccine failures and serious AEFIs must be rapidly shared both within the province/territory and nationally. The public must see these detection and assessment systems as strong, evidence-based, and transparent processes. Who is assessing if the COVID-19 case is a vaccine failure and why this happened needs to be made known, as does the information on who is carrying out the AEFI causality assessment, their independence and ultimately, their findings. Mixed messages must be avoided. There must be clarity on which vaccine is associated with the failure, the rate of that failure compared to other vaccines, whether the AEFI is vaccine-related and to which vaccine, and information on whether such findings have been seen in other countries. There must also be transparency about how these findings will, or will not, lead to changes in the COVID-19 program in Canada, and about who will make these decisions and on what basis. Not getting this right is not an option.

Conclusion

In conclusion, as Canada eagerly awaits the arrival of one or several COVID-19 vaccines, we need to ensure that we have robust monitoring systems in place to detect vaccine failures and serious AEFIs in a timely fashion. Moreover, these events must be dealt with in an evidence-based manner, and our communications to both the public and to health care providers must be informed by science and implemented in such a manner as to engender trust in COVID-19 vaccines and the immunization program. Changes in the COVID-19 vaccine program must be made swiftly and transparently if the evidence warrants it.

References

1. Graham BS. Rapid COVID-19 vaccine development. Science. 2020 May 29;368 (6494): 945-6.
2. Vaccine development, testing and regulation [Online]. History of Vaccines. January 17, 2018 [cited: September 9, 2020]. Available from: https://www.historyofvaccines.org/content/articles/vaccine-development-testing-and-regulation
3. Lurie N, Saville M, Hatchette R, Halton J. Developing Covid-19 vaccines at pandemic speed. New England Journal of Medicine. 2020;382 (21): 1969-1973.
4. Juno JA, Tan HX, Lee WS, Reynaldi A, Kelly HG, Wragg K, Esterbauer R, Kent HE, Battwn CJ, Mordant FL, Gherardin NA, Pymm P, Dietrich MH, Scott NE, Than W-H, Godfrey DI, Subbarao K, Davenport MP, Kent SJ, Wheatley AK. Humoral and circulating follicular helper T cell responses in recovered patients with COVID-19. Nature medicine. 2020 online ahead of print, 2020 Jul 13. doi:10.1038/s41591-020-0995-0.
5. Corey L, Mascola JR, Fauci A, Collins FS. A strategic approach to COVID-19 vaccine R&D. Science. 2020;368 (6494): 948-950.
6. New measures to ensure the supply of future vaccines and therapies against COVID-19 [Online]. Prime Minister of Canada. August 31, 2020 [cited: September 10, 2020]. Available from: https://pm.gc.ca/en/news/news-releases/2020/08/31/new-measures-ensure-supply-future-vaccines-and-therapies-against
7. MacDonald NE, Law BJ. Canada's eight-component vaccine safety system: A primer for health care workers. Paediatrics & Child Health. 2017;22 (4): e13-e16.
8. Ritchie H, Ortiz-Ospina E, Beltekian D, Mathieu E, Hasell J, Macdonald B, Giattino C, Roser M. Coronavirus (COVID-19) Testing [Online]. Our World in Data. September 9, 2020 [cited: September 9, 2020]. Available from: https://ourworldindata.org/coronavirus-testing
9. Nichols MK, Andrew MK, Hatchette TF, Ambrose A, Boivin G, Bowie W, Chit A, Dos Santos G, ElSherif M, Green K, Haguinet F. Influenza vaccine effectiveness to prevent influenza-related hospitalizations and serious outcomes in Canadian adults over the 2011/12 through 2013/14 influenza seasons: a pooled analysis from the Canadian Immunization Research Network (CIRN) Serious Outcomes Surveillance (SOS Network). Vaccine. 2018 Apr 12;36 (16): 2166-75.
10. Graven M, Allen P, Smith I, MacDonald NE. Decline in mortality with the Belize integrated patient-centred country wide health information system (BHIS) with embedded program management. International Journal of Medical Informatics. 2013 Oct 1;82 (10): 954-63.
11. Persaud N. A national electronic health record for primary care. CMAJ Jan 2019;191 (2): E28-E29; DOI: https://doi.org/10.1503/cmaj.181647
12. World Health Organization [Online]. Causality assessment of an adverse event following immunization (AEFI): user manual for the revised WHO classification (second edition). Geneva: World Health Organization, 2018. 62 p. Available from: https://apps.who.int/iris/bitstream/handle/10665/259959/9789241513654-eng.pdf
13. Adverse Events Following Immunization Reporting Form [Online]. Public Health Agency of Canada. August 5, 2020 [cited: September 10, 2020.]. Available from: https://www.canada.ca/en/public-health/services/immunization/reporting-adverse-events-following-immunization.html
14. Bettinger JA, Halperin SA, Vaudry W, Law BJ, Scheifele DW. The Canadian Immunization Monitoring Program, ACTive (IMPACT): Active surveillance for vaccine adverse events and vaccine-preventable diseases. Canada Communicable Disease Report. 2014;40 (Suppl 3): 41-44.
15. Gold MS, MacDonald NE, McMurtry CM, Balakrishnan MR, Heininger U, Menning L, Benes O, Pless R, Zuber PLF. Immunization stress-related response – Redefining immunization anxiety-related reaction as an adverse event following immunization. Vaccine. 2020;38 (14): 3015-3020.