Vaccines for preventing influenza in healthy adults (Review) Jefferson T, Di Pietrantonj C, Rivetti A, et al.

The stated objectives of this update of three previous reviews dating back to 1999 (1) were “To identify, retrieve and assess all studies evaluating the effects (efficacy, effectiveness and harm) of vaccines against influenza in healthy adults.” Efficacy is defined by the authors as the capacity of the vaccines to prevent influenza A or B and its complications, effectiveness as the capacity of the vaccines to prevent influenza-like illness (ILI) and its consequences and harm as any harmful event potentially associated with exposure to influenza vaccines.” To do so, the authors searched in three different sources: the Cochrane Central Register of Controlled Trials (CENTRAL) (The Cochrane Library, 2010, issue 2), MEDLINE (January 1966 to June 2010) and EMBASE (1990 to June 2010). In addition they searched reference lists and hand-searched the journal Vaccine. The studies incorporated in the review to investigate efficacy and effectiveness were restricted to randomised controlled trials (RCT’s) or ‘quasi-RCTs’ comparing influenza vaccines with placebo or no intervention in naturally-occurring influenza in healthy individuals aged 16 to 65 years.  Observational (non-randomised) studies were excluded. Both observational studies and randomised controlled trials were used in the review of harmful effects. 

In brief from their work the authors estimated that vaccine efficacy for laboratory confirmed influenza was 73% (95% C.I. 54%-84%) in years of good match between the WHO recommended vaccine strains and the circulating viruses. This decreased to 44% (95% C.I. 23%-59%) in years of poor vaccine-virus match. Vaccine efficacy was much lower but still positive if clinical outcomes (ILI) were used [VE 30% (95% C.I. 17%-41%)]. When comparing risks of laboratory confirmed influenza among vaccinated and unvaccinated individuals 1% of vaccinated people developed influenza symptoms versus 4% of unvaccinated people if there was a good match of the vaccine to the circulating influenza strains.  When the match was not so good, they considered the percentages were only 1% and 2% respectively.  The authors concluded that vaccination had a modest beneficial effect in reducing time off from work but no beneficial effect in reducing hospital admissions, complication rates or deaths.  A statement is made to the effect that readers should put less faith in industry sponsored than other trials whether or not they appear in peer-reviewed journals.   

Concerning harm that may be caused by influenza vaccines the authors include both observational studies and trial data. They conclude that inactivated vaccines caused local harm, an excess of mild oculo-respiratory syndrome and an estimated 1.6 additional cases of Guillain-Barré Syndrome (GBS) per million vaccinations.  

ECDC Comment (12 December 2010):

Vaccine Efficacy and Effectiveness
The main difference between this and the last update is the use of an alternative way of presenting influenza vaccine efficacy which the authors expresses as a risk difference and includes a number of case-control studies investigating possible serious side effects of influenza vaccines. There are few new trial data.(1,2)

There are various reasons for concern with this literature review.  The terms vaccine efficacy and effectiveness were used incorrectly. The authors define them as stated above.  Conventionally the term vaccine efficacy refers to estimates of protection obtained under the ideal conditions of randomised control trials (RCTs) against laboratory confirmed end-point (3,4). Since assessed in a rather limited trial population, this estimate usually does not take the ‘herd immunity’ effect into account, i.e. the indirect protection of the unvaccinated offered by the vaccinated not being infectious. For influenza vaccines, a proxy measure of efficacy is given by their ability to cause rises of antibody titres to levels considered to give protection (seroconversion and seroprevalence studies) according to criteria of influenza vaccine immunogenicity set by the European Medicines Agency Committee for Human Medical Products (CHMP).  The serological criteria have to be met before authorization of the annual seasonal influenza vaccines.(5)  Vaccine effectiveness refers to estimates of protection that can be expected when the vaccine is used under the less-than-ideal conditions of a general immunisation programme (6), and usually also includes the indirect protection offered by herd immunity. Most often, effectiveness is estimated through observational studies which can be influenced by various field conditions and biases have to be allowed for (3,7). The distinction between efficacy and effectiveness studies is not usually related to the specificity of the outcome as the authors state. Influenza vaccine efficacy and effectiveness should preferably be measured against laboratory confirmed outcomes, otherwise they will be seriously underestimated (3,7). Also if they are measured against unconfirmed clinical outcomes the estimates can seem to fluctuate greatly according to the mix of viruses causing respiratory symptoms at the time.

By convention vaccine efficacy and effectiveness are expressed as a percentage obtained using the formula (1-RR)*100. This percentage can be interpreted as the proportion reduction of incidence of disease among vaccinated compared with unvaccinated that is attributable to vaccination assuming that the vaccine is causing such reduction. Jefferson et al present absolute risk differences which are mistakenly interpreted as proportional reductions in the risk of developing influenza. Absolute differences are influenced by the frequency of the outcome and can be very low if the outcome is rare, as in the studies included in this literature review. In addition the absolute difference presented, i.e. 4% risk of developing influenza in unvaccinated minus 1% in vaccinated, should be 3 and not 4 as described in the discussion of the review. This correspond to a relative risk (RR) of 0.25 and a respectable vaccine efficacy or effectiveness of 75% (VE = (1 – 0,25) *100) = 75%.

Influenza like illness (ILI) is not the same as influenza. As the authors state, influenza viruses account for a minority of the viruses causing ILI symptoms, especially outside of epidemics. Therefore the conclusion that influenza vaccines “impact on the global incidence of clinical cases of influenza is limited” is misleading since it is judging influenza vaccines against the wrong outcome. Given that severe outcomes, hospitalisations and deaths are rare outcomes of influenza in healthy adults, randomised trials (which must be limited in size) will not be able to test whether vaccination reduces such outcomes and there is a danger that reviews like this mistake absence of evidence (of impact on severe outcomes) with evidence of absence. To do that, observational analyses must be included. Though they will be more subject to bias, that can be overcome (3,7-9).  Since influenza vaccine efficacy was consistently demonstrated by a number of studies, it became unethical to withhold vaccination from those at risk of severe influenza illness in RCTs. Therefore in the past 2 decades evidence on influenza vaccine effectiveness has mostly had to come from observational studies. Many of these studies applied sound and rigorous methodological approaches which minimised the risk of bias and the effect of confounding variables.(3,7-9) Not to include those studies is ignoring the bulk of the modern scientific literature and it is unfortunate that these studies were consciously excluded in this review. Equally it is does not seem even-handed to include observational studies for detecting harmful effects while at the same time excluding them for detecting benefits.

The definition of ‘healthy adults’ (all healthy persons aged 16 to 65 years) is unusual. Are all pregnant women unhealthy? Are all men and women age 65 years and over unhealthy? The greatest impact of seasonal influenza up to 2008 has been on deaths in men and women aged over 65 years (10). To exclude healthy ‘seniors’ (persons aged 65 years and over) will tend to underestimate the effect of immunisation on severe disease and mortality in health adults.

Vaccine Safety
This review tackles safety influenza vaccines for the first time in the series. Full evaluation of the safety of a certain vaccine requires various steps and may take years. It also has to continue beyond trials and marketing to detect possible rare adverse events following immunisation (so called AEFIs) and to investigate these particularly using linkage between large-scale data-sets.(11)  After initial studies (animal studies, phase 1 and 2 trials) there are usually randomised controlled efficacy trials which can also exclude commoner side-effects.  After licensure and marketing, data from passive surveillance systems and post-licensure epidemiological studies are used to detect rare adverse events, an example of this is the European Medicine’s Agency Pharmacovigilance mechanism .  Where there are reports of plausible AEFIs investigators have to determine whether these are due to causative relations, confounding or chance association using approaches such as large-scale data linkage. This review does not seem to encompass these mechanisms. From phamacovigilance and subsequent data-linkage studies influenza vaccines have a strong record of safety 11, 12). However vigilance has to continue because of the introduction of new products and the changes that take place in the vaccine composition. 

As this review includes only a very limited number of safety studies, it does not readily correspond to the description of a systematic literature review of influenza vaccines safety. The authors’ search for references pertinent to harmful effects missed a number of studies and topics. In some of the safety studies included it was not possible to discriminate whether events were associated to vaccination or resulted from influenza infection.  It is unusual to find no mention of anaphylaxis in any review of severe adverse effects following immunisation and looking at one other example: Guillain Barre Syndrome (GBS), the important study of Stowe et al and a recent American analysis are missing.(13,14) Stowe et al investigated more than one hypothesis and found a significant relationship between a diagnosis of influenza infection in GP records in the UK and the development of GBS. Influenza vaccination had a non-significant protective effect consistent with vaccination protecting the recipient again influenza infection (13). The latter found a weak relationship between the new A(H1N1) vaccine (0.8 cases per million immunisations) though it has yet to be adjusted for confounding and that work is now underway for the 2009 pandemic vaccine in the American study and in Europe by ECDC and the publicly-funded Brighton Collaboration. Accurate case ascertainment is of utmost importance in assessing reported adverse events. Older studies did not have these acknowledged case-definitions and should therefore be interpreted with caution. Over the last decade a series of case-definitions of adverse events including some that may be observed after influenza vaccination (GBS, anaphylaxis etc) have been developed under the Brighton Collaboration.  That contribution is also not mentioned in this review which overall seems to have fallen below the standards expected of the Cochrane Collaboration.

References

1. Demicheli V, Rivetti D, Deeks JJ, Jefferson TO. Vaccines for preventing influenza in healthy adults. Cochrane Database of Systematic Reviews 1999, Issue 4. [DOI: 10.1002/14651858.CD001269.pub3]

2. TO, Rivetti D, Di Pietrantonj C, Rivetti A, Demicheli V. Vaccines for preventing influenza in healthy adults. Cochrane Database of Systematic Reviews 2007, Issue 2. [DOI: 10.1002/14651858.CD001269.pub3]

3. Halloran ME, Haber M, Longini IM Jr, et al. Direct and indirect effects in vaccine efficacy and effectiveness. Am J Epidemiol 1991:133:323-31.

4. Last JM, Dictionary of epidemiology International Epidemiological Association, Oxford University Press ^4th edition 2001. 

5.  The European Agency for the Evaluation of Medicinal Products (EMEA). Note for guidance on harmonisation of requirements for influenza vaccines. Committee for proprietary medicinal products 1997; CPMP/BWP/214/96.

6. Cochrane AL. Effectiveness and efficiency; Random reflections on health services. London. Nuffield Hospital Press 1972.

7. Valenciano M, et al. Study designs for timely estimation of influenza vaccine effectiveness using European sentinel practitioner networks. Vaccine (2010), doi:10.1016/j.vaccine.2010.09.010

8. Nichol KL. Challenges in evaluating influenza vaccine effectiveness and the mortality benefits controversy.Vaccine2009; 27:6305–11.

9. OrensteinWA, Bernier RH, Dondero TJ, et al. Field evaluation of vaccine efficacy. Bull World Health Organ 1985; 63:1055–68.

10. CDC, Estimates of Deaths Associated with Seasonal Influenza United States, 1976 to 2007. MMWR Weekly August 27, 2010 / 59(33);1057-1062

11. Greene SK, et al, Near real-time surveillance for influenza vaccine safety: proof-of-concept in the Vaccine Safety Datalink Project. Am J Epidemiol. 2010 Jan 15;171(2):177-88. Epub 2009 Dec 4.