Is there a role for using the screening method to calculate vaccine effectiveness of PCV7? Yes, but it is a limited role. The term “screening method” was coined because this method was designed to be used as a quick, preliminary analysis when incidence and attack rate data are not available. In clinical medicine, screening tests are not expected to be diagnostic but rather to identify patients who need further evaluation for diagnosis and treatment. Similarly, vaccine effectiveness estimates derived from the screening method may require subsequent confirmation with more accurate and valid methods. Similar to diagnostic screening, a plausible result does not necessarily equate to a valid estimate of vaccine effectiveness. The computed and corresponding attack rates are an additional *de facto* screen on use of the same data inputs.

Based on results of randomized clinical trials and case-control studies, for the full-vaccine regimen and the vaccine-specific outcome PCV effectiveness estimates generally above 90% are credible. At the reduced dosage but still for the specific outcome, vaccine effectiveness estimates in the range of 80% are also credible. After the first 2–3 years and when using a non-specific endpoint such as IPD caused by all serotypes, the vaccine effectiveness estimates are highly variable and not what would be expected theoretically. We would expect vaccine effectiveness estimates to be higher against PCV7-type disease compared with all-serotype IPD. We would also expect that the complete 4-dose schedule would be more effective than receiving only 1 or more doses. Further, we would expect that the vaccine effectiveness estimates against all-serotype IPD would decline over time as fewer IPD cases are caused by serotypes included in the vaccine. These expectations were fulfilled, suggesting that the screening method may yield valid estimates of vaccine effectiveness, particularly for PCV7-type IPD during the first few years post-introduction, when our estimates were similar to those from published randomized controlled trials and comparable case-control studies

[2],

[3].

The estimated attack rates provide a critical opportunity to evaluate the believability of the estimates. If the screening results are to be credible, then the corresponding ARs must also be plausible. Even in the case of “good” effectiveness estimates, if the attack rates are impossible or unlikely, then the VE rates are suspect. For vaccines known by other means to be effective, such as PCV7, we would expect the attack rate to be lower in the vaccinated than the unvaccinated. The near-zero attack rate for PCV7-serotype IPD for the 4-dose schedule is what one would expect, as is the gradually declining attack rate in the unvaccinated, presumably due to herd effect. Similarly, the attack rates for PCV7-type IPD for the ≥1-dose schedule are low and gradually declining over the study period. The implausible attack rates for all-serotype IPD more than two years after vaccine introduction suggest that the screening method cannot be used reliably to estimate effectiveness of PCV7 against all-serotype IPD once the proportion of all IPD caused by PCV7 serotypes falls precipitously and non-PCV7 type disease increases.

Although the screening method simply applies the basic equation for calculating vaccine effectiveness, it is subject to biased estimates as is any secondary data analysis or observational study. However, because the screening method requires data that may be collected for other purposes, future PCV-adopting countries may consider applying this method as PCV immunization programs are rolled out worldwide. Accordingly, since we’ve shown vaccine effectiveness estimates using this method to be erratic, we submit that it is critical to understand and review the underlying assumptions and features of the method

[4],

[5].

Estimates of effectiveness from the screening method are very sensitive to minor errors in input estimates. For example, vaccine effectiveness will be overestimated if the population vaccine coverage is overestimated (, adapted from

[4]). For pneumococcal vaccine and other vaccines incorporated into the routine infant immunization system in the U.S. and many other countries, vaccine coverage rises quickly, which will likely adversely affect vaccine effectiveness estimates produced by the screening method 2–4 years after vaccine introduction. In addition, PCV7-serotype IPD declined dramatically after PCV7 introduction. Extrapolating from small numbers when case counts decline after vaccine introduction can lead to wide variability in vaccine effectiveness estimates. An additional contributing factor is that widespread PCV7 use leads to herd immunity, which could account for lower attack rates in the unvaccinated than otherwise expected and translate to lower estimates of vaccine effectiveness. A potential weakness of the screening method is that it may not be as reliable for evaluating vaccines that create high levels of herd immunity, such as pneumococcal in contrast to measles vaccines. We also assumed that the vaccinated and unvaccinated cases arise from the same population, which may not be the case in many developing countries.

The screening method assumes that population vaccine coverage is relatively stable and a sufficiently large proportion of the population is unvaccinated to allow the disease to remain endemic. This applies to the epidemiology of measles vaccination, a disease for which the screening method has been successfully used and for which the method was originally described

[4]. However, based on our experience in the U.S., this was not the case with PCV7 coverage (which rose quickly) and IPD (which dropped soon after introduction). In addition, increases in non-vaccine serotype IPD in the years following vaccine introduction may lead to inaccurate estimates of vaccine effectiveness because such increases violate the “steady state” assumption inherent in single-point vaccine coverage or disease incidence inputs

[18].

This analysis is subject to the following limitations. First, because the NIS limits its survey to children 19–35 months of age, we limited this analysis to children with IPD who were 19–35 months of age, which is slightly older than the peak age of IPD in young children

[19]; vaccine effectiveness estimates may differ by age, and we were unable to calculate vaccine effectiveness for the commonly studied age group of children <5 years of age. Second, we were limited by the post-hoc case-control analysis in this manuscript, and we were unable to calculate effectiveness estimates using case-control methodology for the years 2005–2006 and for 4-dose PCV7 schedules. This also affected the number of cases with a complete vaccine history, which was only gathered reliably for patients enrolled in the case-control study and in the later years of the IPD surveillance. We limited our analysis to patients with available vaccine histories, which could bias trends shown in IPD. Third, vaccine coverage estimates should represent the populations from which the cases come. In our situation, the vaccine coverage estimates included the population at risk from the pneumococcal surveillance areas, but the sampling methodology of ABCs surveillance and the NIS precluded exactly matching populations. Fourth, we did not calculate confidence intervals for the screening method vaccine effectiveness estimates because the method uses a simple algebraic and deterministic equation. Fifth, the estimated attack rates assume a stable population, which may not be the case. Lastly, we were not able to adjust for known confounders such as underlying medical conditions of case-patients with IPD with the screening method

[3]. The authors note that the overall vaccine effectiveness estimates may seem counterintuitively high compared with the annual estimates; this is a result of pooling the data overall to calculate the vaccine effectiveness and not calculating a mean of the annual estimates.

With higher valency (10- and 13-valent) pneumococcal conjugate vaccines now being introduced in many countries worldwide, there will be an increased need for evaluations of the impact and effectiveness of pneumococcal conjugate vaccine. The screening method should be used cautiously to evaluate VE of PCVs. Those using the screening method should remember that small differences in the data due to poor quality surveillance can lead to large differences in vaccine effectiveness estimates. Other less resource-intensive methods, such as the indirect cohort method, have been used successfully; however, it requires serotyping of pneumococcal isolates, cannot estimate vaccine effectiveness against all-serotype IPD, and may not be able to account for herd immunity

[20]. In summary, the screening method should only be used as a preliminary test in situations where the data inputs are valid, more reliable evaluations of vaccine impact are not feasible, where sufficient cases are available, and vaccine coverage has not peaked in the general population.