We systematically calibrated a microsimulation model of HPV and cervical cancer in the United States to multiple epidemiologic studies using a likelihood-based approach. By characterizing uncertainty in disease natural history using a model that includes both vaccination and screening capabilities, we demonstrated the impact of the approach in terms of quantifying the uncertainty about one policy-relevant outcome, the reduction of cervical cancer incidence.
Concordance between modeled age- and HPV type-specific prevalence outcomes was reasonable, with consistent fits achieved for important calibration targets including high-risk HPV prevalence, HPV type distribution within neoplasia and cancer, and cervical cancer incidence. In terms of external consistency, model performance was deemed reasonable by comparison of additional modeled outcomes in the presence of screening to independent studies not used to parameterize the model.
Prior research on model calibration and validation and the quantification of uncertainty has spanned many disease areas and disciplines. Important aspects of this research include data synthesis useful for model parameterization, calibration, and evaluation [105
]; methods to use evidence in model calibration and quantification of parameter uncertainty [111
]; and methods to alleviate the computational burden associated with all parts of model calibration and validation [114
A broad range of modeling techniques has been used to evaluate cervical cancer screening and vaccination, though researchers have only recently begun to integrate, apply, and extend work on model calibration and validation techniques to models of HPV and cervical cancer [22
]. Numerous computer models based on Markov cohorts have evaluated alternative cervical screening policies in the absence of HPV vaccination [10
]. Additionally a European microsimulation of cervical cancer has been developed [119
]. Subsequent models considered both screening and vaccination [121
Dynamic transmission models have been reported for Finland [126
] and the United States [103
]. Key features of dynamic transmission models, which will become more important as data become available on vaccine efficacy in males, include their ability to incorporate the natural history of HPV in men and women, reflect transmission dynamics over time, and endogenously capture herd immunity. Because such models require data on HPV transmission and sexual partnerships data in addition to data on disease natural history, they involve more uncertain parameters, and thus far, have not considered oncogenic HPV types other than HPV-16 and HPV-18.
Several recent studies have undertaken methodological efforts related to those in this study. In a model of HPV in Brazil, Kim et al. [22
] used an iterative approach to calibrate a natural history model of cervical cancer to longitudinal data to elucidate the differential impact of selected uncertain assumptions and then employed likelihood-based calibration. Based on data from a Canadian study, Burchell et al. [131
] explored heterosexual HPV transmission probabilities, using a stochastic computer simulation to search for transmission probabilities consistent with the study's 95% confidence intervals. The model identified parameter uncertainty for transmission parameters in comparison to one epidemiologic study. Van de Velde et al. [125
] calibrated a model of HPV and cervical cancer, identifying multiple parameter sets that fit North American epidemiologic data, including Canadian cancer data. However, their evidence combination methods did not account for differences in study size, and their fitting procedure was not likelihood-based. Like Van de Velde and colleagues, our model's unknown and uncertain inputs are calibrated to observed data, and multiple parameterizations are identified. Like Burchell et al., we identify plausible parameter combinations, relying on a likelihood-based approach to fit model outputs to confidence intervals derived from multiple data sources. Because our model simulates individual women and is analyzed as a first-order Monte Carlo simulation, in addition to capturing the effects of vaccination, it can account for each woman's previous history and allow her history of screening, vaccination, health, and behavior to affect her future risk.
Our study has a number of limitations. While we searched 1,000,000 parameter sets, searching more extensively might yield marginal improvements in fit to the observed data. On the other hand, the data used for calibration come from heterogeneous sources whose evidence is not entirely consistent. Insistence on an exact match to available data may lead to over-fitting and thereby underestimate uncertainty in model outcomes. Another limitation is the availability of data for model design and parameterization. Areas of particular importance include sexual activity patterns and HPV transmission in adolescents, characteristics of natural immunity, and host genetic heterogeneity. Results from our study support the need for further studies of HPV natural history in order to narrow parameter uncertainty. Findings from future studies will likely necessitate updating our current model and parameter estimates through recalibration. While our model can be linked indirectly to a separate dynamic transmission model, it does not directly capture herd immunity effects [103
]; we opt instead for a detailed monthly microsimulation that includes multiple HPV types and allows for individual differences in risks and screening patterns based on past history – features that are less readily incorporated into standard dynamic transmission models. Finally, we recognize that a focus on cervical cancer omits the impacts of preventing other rarer cancers that occur in both men and women [132
]. This is an important area of future work as better data become available.
Our model makes a set of causal assumptions about the natural history of HPV and cervical cancer, embodied in the model's structure, which are consistent with current biologic understanding of cervical disease. Systematic model calibration results in multiple alternative model parameterizations consistent with epidemiologic data. With our calibrated model, we were able to generate predictions about the effects of screening patterns in the U.S. on HPV, CIN, and cancer that were reasonably consistent with multiple independent studies. The model is not, however, intended as a formal model of causal effect that proves specific relationships in the biology and epidemiology of HPV and cervical cancer. Rather, the model is intended to comment on current questions of policy relevance from a decision analytic standpoint, reflecting the uncertainty in predicted outcomes that exists even in models that are consistent with observed population data.
While we report reasonably favorable model performance results with respect to three large studies not used in parameterization or calibration, model fits to sometimes wide confidence intervals are imperfect. One important future direction would be to continue to refine and expand the set of evaluation targets. When newer data from ongoing population studies of HPV vaccines emerge, they may provide useful opportunities to conduct evaluation exercises with respect to possible changes in the HPV type distribution between unvaccinated and vaccinated cohorts [133
]. Another future direction is the assessment of model structure uncertainty. Research in other disease areas has relied largely on review and comparison of cost-effectiveness results to assess the differential effects of model structure assumptions [134
]. Recent studies have compared alternative model structures developed by different groups [137
]. Few studies in health-related areas have simultaneously considered model structure and parameter uncertainty [139