|Home | About | Journals | Submit | Contact Us | Français|
Estimate the accuracy and cost-effectiveness of cervical cancer screening strategies based on high-risk HPV DNA testing of self-collected vaginal samples.
A subset of 1,665 women (18-50 years of age) participating in a cervical cancer screening study were screened by liquid-based cytology and by high-risk HPV DNA testing of both self-collected vaginal swab samples and clinician-collected cervical samples. Women with positive/abnormal screening test results and a subset of women with negative screening test results were triaged to colposcopy. Based on individual and combined test results, five screening strategies were defined. Estimates of sensitivity and specificity for cervical intraepithelial neoplasia grade 2 or worse were calculated and a Markov model was used to estimate the incremental cost-effectiveness ratios (ICERs) for each strategy.
Compared to cytology-based screening, high-risk HPV DNA testing of self-collected vaginal samples was more sensitive (68%, 95%CI=58%-78% versus 85%, 95%CI=76%-94%) but less specific (89%, 95%CI=86%-91% versus 73%, 95%CI=67%-79%). A strategy of high-risk HPV DNA testing of self-collected vaginal samples followed by cytology triage of HPV positive women, was comparably sensitive (75%, 95%CI=64%-86%) and specific (88%, 95%CI=85%-92%) to cytology-based screening. In-home self-collection for high-risk HPV DNA detection followed by in-clinic cytology triage had a slightly lower lifetime cost and a slightly higher quality-adjusted life expectancy than did cytology-based screening (ICER of triennial screening compared to no screening was $9,871/QALY and $12,878/QALY, respectively).
Triennial screening by high-risk HPV DNA testing of in-home, self-collected vaginal samples followed by in-clinic cytology triage was cost-effective.
Cytology-based screening programs have led to a dramatic reduction in cervical cancer incidence over the last five decades. Yet, these programs have limitations that have prompted research into alternative screening technologies and strategies. Cytology-based screening is labor- and time-intensive and lacks sensitivity and reproducibility in detecting precancerous cervical lesions or cervical cancer (1-4). Lack of participation in screening is also a challenge. A number of novel approaches to screening are being investigated, including the use of self-collected specimens for high-risk human papillomavirus (HPV) DNA testing.
In the United States, HPV DNA testing is recommended as an adjunct to cytology screening for all women 30 years and older, and reflex testing for HPV DNA is recommended for women with a cytologic finding of atypical squamous cells of undetermined significance (ASC-US) (5). HPV DNA testing has been shown to have higher sensitivity in screening (2, 6-11) and greater reproducibility (12) than cytology testing. It also lends itself to self-testing options. Studies using cervico-vaginal lavage, vaginal swabs or brushes, and tampons (13-29) suggest that there is good agreement between high-risk HPV DNA detection from self-collected vaginal samples vs. clinician-collected cervical samples and high acceptability of vaginal self-collection (20, 26-28). On the other hand, high-risk HPV testing of self-collected vaginal samples, like HPV testing of clinician-collected cervical samples, is less specific than cytology (23, 30), which leads to increased unnecessary diagnostic and treatment costs associated with false-positive test results.
We conducted a screening study in a population of young, sexually active women to determine the accuracy of high-risk HPV DNA testing of self-collected specimens alone or in combination with cytology for cervical cancer screening. We then used the resulting test characteristics as parameters in a mathematical model to estimate the cost-effectiveness of the different screening strategies.
This project was part of a previously described (7) larger cervical cancer screening study. 1,665 women were recruited between 2000 and 2002 from among women attending routine gynecologic examinations at Planned Parenthood of Western Washington clinics and by advertisements in local colleges and universities and in newspapers. Women were eligible for participation if they were between 18 and 50 years old, not pregnant, not chronically immunocompromised, and reported no prior treatments for cervical neoplasia. All study procedures were approved by the Human Subjects Division at the University of Washington, and written informed consent was obtained from all research participants.
Clinical procedures for screening, follow-up, treatment, and laboratory methods have been described previously (7). Briefly, women were screened by liquid-based cervical cytology and by high-risk HPV DNA testing (using PCR-based and Hybrid Capture II™ (hc2, Qiagen, Valencia, CA) tests) of cervical samples. For the time period of the study included in this analysis, participants also provided a self-collected vaginal swab sample for HPV DNA testing prior to the clinical examination. Details on self-sample collection and processing are the same as those previously described by our group in a different study (31). Women with a positive/abnormal result on any screening test and a random subset of those with normal screening test results were referred to diagnostic colposcopy and biopsy. At the colposcopy visit, colposcopically-directed biopsy and/or endocervical curettage (ECC) samples were collected from all women; for women with normal colposcopic findings an ectocervical biopsy sample at the 12 o’clock position and/or ECC were collected.
The most severe histologic finding from diagnostic and/or treatment visits was used to define disease outcomes. Estimates of sensitivity and specificity for detection of cervical intraepithelial neoplasia grade 2 or more severe disease (≥CIN2) were obtained for five screening strategies that included referral to colposcopy of women with the following test results: (1) a screening cytology showing low-grade squamous intraepithelial lesions (LSIL), atypical glandular cells (AGC) or ≥high-grade SIL (HSIL), or a screening cytology showing ASC-US with positive hc2 test results on the cervical sample (referred to as ‘cytology screening with reflex HPV testing for ASC-US’); (2) a screening cytology showing LSIL, AGC or ≥HSIL, or a screening cytology showing ASC-US with a follow-up cytology showing ASC-US, AGC, LSIL or HSIL (referred to as ‘cytology screening with repeat cytology testing for ASC-US’); (3) positive hc2 test of the self-collected vaginal sample; (4) positive hc2 test of the clinician-collected cervical sample; (5) positive hc2 results on the self-collected vaginal sample and a follow-up cytology showing ASC-US, AGC, LSIL or ≥HSIL (referred to as ‘vaginal HPV DNA screening with cytology triage’).
Sensitivity and specificity estimates were calculated for each screening strategy, and were corrected for verification bias using inverse probability weighting (IPW) as previously described (7). The five ‘screening strata’ used for IPW included women with: (1) negative results on all screening tests; (2) negative result on LBC but positive results for HR HPV DNA in prospectively tested cervical or vaginal sample using hc2 or PCR-based methods; (3) LBC result of ASC-US; (4) LBC result of LSIL; (5) LBC results of AGC or ≥HSIL. HPV DNA and cytology results were the only factors found to be independently associated with both colposcopy attendance and detection of ≥CIN2.
Weighted robust variance estimators were used for calculating 95% confidence intervals (C.I.s) of corrected sensitivity and specificity estimates. A weighted McNemar’s test was used to compare differences between the weighted sensitivity and specificity estimates from different screening strategies. Weights were applied using the pweights command in STATA 9.0 (STATA Corp, College Station, TX), which was used for data analyses.
A previously described state-transition Markov model (32) was used to estimate the lifetime costs, lifetime risks of cervical cancer, quality-adjusted life expectancies and incremental cost-effectiveness ratios (ICER) of the screening strategies described above, using a societal perspective. The model simulates a cohort of women beginning at age 12 years and followed through age 85 years. Each year women are at risk of being infected with HPV. Women who are infected can either progress to CIN (1, 2-3) or have their infection clear. Women with CIN can either clear their disease or have their disease progress to a more severe disease state (CIN 2-3 or cancer). Women with cancer can have their disease detected during screening or, in the absence of screening, through presentation with symptoms. Women with cancer can die of cancer based on 5-year survival probabilities. Each year women are also at risk of dying of other causes or undergoing hysterectomy for benign reasons. Stage-specific survival estimates for cervical cancer (33) and age-specific probability of death due to causes other than cervical cancer (34,35) were taken from the SEER registry. Age-specific rates of benign hysterectomy were taken from the published literature (36).
Women were assumed to be screened between the ages of 18 and 85 years, with one of the five previously described strategies conducted at intervals of every 1, 2 or 3 years. It was assumed that cervical screening test samples would be collected by a clinician while vaginal test samples would be self-collected by women in their homes. The compliance rate with screening was assumed to be 80% in our simulated population, based on reported screening rates in the U.S.(37, 38). Women with abnormal screening test results were referred for colposcopy and biopsy or assumed to undergo another screening test prior to referral to colposcopy. Compliance with colposcopy and treatment was assumed to be 100% in keeping with the literature: this allows for the assumption that screening strategies are being compared while holding constant external factors not related to the screening test performance. Disease status was assumed to be histologically verified among all women attending the colposcopy visit. Due to the variable management guidelines for women with CIN 1, which recommend treatment only in a subset of cases (39), these women were assumed not to undergo treatment but to return to regular screening. All women with a cytology result of HSIL or histology results ≥CIN2 were assumed to receive treatment (Loop Electrosurgical Excision Procedure [LEEP] for HSIL cytology, cone biopsies or LEEP for CIN2 and CIN3 and stage appropriate treatment for cancer). Women with cancer were assumed to receive stage-specific treatment.
Base case assumptions and ranges for relevant cost parameters are presented in table 1. Direct costs included costs for medical procedures and clinicians’ time. Indirect costs included the cost of time spent in clinic and in transportation for clinic-based procedures, and time spent on self-collection and mailing of specimens for home-based procedures.
Estimates of utilities and disutilities for calculation of QALYs were based on a time trade-off study of women’s perceptions of health states associated with cervical cancer (40). Utilities associated with cancer were 0.76 for stage I and 0.67 for stages (II-IV); these utilities were applied for 5 years. A disutility of 0.02, which is associated with undergoing cervical cancer screening (40), was applied towards false-positive screening test results. This disutility was applied for a period of 54 days, the median time interval between screening and colposcopy visits for women with false-positive screening test results in our study. All costs and outcomes were discounted at 3% (41).
Sensitivity analyses were performed to determine the impact of univariately varying sensitivity and specificity estimates, compliance with screening, compliance with follow-up, and individual costs (see table 1 for ranges of values evaluated). The estimates of sensitivity and specificity were simultaneously varied between the bounds of their 95% confidence intervals under the assumption that gains in sensitivity are at the expense of specificity and vice versa.
Additional analyses included: (i) cost of clinician time spent counseling HPV positive women (estimated to be twenty minutes), (ii) disutility associated with lack of clinician contact in home-based tests (a disutility of 0.02, which is associated with undergoing screening, was applied as an estimate for a one-month period following screening), and (iii) cost of a follow-up cytology test for women with HPV-positive, cytology-negative results for the vaginal HPV screening with cytology triage strategy. (As this strategy is not clinically recommended and no relevant guidelines exist, we considered a follow-up cytology test with the assumption that women with HPV-positive, cytology-negative results might require some type of follow-up. In the base-case model, such women are assumed to return to regular screening.)
The 1,665 study participants were predominantly non-Hispanic (95.8%) and White (73.8%), with a median age of 23 years. They had a lifetime median of 6 male sex partners and 36.8% reported having had 10 or more sex partners. Cytology showed ASC-US or greater in 21.8%, and high-risk HPV DNA was detected by hc2 in 28.4% of cervical and 33.1% of vaginal samples at screening. 504 women (30.3%) met the referral criteria and underwent colpo-biopsy. 100 women (Figure 1) were found to have histologically-confirmed ≥CIN2, indicating that the prevalence of ≥CIN2 was high (7.8%) in this young, sexually experienced population.
The sensitivity of self-collected vaginal HPV DNA screening (84.8%) was significantly greater than that of the cytology-based strategies (68.1%) and similar to that of cervical HPV DNA screening (93.8%). Vaginal and cervical HPV DNA screening had similar specificities (72.9% to 73.6%), which were significantly lower than those of the cytology-based strategies (88.6% to 90.6%). The sensitivity (75.2%) and specificity (88.3%) of vaginal HPV DNA screening with cytology triage was comparable to those of cytology-based strategies (Table 2).
In-home vaginal HPV DNA screening with in-clinic cytology triage was the least costly strategy and also more effective in terms of QALYs than all other strategies. The only exception to this was in the case of the triennial screening interval, where in-clinic cervical HPV DNA screening was associated with the highest effectiveness. Cervical HPV DNA screening was, however, also the most costly strategy: compared to triennial vaginal HPV DNA screening with cytology triage, triennial cervical HPV DNA screening had an ICER of $238,706/QALY (Figure 2). Compared to no screening, triennial vaginal HPV DNA screening with cytology triage was associated with a 75% reduction in lifetime risk of cervical cancer (3.61% vs. 0.91%) and an ICER of $9,871/QALY. Biennial screening with this strategy had an ICER of $70,151/QALY compared to triennial screening. Annual screening was very costly for this and all other strategies, with ICERs exceeding $300,000/QALY.
If compared directly to no screening, then both cytology-based strategies were cost-effective, with the reflex HPV DNA triage option being the least costly. Triennial and biennial screening with this strategy had ICERs of $12,878/QALY and $18,051/QALY, respectively, compared to no screening. Compared to biennial cytology screening with HPV DNA triage for ASC-US, biennial vaginal HPV DNA screening with cytology triage was associated with a gain in quality-adjusted life expectancy of 1.10 days and a reduction in lifetime costs of $199.
Our findings were most sensitive to assumptions about compliance with follow-up. If compliance with colposcopy was progressively lowered from the base-case value of 100% to 80%, triennial and biennial stand-alone in-home vaginal HPV screening without cytology triage were identified as potentially cost-effective, with an ICER of $10,508/QALY for triennial screening compared to no screening. When we applied the compliance rates for colposcopy observed in our screening study (specified in Table 1), which varied between women with different types of screening test results, vaginal HPV-based strategies with or without cytology triage remained cost-effective, and cytology screening with repeat cytology testing for ASC-US also became cost-effective with ICERs ranging from $78,770/QALY to $187,540/QALY depending on the screening interval.
Altering assumptions regarding the administration of HPV- and home-based strategies affected our findings in the following ways: (1) if it was assumed that women in the vaginal HPV screening with cytology triage strategy would undergo repeat cytology testing for HPV-positive, cytology-negative screening test results, then triennial stand-alone vaginal HPV screening became the least costly strategy but triennial and biennial vaginal HPV with cytology triage also remained cost-effective, with ICERs of $19,127/QALY and $75,839/QALY, respectively; (2) if a disutility associated with lack of clinician contact was included in home-based strategies then biennial vaginal HPV screening with cytology triage became less effective and more costly than triennial in-clinic Pap- or HPV-based strategies; (3) including a cost for clinician time for counseling HPV-positive women did not appreciably alter the results.
Of the 1,665 eligible women, 249 (14.9%) did not provide a self-collected vaginal sample, which most often occurred because of clinical time constraints. Women who did and did not provide a self-collected sample had similar demographic and clinical characteristics. Four hundred and four participants of the 504 who attended colposcopy were interviewed regarding their experience with self-collecting the vaginal sample during their previous screening visit and 246 (61%) responded that if given the option of a safe and accurate home-based self-test, they would prefer it over clinic-based testing, the reasons being greater ease and convenience; avoidance of the inconveniences of transportation and time required for a clinic visit; greater privacy of the home-based self-test; and fear or embarrassment associated with doctors and gynecologic exams. Of the 158 women who preferred a clinic-based test, 98 (62%) felt that it would be more accurate. Twelve women reported that they would want to benefit from having questions answered by the clinician or of undergoing a more thorough examination in which other, unrelated health problems might potentially be diagnosed.
In this population of young, sexually active women in the U.S., a combined strategy of vaginal high-risk HPV DNA screening with cytology triage of HPV positive women had comparable sensitivity and specificity to cytology-based screening. Assuming that vaginal self-sampling would be home-based and cytology testing clinic-based, this combined strategy was potentially cost-effective when performed triennially or biennially. Compared to absence of screening, triennial screening of home-based, self-collected vaginal samples for HPV DNA with in-clinic cytology triage had an ICER of $9,871/QALY. Triennial in-clinic cytology screening with reflex HPV DNA testing for ASC-US had an ICER of $12,878/QALY compared to no screening. Home-based self-collected HPV screening may be considered as a cost-effective screening alternative to increase convenience and accessibility of existing cytology-based screening programs.
To our knowledge, there have been no published studies of HPV screening of self-collected vaginal samples in a screening population in the United States. Investigations in China (23) and South Africa (30) indicated that this strategy performed favorably as an alternative to cytology-based screening. These studies were conducted among populations that were markedly older than ours (median ages of 42 and 39 years, respectively, versus 23 years in our study) and, given screening practices in these countries, less frequently screened. The estimated sensitivity of cytology screening in our study was similar to estimates reported for other well-screened populations in North America and Europe (6). In keeping with our findings, Belinson et al and Wright et al reported that vaginal HPV DNA screening was less specific than cytology screening. Neither of these studies investigated a combined strategy of vaginal HPV DNA testing with cytology triage and referral to colposcopy on the basis of positive/abnormal results from both these tests. In our study, this strategy showed similar sensitivity and specificity to cytology-based strategies.
Cost-effectiveness studies of HPV-based cervical cancer screening in industrialized countries have generally focused on incorporating HPV testing into existing cytology-based screening programs. One U.S study of primary screening with HPV showed that, compared to cytology-based screening, HPV screening of clinician-collected cervical samples was associated with a greater reduction in lifetime risk of cervical cancer but was not cost-effective (51). This observation was replicated in our study. When HPV screening is clinic-based, its cost is comparable to that of cytology screening. Furthermore, the relatively low specificity of HPV screening leads to a high cost of unnecessary diagnostic tests. By using in-home, self-collected vaginal samples for HPV screening, the large reduction in lifetime risk of cervical cancer is maintained by the high sensitivity of the strategy, and the cost of screening is reduced substantially by eliminating the need for an office visit. The specificity of vaginal HPV DNA-based screening was improved to the level of cytology screening by using cytology testing to triage HPV-positive women to colposcopy. This improved specificity further reduced the costs associated with this screening strategy. This combination of relatively low costs and high sensitivity and specificity made this strategy less costly and more effective than all others investigated.
There were limitations to this study. First, our study participants performed the vaginal self-collection in the clinic rather than at home. Our conclusions regarding this test in a home-based setting are based on an assumption that it would perform similarly in both settings. In a previous study by our group, the prevalence of HPV DNA was statistically similar between vaginal samples self-collected at home vs. in the clinic (31). Other investigators have reported high specimen adequacy as indicated by the level of β-globin in vaginal samples self-collected at home and transported to the clinic (52, 53), as well as increased HPV detection in these samples compared to clinician-collected cervical samples (53). Second, our model did not take into account the potential impacts of HPV vaccination of young women on the design, implementation and performance of screening. It is beyond the scope of this article to discuss the effect of vaccination on screening performance, and readers are referred to two detailed analyses in the literature (54, 55). Briefly, the positive predictive values of both cytology and HPV screening are likely to decrease in the era of HPV vaccination. It is more difficult to predict whether and how the sensitivities and specificities of screening tests would change, but it seems reasonable that the direction of change would be similar for cytology and HPV screening. More importantly, researchers predict that in the era of HPV vaccines, HPV testing will take on an increasingly important role in primary screening as well as in disease surveillance and monitoring of vaccine effectiveness (54, 55). Under these circumstances, a self-test for HPV could be advantageous. It is likely that once vaccine coverage is widespread, the interval of screening will be extended and screening might begin at an older age (54, 56). Third, it is possible that some disease may have remained undetected due to the fact that histology-confirmed disease outcome was not ascertained in all women, and only a single biopsy was used to confirm disease status in women with normal colposcopic findings. Incomplete ascertainment of disease outcome could bias sensitivity estimates towards being artificially inflated for all screening tests. Also, our cost-effectiveness model applied the same estimates of sensitivity and specificity across all age groups. Our sensitivity analyses showed, however, that vaginal HPV DNA screening at home with cytology triage was consistently cost-effective when estimates of sensitivity and specificity were varied. A higher estimate of specificity for the vaginal HPV DNA test, which would be expected in older age groups, caused home-based vaginal HPV DNA screening to become cost-effective even without cytology triage. Fourth, our model did not account for the possibility that home-based HPV screening with cytology triage could lead to fewer interactions with health care providers, and thus might be associated with increased costs due to missed opportunities to diagnose and manage unrelated medical conditions. On the other hand, clinic visits are also associated with an unknown risk of receiving unnecessary and potentially harmful interventions (e.g., HRT). Understanding the impact of these difficult-to-quantify indirect costs of gynecological visits will become more important as clinical recommendations for age of initiation and intervals of screening are increased. Finally, the results of our study may not be generalizable to other populations of women who are older, have a lower risk of HPV infection, or have never been screened..
The field of sexually transmitted diseases is increasingly moving towards the use of self-collected vaginal swab samples in screening and surveillance. The findings of this study suggest that cervical cancer screening strategies that are based on self-collection of vaginal swab samples at home could perform with comparable, and possibly even greater, effectiveness and cost-effectiveness compared to existing clinic-based cytology programs. We found that participants were receptive to the idea of a self-test if this test were safe and reliable. Other investigators have reported that self-sampling for HPV is not only acceptable but often preferred by women compared to clinician-sampling (20, 21, 26, 28, 57, 58). In a recent workshop held at the National Institutes of Health, a panel of experts recommended the use of self-collected vaginal specimens for detecting chlamydial and gonorrheal infections (59). They further emphasized a more widespread use of self-sampling to detect other genital pathogens. The FDA has already approved self-sampling kits for HIV and Hepatitis C testing (59). The natural next step would be to develop a common sampling device and collection medium that could aid in integrating testing for various genital infections including HPV. With the advent of the HPV vaccine, many changes are anticipated in the protocol and schedule for cervical cancer screening. This is an opportune time to determine the feasibility and impact of incorporating self-testing for high-risk HPV DNA and for other genital infections into our public health programs. Studies to evaluate effectiveness of HPV self-screening in other populations and assess women’s and clinicians’ preferences, as well as larger field trials and feasibility assessments are needed as the next steps before translating the findings of our study into public health policy and programming.
The authors would like to acknowledge Amy Bonney, Laura Merrell, Kimberle Tomlinson, and Nicolas Seal for their contributions towards patient recruitment and clinical and logistical support. We also acknowledge Qinghua Feng, Donna Kenney, Jane Kuypers, and the staff at the HPV virology laboratory at Harborview Medical Center, Seattle, WA.
Source of Financial Support: This study was funded through the National Cancer Institute, grant number CA34493