TIS automated screening practices have been compared to conventional Pap cytology and found to have similar or increased detection rates for high-grade lesions.[16
] When comparing liquid based Pap cytology by screening method, the majority of previously published studies have indicated that automated screening increases productivity while meeting or exceeding the rate of detection of ASC-H and HGSIL lesions compared to manual historic controls. Comparing a newly implemented TIS system to a manually screened historical control, Loanzo et al
., described a 38% increase in HGSIL detection rates using the TIS automated system.[7
] In a similar study, again using the comparison of a historic control, Miller et al
., reported that the TIS system significantly increased the rate of HGSIL diagnoses, but did not show a similar significant increase in the ASC-H diagnoses.[19
] Papillo et al
., found that TIS screened cytology increased the ASC-H rate of detection by 48% and the HGSIL rate by 24% compared to historical manual controls.[9
] Chivukula et al
., and Duby et al
., found a statistically significant increase in HGSIL rates and an increase in ASC-H rates when comparing TIS screened cases to their pre-implementation cohorts.[10
In the previously published studies, the control for TIS screening was cytology cases screened manually previous to implementation of the automated system. While this design has the advantage of detecting changes in Pap screening brought on by the TIS, there are some potential problems with this methodology. Changes in clinical practice, the dynamics of the patients being screened, or the interpretative threshold of the cytotechnologist and cytopathologists, as well as staffing changes from 1 time period to the next may affect the results and confound the effects of TIS alone. We designed our study with the goal of avoiding these possible confounding factors and analyzed manual and TIS screened cases over the same time period and follow-up interval.
In an attempt to ascertain possible difficulties with TIS use, previous studies have reviewed their detection failures. In a study by Halford et al
., comparing conventional cervical cytology to liquid based TIS screened cases, more than half of the false negative TIS screened cases on subsequent review of the slides displayed the abnormal cells in the extreme periphery of the 22 fields of view and were missed by the cytotechnologist.[16
] In addition, a cytotechnologist survey in a study by Kitchener et al
., reported monotony as a factor in using the automated systems for screening, which could lead to decreased concentration and focus by the cytotechnologist when reviewing fields of interest.[21
] We did not find a statistically significant difference in the rate of ASC-H and HGSIL diagnoses when comparing manual to the TIS screening system. Although reasons for failure of abnormal cell detection in our cohort were not specifically studied, we can ascertain that the TIS system did not introduce enough screening diagnostic difficulties to affect the rate of high-grade detection.
The majority of the previously published studies were not designed for histologic follow-up after cytology screening, but a few previously published studies did review surgical follow-up after an ASC-H or HGSIL diagnoses with varied results in the HGSIL category. Duby et al
., found similar rates of high-grade histology in the follow-up of the ASC-H Pap smears manually and TIS screened, but reported an increase in high-grade histology on TIS screened cases compared to the manual cohort.[20
] Papillo et al
., did not find a statistically significant difference in the positive predictive value in the ASC-H category, but found a significant increase in the HGSIL category.[9
] Similar to our results, Loanzo et al
., also reported statistically equivalent histologic follow-up diagnoses for the HGSIL category on manual and TIS screening.[7
Histologic follow-up data provided the basis for relative sensitivity and specificity calculations as well as valuable information on the accuracy and clinical significance of both screening methodologies in our study. Histologic results were used as the standard for comparison to determine the accuracy of the Pap diagnoses by screening method. An increased rate of ASC-H and HGSIL lesions using manual screening signifies little clinically if the histologic correlations are not accurate or if manual screening leads to an increase in false positive high-grade lesions. When analyzing the surgical diagnoses, the TIS and manual screened groups did not show a statistically significant difference in the correlation percentage of high-grade histology in either the ASC-H or HGSIL categories. Once a high-grade lesion was detected on a Pap smear, the correlation on tissue was similar no matter how the cytology was screened; the accuracy of manual and TIS screening was not significantly different. A high-grade lesion detected by TIS is just as likely as a manual screened case to be a true high-grade lesion.
Varied methodologies were used to calculate sensitivity and specificity of manual and TIS screening in previously published studies. Histologic follow-up was rarely used as a means of comparison. Using retrospective reviews of the same slides for reclassification, or comparing original diagnoses to an adjudicated group of negative cytology slides, the majority of studies based the sensitivity and specificity calculations on cytologic outcomes. Using these cytologic comparisons, all studies found that the sensitivity of TIS screening either met or exceeded by varying degrees the sensitivity of manual screening in the high-grade diagnostic category.[6
] A recently published randomized controlled trial by Kitchener et al
., studied the sensitivity and specificity of two automated systems, the TIS and the BD FocalPoint GS Imaging System (BD Diagnostics, Franklin Lakes, NJ, USA), relative to manual screening by comparing cytologic screening to histologic follow-up. Using CIN II histology as a cut-off point for high-grade lesions, Kitchener et al
., reported a decreased sensitivity by 8% of TIS relative to manual reading and a modest 0.6% specificity increase.[21
] Based on their findings, the authors concluded that automatic screening could not be recommended for primary cervical screening in the UK screening programs.
Our relative sensitivity calculations showed almost equal sensitivity and similar specificity in HGSIL diagnostic category when comparing TIS to manual screening. The differences were wider in the ASC-H category. Due to the small detection rates of HGSIL and ASC-H in our patient population and therefore small numbers of patients with histologic follow-up, the relative differences in sensitivity and specificity between manual and TIS screened cases is effected by the small number of cases for comparison. Though unattainable, increased patients numbers and follow-up information on those negative patients would better indicate the sensitivity and specificity differences between screening methods. However, although absolute calculations of sensitivity and specificity were unfeasible, the relative sensitivity and specificity still allows for a direct comparison between methodologies. While our study was limited by the retrospective design and small histologic data, the histologic follow-up and standard of comparison allows for a more rigorous analysis of the accuracy, sensitivity, and specificity of TIS relative to manual screening. Our findings and calculations indicate that either manual or TIS screening methodologies are acceptable for detecting patients with high-grade or possible high-grade cervical lesions.
Although our data suggests that manual screening is equal to TIS in the detection of high-grade lesions, several relevant issues warrant consideration. First, as the cervical cytology specimens are screened by cytotechnologists even when aided by the TIS system, their role as a factor in our results deserves attention. Manual screening will only be as effective as the comfort, skill, and experience of the cytotechnologist. If cytotechnologists have a high index of suspicion, a low threshold for abnormality, and the added benefit of experience, it is not unreasonable to believe that those individuals may be more effective at detecting high-grade abnormalities on screening, thus making manual screening as sensitive as automated screening for those individuals. Of additional interest is that the amount of cervical cytology slides screened per day by the cytotechnologists at our institution averages less than the limit set forth by regulatory bodies. The additional time available per screened slide may allow for more rigorous attention and focus to allow for a greater detection of abnormalities.
Due to the amount of time necessary to allow for histologic follow-up, our data is from 5 years ago when the TIS was relatively new to the institution. Learning curves have been described with automated systems, and we attempted to adjust for this phenomenon by designing our study period to begin almost a year from when the TIS was implemented. In addition, to further determine the effect of timing and learning curves on our automated laboratory rates of high-grade lesions, we collected the additional detection rate data from 3 years subsequent to our study period. The detection rates of ASC-H and HGSIL lesions were consistently similar by manual screening than TIS over the years. Learning curves did not appear to be a factor in detection rates of the TIS automated system indicated by our study.
Our data was examined for potential sources of bias, and attempts were made to adjust the data to exclude these factors. The distribution of cases assigned to manual or TIS screening was performed at random, except for cases from a group of gynecologic oncology physicians, which were always manually screened. These cases were excluded from inclusion in our data. At the time of the study period, three cytotechnologists preferred the TIS screening method and performed 75% of the TIS screening; therefore, the reading of the cases by the cytotechnologists was not completely randomized.
Of note is that our results were from an older TIS system, and an updated version is currently available. The benefits and changes of the updated system are not yet within our experience.
A final consideration of future study includes the group of women with negative diagnoses on cervical screening by both manual and TIS methods. Due to the large number of women in these subgroups, we are unable to estimate or determine false negative rates and determine the clinical significance of false negatives. Further study directions include determining the follow-up screening interval, subsequent diagnoses and comparison with prior negative results to determine the outcome in those manually and TIS screened with negative cytology. By determining the clinical significance of those false negatives cases, more valuable information could be gained about the relevant differences and clinical impact of manual versus automated screening.