PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of jcdrAbout UsAuthor ServicesArchiveSubmitJCDR
 
J Clin Diagn Res. 2017 January; 11(1): XC04–XC06.
Published online 2017 January 1. doi:  10.7860/JCDR/2017/21943.9168
PMCID: PMC5324475

Intra and Inter-Observer Variability of Transformation Zone Assessment in Colposcopy: A Qualitative and Quantitative Study

Abstract

Introduction

Colposcopy is an important tool in the diagnosis of cervical precancer and early cancer. The assessment of women with abnormal cytology and selection of those who require further therapy or follow up depends on the colposcopic assessment of the Transformation Zone (TZ). Identification of the TZ is thus an important part of this examination. Intra and inter-observer variability is known to be relatively high in the colposcopic interpretation of abnormal features. However, there are hardly any studies on the observer variability in the assessment of the type of TZ.

Aim

The present study was conducted with the aim to compare the intra and inter-observer variability of the TZ type classification and the Squamo-Columnar Junction (SCJ) visibility and to quantitatively measure the intra and inter-observer correlations of tracing of the TZ contours.

Materials and Methods

Colposcopy images were obtained for a total of 170 cases. They were reviewed by three colposcopists independently. The colposcopists classified the TZ type and also marked the SCJ contours on the images. Each observer independently reviewed the cases on two different instances (few weeks apart) and the result was compiled for intra-observer variation. The intra and inter observer variability on the TZ type was compared using Cohen’s Kappa. This was followed by a quantitative measurement of TZ observation variability using Hausdorff distance.

Results

The inter-observer agreement for the TZ type classification was moderate (Kappa= 0.53 to 0.66). The Intra-observer agreement was moderate to strong (0.60 to 0.86).

Conclusion

Colposcopic in vivo examination increases the variability in the identification of the TZ when compared to the analysis of recorded images. The disagreement in the TZ type was mostly in categories Type 2 vs Type 3 followed by Type 1 vs Type 2. A computerized quantification method can be used for quality control and training purpose in colposcopy.

Keywords: Colposcopic interpretation, Squamo-columnar junction, Transformation zone type

Introduction

Cervical cancer is the second most common cancer in women in developing countries [1]. Cervical cancer is the only preventable cancer among the female genital cancers. Colposcopy is an important tool in the diagnosis of cervical precancer and early cancer. Although, colposcopy was introduced by Hinselmann in 1924 as a clinical method to identify morphological features of various cervical diseases [2], Pap smear replaced it as a screening tool in the 1940s [3]. Colposcopy is currently used for management of abnormal Pap smears and is the second step of the diagnostic approach. The assessment of women with abnormal cytology and selection of those who require further therapy or follow up depends on the colposcopic assessment of the TZ. Identification of the TZ is thus an important part of this examination. The International Federation for Cervical Pathology and Colposcopy (IFCPC) 2011 nomenclature for colposcopy also emphasizes on the TZ type, visibility of SCJ and lesion location with regard to the TZ [4].

Intra- and inter-observer variability is known to be relatively high in the colposcopic interpretation of abnormal features. However, there are hardly any studies on the observer variability in the assessment of the type of TZ [57].

The aim of this study was to compare the intra and inter-observer variability of the classification of the type of the TZ and the SCJ visibility and to quantitatively measure the intra and inter-observer correlations of the tracing of the TZ contours.

Materials and Methods

The study was an observational study done in which women aged between 18-70 years, visiting the outpatient clinic for either routine or opportunistic screening were included. Women who had a visible growth on the cervix or had undergone hysterectomy or conization were excluded. The study was approved by the institutional ethical committee and an informed patient consent was taken before the examination. The procedures followed were in accordance with the ethical standards of the institutional committee on human experimentation and with the Helsinki Declaration of 1975 that was revised in 2000. Cervical cytology and colposcopy were done for all subjects. Colposcopy images were obtained using the Philips Goldway digital colposcope at the gynecologic oncology department in a tertiary care hospital.

Women, who visited the colposcopy clinic and consented, were included in the study from December 2011 to February 2013 and underwent colposcopic examination. A TZ was classified as Type 1 when it is entirely ectocervical (without any endocervical portion). Transformation zones Type 2 and 3 always have an endocervical portion. If the new SCJ was completely visible, it was referred to as a Type 2 TZ. If the new SCJ was not completely visible even with the help of additional instruments it was classified as Type 3 [4]. The colposcopists also annotated the SCJ contours on the image. The colposcopy images from these cases were reviewed by three colposcopists independently. Each observer independently reviewed the cases on two different occasions (a few weeks apart) and the result was compiled for intra-observer variation.

The statistical method used to compare the intra- and inter- observer variability on the TZ type was Cohen’s Kappa. This was followed by the quantitative measurement of the annotation variability using Hausdorff distance [Table/Fig-1] in pixels with image resolution of 640x480. Hausdorff distance measures how far two subsets of a metric space are from each other. Two sets are close in the Hausdorff distance if every point of either set is close to some point of the other set, in other words it is the greatest of all the distances from a point in one set to the closest point in the other set [8]. The Hausdorff distance between the three colposcopists was computed by taking the mean of the Hausdorff distances between colposcopist 1 and colposcopist 2, colposcopist 1 and colposcopist 3 and colposcopist 2 and colposcopist 3.

[Table/Fig-1]:
Hausdroff distance.

The data were recorded using Excel 2013. The calculation of Kappa statistics and p-values were done in SPSS 16.0 [9,10].

Results

While 203 women consented for the study and were examined, complete information about the annotation of the TZ by the colposcopists was available only in 170 women. The inter-observer agreement for the TZ type classification of the 170 cases was moderate (Kappa= 0.53 to 0.66, [Table/Fig-2]). The intra-observer agreement was moderate to strong (Kappa =0.60 to 0.86, [Table/Fig-3]). The disagreement in the TZ type was mostly for categories Type 2 vs Type 3 followed by Type 1 vs Type 2. The SCJ visibility was assessed once and only by observers 1 and 3. The corresponding inter-observer agreement for the SCJ visibility was moderate (Kappa= 0.61).

[Table/Fig-2]:
Inter-observer agreements for the TZ type classification.
[Table/Fig-3]:
Intra-observer agreements for the TZ type classification.

When the cases with Type 3 TZ were excluded, the mean and the SD of the Hausdorff distance in the remaining 99 cases were found to be 34 and 21 pixels respectively. The mean and standard deviation of the cases where there were disagreements (SD: 24) [Table/Fig-4,,5,5, and and6]6] on the TZ type were higher than in the agreed cases (SD: 15) [Table/Fig-7,,88 and and99].

[Table/Fig-4]:
The Hausdorff distance is 38 pixels for this patient where there is disagreement for the TZ type colposcopist 1.
[Table/Fig-5]:
The Hausdorff distance is 38 pixels for this patient where there is disagreement for the TZ type colposcopist 2.
[Table/Fig-6]:
The Hausdorff distance is 38 pixels for this patient where there is disagreement for the TZ type colposcopist 3.
[Table/Fig-7]:
The Hausdorff distance is 12 pixels for this patient where there is agreement for the TZ type colposcopist 1.
[Table/Fig-8]:
The Hausdorff distance is 12 pixels for this patient where there is agreement for the TZ type colposcopist 2.
[Table/Fig-9]:
The Hausdorff distance is 12 pixels for this patient where there is agreement for the TZ type colposcopist 3.

Discussion

Colposcopy is an essential tool in the diagnosis of early cervical disease. It is used for the evaluation of a patient with an abnormal cytology; however, it is not a definitive diagnostic test. There can be variations when performed by the same colposcopists or when performed by different colposcopists. Thus, the main drawback in the use of colposcopy as a diagnostic tool is the skill and experience of the clinician. Various studies show good sensitivity and poor specificity for invasive and pre invasive lesions of the cervix diagnosed by colposcopy [11].

The TZ is defined colposcopically as the area between the original and the new SCJ [12]. Identifying the TZ is an essential knowledge required by all colposcopists. Firstly, the new border between the squamous and columnar epithelium has to be localized for classifying the TZ as Type 1, 2 or 3.

The new SCJ constitutes the “inner” margin of the TZ. It can be either “completely visible”, “partially visible” or even “not visible”. Evaluation of the TZ helps in planning the type of excision. The colposcopic impression and histopathological diagnosis following a colposcopically directed biopsy provides the basis for the management of patients suspected to have cervical disease.

Studies on the TZ identification are limited and most of them have compared inter-observer variability in colposcopy for lesions and abnormal findings [5,11]. However, in this study, we were mainly interested in comparing the intra and inter-observer variability of the TZ type classification and the SCJ visibility.

The intra and inter-observer agreements for the TZ type classification and the SCJ visibility were within the range of agreements published for the SCJ visibility and the abnormal colposcopic parameters [13]. Colposcopist 3 who had an excellent intra-observer agreement interpreted only images recorded (offline) while colposcopist 1 and 2 had mostly done the first interpretation live. This confirmed that in vivo examination increases variability compared to analysis of recorded images by introducing other subjective factors like position, lighting, and magnification of the uterine cervix and other disruptive factors related to the outpatient department workflow [14].

According to Luyten A et al., Type 3 TZ seems to be a reproducible finding, the distribution of Types 1 and 2 TZs showed significant heterogeneity [15].

The tracing of the SCJ were done independently by the three colposcopists, sometimes on different sets of images of the same patient. Hence, to compute the Hausdorff distance for experiments, the tracing were repeated on the same image for a particular patient. The Hausdorff distance in our study was able to distinguish cases where disagreement was present and could be used as a tool to quantify and diagnose automatically the intra and inter-observer variation of selected areas whether it is the SCJ or an abnormal zone, provided that the set of images is identical for all observers.

Limitation

The limitation of this study is that a significant number of the cases had Type 3 TZ as a large proportion of cases attending a referral gynecologic oncology unit, are postmenopausal. Annotation was not possible in women with Type 3 TZ.

Conclusion

Colposcopic in vivo examination increases the variability in the identification of the TZ when compared to the analysis of recorded images. The disagreement in the TZ type was mostly in categories Type 2 vs Type 3 followed by Type 1 vs Type 2. This study has attempted to assess the intra and inter-observer variability of the TZ assessments and demonstrates that a computerized quantification method can be used for this purpose. This result could be used for quality control and training purpose in colposcopy.

Contribution: The first author (EV), the principal investigator contributed to patient care, intellectual content, conception and design of the work, inter-observer and intra-observer interpretation of the colpo-photographs, first drafting and final editing. The second and third authors (TSP and KK) have contributed equally and were involved in the patient care, colposcopic examination, colpo-photography, documentation and record keeping, inter-observer interpretation of the data, manuscript preparation, editing, referencing. The fourth and the fifth author (CF and PK) contributed in intellectual content, design of research and writing manuscript and CF was also an inter observer of the colpo-photography. The sixth author and the seventh authors (PV and SN) analyzed the results; the eighth author (LG) was responsible for maintaining the de-identified data and proof reading the manuscript.

Notes

Financial or Other Competing Interests

None.

References

[1] Human papillomavirus (HPV) and cervical cancer [Internet]. World Health Organization. 2016 [cited 17 July 2016]. Available from: http://www.who.int/mediacentre/factsheets/fs380/en/
[2] Fusco E, Padula F, Mancini E, Cavaliere A, Grubisic G. History of colposcopy: A brief biography of Hinselmann. J Prenat Med. 2008;2(2):19–23. [PMC free article] [PubMed]
[3] Safaeian M, Solomon D. Cervical Cancer Prevention-Cervical Screening: Science in Evolution. Obstet Gynecol Clin North Am. 2007;34(4):739–ix. [PMC free article] [PubMed]
[4] Quaas J, Reich O, Küppers V. Explanation and use of the Rio 2011 colposcopy nomenclature of the IFCPC (International Federation for Cervical Pathology and Colposcopy) Geburtshilfe und Frauenheilkunde. 2014;74(12):1090–92. [PubMed]
[5] Sellors J, Nieminen P, Vesterinen E, Paavonen J. Observer variability in the scoring of colpophotographs. Obstet Gynecol. 1990;76(6):1006–08. [PubMed]
[6] Bucchi L, Cristiani P, Costa S, Schincaglia P, Garutti P, Sassoli de Bianchi P, et al. Rationale and development of an online quality assurance programme for colposcopy in a population-based cervical screening setting in Italy. BMC Health Serv Res. 2013;13:237. [PMC free article] [PubMed]
[7] Etherington IJ, Luesley DM, Shafi MI, Dunn J, Hiller L, Jordan JA. Observer variability among colposcopists from the West Midlands region. Br J Obstet Gynaecol. 1997;104(12):1380–84. [PubMed]
[8] Rockafellar R. Tyrrell, Wets Roger J-B. Variational Analysis. 3rd print. Springer-Verlag; 2009.
[9] Gwet KL. Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol. 2008;61:29–48. [PubMed]
[10] Komagata N. Chance agreement and significance of the kappa statistic. http://nobo.komagata.net/pub/Komagata02-Kappa.pdf.
[11] Olaniyan OB. Validity of colposcopy in the diagnosis of early cervical neoplasia-A review. Afr J Reprod Health. 2002;6(3):59–69. [PubMed]
[12] Kolstad P, Stafl A. Atlas of Colposcopy. Baltimore: University Park Press; 1972. pp. 35–56.
[13] Sideri M, Spolti N, Spinaci L, Sanvito F, Ribaldone R, Surico N, et al. Interobserver variability of colposcopic interpretations and consistency with final histologic results. J Low Genit Tract Dis. 2004;8(3):212–16. [PubMed]
[14] Liu AH, Gold MA, Schiffman M, Smith KM, Zuna RE, Dunn ST, et al. Comparison of colposcopic impression based on live colposcopy and evaluation of static digital images. J Low Genit Tract Dis. 2016;20(2):154–61. [PMC free article] [PubMed]
[15] Luyten A, Buttmann-Schweiger N, Hagemann I, Scherbring S, Boehmer G, Gieseking F, et al. Utility and reproducibility of the International Federation for Cervical Pathology and Colposcopy classification of transformation zones in daily practice: A multicenter study of the german colposcopy network. J Low Genit Tract Dis. 2015;19(3):185–88. [PubMed]

Articles from Journal of Clinical and Diagnostic Research : JCDR are provided here courtesy of JCDR Research & Publications Private Limited