|Home | About | Journals | Submit | Contact Us | Français|
Although trachoma control programs frequently use the World Health Organization (WHO) simplified grading system for trachoma to monitor the clinical response after repeated mass azithromycin treatments, the programmatic relevance of this evaluation after multiple rounds of antibiotic treatments is unclear.
Three rounds of annual mass azithromycin were distributed to 12 villages in Ethiopia. Twelve months after the third treatment, children were assessed for follicular trachomatous inflammation (TF) and intense trachomatous inflammation (TI) using the WHO simplified grading system and for ocular chlamydial infection using DNA-based and RNA-based tests. Test characteristics for predicting chlamydial infection were computed assuming a chlamydial RNA-based gold standard. As a secondary analysis, test characteristics were also assessed using a latent class analysis.
The prevalence of RNA evidence of ocular chlamydia was 7.1% (95% confidence interval [CI], 2.7–17.4). A DNA-based test and TF had sensitivities of 61.0% (95% CI, 47.1–73.3) and 65.9% (95% CI, 41.6–83.9), specificities of 100% (95% CI, 99.3–100) and 67.5% (95% CI, 61.0–73.5), and positive predictive values of 100% (95% CI, 86.3–100) and 13.4% (95% CI, 5.5–29.3) compared with an RNA-based gold standard. The latent class analysis confirmed that the RNA-based test was a reasonable choice for a gold standard, with a sensitivity of 100% (95% CI, 67.1–100) and specificity of 99.6% (95% CI, 98.1–100).
Basing treatment decisions after mass azithromycin distributions on the WHO simplified grading system will maximize the treatment of infected persons compared with a DNA-based test but will also result in more uninfected persons being treated. The RNA-based test was considerably more sensitive, and almost equivalently specific, compared with a DNA-based test. (ClinicalTrials.gov number, NCT00322972.)
Trachoma is a blinding disease caused by ocular infection with Chlamydia trachomatis.1 Ocular chlamydial infection manifests clinically as conjunctival inflammation, characterized by conjunctival hyperemia and germinal lymphoid follicles on the upper tarsal conjunctiva. Repeated or persistent episodes of infection are associated with conjunctival scarring, which in turn may lead to entropion, trichiasis, and blindness.2–4
Trachoma-endemic districts are treated with mass administration of azithromycin to the entire population, regardless of each person's disease status. Given the central role of antibiotic therapy for trachoma, an ideal test for trachoma would assess chlamydial infection to provide information about the usefulness of future antibiotic treatments. The most widely used test for monitoring trachoma is conjunctival examination using the World Health Organization (WHO) simplified grading system.5 In this system, a grade of follicular trachomatous inflammation (TF) indicates the presence of five or more follicles at least 0.5 mm in diameter in the upper tarsal conjunctiva. A grade of intense trachomatous inflammation (TI) indicates obscuration of more than half the deep tarsal blood vessels. Both TF and TI are used to assess the burden of trachoma in a community.
Laboratory tests for chlamydial infection are used primarily for trachoma research.6 Most commonly used is a nucleic acid amplification test (NAAT) that targets the cryptic DNA plasmid of chlamydia (Amplicor; Roche Diagnostics, Indianapolis, IN). A second-generation rRNA-based NAAT with an additional target capture step has been well characterized in the sexually transmitted disease literature. In a duplex format that detects both Chlamydia trachomatis and Neisseria gonorrhoeae, this rRNA-based test (Aptima Combo 2; Gen-Probe, San Diego, CA) has been shown to be among the most sensitive and specific of the NAATs for the detection of chlamydia.7–15 A version of this test that detects only chlamydia (Aptima CT) has proven to be an even more sensitive indicator of chlamydial infection.8,11,16 NAATs targeting chlamydial rRNA are not widely used by trachoma control programs or researchers, though several trachoma studies have reported results of conjunctival swabs using Aptima CT.17–19 Because of its superior sensitivity, the RNA test is a logical gold standard test against which to compare other tests for their ability to predict ocular chlamydia.
The WHO recommends treating trachoma-endemic communities with annual mass azithromycin treatment, followed by reassessment after at least 3 years of treatment using the simplified grading system.20,21 However, the results of clinical examination are often discordant with results from laboratory tests for chlamydia, especially after mass azithromycin treatments.22 This finding may reflect differences in the kinetics of infection versus inflammation (i.e., there is an incubation period before inflammatory clinical signs develop and a period of persistent clinical signs after infection has cleared).6,23 Because the simplified grading system is used to determine whether mass antibiotic treatments should continue, its ability to predict chlamydial infection is important. We compared the diagnostic test performance (defined as the ability to correctly identify the presence of chlamydia) of the WHO simplified grading system with that of a DNA-based chlamydial test after three rounds of mass azithromycin using an RNA-based chlamydial test as a gold standard.
In a cluster-randomized clinical trial for trachoma conducted in the Goncha Siso Enese woreda of Ethiopia from 2006 to 2009, 72 contiguous subkebeles (government administrative units) were randomly assigned to 1 of 6 treatment arms.24–27 In one of the treatment arms, 12 subkebeles were randomly assigned to receive annual mass azithromycin treatments of the entire community. At the 36-month visit (1 year after the third mass treatment), conjunctival examination and swabbing for chlamydia testing were performed on a random sample of children from each subkebele.28,29
An enumerative census and mass azithromycin distribution were performed in all 12 subkebeles at months 0, 12, and 24 of the study. At each distribution, a single dose of directly observed azithromycin (1 g for adults, 20 mg/kg approximated with height-based dosing for children)30 was offered to all persons in the subkebele aged 1 year and older. Infants younger than 1 year, pregnant women, and anyone allergic to macrolides were given two tubes of tetracycline ophthalmic ointment to be used twice daily for 6 weeks.
Each of the 12 subkebeles consisted of approximately four smaller administrative units known as state teams (interquartile range, 3.5–5 state teams per subkebele), which we refer to as villages for the purposes of this report. We randomly selected one sentinel village from each subkebele for monitoring. At the 36-month study visit, conjunctival examination and swabbing were performed on 50 randomly selected children aged 0 to 9 years from each sentinel village. Trained, experienced examiners assessed the upper right tarsal conjunctiva for TF and TI according to the WHO simplified grading system.5 Then a dacron swab was passed firmly three times over the upper right tarsal conjunctiva, rotating 120° between each pass. Finally, a swab for chlamydial RNA was collected in a similar fashion, using swabs and transport media from a swab collection kit (Aptima CT [ACT] Unisex Swab Specimen Collection Kit; Gen-Probe, Inc.). RNA swabs were always collected after DNA swabs. We collected five negative field control swabs per sentinel village for each laboratory test by passing the swab within 1 inch of, but not touching, the tarsal conjunctiva. Examiners used a new pair of gloves for each study participant.
We trained all examiners in trachoma grading and conjunctival swabbing at the beginning of each study visit. To grade the clinical signs of trachoma for this study, examiners were required to reach sufficient agreement (Cohen's kappa ≥0.6) with a consensus grade from three experienced trachoma graders (BA, BDG, TML) regarding the presence of clinically active trachoma (TF, TI, or both) on a set of 50 conjunctival photographs. The kappa coefficient for clinically active trachoma for clinically active trachoma for the eight graders in this study ranged from 0.66 to 0.88; the mean kappa coefficient, inverse probability weighted based on the number of children examined, was 0.87 (95% confidence interval [CI], 0.75–0.99) for TF and 0.73 (95% CI, 0.69–0.77) for TI.
The dacron swabs were stored at 4°C in the field and at −20°C within 6 hours and then were shipped at 4°C to San Francisco, where they were stored at −80°C until processing. The dacron swabs were processed using an assay (Amplicor PCR assay; Roche Diagnostics) to detect Chlamydia trachomatis DNA within the cryptic plasmid. This assay was performed on pools of five swabs, and individual swabs from any pools with positive results were tested further. RNA swabs were stored and transported to San Francisco at room temperature. Nucleic acid amplification of C. trachomatis 16s rRNA using the Aptima CT assay was also performed on pools of five swabs, and individual swabs from any pools with positive results were tested further.
Any samples with discrepant NAAT results were retested with the RNA-based ACT assay. Discrepant samples were also tested with the Aptima Combo 2 assay (Gen-Probe, Inc.), a different NAAT that targets the 23S chlamydial rRNA. A random sample of nine swabs that tested positive by the ACT and Amplicor tests and five swabs that tested negative by ACT and Amplicor were also processed with both RNA-based tests as positive and negative controls. Amplicor was performed on any discrepant samples from pools that had tested negative for chlamydia. Laboratory personnel were masked to the results of the original tests.
We calculated the sensitivity, specificity, predictive values, and likelihood ratios of the DNA-based test (Amplicor), TF, TI, and TF/TI (TF, TI, or both) using the RNA-based test (ACT) as the gold standard and accounting for within-village correlation when computing 95% CIs. We tested whether the sensitivity of the DNA-based test differed among those with and those without TF/TI using a mixed-effects logistic regression of the RNA-positive population, with community as a random effect. We estimated likelihood ratios from a log Poisson model using a robust variance estimate that adjusted for within-village correlation.31 Exact confidence intervals were computed for the likelihood ratio of any test with perfect sensitivity or specificity, ignoring the clustered study design (StatXact 3; Cytel Inc., Cambridge, MA).
Based on numerous previous studies, we assumed that the RNA-based test was the gold standard for our initial analyses.7–15 As a secondary analysis, we also determined the diagnostic characteristics of four tests (TF, TI, DNA, and RNA) using a latent class analysis that does not assume any of the tests is a gold standard.32,33 We implemented a clustered variant of the classical latent class model; we assigned a parameter to the latent gold standard prevalence of each village and the sensitivity and specificity of each of the four tests for this latent gold standard.34 To fit the model, we minimized the sum of the Kullback-Leibler discrepancies for each village.32,35 We used bootstrap resampling at the village level with 999 replicates and the percentile method to calculate confidence intervals. Pairwise P values were calculated by paired t-test on estimates from each village-level resample. The latent class analysis was performed with technical computing software (Mathematica 7; Wolfram Inc., Champaign, IL); all other analyses were performed with data analysis and statistical software (Stata 10; StataCorp, College Station, TX).
Ethical approval for the study was obtained by the Committee for Human Research at the University of California at San Francisco, the Institutional Review Board at Emory University, and the Ethiopian Science and Technology Commission. Informed consent in Amharic was obtained from all study participants. The research followed the tenets of the Declaration of Helsinki.
One year after a third annual mass azithromycin treatment, we assessed 583 children aged 0 to 9 years. Six children were excluded from further analysis because either they did not have a documented trachoma grade or no one performed a conjunctival swab on them. A grade of TF was assigned to 201 children (34.8%; 95% CI, 27.7–42.8), TI to 95 children (16.5%; 95% CI, 9.4–27.2), and TF/TI (TF, TI, or both) to 251 children (43.5%; 95% CI, 35.0–52.4). Laboratory evidence of chlamydia was found in 25 children (4.3%; 95% CI, 1.7–10.5) using the DNA-based test and in 41 children (7.1%; 95% CI, 2.7–17.4) using the RNA-based test (Table 1). None of 60 negative control dacron swabs or 59 negative control RNA swabs tested positive for chlamydial DNA or RNA, respectively.
We calculated the sensitivity, specificity, predictive values, and likelihood ratios of four different measures of trachoma using RNA evidence of chlamydial infection as the gold standard (Table 2). The simplified grading system had sensitivities ranging from 53.7% to 75.6% and specificities ranging from 59.0% to 86.4%, depending on the grade. The test for chlamydial DNA had a sensitivity of 61.0% (95% CI, 47.1–73.3) and a specificity of 100% (95% CI, 99.3–100). The sensitivity of the DNA test was higher in subjects with TF/TI (74.2%; 95% CI, 48.2–89.9) than in those without TF/TI (20.0%; 95% CI, 4.3–58.2; P = 0.01). Positive predictive values were much lower for the simplified grading system (range, 12.4%–23.2%) than for the DNA-based test (100%).
Discrepancies between the DNA-based and the RNA-based test were limited to 16 cases that were positive for chlamydial RNA but negative for chlamydial DNA (Table 1). On confirmatory RNA testing of these 16 discrepant samples, ACT confirmed 13 as positive, and AC2 confirmed four as positive and one as equivocal. For comparison, five negative control swabs (negative for ACT and Amplicor) tested negative by both confirmatory RNA-based tests, and nine positive control swabs (positive for ACT and Amplicor) tested positive by both confirmatory RNA-based tests. Amplicor testing was performed on 10 swabs that had come from a negative pool and therefore had not been tested individually. Of these 10 swabs, one tested positive for chlamydial DNA. The Amplicor-positive confirmatory test was also positive for both confirmatory RNA tests, and the five confirmatory samples that tested positive or equivocal by AC2 tested positive by the confirmatory ACT test.
Our analyses assumed that the RNA-based chlamydial assay was the gold standard. Recognizing the imperfect sensitivity or specificity of this assay, we also performed a latent class analysis in which we allowed an unobservable gold standard. This latent class analysis estimated 12 prevalence parameters and eight test characteristic parameters, including those for the RNA-based test (Table 3). The model estimated the RNA-based test to be considerably more sensitive than the DNA-based test (100% vs. 64.2%; P = 0.004) and slightly less specific (99.6% vs. 100%; P = 0.03). TF was significantly more sensitive than TI (P = 0.04), and TI was more specific than TF (P < 0.001).
After three rounds of mass azithromycin, the WHO simplified grading system appeared to have comparable or better sensitivity but lower specificity than a DNA-based laboratory test. In other words, clinical examination identified more infected persons than the DNA-based test but at the cost of more false positives. Both TF and TI significantly increase the posttest probability of infection in a person but to a lesser degree than the DNA-based test.
Consistent with two previous reports, ocular swabs from this trachoma-endemic region were more frequently positive using the test for chlamydial RNA compared with the test for chlamydial DNA.17,18 Although it is possible that results of the 16 excess RNA-positive tests are false positives, there are several reasons to think they represent a true measure of chlamydial infection. First, the Aptima RNA-based tests have consistently been shown to be more sensitive, and equally or more specific, than DNA-based tests in the sexually transmitted disease literature.7,10–15 Second, a latent class analysis, which makes no assumptions about the gold standard, suggested that the data were most consistent with a highly specific RNA-based test and a less sensitive DNA-based test. Third, none of the negative control swabs tested positive for RNA, arguing against the possibility of false positives through contamination in the field or in the laboratory. Finally, confirmatory testing with RNA-based tests demonstrated infection in 81% of discrepant cases using ACT and in 31% of cases using AC2. Although one might have expected a higher proportion of discrepant cases to have tested positive with the ACT test, this degree of discrepancy on repeat testing is consistent with the 90% reproducibility of chlamydial NAATs demonstrated in other studies.16,36 The lower number of positive confirmatory tests with the AC2 assay was expected given that this test has been shown to be less sensitive than the ACT assay.8,11,16
Compared with a gold standard of chlamydial RNA, the DNA-based chlamydial NAAT had a sensitivity of only 61%, albeit with wide confidence intervals. This may seem to be lower than expected. However, these results are consistent with those of other studies of these particular RNA- and DNA-based tests in which the DNA-based test detected 64% to 86% of the RNA-positive cases.11–13 RNA-based methods are thought to more easily detect chlamydia than DNA-based methods, likely because chlamydial organisms contain approximately 1000 copies of 16s rRNA but only 7 to 10 copies of the cryptic DNA plasmid.9,37–39 In addition, the target capture step in the Aptima assays probably contributes to enhanced sensitivity. This increased sensitivity of the RNA target may be most relevant in the setting of a low load of infectious organisms, which was likely the case in this study of communities monitored after repeated mass azithromycin treatments. A separate study in Ethiopia found evidence for this hypothesis: the sensitivity of the DNA-based test was lower in villages treated with mass azithromycin than in untreated villages.18 In the present study, the sensitivity of the DNA test was lowest in those without clinically active trachoma, perhaps because these persons have a lower infectious load of chlamydia.40,41 The increased sensitivity of the RNA-based test—in the face of nearly equivalent specificity—suggests that the RNA-based test is the NAAT of choice for monitoring ocular chlamydia, especially for researchers and trachoma programs interested in trachoma elimination.
The WHO recommends conjunctival examination of children aged 1 to 9 years in communities treated with three annual mass azithromycin treatments, with further mass treatments until the prevalence of TF in children falls below 5%.20,21 Although the WHO recommendations are widely implemented, few studies have assessed the diagnostic test performance of the WHO simplified grading system after mass azithromycin treatment (Table 4). Here, we showed that both TF and TI provide diagnostic information regarding chlamydial infection after three mass azithromycin distributions, yet TF and TI are not interchangeable tests. In this and other studies, TF appears to be a more sensitive test than TI for ocular chlamydia infection after mass azithromycin treatments.42 In this study, TF even had a higher sensitivity than a DNA-based laboratory test. TF is, therefore, a sensible metric if the goal is trachoma elimination and identification of all possible cases of ocular chlamydia is desired.
It is important to note that the positive predictive values for both TF and TI were far lower than those of the DNA-based NAAT. This indicates that basing treatment decisions solely on the simplified grading system will necessarily result in treatment of a considerable number of uninfected persons. For example, in these communities, three mass treatments had a dramatic effect on ocular chlamydia, reducing the prevalence among 0 to 9-year-old children from 42% to 4% (reported elsewhere).27 The prevalence of TF declined much more slowly, to approximately 35%, much higher than the <5% prevalence required for discontinuing mass antibiotic treatments. Thus, many uninfected persons would receive antibiotics if mass treatments were continued. This is probably appropriate in areas with hyperendemic trachoma, where reinfection can rapidly occur after the cessation of mass treatments, even when these treatments have reduced the prevalence of chlamydial infection to very low levels.44 In areas with less prevalent trachoma, however, reliance on the WHO simplified grading system will likely result in the overtreatment of communities with low levels of ocular chlamydia. Use of NAATs for trachoma monitoring could reduce unnecessary antibiotic treatments. The perceived costs of NAAT are likely to prevent its use by many trachoma programs though, especially because the expenses of using the simplified grading system are so low. More expensive nucleic acid amplification tests may be worth the added cost in certain situations, such as in communities with a low pretreatment prevalence of trachoma, where reinfection is less likely to occur after mass antibiotics have reduced ocular chlamydia to low levels.42 Formal cost-effectiveness studies might be helpful in more clearly defining the role of laboratory tests after mass antibiotic treatments.
A limitation of this study is its cross-sectional nature. We had data for the WHO simplified grading system, the DNA-based NAAT, and the RNA-based NAAT only at a single time point, which did not allow us to comment on the clinical significance of infections detected only by the RNA-based test but not the DNA-based test (e.g., whether these infections are new, almost resolved, or low-level persistent).
In conclusion, we report the diagnostic test characteristics of the WHO simplified grading system and a DNA-based nucleic acid amplification test after three repeated mass antibiotic distributions in a hyperendemic region of Ethiopia compared with an RNA-based gold standard test. TF was the most sensitive test and appears to be an appropriate test for detecting the maximum amount of ocular chlamydia. However, both TF and TI had much lower positive predictive values than the DNA-based laboratory test, indicating that use of the WHO simplified grading system could result in overtreatment. The positive predictive value of the NAATs remained high even after repeated mass treatments, suggesting these tests would minimize unnecessary antibiotic distributions. If resources allow for the use of NAAT, the RNA-based test, given its superior sensitivity, is the test of choice.
Donald Everett (National Eye Institute, Bethesda, MD) was the program officer for the underlying clinical trial. The data safety and monitoring committee for the underlying clinical trial included William Barlow (University of Washington, Seattle, WA; Chair), Donald Everett, Larry Schwab (International Eye Foundation, Kensington, MD), Arthur Reingold (University of California, Berkeley, CA), and Serge Resnikoff (Brien Holden Vision Institute, Sydney, Australia, and International Health and Development, Geneva, Switzerland). Logistical support was provided by the head of the Goncha woreda health office, Tadege Alemayehu; the head of the Amhara Regional Health Bureau, Asrat Genet Amnie; and the Ethiopian Ministry of Health. Trachoma monitoring was performed by Mitselal Abrahale, Melkam Andualem, Rebecca Beauregard, Manahlosh Berihun, Michael Chen, Temesgen Demile, Tessema Eneyew, Banchu Gedamu, and Melese Temesgen.
Supported by National Institutes of Health Grants NEI U10 EY016214 (TML), NEI K23EY019071 (JDK), and NCRR/OD UCSF-CTSI KL2 RR024130, the Bernard Osher Foundation, That Man May See, the Harper Inglis Trust, the Bodri Foundation, the South Asia Research Fund, Research to Prevent Blindness, and the International Trachoma Initiative.
Disclosure: J.D. Keenan, None; C.W. See, None; J. Moncada, None; B. Ayele, None; T. Gebre, None; N.E. Stoller, None; C.E. McCulloch, None; T.C. Porco, None; B.D. Gaynor, None; P.M. Emerson, None; J. Schachter, None; T.M. Lietman, None