|Home | About | Journals | Submit | Contact Us | Français|
Treatment of recent tuberculosis infection in children <2 years old is essential because of high risk of progression to disease, but diagnosis is hindered by the inaccuracy of the tuberculin skin test (TST). More accurate T cell-based tests for infection could enhance diagnosis by optimizing TST interpretation.
979 child tuberculosis contacts in Istanbul underwent TST and enzyme-linked immunospot (ELISpot) testing. Using ELISpot results as a reference standard, we assessed the effect of age and BCG-vaccination on sensitivity and specificity of TST, and computed optimal TST cut-off points (OCPs) using receiver operator characteristic curves.
Using a ≥10mm TST cut-off point, sensitivity of TST was 66% in children <2y, lower than in older children (p=0.006). Specificity was 75% in BCG-vaccinated children, compared with 92% in unvaccinated children (p=0.001). OCPs improved TST specificity in children with 1 BCG scar with little loss of sensitivity. Despite use of OCPs, sensitivity of TST remained <70% in children <2y, specificity remained <87% in BCG-vaccinated children >2y and overall accuracy was low in children with >1 BCG scar.
Negative TST results cannot exclude tuberculosis infection in child tuberculosis contacts <2 years old, supporting use of preventive therapy regardless of TST results in this age group. In children >2 years old, accuracy of TST can be improved by adjustment of cut-off points in BCG-vaccinated children but remains poor in children with >1 BCG scar. This methodology can define optimal TST cut-off points for diagnosis of tuberculosis infection tailored to target populations.
Children with Mycobacterium tuberculosis infection are at high risk of developing active tuberculosis [1, 2]. Infected children <2 years of age have a ≥20% risk of progression to active disease, which rises to 50% in infants . The risk falls in children >2 years and is approximately 5% in the 2-5 year old age group . Given that isoniazid therapy prevents progression to tuberculosis , rapid, accurate diagnosis of childhood M tuberculosis infection is a global public health priority.
However, routine diagnosis of M. tuberculosis infection is based on the tuberculin skin test (TST), which handicaps management of childhood tuberculosis infection. Prior BCG vaccination may affect specificity of the TST , and some countries adjust TST cut-off points in vaccinated children. However, there is no consensus on whether, or how much, the TST cut-off point should be increased [5, 6].
Sensitivity of the TST in young children is unknown and guidelines for the management of child tuberculosis contacts therefore vary widely. Some national tuberculosis control programmes recommend isoniazid preventive therapy for young children on the basis of TST results [5, 7], while others recommend it for all child tuberculosis contacts under a certain age, regardless of TST results [6, 8-11].
T cell-based interferon-gamma release assays (TIGRA) represent a significant advance on the TST [12-15], and in this study, the interferon-gamma enzyme-linked immunospot (ELISpot) assay was utilized. Although further research is required, much published evidence indicates that in adults the ELISpot is more specific, and more sensitive than the TST [16-28]. Although there are fewer studies in children, the available evidence shows that the assay is more sensitive than TST in children with active tuberculosis [29-31], and although there is no gold-standard test for latent tuberculosis infection (LTBI), ELISpot appears more sensitive than TST in contact investigations in children and infants [17, 30, 32-34]. Thus, available evidence suggests that this ELISpot assay is more specific and more sensitive than TST for detection of LTBI in children.
However, TIGRAs are not yet suitable for all populations because they are expensive and require a blood sample and laboratory equipment. Therefore, most children may fail to benefit from this important medical advance. We reasoned that TIGRAs might be used to inform and improve the interpretation and diagnostic accuracy of TST. In the absence of a gold-standard test, we used ELISpot as a surrogate reference standard for LTBI to optimize use of TST in children with recent tuberculosis exposure. ELISpot was applied in parallel with TST in 979 child household contacts in Istanbul, Turkey, which has an intermediate prevalence of tuberculosis (40/100,000) , a very low prevalence of HIV-infection in children , and a policy of universal childhood BCG vaccination. We classified ELISpot-positive children as infected and ELISpot-negative children as uninfected and then assessed the effect of young age and BCG-vaccination on sensitivity and specificity of TST.
All adults diagnosed with sputum smear-positive pulmonary tuberculosis at the 7 government-funded tuberculosis clinics in east Istanbul between October 2002 and May 2004 and who had children living in the household were invited to participate as previously described .
Ethical approval was granted by the Institutional Review Board of Marmara University School of Medicine, Istanbul, The Turkish Ministry of Health, Ankara and the World Health Organization Steering Committee on Research Involving Human Subjects, Geneva.
The Turkish Ministry of Health guidelines for BCG vaccination are as follows: all children are vaccinated intradermally with BCG Pasteur 1173-P2 (Serum Institute of India Ltd., Pune, India) between 2 and 3 months of age and a booster vaccination is administered in the first year of primary school, at 6 to 7 years of age. BCG vaccination coverage in Turkish children was 79% in 2004 .
1,024 child contacts of the 443 index patients with sputum smear-positive pulmonary tuberculosis were enrolled at the Paediatric Infectious Diseases Clinic, Marmara University School of Medicine where medical histories were taken, physical examination and investigations performed and demographic information recorded, as previously described . Out of a total of 1024 children enrolled, complete demographic, clinical, ELISpot and TST data were available for 979 as previously described .
TST was administered by the Mantoux method as previously described . For interpretation and analysis of TST induration, three different universal cut-off points of induration were used to define a positive TST result: ≥5mm; ≥10mm; ≥15mm.
ELISpot assays were performed as previously described [30, 32]. This assay has subsequently been developed into the regulatory approved commercially available T-SPOT.TB® assay (Oxford Immunotec, Abingdon, UK), which uses the same ESAT-6 and CFP-10 peptides. Our predefined cut-off point is the standard used in all previous studies based on our assay, amounting to 9 studies in 1916 participants [16, 17, 19, 20, 22, 29, 32, 38, 39]. Specificity and sensitivity of this assay are described in the corresponding studies [16, 17, 19, 20, 22, 29, 32, 38, 39] and recent reviews [13, 14, 34, 40].
Determinants of disease prevalence, test sensitivity and specificity were identified using logistic regression modeling, the significance of each factor assessed using likelihood ratio and Wald tests. Observed receiver operator characteristic (ROC) curves were plotted for each subgroup, the areas under curves estimated using the trapezoidal rule, and the significance of differences in areas under the curve were tested.
Smoothed ROC curves were constructed to minimize random error and remove digit preference, before identifying optimal cut-off points (OCPs). Curves were fitted using the Dorfman and Alf maximum likelihood latent scale binormal model , and fitted values of sensitivity and specificity obtained at each observed cut-off point. Estimates of the probability of false-negative (Prob(FN)) and false-positive (Prob(FP)) diagnoses accounting for disease prevalence were computed for each cut-off point as:
OCPs were computed using prevalence of infection estimates obtained from the logistic regression model for each subgroup.
The first optimal cut-off point criteria minimized total diagnostic error, identifying the cut-off point at which the sum of the probabilities of false positive and false negative diagnoses of latent tuberculosis infection was lowest, which allowed prevalence of infection to be taken into account. We also report the range of cut-off points within which the probability of diagnostic error was within 2% of this minimum value to describe the sensitivity of overall performance to cut-off point selection. A second optimal cut-off point was defined as the cut-off point at which the consequences of diagnostic error were minimized. Whilst it is clear that false negative diagnoses (missing a case of LTBI) have greater consequences than false positive diagnoses (incorrectly diagnosing LTBI and giving unnecessary treatment), the relative disutility of these consequences is not known. We created a disutility score as:
where k was given values of 2, 5 and 10 corresponding to assumptions of consequences of false negative results being 2, 5 and 10 times more catastrophic than consequences of false positives and identified the cut-off points at which the score was minimized. Results are presented for a value of k=2. We report ranges of cut-off points with disutility values within 2% (relative to the maximum theoretical score) of the optimal cut-off point score.
Analyses were undertaken in Stata V9·0 (Stata Corporation, College Station, TX, USA) using roccomp and rocfit commands, and in Microsoft® Office Excel (Microsoft Corporation, Seattle, WA, USA).
Median age of the 979 child contacts of the 414 sputum-smear positive TB cases was 7 years [IQR 3, 11], 50·2% were male, and the average number of contacts per household was 2·5. 770 contacts (78·7%) were BCG-vaccinated, based on presence or absence of a BCG scar, including 115 contacts that had 2 scars and 3 contacts with 3 BCG scars.
Based on positive ELISpot results, 416 (42·5%) children were deemed infected with M tuberculosis. Proportions of TST-positive children were 60·5%, 50·8%, and 40·1% using cut-off points of ≥5mm, ≥10mm, and ≥15mm, respectively.
TST indurations for ELISpot-negative children (figure 1b) were lower than for ELISpot-positive children (figure 1c) (median [IQR]: 0 [0,10] mm and 20 [16,23] mm respectively, P<0·0001, Mann Whitney U-test).
With a cut-off point of ≥1mm, the relative specificity remains above 50%, reaching a plateau at 25 to 30mm and the relative sensitivity starts at 89·4% gradually declining to 78·6% at 15mm and more steeply thereafter (figure 2).
The shape of the ROC curve was different for children aged <2 years compared with older children (figure 3a). Using a ≥15mm cut-off point, relative sensitivity decreased from 80% in children aged 2-16 years to 63% in children aged <2 years (P=0·024) and from 85% to 66% using the ≥10mm cut-off point (P=0·006) (table 1). The ≥5mm cut-off point gave the best relative sensitivity of 78·1% in the under 2 year olds, which was still significantly lower than in older children (P=0·049), at the cost of a very low relative specificity of 59·2% (table 1). Thus at all cut-off points, relative sensitivity of TST was lower in children <2 years old, and even using the ≥5mm cut-off point, 7 infected infants would have been missed because of false-negative TST results.
Table 1 shows a trend towards lower relative specificity in older children which remained significant after adjustment for BCG vaccination status (test for trend across age-categories at a ≥15mm cut-off point: P=0·013). Despite the strong relationships between relative TST sensitivity and specificity with age, the area under the ROC curve, a measure of overall test accuracy, did not significantly differ between the three age-groups (0·81, 0·87 and 0·87 for 0-1, 2-5, 6-16 years respectively; test for difference P=0·54), as can occur when the relationships act in contrasting directions.
The area under the ROC curve became significantly smaller with increasing number of BCG scars (figure 3b), indicating a reduction in overall test accuracy (0·93, 0·86 and 0·79 for 0, 1, and >1 BCG scars respectively; test for difference P=0·003). The number of BCG scars significantly adversely affected TST specificity at all cut-off points but did not affect sensitivity (table 1). To investigate this further we stratified the distribution of Mantoux indurations by ELISpot results and number of BCG scars (figure 4).
OCPs that minimized the sum of the probabilities of false-positive and false-negative diagnoses taking into account prevalence of infection were identified from smoothed ROC curves generated for groups defined by age (<2 vs. 2-16) and BCG scar status (0 vs. 1 vs. >1) (figure 5 and table 2). The 2-5 and 6-16 age-groups were combined as there was no significant difference in prevalence of infection or TST performance between the groups (table 1 and figure 3a).
Table 2 shows the OCPs for prevalences of infection estimated from the observed data. Results are presented for two different analytical strategies (see Statistical Methods). Estimated performance of TST relative to ELISpot at these OCPs and the range of cut-off points adjacent to the OCPs with similar overall performance are shown in table 2.
In unvaccinated children >2 years where relative TST specificity and the prevalence of infection are high (table 2), the probability of false-positive results is low. Optimization therefore computed a low cut-off point of ≥2mm to minimize the summed probabilities of false-negative and false-positive results.
Because of the serious adverse clinical consequences of false-negative results in children aged <2 years, we calculated OCPs for disutility multipliers of k=5 and k=10 (see Statistical Methods). In vaccinated children aged <2 years, at the observed population infection prevalence of 22%, OCPs using disutility multipliers of 5 and 10 were, ≥13mm and ≥11mm, respectively. In unvaccinated children aged <2 years, a disutility multiplier of 10 gave an OCP of ≥4mm, which amounted to a cut-off of any observed response (as 4mm was the smallest observed response in this group); even this OCP yielded a relative diagnostic sensitivity of only 81%. If the disutility multiplier (k) exceeded 10.7, treating all patients would be a preferable strategy than even this lowest cut-off point option, given the observed study prevalence.
However, even using computed OCPs, the overall error rate of TST relative to ELISpot, rises from 10% in unvaccinated children >2 years old to 20% or more in children with ≥1 BCG vaccination scars (table 2).
OCPs were also calculated for the 5 groups using arbitrary prevalences of 5%, 20% and 40%. Cut-off point selection for children <2 years old was unaffected by prevalence of infection, while OCP selection for children aged >2 years was related to prevalence (Table 2, footnote).
The lack of a gold standard for LTBI greatly complicates interpretation of TST results which in turn represents a substantial obstacle to improving tuberculosis control. Setting cut-off points has therefore relied upon empirical comparison of TST indurations in presumptively infected and uninfected populations. Such analyses have hitherto been based on population distributions of TST results from patients with active tuberculosis or hypothetical distributions of TST results computed by subtracting the distribution of TST results in unexposed individuals from that in tuberculosis contacts [42, 43]. We used ELISpot results as a surrogate reference standard in child tuberculosis contacts to assess the effect of BCG vaccination and young age on specificity and sensitivity of TST and optimize TST cut-off points.
The lower relative sensitivity of TST in children <2 years old suggests that the delayed type hypersensitivity response to M. tuberculosis infection in infants is weaker than in older children . However, ELISpot can detect very low levels of T cell responses to M. tuberculosis infection. This explains its high diagnostic sensitivity relative to TST in infants [17, 33] and young children  with immature cellular immune systems, and in HIV-infected individuals [12, 22, 29].
Our analysis to minimize diagnostic error assumed that the consequences of false-positive and false-negative results were equal, but in practice the clinical consequences of not treating infected children aged <2 years are more severe than treating uninfected children, because of the high risk of progression to tuberculosis and its attendant high morbidity and mortality [1, 45, 46]. In order to account for how different clinical outcomes affect clinicians' interpretation of diagnostic test results, we computed an alternative set of OCPs which minimized a disutility score. This weighted the cut-off point selection process in favor of minimizing false-negative results and generated cut-off points that were equal to or lower than cut-off points for minimizing the diagnostic error score, as relative TST sensitivity was maximized at the cost of relative specificity. Given the imperative for early treatment of tuberculosis infection in very young children, our results suggest that TST lacks sufficient sensitivity to reliably rule out a diagnosis of tuberculosis infection amongst household contacts in this age group. Some national guidelines recommend that child contacts <2 years old should receive isoniazid preventive therapy on the basis of positive TST results . Our results lend greater support to a policy of giving universal preventative therapy to all child tuberculosis contacts <2 years, regardless of TST results.
In children aged 2-5 years, sensitivity of TST relative to ELISpot was not significantly different to its performance amongst children aged over 6 years. Therefore, given that children >2 years are at substantially lower risk of primary progression to active disease than younger children , and given that the relative sensitivity of TST did not increase further with increasing age, 2 years may be a suitable age threshold above which TST results can be used to guide targeting of isoniazid preventive therapy to child contacts.
In unvaccinated children, TST specificity was high but declined progressively as the number of previous BCG vaccinations increased. In ELISpot-negative children with >1 scar, indurations of ≥20mm were not uncommon. The recommended cut-off points in Turkey, while slightly different to our OCPs, nonetheless performed within a 2% margin of error compared with the OCPs that were computed to minimize disutility where k=2.
The OCPs represent the best possible performance for the TST in this population, given ELISpot as the reference standard; however, even these gave error rates between 10 and 27%. The problem was most pronounced in children with >1 BCG scar. Several countries perform a second BCG vaccination in children (e.g. Turkey and Russia) and use TST to diagnose LTBI . Previous studies reached contradictory conclusions about the impact of repeat BCG vaccination on TST [47-49]. Our results indicate that repeat vaccination has a substantial impact on TST induration which renders interpretation of TST unreliable, and even use of OCPs results in a high proportion of false-positive results amongst child contacts.
In contrast, TST specificity in children under 2 years was high, despite the close proximity in time to BCG vaccination (table 1). Thus, notwithstanding the poor sensitivity of TST in children under 2 years, its high specificity, using a ≥15mm cut-off point, makes a positive TST result a reliable marker of tuberculosis infection in very young children.
Specificity of TST varies across different populations and regions of the world and depends, in part, on the level of environmental mycobacterial exposure, as well as BCG vaccination status [4, 50]. Thus, while our OCPs are of direct relevance to clinical practice in Turkey, they cannot be extrapolated to other populations. Our approach for deriving OCPs, however, is generalizable. Where deployment of TIGRAs is not yet possible, testing sentinel populations by TST and TIGRA would enable tuberculosis control programmes to set more accurate TST cut-off points tailored to the whole target population.
In contrast to the consistent evidence from low and medium prevalence countries for correlation of TIGRA results with tuberculosis exposure, a recent Gambian study using a variation of our assay with different thresholds for scoring results, sensitivity and specificity [51, 52] found poor correlation and further work is required in high-burden settings. Although the ELISpot assay used in our study represents an improvement on TST [12, 16-19, 21, 22, 25, 32, 38] and is already recommend by several national guidelines, it is not a perfect test of LTBI and our OCPs will have been based on imperfect diagnoses in a proportion of our population,
Our findings could inform tuberculosis control policy. The low sensitivity of TST in children <2 years old supports the use of isoniazid preventive therapy in these household contacts regardless of TST results. In children with two BCG scars, TST specificity was very poor even after adjusting the cut-off point. Given that a second BCG vaccination confers no additional protection against tuberculosis disease  or infection  yet renders the TST almost uninterpretable, it may be impeding tuberculosis control efforts. Finally, our study provides a mechanism through which the scientific advance of T cell-based testing could be used to improve management of childhood tuberculosis infection in resource-limited settings.
We dedicate this article to Prof. Mujdat Basaran, who died 6 years ago. He was the founder of the Department of Paediatrics, Marmara University School of Medicine, and contributed to tuberculosis immunology research with many publications. We thank all the children who took part in the study, and their parents. We thank Zeliha Arslan and Filiz Ozturk for referring contacts from the tuberculosis clinics in Istanbul and Imogen Staveley for contributing to the reading and analyzing of ELISpot assays. We acknowledge the crucial support of the Istanbul Association for the Fight Against Tuberculosis, and the physicians and nurses of the seven government-run tuberculosis clinics in the Asian side of Istanbul. We also thank The Department of Tuberculosis of The Ministry of Health of Turkey for their support. We are very grateful to Ben Marais, Luca Richeldi, Frank Cobelens, Hans Rieder, Beate Kampman, Susan Liebeshuetz, Paul Newton, Paul Heath, Paul Fine and John Innes for critical appraisal of the manuscript and helpful discussions. This work was funded by the Wellcome Trust and the UNICEF/UNDP/World Bank/WHO Special Programme for Research and Training in Tropical Diseases (TDR). DPSD's PhD studentship was supported by the Sir Halley Stewart Trust. JJD was funded by a Department of Health Senior Research Scientist in Evidence Synthesis Award. KAM was supported by a Wellcome Trust Prize studentship. AL is a Wellcome Senior Research Fellow in Clinical Science.
Conflict of Interest Statement Professor Lalvani is lead inventor for several patents underpinning T cell-based diagnosis. The Lalvani ELISpot was commercialised by an Oxford University spin-out company (T-SPOT.TB®, Oxford Immunotec Ltd, Abingdon, UK) in which Oxford University and Professor Lalvani have minority shares of equity. Professor Lalvani acted as non-executive director to Oxford Immunotec from 2003-07. Dr Dosanjh and Dr Millington are named inventors on patents relating to T cell-based diagnosis. No other authors have a potential conflict of interest.