|Home | About | Journals | Submit | Contact Us | Français|
The clinical utility of real-time PCR screening assays for methicillin (meticillin)-resistant Staphylococcus aureus (MRSA) colonization is constrained by the predictive values of their results: as MRSA prevalence falls, the assay's positive predictive value (PPV) drops, and a rising proportion of positive PCR assays will not be confirmed by culture. We provide a quantitative analysis of universal PCR screening of critical care and emergency surgical patients using the BD GeneOhm MRSA PCR system, involving 3,294 assays over six months. A total of 248 PCR assays (7.7%) were positive; however, 88 failed to be confirmed by culture, giving a PPV of 65%. Multivariate analysis was performed to compare PCR-positive culture-positive (P+C+) and PCR-positive culture-negative (P+C−) assays. P+C− results were positively associated with a history of methicillin-sensitive Staphylococcus aureus infection or colonization (odds ratio [OR], 3.15; 95% confidence interval [CI], 1.32 to 7.54) and high PCR thresholds of signal intensity, indicative of a low concentration of target DNA (OR, 1.19 per cycle; 95% CI, 1.11 to 1.26). P+C− results were negatively associated with a history of MRSA infection or colonization (OR, 0.19; 95% CI, 0.09 to 0.42) and male sex (OR, 0.40; 95% CI, 0.20 to 0.81). P+C+ patients were significantly more likely to have subsequent positive MRSA culture assays and microbiological evidence of clinical MRSA infection. The risk of subsequent MRSA infection in P+C− patients was not significantly different from that in case-matched PCR-negative controls. We conclude that, given the low PPV and poor correlation between a PCR-positive assay and the clinical outcome, it would be prudent to await culture confirmation before altering infection control measures on the basis of a positive PCR result.
Methicillin (meticillin)-resistant Staphylococcus aureus (MRSA) is endemic in hospitals and health care facilities in most countries of the world (5). It is frequently carried into the community on colonized, discharged patients, forming a reservoir which then returns to the health care facility when asymptomatic carriers are readmitted (9). Admitted carriers may then develop endogenous infection or become sources of nosocomial transmission to other patients. Screening for MRSA carriage on admission has therefore become a component of many infection control programs (2, 8, 26, 27, 38, 42). This facilitates targeted treatment and infection control measures for MRSA-positive patients while avoiding unnecessary isolation and treatment of noncarriers.
Conventional culture-based screening may take several days to produce a result, and it is widely hypothesized that a reduced turnaround time would allow faster implementation of appropriate patient management. PCR-based MRSA screening assays have the potential to provide a result in 2 to 3 h of laboratory time and have a total turnaround time from specimen collection to ward report of approximately 20 h (1, 16, 21). The BD GeneOhm MRSA assay (previously known as IDI-MRSA; BD Diagnostics, San Diego, CA) and the Cepheid GeneXpert MRSA assay (Cepheid, Sunnyvale, CA) are two such PCR tests that detect the presence of characteristic MRSA DNA sequences bridging the SCCmec resistance cassette and the S. aureus-specific orfX open reading frame gene (18). However, concerns have been raised over false-positive and -negative results with these tests (11, 36).
Previous estimates of the sensitivities, specificities, negative predictive values (NPV), and positive predictive values (PPV) of the BD IDI-MRSA test and other PCR-based assays have varied considerably (for a summary, see Table S1 in the supplemental material). In our own cluster-randomized crossover study of adult general ward patients admitted during 2006 and 2007, the overall prevalence of MRSA carriage on admission was 6.7% (4.9% of assays) and, in comparison with parallel culture screenings, the BD IDI-MRSA PCR test had a sensitivity of 88%, a specificity of 96%, an NPV of 99%, and a PPV of 55%. (21). With low carriage rates, a high NPV is to be expected, but the relatively low PPV exacerbated concerns about high rates of false positivity with this test.
Our previous study did not demonstrate a difference in MRSA acquisition rates between admitted patients screened by PCR and by culture on general medical and surgical wards (21). Therefore, we now limit the use of PCR assays to the high-risk populations of patients in adult and pediatric critical care units and those admitted as surgical emergencies. In the present study, we investigate the performance of the BD GeneOhm MRSA PCR test for admission screening of these groups and consider its usefulness in guiding patient management.
Guy's and St. Thomas' NHS Foundation Trust is a London Teaching Hospital, comprising two adult hospitals with about 1,150 beds and the Evelina Children's Hospital with about 140 beds. Its adult critical care facilities are comprised of three general adult intensive care units, a specialized respiratory intensive care unit, high-dependency units for general medical and surgical patients, and a postoperative overnight intensive recovery unit (about 90 beds in total). Its pediatric critical care facilities are comprised of about 50 cots for general neonatal patients and 21 beds for pediatric intensive care or high-dependency patients. This investigation was conducted during the six months between 1 April and 30 September 2008. Data from patients who remained in the hospital on 30 September were collected until 31 October 2008.
Standard infection control precautions and training policies for MRSA reduction were in place throughout the hospital and were in accordance with United Kingdom national guidelines (8, 32). Patients with a history of MRSA within the past 18 months were barrier nursed on admission and isolated or nursed in cohorts. Known MRSA-positive patients received skin decolonization treatment with 2% chlorhexidine gluconate bathing cloths (Sage Products, Ltd., Cary, IL), and chlorhexidine powder was applied to the groin and axillae. Children aged 6 months to 12 years were decolonized with octenidine dihydrochloride in place of chlorhexidine, and infants under 6 months did not receive skin decontamination. Colonized wounds were treated with povidone-iodine or silver sulfadiazine where possible. The additional application of a 5-day course of intranasal mupirocin ointment was guided by susceptibility results.
Further precautions were in place in critical care for patients with unknown or negative MRSA status. Immediately after the collection of admission screening swabs, prophylactic treatment with chlorhexidine cloths and powder (octenidine for children) was initiated, and separate nursing cohorts were maintained for patients known to be MRSA positive or negative. When a positive PCR result was received, repeat culture screening swabs were taken and patients were treated with chlorhexidine ointment (Hibitane) to the anterior nares and (where applicable) by tracheostomy. Upon culture confirmation from either the first or repeat swabbing, the patient was classified as MRSA positive; the patient and family were informed of the result, and cohort nursing or side room isolation was initiated. If confirmatory culture results were negative, the patient was then classified as MRSA negative.
A policy of near-universal MRSA admission screening was introduced into the hospital on 1 April 2008. Culture-based MRSA screening was conducted in preadmission clinics or upon admission to the hospital. Patients admitted as surgical emergencies or as admissions or transfers to critical care units were screened for MRSA by PCR. In all settings, the anterior nares, throat, and perineum, plus any skin breaches or surgical wounds, were screened routinely with moist, sterile, rayon-tipped swabs in Amies transport medium without charcoal (Barloworld Scientific, Stone, United Kingdom). In critical care environments, screening swabs, which included an additional rectal swab, were taken from adult and pediatric patients prior to the first application of topical antiseptics upon admission to the unit (3). All critical care patients were rescreened weekly after the admission screening.
Upon arrival in the laboratory, all screening swabs except those from critical care or emergency surgery were cultured on selective chromogenic Brilliance MRSA agar (Oxoid, Basingstoke, United Kingdom) according to the manufacturer's instructions. Rectal swabs from critical care were processed in the same manner. Suspected MRSA colonies were confirmed by a variety of standard laboratory techniques, including latex agglutination, catalase testing, and antibiotic sensitivity profiling.
MRSA admission screening data from critical care patients and emergency surgical admissions were analyzed using a BD GeneOhm MRSA real-time PCR assay (BD Diagnostics, San Diego, CA), laboratory batch testing being performed three times daily on weekdays and once daily on weekends and public holidays. Swabs from the nose, throat, and perineum were placed in 0.3 ml of sample buffer each, and then suspensions were pooled into a single lysis tube. Suspensions from swabs of other clinical sites were placed in 1 ml of sample buffer and processed separately. Suspension processing and analyses were then performed according to the manufacturer's instructions. This multiple-site processing has been validated, providing a sensitivity of 88% and specificity of 96% compared to chromogenic culture (21). Pending the result of the PCR assay, swab heads were stored at room temperature. If the PCR result was positive, swab heads were subjected to overnight enrichment in salt broth 7.0% HPA (Oxoid) at 37°C. Broth cultures were then plated on Oxoid chromogenic agar and processed as described above. Specimens with an unresolved PCR assay were subjected to a freeze-thaw cycle according to the manufacturer's instructions and processed again using the same protocol. Specimens which remained unresolved after the repeat assay were reported as such and subjected to broth enrichment and culture as described above.
The following anonymized patient and sample data were collated from computerized records: age, sex, clinical specialty of the treating team, clinical environment, sample site, result, and turnaround time (from the generation of the request to the electronic reporting of the result to the ward). For each patient with one or more PCR-positive results, data were also collected from a corresponding patient with only negative PCR results and matched by age, sex, and clinical specialty. Isolation of MRSA or methicillin-susceptible S. aureus (MSSA) from any sample collected in the 18 months before and 1 month after the end of the study period was also recorded for PCR-positive patients and their matched controls. Dates of discharge were obtained from the hospital electronic patient record. The PCR cycle at which amplification of the target sequence reached a threshold of signal intensity (PCR Ct value) was collected for each specimen swab as a surrogate for the quantity of DNA with the PCR target sequence.
Overall agreement between PCR and culture assays was quantified using the kappa coefficient calculated with the GraphPad Software online application at http://www.graphpad.com/quickcalcs/kappa1.cfm. Factors possibly associated with discrepancies between PCR and culture results were analyzed using STATA statistical software. Univariate analysis was performed for each variable to obtain the odds ratio (OR) of a PCR-positive specimen being found culture negative, and 95% confidence intervals (CIs) and the P value of the OR significance were calculated. Variables with ORs attaining or approaching statistical significance (P value, ≤0.05) were identified and subjected to multivariate analysis using stepwise regression.
PCR Ct values for PCR-positive culture-positive (P+C+) and PCR-positive culture-negative (P+C−) assays were compared using the two-tailed Mann-Whitney test (because a Gaussian distribution of values could not be assumed), which was performed with Prism software (v.5.0.a).
Analysis of the time to culture confirmation of PCR assays or the isolation of MRSA from a clinically significant site was performed using Kaplan-Meier analysis, performed with Prism software. Patients were either scored for elapsed time between the first paired assay result and any screening culture confirmation (up to 75 days from swab collection) or censored at the elapsed time of discharge, death, or the end of the study period (up to 30 days after the end of the data collection period). For each PCR-positive patient (both culture positive and culture negative), a PCR-negative patient was case-matched from the database on the basis of age, sex, and clinical specialty. The Mantel-Cox log-rank test was used to compare each pair of Kaplan-Meier curves in turn; the level of significance was set at 0.05.
Over the period of this study, 3,294 PCR assays were performed, of which 43 were technically unresolved. The remaining 3,251 assays were obtained from 1,788 patients on 1,850 separate admissions to critical care (94% of specimens) or emergency surgery (6%). Overall, 248 (7.7%) PCR assays were positive; these were from 141 patients (7.9%) on 143 admissions (7.7%). Positive PCR results were obtained for 9.9% of adults tested and 2.3% of pediatric patients. The median turnaround time for both positive and negative PCR results was 16 h (interquartile range [IQR], 11 to 21 h).
Over the same period, across the rest of the hospital, 29,579 culture assays were performed with samples from 19,210 patients during 22,641 separate admissions. Growth of MRSA was obtained in 1,048 assays (3.54%), representing one or more positive results from 539 patients (2.81%) on 671 admissions (2.96%). The median turnaround time for culture assays was 35 h (IQR, 25 to 41 h).
Of the 248 positive PCR assays, 160 were confirmed by culture, giving a PPV for PCR of 65% (assuming culture is taken to be the best available indication of true MRSA positivity). Because PCR-negative specimens were not processed for culture, it is not possible to make a precise assessment of the NPV. However, a conservative estimate can be obtained from analysis restricted to patients tested by both assays during the same admission. Of the 1,386 patient admissions subjected to screening by both assays, 1,243 were PCR negative. Of these, 1,214 yielded only negative MRSA culture results during the same admission, giving an NPV of approximately 98%. Compared in this manner, the assays (positive and negative) are in agreement in 94% of cases, with a kappa coefficient of 0.68 (95% CI, 0.61 to 0.75), representing a substantial overall agreement between the assays.
Each PCR cycle approximately doubles the quantity of the target DNA sequence. There is therefore a logarithmic relationship between the quantity of target DNA and the point at which the threshold of signal intensity will be attained: the higher the quantity of target DNA in the specimen, the lower the Ct value. Figure Figure11 shows that specimens in the P+C+ group tended to yield lower Ct values than P+C− specimens, indicating a higher volume of target DNA among positive PCR specimens which go on to be confirmed by culture. This correlation reached statistical significance, with a P value of <0.0001. The Ct value of the PCR assay was incorporated, along with other relevant variables, into a multivariate analysis of the P+C+ and P+C− assay groups.
Variables which might contribute to the culture discrepancy between the P+C+ and P+C− groups were subjected initially to univariate analysis. This analysis revealed that a history of a previous positive MRSA culture, the use of chlorhexidine, and male sex were each associated with P+C+ results, and a history of a previous MSSA culture and a high Ct value (indicating a low concentration of the target DNA sequence) were each associated with P+C− results (Table (Table1).1). When these five variables were taken forward to multivariate analysis, all—apart from chlorhexidine use—remained independently significant, with MRSA history and male sex decreasing the risk of a discrepant result and MSSA history and high Ct value increasing the risk of discrepancy (Table (Table22).
The frequent discrepancies between the PCR and culture assays do not in themselves demonstrate the superiority of one technique over the other: such discrepant results must be understood in terms of the clinical outcomes they predict. If P+C− patients are truly carriers of MRSA with false-negative culture results (rather than noncarriers), we hypothesized that there would be no difference in MRSA infection/colonization outcomes from P+C+ patients. The P+C+ and P+C− groups therefore were compared for risk of future culture-confirmed MRSA status. Subsequent isolation of MRSA was compared for 82 P+C+ patients and 59 P+C− patients; 141 case-matched patients with PCR-negative (P−) results served as a control population.
P+C+ patients were significantly more likely to have repeated culture screening confirmation of carriage than P+C− patients, with a P value of <0.0001 (Fig. (Fig.2A).2A). Overall, 63 of the 82 P+C+ patients (77%) had subsequent culture positive screenings, compared with 11 of 59 P+C− patients (19%) and 8 of 141 P− matched control patients (6%). The P+C− curve for time to confirmation of carriage is statistically more similar to the P− curve than to the P+C+ curve (with P values of 0.0085 and <0.0001, respectively).
Figure Figure2B2B shows that there was also a highly significant difference in the risk of subsequent MRSA culture from a clinical site between P+C+ and P+C− patients (P value, <0.0001). In total, 34 of 82 P+C+ patients (41%) subsequently grew MRSA from clinical sites (nine samples identified from sputum, seven from bronchoalveolar lavage fluid, four from ulcer swabs, three from tracheostomy swabs, two from urine, two from line tips, one from blood culture, one from pleural fluid, one from a penile swab, and four from multiple sites), while this occurred in 5 of 59 P+C− patients (8%) (two samples identified from bronchoalveolar lavage fluid, two from wound swabs, and one from pleural fluid) and in 5 of 141 P− matched controls (4%) (three samples identified from wound swabs and two from tracheostomy swabs). There was no significant difference between the P+C− and P− curves (P value, 0.2611).
When analysis was restricted to patients with no prior history of MRSA—those whose management would be immediately influenced by the assay outcome—there was a highly significant difference between P+C+ and P+C− groups (20 of 49 versus 4 of 54, respectively; P value, <0.0001) (Fig. (Fig.2C)2C) but not between the P+C− and P− groups (P value, 0.3745).
To avoid any confounding due to differences in duration of stay, a subanalysis was performed to assess the risk of repeat MRSA positivity within 1 week of the initial PCR result. The risk of repeat MRSA culture within 7 days was significantly greater for the P+C+ than for the P+C− group (60% versus 15%, respectively; P value, <0.0001). Twenty-nine percent of P+C+ patients and 5% of P+C− patients grew MRSA from clinical sites within 7 days (P value = 0.0002), falling to 24% and 4%, respectively, when restricted to patients with no previous history of MRSA (P value, 0.0014).
This investigation reports on the first 6 months of using the BD GeneOhm MRSA PCR to screen critical care and emergency surgical admissions as part of a universal MRSA screening program. During a previous cluster control study of MRSA screening for patients on general wards, we noted a low PPV of the PCR assay compared to culture (21). This finding was replicated during the present investigation: more than a third (35%) of the positive PCR specimens failed to be confirmed by culture. This is a high rate of discrepancy, and further investigation into its validity and cause was required.
We considered whether this result reflected PCR false positivity, culture false negativity, or a combination of the two. In the absence of an accepted gold standard against which to compare the two screening methods, this question is not easily answered. It is necessary to consider the complex interplay of clinical and laboratory variables which influence the outcome of both PCR and culture and determine which provides the better guide to true MRSA status and future risk of MRSA infection.
There is another consideration, which pertains to any predictive test. For any assay with fixed sensitivity and specificity, as prevalence falls, the NPV rises and the PPV falls (24). Improvements in sensitivity and specificity will produce a squaring off of the curve, but a drop in prevalence necessarily produces a drop in PPV. Such a trend is evident from previously published reports of PCR assays in a range of settings: a fall in the endemicity of MRSA is associated with an exponential fall in the PCR assay's PPV (Fig. (Fig.3).3). Thus, our findings of a low PPV are in keeping with local and United Kingdom national trends of falling endemicity, attributed to enhanced infection control policies (http://www.hpa.org.uk/webw/HPAweb&HPAwebStandard/HPAweb_C/1233906819629?P=1191942169773). This decline in MRSA prevalence must be taken into account when assessing the utility of a universal PCR screening program, for it predictably leads to an increase in the number of PCR assays that will fail to be confirmed by culture.
In accepting that a low PPV is in part a by-product of falling prevalence, it remains important to recognize factors which predict or protect against discrepancies between PCR and culture, so that risk can be assessed for patients with positive PCR results while culture confirmation is awaited and so that discrepant results are correctly interpreted when they occur. In the present investigation, two approaches were undertaken to analyze the P+C− results: one assessing the group a priori, by identifying variables predicting a discrepant result though multivariate analysis, and the other post hoc, by linking results to clinical outcome through Kaplan-Meier analysis.
The multivariate analysis presented in Table Table22 provides some guidance in interpreting discrepant results, though it does not point to a single explanation. The significant association between P+C− results and a low DNA copy number (evident from a high Ct number) supports the hypothesis that PCR can detect MRSA at concentrations too low for reliable detection by the broth-enrichment culture which follows. Moreover, we have found that processing swabs for PCR produces a decrease in the number of organisms subsequently cultured (data not shown). This suggests that in cases where the concentration of MRSA is low, some P+C− results are due to insufficient sensitivity of culture rather than insufficient specificity of PCR. However, the lack of a significant association with chlorhexidine decolonization argues against the related hypothesis that discrepancies arise when organisms are no longer viable but their DNA is still present (10): P+C− results are no more likely among patients who have been treated with chlorhexidine at the time of swab collection than those who have not.
P+C− results were significantly less likely to occur in patients with a history of MRSA and more likely to occur in patients with a history of MSSA. While this investigation did not directly assess P+C− specimens for the presence of MSSA, such an assessment would be a valuable component of future studies of this and other molecular assays. Previous studies using the BD GeneOhm assay have isolated MSSA in up to 55% of P+C− specimens (13). Our findings add to previous reports that this PCR assay may give false-positive results by amplifying DNA from residual SCCmec target sequences in MSSA strains (11, 13, 36). Such strains may arise from MRSA strains that have lost the mecA resistance gene and become methicillin susceptible but retain the region of the SCCmec sequence to which the PCR primers bind (18, 25, 37). The propensity of the BD IDI-MRSA PCR assay to detect certain strains of MSSA is established in vivo and in vitro and remains an important caveat to the interpretation of the assay (11, 14, 18).
Having concluded that both false-positive PCR and false-negative culture results may contribute to discrepancies between the assays, we evaluated how P+C− results relate to future clinical risk. We hypothesized that if PCR is able to detect clinically significant MRSA colonization missed by culture-based screening, the risk of subsequent positive MRSA culture results (carriage or infection) should be similar between P+C− and P+C+ patients. This was not the case. MRSA was significantly less likely to be cultured from patients with an initial P+C− result than from those with a P+C+ result. Moreover, the disparity in outcome remained highly significant when we discarded those patients with a history of MRSA (a group for which such a result had no immediate consequences for clinical management). Thus, a patient with a P+C− result does not have the same risk of subsequent MRSA colonization or infection as one with a culture-positive screening. This is an important observation, since among patients with no history of MRSA, more than half of the positive PCR results (52%) failed to be confirmed by culture.
Thus, in a setting of low MRSA endemicity, a negative PCR result is a reliable predictor of true MRSA status and future risk, but a positive result is not. In this setting, the pragmatic stance is to wait for culture confirmation of a positive PCR result before modifying patient management. Infection control measures that carry no risk to the patient, such as hand hygiene and barrier nursing, must be universal anyway and do not hinge on the PCR result. Measures that subject patients to added risk, such as cohort nursing with confirmed MRSA carriers, delaying essential procedures while implementing eradication protocols, or labeling patients wrongly as MRSA positive (which may be alarming and stigmatizing), are not justified by the low PPV before the culture result is known. In contrast, the high NPV of the assay allows the clinical team to act with confidence when a negative PCR result is obtained, scaling down unnecessary infection control policies, ensuring appropriate selection of antimicrobial agents, and aiding the patient's mobility between clinical environments.
At the current level of MRSA endemicity, the decision of whether to continue use of PCR-based screening is finely balanced. The consensus view is that its high NPV, rapid turnaround time, and ability to inform optimal bed usage in critical care outweigh the uncertainty of its low PPV and its higher cost, although this decision remains under review as local clinical circumstances change. Clinical decisions which hinge upon a negative result (such as bed placement upon discharge from critical care) are made on the basis of the PCR result alone, whereas decisions based on a positive result take the low PPV into consideration. Some interventions, such as choice of empirical gram-positive antimicrobial therapy, can be based upon an interim PCR-positive result; others, such as placement in an MRSA-positive nursing cohort, are introduced only if culture confirmation is obtained.
In conclusion, PCR screening tests for MRSA that target SCCmec sequences may give false-positive results. Furthermore, the PPV of any screening test declines as the prevalence of MRSA carriage falls. The rapid PCR assay is, therefore, of greatest value when a negative result is obtained. A positive result must be interpreted with caution, since if culture confirmation is not obtained, the risk of subsequent MRSA infection is not significantly greater than that of a negative PCR result. Both clinicians and laboratory directors should be aware of the limitations of this test in its present form before adopting it for routine practice.
Financial support was provided by Guy's & St. Thomas' Charity (J.D.E.) and the Department of Health via an NIHR comprehensive Biomedical Research Centre award to Guy's and St. Thomas' NHS Foundation Trust in partnership with King's College London (J.D.E.).
Published ahead of print on 21 October 2009.
†Supplemental material for this article may be found at http://jcm.asm.org/.