|Home | About | Journals | Submit | Contact Us | Français|
Rapid Ebola virus (EBOV) detection is crucial for appropriate patient management and care. The performance of the FilmArray BioThreat-E test (v2.5) using whole-blood samples was evaluated in Sierra Leone and the United Kingdom and was compared with results generated by a real-time Ebola Zaire PCR reference method. Samples were tested in diagnostic laboratories upon availability, included successive samples from individual patients, and were heat treated to facilitate EBOV inactivation prior to PCR. The BioThreat-E test had a sensitivity of 84% (confidence interval [CI], 64% to 95%) and a specificity of 89% (CI, 73% to 97%) in Sierra Leone (n = 60; 44 patients) and a sensitivity of 75% (CI, 19% to 99%) and a specificity of 100% (CI, 97% to 100%) in the United Kingdom (n = 108; 70 patients) compared to the reference real-time PCR. Statistical analysis (Fisher's exact test) indicated there was no significant difference between the methods at the 99% confidence level in either country. In 9 discrepant results (5 real-time PCR positives and BioThreat-E test negatives and 4 real-time PCR negatives and BioThreat-E test positives), the majority (n = 8) were obtained from samples with an observed or probable low viral load. The FilmArray BioThreat-E test (v2.5) therefore provides an attractive option for laboratories (either in austere field settings or in countries with an advanced technological infrastructure) which do not routinely offer an EBOV diagnostic capability.
An Ebola virus (EBOV) outbreak has been ongoing in West Africa since December 2013 and was declared a Public Health Emergency of International Concern (PHEIC) by the World Health Organization (WHO) in August 2014 (1). Since the outbreak began, the designated United Kingdom EBOV testing laboratory, the Rare and Imported Pathogens Laboratory, Public Health England (PHE), has been testing 10 to 15 samples per week for EBOV, an approximate 10-fold increase over the normal testing frequency. These samples have been taken primarily from United Kingdom citizens with a history of recent travel to West Africa, and the vast majority of the samples have not been infected with EBOV.
The preferred sample for detecting EBOV, an enveloped negative-sense single-stranded RNA virus, is EDTA with blood, serum, or plasma, with the primary diagnostic technology being real-time PCR (2). EBOV is designated in the United Kingdom by the Advisory Committee on Dangerous Pathogens (ACDP) a hazard group 4 (HG4) pathogen that must be handled under containment level 4 (CL-4) conditions (biosafety level 4 [BSL-4] in other countries). As such, stringent laboratory infrastructure and containment procedures are required to handle viable EBOV material, and only a few laboratories in Europe and elsewhere are equipped to do so (3), though United Kingdom guidelines do allow primary clinical analyses of samples from individuals with possible EBOV infection to be carried out under less-stringent containment conditions (4).
The FilmArray PCR platform (Biofire Diagnostics, UT) integrates sample processing, nucleic acid extraction and purification, and a multiple PCR analysis in a single-use, disposable pouch (5). The FilmArray was developed to provide low-operative-burden, point-of-care (PoC), diagnostic screening capability within a health care setting. Results are available around an hour after a crude sample is added to the pouch. Evaluations of the FilmArray Biothreat Panel pouch (a multivalent pouch which contains an EBOV assay) in Sierra Leone (6) and the United States (7) have demonstrated its utility in the identification and treatment of Ebola-positive patients.
A BioThreat-E test (v2.5) pouch (here referred as the BioThreat-E test) was released in 2014 (8). The pouch detects EBOV only and has been optimized for detection of the current strain (Makona) in clinical sample types. This pouch was awarded an emergency use authorization (EUA) by the U.S. Food and Drug Administration (FDA) for the presumptive detection of EBOV in whole-blood or urine specimens (9) and a conditional, temporary derogation from regulation 39 of the regulations governing medical devices in 2002 (2) by the United Kingdom regulatory body, Medicines and Healthcare Products Regulatory Agency, permitting supply and use of a non-CE-marked device.
In this study, we evaluated the performance of the BioThreat-E test using whole-blood samples from patients presenting in both Sierra Leone and the United Kingdom. In both countries, a range of samples (including successive samples from individual patients) were tested by each method to generate data on the sensitivity (i.e., proportions of positive results from various stages of infection with high and low viral loads) and specificity (i.e., proportions of negative samples) of the BioThreat-E test. Samples were also heat treated to address concerns associated with handling and processing of specimens potentially containing an HG4 pathogen under conditions of lower levels of containment. Therefore, the test conditions were outside the parameters stated by the manufacturer. The overall aim was to provide further confidence, in addition to that supplied by the manufacturer's validation data and the emergency use orders, that the BioThreat-E test could be used to provide a safe, reliable, and sensitive diagnostic for EBOV detection in whole-blood samples by laboratories (both in austere field settings and in countries with an advanced technological infrastructure) which do not routinely offer an EBOV diagnostic capability.
This study was conducted at the PHE and the United Kingdom Ministry of Defense (MOD) laboratory at the Kerry Town Ebola Treatment Centre in the Western Area Urban District of Sierra Leone and at the PHE laboratories in the United Kingdom. This evaluation was performed as a field evaluation in the context of an ongoing emergency. Consequently, operational and logistical requirements in both the United Kingdom and Sierra Leone considerably limited the scope of comparative testing that could be employed during the evaluation period.
Whole-blood samples (taken by venipuncture and collected into EDTA tubes) submitted for Ebola testing were tested on receipt by a method (RNA extraction from plasma and real-time PCR) validated for routine use in PHE laboratories in Sierra Leone and the United Kingdom and were then stored at 4°C as whole-blood samples. These samples were then tested by BioThreat-E test within 0 to 6 days of the diagnostic real-time PCR. The samples were selected and tested according to availability rather than through the application of specific selection criteria and included successive samples from the same patients.
The United Kingdom study was conducted under Royal College of Pathology guidelines (10) for the in-service development of diagnostic capability (i.e., performance assessment), and, as such, ethical approval was not sought from individual organizational ethics committees. The Sierra Leone Ethics and Scientific Review Committee was consulted but, after considering the parameters of the study and the United Kingdom guidance, determined that they did not need to provide approval.
In the Sierra Leone laboratory, and within a flexible film isolator, EDTA-blood samples were centrifuged to produce plasma. In a fresh 2-ml screw cap microtube, 80 μl of plasma was mixed with 320 μl of Buffer AVL (Qiagen); the tube surface was decontaminated by wiping with 5% chlorine solution, and the tube was left to stand for 10 min. The tube was removed from the isolator, and the sample was heated for 15 min to 60°C, the temperature required to inactivate EBOV in a blood sample mixed with Buffer AVL (11). RNA extraction was performed using the entire heat-treated 400-μl plasma/Buffer AVL sample, Qiagen EZ1 virus minikit v2.0, and a Qiagen EZ1 Advanced XL platform to generate a final eluted RNA extract volume of 60 μl. Whole MS2 bacteriophage was incorporated during the extraction procedure as an internal and inhibition control.
Real-time PCR comprised the Ebola Zaire minor groove binder (Ebola Zaire-MGB) PCR (12) multiplexed with an MS2 control PCR (13). A total reaction volume of 25 μl comprised 5 μl of template; Ebola Zaire-MGB primers (0.9 μM F565 [TCTGACATGGATTACCACAAGATC] and 0.9 μM R6405 [GGATGACTCTTTGCCGAACAATC]); probe (0.25 μM P597S [6-carboxyfluorescein [FAM]–AGGTCTGTCCGTTCAA–MGB-NFQ]); MS2 PCR primers (0.08 μM MS2 F1 [TGGCACTACCCCTCTCCGTATTCACG]; 0.08 μM MS2 R1 [GTACGGGCGACCCCACGATGAC]); probe (0.16 μM MS2 [VIC-CACATCGATAGATCAAGGTGCCTACAAGC-QSY]); 6.25 μl TaqMan Fast Virus 1-Step master mix (Thermo Fisher Scientific); and water. Procedures using controls (positive; no template [H2O]; and negative extraction) were performed with each run. Positive results were recorded as a Cq value (corresponding to the number of the cycle during which fluorescence was first detected above the threshold during the PCR). Tests conducted in the United Kingdom were performed similarly, though initial sample inactivation occurred under BSL-3 standards.
Within the isolator, comparator whole-blood samples (200 μl) were added to a 2-ml microtube containing the contents of a FilmArray Sample Buffer ampoule (800 μl) which had previously been used to resuspend the protease vial also supplied with the BioThreat-E test pouch. The sample was mixed, the tube surface was decontaminated by wiping with 5% chlorine solution, and the sample was left to stand for 10 min. The tube was removed from the isolator and heated to 60°C for 15 min. The sample was then added to a hydrated BioThreat-E test per the manufacturer's instructions and run on a benchtop FilmArray. The blood/buffer suspension (300 μl) was drawn into a FilmArray pouch (5) indicating that around 60 μl of blood was processed and was tested by the BioThreat-E test. The FilmArray presents results in a qualitative Positive/Negative interpretation. No Cq data are available, although melt curve peaks are viewable by the operator. Internal control PCRs (RNA extraction and PCR) monitor the success of the FilmArray process. To maintain the comparison between the results from the two countries, United Kingdom FilmArray testing also included the pre-PCR heat treatment, though all work occurred under BSL-3 standards. Data from both methods and countries were analyzed using the R language and environment for statistical computing and graphics (14).
PCR results from 60 samples, obtained from 44 patients, are summarized in Table 1. EBOV was detected in 25 samples by conventional real-time PCR and in 25 samples by BioThreat-E test. Discrepant PCR results were returned from 8 samples: four real-time PCR positives and BioThreat-E test negatives and four real-time PCR negatives and BioThreat-E test positives. In a further five samples, the control PCRs in the BioThreat-E test pouch failed, and each run, including one which returned a EBOV real-time PCR positive, was declared invalid by the FilmArray software. These five samples were excluded from the statistical analysis. The respective distributions of the Sierra Leone BioThreat-E test PCR results against positive real-time Cq values are included in Table 2.
In Sierra Leone, the BioThreat-E test had a sensitivity of 84% (confidence interval [CI], 64% to 95%) and a specificity of 89% (CI, 73% to 97%) compared to the reference real-time PCR method. Statistical analysis (Fisher's exact test) indicated there was no significant difference between the methods at the 99% confidence level.
PCR results from 108 samples, obtained from 70 patients, are summarized in Table 1. In total, EBOV was detected in four samples by either method, with one discrepant result (real-time PCR positive and BioThreat-E test negative). The respective distributions of United Kingdom BioThreat-E test PCR results and positive real-time Cq values are included in Table 2. In the United Kingdom (n = 108; 70 patients), the BioThreat-E test had a sensitivity of 75% (CI, 19% to 99%) and a specificity of 100% (CI, 97% to 100%) compared to the reference real-time PCR. Statistical analysis (Fisher's exact test) indicated there was no significant difference between the methods at the 99% confidence level.
The West African Ebola outbreak has led the international community to deploy a number of Ebola diagnostic laboratories in the countries mainly affected. There has also been pressure to increase the number of laboratories in the United Kingdom able to provide EBOV diagnostic capability (for suspect patients with a history of recent travel to West Africa), thereby improving test turnaround times irrespective of where in the United Kingdom patients present for medical attention.
In West Africa, it has been impossible to rapidly create a BSL-4 laboratory infrastructure and to ensure that operative safety laboratories have used methods which rapidly inactivate EBOV prior to testing. A separate study (11) has indicated that a combination of Buffer AVL (containing a chaotropic guanidine salt) and heat is required to inactivate EBOV in blood samples—with individual Buffer AVL or heat treatments not inactivating EBOV in samples. In Sierra Leone, the EZ1 RNA extraction and SmartCycler PCR platforms were operated outside biological containment. Therefore, using a flexible film isolator, all plasma samples were prepared and inactivated with a combination of Buffer AVL and heat prior to RNA extraction and real-time PCR.
This heat treatment step was also included for whole-blood samples mixed with FilmArray sample buffer and is outside the parameters stated by the manufacturer and the FDA EUA. We have not experimentally evaluated the EBOV virucidal efficacy of FilmArray sample buffer (which, like Buffer AVL, contains a guanidine salt), with or without an additional heat treatment. Without any relevant experimental evidence, we included the heat treatment, as a previous study (11) indicated that a dual treatment (guanidine-based buffer and heat) was required to ensure EBOV inactivation. This provided confidence to allow operation of the Sierra Leone FilmArray outside containment.
Overall, there was good concordance between the results from BioThreat-E test and real-time EBOV PCR assays, particularly under conditions of use with high-load (low-Cq) samples (Table 2). The Sierra Leone sample set, with a range of positive samples, confirmed the diagnostic sensitivity of the BioThreat-E test assay, while the United Kingdom sample set, with a large number of true negatives, helped confirm the specificity of the BioThreat-E test assay.
In total, nine samples, all of which came from patients who were confirmed to be infected with EBOV on the basis of real-time PCR results and clinical findings, provided discrepant results. The results comprised 4 potential BioThreat-E test false positives and 5 false negatives in measuring the performance of the FilmArray against that of the reference PCR method. Timeline patient sample data are summarized in Fig. 1 to provide contextual information for each discrepancy.
Among the potentially false-positive results, one was from a patient sample (from Sierra Leone patient 6) whose repeat and next-day samples generated real-time PCR Cq values above 36. Another sample (from Sierra Leone patient 5) was returned 2 days after a previous sample had generated a real-time Cq result of 31.5. Two other samples (from Sierra Leone patients 2 and 7) tested positive by BioThreat-E test 10 and 14 days, respectively, after the initial samples had provided strong EBOV real-time PCR positives. Although it cannot be definitively proven that the last three results do not represent BioThreat-E test false-positive reactions, a previous study (7) testing patient samples also generated real-time PCR negatives (from plasma samples) but FilmArray PCR positives (from whole-blood samples). These were subsequently confirmed to be from patients whose viral load was waning (Cq of >36). Analytical sensitivity studies in the same paper produced positive (Cq = 37) and negative real-time PCR results at the EBOV titer of 4 × 101 50% tissue culture infective doses [TCID50] · ml−1. Therefore, the four discrepant BioThreat-E test-positive results discussed above are consistent with the presence of residual low EBOV concentrations in blood, as was also observed in another study (7).
To support this hypothesis, the four potential FilmArray false-negative results (from Sierra Leone patients 1 and 4 and United Kingdom patient 1) were generated from samples where corresponding real-time PCR values were high (>36), again indicating that these samples contained small amounts of target nucleic acid, but, in this case, a potential BioThreat-E test false-negative result occurred. Of note, the United Kingdom discrepant result (real-time Cq = 37.53) was generated in a sample where a repeat sample (taken at the same time but using a different EDTA blood tube) tested positive by FilmArray PCR (comparator Cq = 36.7).
The remaining discrepant result (a potential BioThreat-E test false negative) was returned from a patient (Sierra Leone patient 3) with a higher viral load (Cq = 29.1) than the other discrepant samples. It is unclear why the FilmArray failed to detect EBOV in this sample, though it is unlikely to have been due to a variation in the BioThreat-E test target sequence, as subsequent samples from the same patient were positive. This is potentially an inhibition effect, with whole blood, the FilmArray sample type, containing more PCR inhibitors (15) than the plasma sample type tested by real-time PCR. Indeed, plasma generated from the same sample and tested by BioThreat-E test did provide an EBOV-positive result (data not shown). Although FilmArray control PCRs performed on the discrepant sample were successful, the performance of these assays is measured by the system using only qualitative melt-curve data (5); therefore, an increase in the control PCR Cq value, which might indicate a general inhibition effect and therefore explain the EBOV negative, would not be detected by the system. Individual PCRs are known to be differentially affected by the same inhibitor (16), supporting the hypothesis of a control PCR success but EBOV PCR failure.
In general, although other potential variables within our test protocol (storage of samples prior to FilmArray testing; differential sample volumes processed by each method; differential viral loads in blood and plasma; laboratory error) could have affected the number and nature of the discrepant results, the majority appear to correlate with samples with observed, or probable, low concentrations of EBOV. At low concentrations, nucleic acids are distributed stochastically (17), and stochastic effects occur commonly in PCR tests, including FilmArray studies (18), when the copy load in the sample is low. This gives rise to apparent discrepancies, even when the same assay is used in duplicate. This approach requires careful interpretation and explanation by the laboratory to clinicians in the context of each case and the likely phase in the evolution of the disease.
While not formally assessed, a number of operational considerations were noted during the operation of the BioThreat-E test. We did not find evidence of cross-contamination between runs, even when high-viral-load positive and negative samples were run in succession. Several pouches also failed to hydrate properly (in both countries), though this event is readily noticed during setup (and does not result in the loss of sample), and the manufacturer has recently released a new version of the pouch-loading station in mitigation. In addition, the FilmArray, designed for low-throughput screening capability and able to test only one sample per run, may not be readily applicable as the primary diagnostic in an Ebola outbreak situation where high numbers of samples need to be rapidly tested. Integrated PCR platforms able to test multiple EBOV samples concurrently (19), or other rapid diagnostic technologies already evaluated in West Africa (20), may provide more-appropriate EBOV diagnostics in this scenario.
This study has indicated that FilmArray BioThreat-E test v2.5 offers performance comparable with that of a validated real-time PCR approach for Ebola diagnosis; therefore, a positive result can be treated as a presumptive diagnosis with confidence. Of equal importance is the confidence in a negative result, as part of a screening process, as this will have considerable significance for both patient care and wider public health responses in both epidemic and nonepidemic countries. The slight differences in results between the two PCR methods likely reflect the stochastic differences occurring at low levels of pathogen during the recovery and sampling stages rather than performance differences between the different assays.
In summary, we show that the FilmArray BioThreat-E test (v2.5) is sensitive and specific and, also considering the low logistic and operative burdens and speed to result, provides an attractive option for researchers in a low-throughput laboratory, without either BSL-4 trained staff or infrastructure, wishing to provide EBOV diagnosis capability. Such a laboratory either could have an existing containment infrastructure, therefore reducing the workload of a centralized reference laboratory, or, if using a method to rapidly inactivate EBOV prior to PCR, could be a field laboratory in an austere or remote environment.
We acknowledge and thank the volunteer and MOD staff in the laboratory in Kerrytown, Sierra Leone, for their contribution to the study execution. Similarly, we thank the staff in the United Kingdom for their role in diagnostic processing of patient specimens.
This study was funded by the United Kingdom Ministry of Defence and Public Health England. The Kerry Town, Sierra Leone, evaluation was performed in the Ebola Treatment Centre laboratory, funded by the United Kingdom Department for International Development.