|Home | About | Journals | Submit | Contact Us | Français|
In an effort to improve the reliability and reproducibility of serological assays for Bordetella pertussis, a collaborative study was conducted to compare four different sources of pertussis toxin (PT) as coating antigens in the immunoglobulin G (IgG) anti-PT enzyme-linked immunosorbent assay (ELISA). Four sources of PT were used as coating antigens in the IgG anti-PT ELISA in four different testing laboratories (labs A to D) to determine whether the different antigen preparations and different laboratories influenced assay results. A panel of 60 sera consisting of deidentified human specimens from previous vaccination trials of healthy adults and infants and clinical specimens from outbreak settings was tested. In the four laboratories, each sample was tested three times with the four PT antigens according to the standard coating optimization and IgG anti-PT ELISA testing procedures used in that laboratory. Differences among the antigens, as well as intra- and interlaboratory variability, were evaluated. Excellent agreement was observed with the test panel results among the four antigens within each laboratory. Concordance correlation coefficient (rc) measurements among the different antigens ranged from 0.99, 0.99 to 1.00, 1.00, and 0.97 to 1.00 for labs A to D, respectively. The comparisons between pairs of laboratories also indicated a high degree of concordance for each PT preparation, with rc measurements between 0.90 and 0.98, 0.93 and 0.99, 0.92 and 0.98, and 0.93 and 0.99 for antigens 1 to 4, respectively. Relatively minor differences in results were observed among laboratories or among antigens, suggesting that the four PT antigens are quite similar and could be considered for acceptance in harmonized immunoassays used for serodiagnosis or vaccine evaluation.
Despite widespread immunization programs, pertussis remains a significant public health problem. Cases are observed in all age groups, including infants, children, adolescents, and adults (4, 5, 11, 26). Immunoassays that measure the antibody response to pertussis antigens have played a major role in the evaluation of the immunologic responses to immunization with whole-cell and acellular pertussis vaccines and continue to serve an important role in diagnosis of pertussis cases and in epidemiological investigations (1, 3, 6, 9, 24, 27, 28). The measurement of immunoglobulin G (IgG) antibodies to pertussis toxin (PT) are particularly important because all acellular pertussis vaccines contain inactivated PT and because the quantitation of IgG anti-PT antibodies has been found to be the most useful assay for serodiagnosis (2, 10, 13, 15, 19, 23). As a result, numerous laboratories have developed IgG anti-PT enzyme-linked immunosorbent assays (ELISAs) (7, 9, 10, 17, 20, 21). Despite their importance, optimal standardization of these assays has not been achieved. This study was designed to obtain data to advance these efforts.
An international meeting was held in 2007 to discuss the current status of harmonization of pertussis immunoassays and to prioritize the activities required to reach this goal (25). Among the priorities were the development of reference materials and evaluation of criteria for acceptance of these reagents. The use of a common reference serum was considered a high priority, and the recent approval of a World Health Organization (WHO) international reference pertussis antiserum was a major step forward in the harmonization efforts (30). The antigens used for coating in immunoassays were identified as critical reagents, and one outstanding question for the IgG anti-PT assay was whether the source of the PT would influence results. In previously conducted collaborative studies (13, 14, 17, 25, 29), each participating laboratory used their own in-house procedures and reagents. These studies suggested that the source of PT coating antigen did not have a major influence on the results. However, none of the studies was designed to allow the sharing of PT antigens among the laboratories and to evaluate the impact of the source of the PT antigen on the assay results.
To address these outstanding questions, four different sources of PT antigens were compared in four different laboratories with prior experience in performing IgG anti-PT ELISAs. This design allowed both an intra- and interlaboratory comparison. Except for the PT coating antigen and the necessity for each laboratory to calibrate their assay to the WHO International Standard (IS) or the U.S. Reference Pertussis Antiserum (human), lot 3 (CBER3; Center for Biologics Research and Review, U.S. Federal Drug Administration, Rockville, MD) and report results in international units (IU), all four laboratories followed their own established procedures and used in-house qualified reagents. The panel of sera for assay standardization included subjects with different ages and vaccination status.
(Part of this research was presented at the 9th International Bordetella Symposium, Baltimore, MD, 30 September to 3 October 2010.)
Four laboratories participated in the study: the Centers for Disease Control and Prevention (CDC), Vanderbilt University Medical Center (VU), Sanofi Pasteur (SP), and GlaxoSmithKline Biologicals (GSK). Laboratories were randomly designated A to D to ensure anonymity of results. A fifth laboratory, Harbor-UCLA Medical Center (UCLA), provided serum samples but did not perform assays. The CDC coordinated testing at the four laboratories and distributed the antigens and panels of sera.
Two of the PT antigen preparations were provided by study participants (SP and GSK), and two preparations were obtained from commercial sources, List Biological Laboratories Inc. (List) (Campbell, CA) and Protein Express, Inc. (PE) (Cincinnati, OH). All antigen preparations, randomly assigned here as antigens 1 to 4, met in-house requirements for purity and storage (Table 1). The antigens were handled per the manufacturer specifications. Purified PT antigens from List, PE, GSK, and SP were available in the following concentrations: 200 μg/ml, 200 μg/ml, 95.5 μg/ml, and 392 μg/ml, respectively.
Collection of human sera for these assays was approved by the appropriate institutional review board (IRB) committees. Specimens were selected to span the expected working ranges of the assays (negative to high). CDC provided 18 clinical sera from suspected cases or outbreak settings, VU supplied 15 serum samples from adults immunized with TdaP by SP, GSK supplied 15 sera from infants immunized with GSK vaccines (DTap and/or DTap IPV), and UCLA provided 10 sera from infants and toddlers immunized with the GSK DTap-HB-IPV vaccine. The primary analyses included results from all 60 subjects; however, additional analyses evaluated subgroups as outlined below. The 60 subjects were divided into two subgroups, 40 subjects who were recently immunized versus 20 who were not recently immunized. The 40 recently vaccinated subjects were further subdivided in two ways; first, by the vaccine received and second, by their age at immunization. When divided by vaccine, there were 25 subjects who had been immunized with the GSK vaccine and 15 immunized with the SP vaccine. When subdivided by age, there were 25 infants/toddlers and 15 adults.
The WHO international human pertussis reference serum lot 06/140 (IS; National Institute of Biological Standards and Control, Potters Bar, United Kingdom) was included in the testing serum panel both undiluted and diluted 1:2 in antibody-negative sera. The IS and CBER3 were provided to each testing laboratory for calibration and control. Laboratories were encouraged to include the IS and CBER3 as test samples at least once per day throughout the study to monitor the calibration.
The study was conducted in two stages. In the first phase, each laboratory was asked to determine the optimal coating concentration for each of the four PT preparations using their routine procedures. In the second phase, each laboratory was requested to test the reference standards (IS and CBER3) and a panel of 60 sera according to the approved protocol. The study protocol was defined and approved by all participants prior to study initiation.
Labs A and D tested PT coating concentrations in 2-fold dilution steps and selected the concentration that was at or near the upper plateau of the response curve, namely, 1 μg/ml and 2 μg/ml, respectively, for each of the four PT preparations (PT antigen 1 [PT1] to PT antigen 4 [PT4]). Lab B tested PT coating concentrations in 2-fold dilution steps and selected the concentration with optimal signal-to-noise ratio, specifically 6, 4, 9, and 7 μg/ml for PT1 to PT4, respectively. Lab C selected the concentration that resulted in curves that most closely matched those for an in-house reference PT preparation; 1.29 μg/ml was selected for each of the four PT preparations (Table 1). For each PT preparation, the study requested three valid, independent assays per test serum. Each laboratory followed their standard operating procedure for generating a reportable value and reported only valid values. Each serum sample was tested against all four antigens at the same time. Thus, each of the 60 sera was tested three times (3 independent assays) against each of the four antigens for a total of 720 reportable values per laboratory.
Each laboratory performed the ELISAs according to the procedures established in their setting. All assays had the following common features. (i) PT was coated directly onto the wells of the plate. (ii) One or more dilutions of human serum were added to the coated wells. (iii) The PT-specific IgG antibodies were detected using enzyme-conjugated anti-human IgG. (iv) The absorbance was measured at the appropriate wavelength following the addition of substrate. (v) Antibody concentrations for test samples were measured using a standard curve tested on each plate. For determining assay validity, each laboratory used their established criteria. The VU ELISA quantitated PT-specific IgG response by a previously described procedure (2, 20). The CDC ELISA was performed following their published protocol with minor modifications (21). ELISA procedures for the four laboratories are summarized in Table 2.
Microtiter plates were coated with optimized concentrations of pertussis antigens diluted in a carbonate-bicarbonate coating buffer (pH 9.6). After coating, the plates were washed and blocked with buffer containing 1.0% goat serum. Samples were prepared as eight 2-fold serial dilutions, added to the plates, and incubated. A goat anti-human (IgG) horseradish peroxidase conjugate was added, the plates were incubated, 3,3′,5,5′-tetramethylbenzidine (TMB) peroxidase substrate was added, and then the reaction was stopped by the addition of 2 N H2SO4. Absorbance was measured at 450 nm using 540 nm as the reference wavelength. Parallel line analysis was used to determine sample titers by comparison to the reference standard.
Microtiter plates were coated at the optimized concentration in carbonate-bicarbonate buffer, pH 9.6. The coated wells on the plates were washed with wash solution (0.9% NaCl containing 0.05% Tween 20) and blocked with blocking solution containing 1% bovine serum albumin (BSA). The samples were prepared as eight 2-fold serial dilutions and added to the plates and incubated. Conjugate was added to each well, the plates were incubated, and substrate was added. Absorbance was measured spectrophotometrically at 490 nm using 620 nm as the reference wavelength. Concentrations were quantitated using the PT IgG SoftMax Pro software.
All valid reportable values were returned to the CDC for analysis. Values below the lower limit of quantitation (LOQ) were assigned a value of one-half the lower limit of the assay, and values above the upper LOQ were assigned a value of two times the upper limit of the assay. All assay values were log transformed prior to analysis. The sera assayed in this study do not have defined concentrations, so “consensus” values were estimated using an analysis of variance (ANOVA) mixed-effect model from the present data and used to quantify reproducibility (between- or interlaboratory variation), repeatability (within- or intralaboratory variation), and bias among the four laboratories. Bias is a measure of the directional error (consistent offset) of the laboratory concentration compared to the consensus concentration.
Scatter plots with lines of identity and Deming regression lines (8) were generated; the Pearson correlation coefficients (r), coefficients of accuracy (Ca), and concordance correlation coefficients (rc) were calculated to determine the agreement between laboratory and PT antigen assay results. Accuracy describes how close the Deming regression line is to the line of identity (intercept of 0 and slope of 1) and is measured using Lin's Ca (16). Precision measures how far a set of observations deviates from a straight line and is quantified using Pearson's r. Lin's rc, which is a combination of Ca and r, was employed to form a single statistic describing both accuracy and precision (16). Linear mixed-effect ANOVA models were used to estimate consensus values for each serum and PT antigen. The relative difference between the consensus value and the actual value reported by the laboratory was then calculated and plotted. All models were fit independently by PT antigen and included the sample, laboratory, and the laboratory-sample interaction as random effects. Since each serum sample was run in triplicate, a single predicted concentration was estimated using the ANOVA models to represent the replicate values for analysis and comparison of laboratories in the figures and tables.
Box plots were created to provide a more complete picture of the agreement and precision among the antigens within the individual laboratories and among the four different laboratories for each of the antigens. The size of the box coupled with the extensions of the vertical lines above and below the box is a direct indicator of the intra- or within-laboratory variability of the relative differences (repeatability). Small boxes centered about the gray dotted line with vertical lines extending between 1/2 and 2 indicate a distribution where the laboratory-reported concentrations were within 1/2 to twice compared to the consensus value. The positioning of the boxes about the gray dotted line is an indicator of the between-laboratory or between antigen variability (reproducibility).
Each of the four testing laboratories provided three valid results for each of the 60 serum samples, as well as the available results for IS and CBER3 when tested in parallel with the test samples. Thus, there were 720 valid results for the panel of sera, 131 values for IS, and 188 values for CBER3. The results of each PT antigen were compared among pairs of laboratories (Fig. 1 and Table 3), and subsequently, the results of each laboratory were compared among pairs of PT preparations (Fig. 2 and Table 4). The data were plotted using scatter plots (Fig. 1 and and2).2). rc, which takes into account both accuracy and precision, was used as the primary measure of agreement. Our analysis demonstrated excellent concordance among the four antigens and the four laboratories; each compared pair of laboratories displayed a high degree of concordance for each PT preparation, with rc measurements between 0.90 and 0.98, 0.93 and 0.99, 0.92 and 0.98, and 0.93 and 0.99 for PT antigens 1 to 4, respectively (Table 3). Similarly, pairs of antigens displayed a high degree of concordance for each laboratory with rc values ranged from 0.99, 0.99 to 1.00, 1.00, and 0.97 to 1.00, for labs A to D, respectively (Table 4).
Box plots are shown in Fig. 3 and and4.4. As illustrated in Fig. 3, for labs A, B, and C, the four PT preparations gave nearly identical results showing almost no mean bias, while for lab D, the results for PT1 and PT2 showed positive mean bias (mean values of 0.184 and 0.094, respectively) and were systematically greater than PT3 and PT4 with negative mean bias (mean values of −0.15 and −0.12, respectively). However, the magnitude of the differences was quite small. Ninety percent of the relative differences were less than 0.5 compared to the consensus values. For each laboratory, the four PTs were similar with respect to within-lab variability. Figure 4 illustrates the differences among the laboratories for each of the PT preparations. Labs B and D exhibited uniformly positive mean bias for all antigens, and labs A and C exhibited uniformly negative mean bias for all antigens. Nevertheless, the magnitude of the differences was rather small. The mean relative differences did not differ by more than 0.20.
As shown in the above analyses, results for all 60 test panel samples were similar regardless of the PT antigen used. Further analyses were conducted to evaluate whether the PT source may have biased the results for any of the subgroups of the sera. For these analyses, the 60 test samples were subdivided into various groups based on subject age, immunization status, and manufacturer of vaccine received as described in Materials and Methods. For all of these subgroups, the potential influence of the PT source was evaluated by determining for all PT pairs the mean and 95% confidence interval (95% CI) for the ratio of the results for that pair of PT preparations. For none of these subgroups and for none of the PT pairs was the ratio significantly different than 1.0, suggesting that the results were not biased by the PT antigen used to coat the wells on the plate (data not shown).
Each of the four laboratories performed testing on the two commonly used reference sera, CBER3 and the IS. Four laboratories included CBER3 and three laboratories included the IS in at least one assay per day (Fig. 5 and and6).6). In addition, the IS, undiluted and diluted 1:2 in negative serum, was included among the test samples. Plotted is the mean and 95% CI for the ratio of the measured value to the expected value based on the assigned amounts in units of the reference preparation. A ratio of 1.0 indicates that the measured value was equal to the expected value. The ratio is considered significantly different than 1.0 when the 95% CI does not include 1.0. For CBER3 (Fig. 5), some of the ratios are significantly different than 1.0; however, the differences appear relatively small, as the means and the extremes of the 95% CIs are generally contained within an interval ranging from a ratio of 0.80 to 1.25. Overall, there do not appear to be any consistent differences among the PT preparations for any of the laboratories. For the IS samples, the ratios were most often above 1.0, indicating that the measured values most commonly exceeded the assigned values (Fig. 6). Furthermore, the ratio was greater than 1.0 for all cases in which the ratio was significantly different than 1.0. Although some of the ratios are significantly greater than 1.0 for labs C and D, the differences are relatively small, as illustrated by the fact that the extremes for all except two of the CIs are contained in the band from 0.67 to 1.5. As with CBER3, there did not appear to be any consistent trends among the four PT preparations.
Immunoassays to detect antibody to B. pertussis antigens, especially PT, are used by many laboratories worldwide either for diagnosis or to evaluate immune responses to vaccines (1, 10, 12, 14, 17, 19, 21, 23, 28, 29). Thus, accurate and precise assays are needed. Consequently, this study was designed to focus on the potential influence of the source of PT as an important parameter for obtaining comparable results in different laboratories. The results presented here indicate that the four evaluated PT antigens showed very similar behaviors in the assay systems used by the participating laboratories. This is, to our knowledge, the first interlaboratory study of pertussis immunoassays carried out thus far in which each of the participating laboratories directly compared the antigenic behavior performance of PT antigens from different manufacturers. In addition to allowing a comparison of the coating performance of the four preparations, this design allowed the comparison of the four PT preparations among four laboratories.
The optimal coating concentration varied among the laboratories. Three of the four laboratories selected the same coating concentration for the four PTs. These laboratories used concentrations from 1 to 2 μg/ml. The fourth laboratory selected 4 to 9 μg/ml for the four PTs. Thus, there was some heterogeneity in the selected coating concentrations; however, these differences appeared to be minor and method specific. Variables that may have impacted the selection of coating concentration include general difference in the assay methodologies, choice of microtiter plates, buffer compositions, coating conditions (Table 2), and the method used to determine the optimal coating concentration (29).
When each of the four PTs was compared within a single laboratory, excellent agreement among the four PT preparations was observed. For the four laboratories, rc values between the pairs of antigens ranged from 0.99, 0.99 to 1.00, 1.00, and 0.97 to 1.00. Because the coefficient takes into account both variability and quantitative agreement, such high values of rc indicate very similar performances between the four PT preparations. This consistency is also illustrated in the box plot (Fig. 3). The largest difference observed was between PT1 and PT3 in lab D. For this comparison, the median value was 0.14 for PT1 and −0.13 for PT3.
The comparisons between pairs of laboratories also displayed a high degree of agreement for each PT preparation, with rc measurements between 0.90 and 0.98, 0.93 and 0.99, 0.92 and 0.98, and 0.93 and 0.99 for the four antigens. By using the same PTs in each laboratory, we were able to eliminate one potential source of variability (i.e., PT) and compare the precision and comparability among laboratories. The small, but consistent, systematic differences among laboratories are also illustrated in the box plot (Fig. 4). The basis for these systematic differences are unknown, but may be related in part to the calibration of in-house standard relative to the IS. More importantly, despite numerous differences in procedures and reagents, good agreement was obtained when laboratories used the same PT as coating antigen and CBER3 or IS for calibration. Together, these concordance analyses suggest that the PT source was not the primary contributor to interlaboratory variability and that any of these four PTs could be considered for a harmonized assay.
By including sera from a variety of sources, this study was able to obtain data relevant to an additional outstanding question, namely, whether the PT source could lead to unintentional bias. To address this, samples for this study included sera from subjects immunized with inactivated PT produced by GSK, sera from subjects immunized with inactivated PT produced by SP, and sera from individuals infected naturally by strains circulating in the population. Additionally, the panel included sera from infants, toddlers, adolescents, and adults. Analyses were done for the various subgroups; in no case was a bias observed. Specifically, no bias was observed when the coating antigen matched the immunizing antigen. Similarly, the four PTs showed very similar behavior performances when the IgG anti-PT antibody levels in recently infected individuals were measured.
Each laboratory evaluated the CBER3 and IS within the collaborative study (Fig. 5 and and6).6). CBER3 was included by all laboratories as a calibrator or control sample, and all laboratories obtained values that were, in general, within 25% of the assigned value. The IS was included both as a control sample and as a blind test sample. The measured values in many cases were greater than the assigned values; however, the extent of the overestimation varied among the laboratories. The highest estimates were obtained by lab B, which was approximately 50% above the assigned value. With respect to the primary purpose of the study, the four PT preparations yielded very similar estimates within each of the laboratories; however, because the units were assigned to the IS such that the unitages of IS and CBER3 are similar (30), the results of our study suggest that there is some heterogeneity among laboratories in the unitage of IS compared to CBER3.
The data reported here are consistent with data reported from previous collaborative studies (13, 14, 17, 25). For example, the results from the collaborative study with 32 participating laboratories (17) suggested that, despite widely different methods and reagents, different laboratories were able to obtain reasonable agreement when a common primary reference was used for calibration. Similarly, there was generally good agreement among 22 participating laboratories in the assignment of unitage of the IS (30). The current study expands on these previous studies in two important ways. First, this study design allowed the direct comparisons of PT antigens within the same laboratory and of laboratories with the same antigen. Second, the study evaluated defined subgroups of sera to evaluate whether the PT source could lead to unintentional bias.
Two limitations of this study should be noted. First, the study was not exhaustive in that it did not evaluate PT from all available sources. This study was designed to include commonly used PT preparations, including two commercially available sources, but was not designed to investigate the impact, if any, of the minor amino acid differences that have been observed among some recently isolated strains (22). The PTs were from three different laboratory strains (BP165, Tohama I, and 10536), but the study did not include PTs representing all strains of B. pertussis currently in circulation. However, recent clinical sera from individuals infected with strains currently in circulation were evaluated. Second, due to reagent limitations, there was a limit to the number of laboratories that could participate. Thus, the study was designed to include four experienced laboratories with well-established methods. All laboratories used in-house qualified reagents, but the procedures varied with respect to the number of sample dilutions, calculation methods, detection reagents, and general methodology.
The reproducibility of results obtained here suggests that future efforts to obtain comparable data may be able to focus on assay harmonization rather than the development of a fully standardized assay that uses a common protocol and shared reagents. Assay harmonization efforts are more focused on the identification of critical parameters that affect assay performance, allowing laboratories to develop or adapt procedures and reagents that conform to those critical variables (18). Many laboratories have well-established pertussis ELISAs, and harmonization of these methods rather than formal standardization may be more feasible and practical. The data obtained here suggest that the harmonized approach requiring each laboratory to characterize and qualify reagents fully, verify methodology, and calibrate carefully to a common reference preparation may be sufficient. The recent availability of an international pertussis reference serum was a major step toward the harmonization of assays (30).
In conclusion, our present findings suggest that despite variation in the PT coating concentrations, excellent agreement among the four PTs within each laboratory and among the laboratories for each PT was observed. Therefore, we conclude that the four PT antigens are quite similar and could be considered for acceptance in harmonized immunoassays used for serodiagnosis or vaccine immunogenicity evaluation. The data from this study, as well as available sera and reagents, can serve as a resource for laboratories that are trying to generate data that can be compared to data from other laboratories.
We thank Chris Anna Mink and Swei-Ju Chang (UCLA) for providing the infant vaccinated sera and Sanofi Pasteur, Vanderbilt University Medical Center, and GlaxoSmithKline Biologicals for supplying clinical samples. We acknowledge GlaxoSmithKline Biologicals, Sanofi Pasteur, and Protein Express for supplying the PT preparations and Teresa Kenney for participating during the initial phase of the project.
Parts of this study were funded by a grant from the National Vaccine Program Office.
The findings and conclusions in this report are those of the author(s) of each section and do not necessarily represent the entire group of participants or the official position of the Centers for Disease Control and Prevention/Agency for Toxic Substances and Disease Registry. The findings and conclusions in this report have not been formally disseminated by the U.S. Food and Drug Administration and should not be construed to represent any agency determination or policy.
Published ahead of print 23 November 2011