|Home | About | Journals | Submit | Contact Us | Français|
Lot 89SF has been the reference standard serum pool used in pneumococcal enzyme-linked immunosorbent assays (ELISAs) since 1990. In 2005, it was estimated that there remained between 2 and 5 years' supply of lot 89SF. Since lot 89SF was the reference standard used in the evaluation of the seven-valent pneumococcal conjugate vaccine Prevnar (PCV7), the link to clinical efficacy would be severed if stocks became completely depleted. Furthermore, demonstration of immune responses comparable to those elicited by PCV7 is a licensure approach used for new pneumococcal conjugate vaccines, so a replacement reference standard was required. A total of 278 volunteers were immunized with the 23-valent unconjugated polysaccharide vaccine Pneumovax II, and a unit of blood was obtained twice within 120 days following immunization. Plasma was prepared, pooled, and confirmed to be free from hepatitis B virus (HBV), hepatitis C virus (HCV), and HIV. The pooled serum was poured at 6 ml per vial into 15,333 vials and lyophilized. Immunological bridging of 007sp to 89SF was used to establish equivalent reference values for 13 pneumococcal capsular serotypes (1, 3, 4, 5, 6A, 6B, 7F, 9V, 14, 18C, 19A, 19F and 23F) by five independent laboratories. Antibody concentrations in 007sp were established relative to the lot 89SF reference preparation using the WHO reference ELISA. Subsequently, 12 existing WHO calibration sera had concentrations reassigned for 13 pneumococcal serotypes using new serum 007sp as the reference, and these were compared to concentrations relative to the original reference serum. Agreement was excellent for the 12 WHO calibration sera. The 007sp preparation has replaced 89SF as the pneumococcal reference standard. Sufficient quantity of this new preparation is available such that, with judicious use, it should be available for at least 25 years.
Streptococcus pneumoniae human reference serum lot 89SF has been used as a standard serum in enzyme-linked immunosorbent assay (ELISA) designed to measure IgG-specific antibody for individual pneumococcal capsular polysaccharides since 1990. This standard serum greatly facilitated the standardization of ELISA methodologies during a critical period when the first polysaccharide-conjugate vaccines were being evaluated for licensure. Lot 89SF consists of pooled, defibrinated plasma collected in the late 1980s from 17 individuals vaccinated with a 23-valent pneumococcal polysaccharide vaccine (Pnu-imune; Lederle), a 4-valent meningococcal polysaccharide vaccine (Menomune; Connaught), and Haemophilus influenzae type b conjugate vaccine (ProHibit; Connaught). The final volume of this collection was approximately 10 liters (3 ml/vial). Serotype-specific weight-based values for IgG, IgA, and IgM were derived for serotypes 1, 3, 4, 5, 6B, 7F, 9V, 14, 18C, 19F, and 23F for lot 89SF by Quateart et al. (5). Assignments for the additional serotypes in the 23-valent pneumococcal polysaccharide vaccine were subsequently bridged from the assignments for the original 11 serotypes (6). The lot 89SF standard has been maintained and shipped to laboratories all over the world from the Center for Biologics and Evaluation Research (CBER), U.S. Food and Drug Administration (FDA), at a rate of approximately 150 vials per year. In 2005, it was estimated that only 2 to 5 years' supply of lot 89SF remained. Since lot 89SF was the reference standard used in the evaluation of the first pneumococcal conjugate vaccine to be licensed, a seven-valent conjugate (Prevnar; Pfizer PCV7), the link to clinical efficacy would be severed if stocks were depleted. Since the use of comparable immune responses to PCV7 is a licensure approach for new pneumococcal conjugate vaccines, retention of the link to the original efficacy trials for PCV7 is critical.
Two hundred seventy-eight volunteers were immunized with Pneumovax II, and blood was collected twice within 120 days following vaccination. Sera were tested, pooled, and aliquoted to establish a new Streptococcus pneumoniae human reference serum (007sp) to replace 89SF. Sera from 16 donors were processed separately and set aside to provide a new set of quality control sera specifically for use for opsonophagocytic assays (OPA). The remaining sera were pooled, aliquoted into 6-ml vials, and lyophilized for future ELISA standardization. This report describes the efforts undertaken by this working group to establish the serotype-specific IgG concentrations for the new reference serum, designated 007sp by the FDA, to validate its performance as a standard, and to reassign values to a set of 12 existing World Health Organization (WHO) calibration sera.
Following written informed consent, 427 individuals were evaluated at the University of Iowa, and 278 healthy male and nonpregnant female volunteers between 18 and 45 years of age met the eligibility requirements for this study (United States National Institutes of Health study protocol no. 06-0093). The study was approved by the University of Iowa institutional review board (IRB, Committee A). Volunteers were vaccinated once with Pneumovax II and returned 10 to 35 days following immunization and 8 to 12 weeks following their first blood donation to donate 1 U of blood. Blood samples were processed in two steps using a three-bag system. First, red blood cells were removed, and the plasma was transferred to a second bag, where it was allowed to coagulate. The sera were collected in the third bag and shipped to CBER, where they were stored at −20°C. Sera from 16 donors were set aside to be filled as new individual calibration sera (to be referred to as FDA OPA calibration sera). Serological and virological testing showed sera to be free from hepatitis B and C viruses (HBV and HCV, respectively) and HIV. Each of the vials was filled with 6 ml of serum, and the serum was lyophilized under contract by Thermo Fisher (Thermo Fisher Scientific, Waltham, MA).
An existing panel of WHO calibration sera established by D. Goldblatt (UCL Institute of Child Health) and distributed by the National Institute for Biological Standards and Control (NIBSC; Potters Bar, Hertfordshire, United Kingdom) was supplied for assessment of the performance of 007sp and for reassignment of concentrations to the calibration serum panel.
Five laboratories participated in the initial phase of the 007sp bridging exercise (University of Alabama, Huntsville; Pfizer Vaccine Research, Pearl River, NY; GlaxoSmithKline Biologicals, Rixensart, Belgium; Institute of Child Health, University College London, London, United Kingdom; and PPD Vaccines & Biologics Laboratory, on behalf of Merck Sharp & Dohme Corp., West Point, PA). A complete description of the different phases of the study is located in the reference materials section at http://www.vaccine.uab.edu. The first phase of the study was designed to assign serotype-specific IgG concentrations to 007sp for 13 serotypes (1, 3, 4, 5, 6A, 6B, 7F, 9V, 14, 18C, 19A, 19F, and 23F) by calibrating it against lot 89SF using an accepted pneumococcal reference ELISA (the “WHO ELISA”) (10). The ELISA protocol followed by participating laboratories can be found at http://www.vaccine.uab.edu/ELISA%20Protocol.pdf although four of the five laboratories incubated serum for between 4 and 18 h at 4°C rather than at room temperature. This protocol includes an absorption of cross-reactive antibodies to cell-wall polysaccharide (CPS) and a heterologous absorption using serotype 22F polysaccharide for the absorption of other common contaminants generated during type-specific polysaccharide purification (2). Four independent sets of serial dilutions of 007sp were made from four randomly selected separate vials. The four sets of eight serial dilutions were run in duplicate as unknown samples on each ELISA plate in a five-plate replicate series. This phase generated a total of 20 concentrations for 007sp from each of the participating laboratories. Each plate also contained seven serial dilutions of lot 89SF run in duplicate and an in-house quality control (QC) serum. This was carried out for each serotype. Raw optical density measurements from the five laboratories were sent to the Centers for Disease Control and Prevention, Atlanta, GA (CDC), and antibody concentrations were derived using the CDC program ELISA for Windows. Basic quality control of the data was carried out by biostatisticians at CDC, Atlanta, GA, and laboratories were requested to rerun selected assays that were deemed outside quality control specifications.
In the second phase of the study, a panel of 12 existing WHO QC sera was assayed and quantified using both 007sp and 89SF as reference standards. Three calibration samples, as well as 007sp and 89SF, were run in duplicate on each ELISA plate of a five-plate replicate series for each serotype, yielding five reported values for each serotype from each laboratory for final analyses. The performance of 007sp was assessed by comparing calculated concentrations using 007sp versus 89SF as the reference standard.
Statistical analysis was carried out at CDC. Through the two phases of this study and the selected repeated assays, there were at least 40 determinations of antibody concentrations for 007sp for each serotype from each laboratory. Antibody concentrations were estimated for the 13 serotypes using a linear mixed-effects analysis of variance (ANOVA) model. All models were fit independently by serotype and included laboratory and plate as random effects. Confidence intervals (95% CI) were estimated by serotype, accounting for the variance components between the laboratories, between plates within a laboratory, and sample variability nested within plates and laboratory. Data were analyzed after natural log transformation of ELISA IgG concentrations. The means of the log concentrations for each serotype were calculated for each laboratory and used to assess agreement and precision among the five laboratories. Agreement is defined as the closeness of the geometric means between two laboratories for each of the 13 serotypes and was measured using Lin's coefficient of accuracy (Ca) (3). Precision measures how far a set of observations deviates from a straight line and was quantified using Pearson's correlation coefficient (r). Lin's concordance correlation coefficient (rc), which is a combination of Ca and r, was employed to form a single statistic describing both agreement and precision.
For each serotype in 007sp, antibody concentrations estimated using ANOVA models adjusting for laboratory were obtained by back-transforming the estimated log-transformed concentration and associated 95% CI. These concentrations served as the “assigned” values for each serotype in 007sp.
Once values for the 13 serotypes of 007sp were finalized, concentrations were determined for the 12 WHO calibration sera. Through the two phases of the study, each laboratory contributed five estimates of antibody concentrations for each WHO calibration sera for each serotype. Twelve WHO calibration sera have existing ELISA concentrations or assignments based on 89SF (see http://www.vaccine.uab.edu). In this study, we used 007sp to derive analogous “consensus” ELISA concentration or assignment values using an ANOVA mixed-effects model for a revised panel of 12 WHO calibration sera. Some of the sera from the original collection are depleted and have been replaced to create a new panel for distribution.
Scatter plots and box plots were employed to assess and evaluate the ability of each of the five laboratories to produce consistent estimates of antibody concentrations for each serotype in 007sp.
Two hundred seventy-eight subjects were immunized, and 253 of these donated blood twice. Twenty-five subjects only donated once (between 10 and 35 days). A total of 15,333 vials were filled with 6.00 ml of serum, and the serum was lyophilized with a coefficient of variation (CV) of fill volume of <1% (gravimetric), moisture content of approximately 1%, and a bioburden below the level of detection. CBER confirmed the moisture content at 0.7% and the sterility of the vial contents.
Quality control of the data submitted for analysis revealed the data to be of high quality, and few were deemed outside QC specifications, leading to minimal reruns. The means of the log concentrations for each serotype were calculated for each laboratory and used to assess the level of agreement among the laboratories. Figure 1 displays scatter plots between all pairs of laboratories with the concordance correlation coefficient (CCC) listed within each plot. The solid diagonal line indicates perfect agreement (where slope=1 and intercept=0). These statistics indicate a high level of agreement, with rc exceeding 0.97 for all plots. For the same data, the Pearson correlation coefficient was ≥0.98, and the accuracy coefficent (Ca) was ≥0.99 in each case. Analysis of variance (ANOVA) models were used to estimate antibody concentrations for each of the serotypes in 007sp, adjusting for laboratory. Variance components necessary for the calculation of confidence intervals were calculated, adjusting for the replicate values within a plate and between plates within a laboratory. Final point estimates and confidence intervals were obtained by back-transforming the estimated log-transformed concentrations and associated 95% confidence intervals (95% CI). These concentrations served as the “assigned” values for each serotype in 007sp and appear in Table 1. These values were derived following double adsorption of 007sp with cell wall polysaccharide (CPS) and polysaccharide 22F, and thus in the future, when used as a standard, both standard and unknown sera should be double adsorbed. (Lot 89SF values were derived following single adsorption, and thus the standard and unknown sera are dealt with differently in the current ELISA protocol.) The values assigned to 007sp compared to the original values assigned to 89SF are shown in Fig. 2.
IgG ELISA concentrations were measured for a panel of 12 WHO calibration sera using both 89SF and 007sp as the reference standards. Table 2 presents the assigned values for the 12 WHO calibration sera (n=25 for each estimate), while Fig. 3 and and44 display the scatter plots and box plots for the first seven serotypes (1, 3, 4, 5, 6A, 6B, and 7F) analyzed, and Fig. 5 and and66 show the same information for the remaining six serotypes (9V, 14, 18C, 19A, 19F, and 23F). These plots illustrate the agreement and precision of the five estimated assigned values for 007sp compared to lot 89SF for each WHO calibration serum and serotype.
The scatter plots (Fig. 3 and and5)5) show the high degree of agreement and correlation among the calculated concentrations for the panel of 12 WHO calibration sera using 007sp (vertical scale) versus lot 89SF (horizontal scale) as reference standards. A perfect level of agreement would yield a straight line with slope of 1 and intercept at 0. With rare exception, all data points cluster tightly about this line of identity.
The box plots (Fig. 4 and and6)6) illustrate the deviation of the 007sp-based estimates from those obtained using lot 89SF as reference standard for the 12 WHO calibration sera. These plots offer more resolution than the scatter plots in that they relay more information regarding the deviation of the 007sp and lot 89SF estimates. In these plots, the box is defined by the 25th and 75th percentiles of the distribution; the horizontal line within the box represents the median or 50th percentile, and the asterisk signifies the mean. Vertical lines extend to the most extreme observation that is less than 1.5 times the interquartile range (75th to 25th percentiles), and the diamonds and boxes correspond to assay values which are progressively distant from the mean. The data above the dotted horizontal line indicate the 007sp-based estimates are greater than estimates using lot 89SF as reference standard. On the vertical axis, the number 2 indicates a point where the 007sp-based estimate was twice the lot 89SF-based estimate. A value of indicates the lot 89SF-based estimate was 4 times the 007sp-based estimate. Boxes centered on the horizontal dotted line indicate a good agreement between the 007sp-based and lot 89SF-based estimates. By and large, the concentrations calculated using 007sp as the reference standard are within 2-fold (1/2 to 2.0) of those calculated using lot 89SF as the reference standard. Notable exceptions include serotypes 6B and 19A for lab 5 and serotype 23F for lab 4.
Table 3 presents accuracy (Ca), precision (r), and concordance (rc) measures of agreement between pairs of laboratories and between laboratories and consensus ELISA concentrations for the WHO calibration sera. In order to form paired data between the labs for these comparisons, the five replicate concentration values were replaced by a single predicted value obtained from a mixed-model analysis of variance. There was an exceptionally high degree of agreement, with all values but one ≥0.90. In general, comparisons with lab 3 yielded the lowest measures of correlation and agreement. This is due to one concentration value for lab 3 (type 14) that is substantially greater than the values reported by the other four labs. If this value were removed, all statistics involving lab 3 improve. As an example, the rc between labs 3 and 5 of 0.882 improves to 0.947 when this single value is deleted from the analysis.
In general there is a high degree of agreement between the 007sp-based and lot 89SF-based estimates. This inspires confidence in the validity of the 007sp assignments.
Assignment of the weight-based antibody units to the human antipneumococcal standard reference serum lot 89SF currently in use was defined by Quateart and colleagues in 1995 (5). They used the Zollinger method (11) to quantitate the amount of total antibody as well as IgA, IgM, and IgG in the reference preparation. Briefly, an ELISA designed to capture immunoglobulin molecules in a known reference preparation was run side by side with a pneumococcal capsule-specific ELISA, and the signal obtained from the reference preparation capture of a known amount of IgG, IgA, and IgM, respectively, could be compared to the signal from the capsule-specific ELISA (the “ELISA quantitation method”). This permitted quantitation of the amounts of capsule-specific IgG, IgA, and IgM in lot 89S for 11 pneumococcal serotypes (1, 3, 4, 5, 6B, 7F, 9V, 14, 18C, 19F, and 23F). In 2004, this was extended through the assignment of weight-based antibody values for 13 additional pneumococcal polysaccharide serotypes (2, 6A, 8, 9N, 10A, 11A, 12F, 15B, 17F, 19A, 20, 22F, and 33F) and for the S. pneumoniae cell wall polysaccharide (CPS) (6) using a an enzyme-linked immunosorbent assay (ELISA)-based equivalence-of-absorbance method suggested by Concepcion and Frasch for cross-standardization (1). This method was also used to assign weight-based IgG1 and IgG2 units for serotypes 3, 6B, 14, 19F, and 23F in lot 89S (9) and serotypes 1, 4, 5, 7F, 9V, and 18C (8).
ANOVA mixed modeling is a flexible framework that allows estimation of ELISA concentrations for 007sp and the 12 WHO calibration sera for each serotype by laboratory. These models may be used to compare and contrast results within and among laboratories. Random-effects ANOVA models allowed us to reduce the replicate measurements to a single predicted value, which was then used to measure levels of accuracy and precision among the laboratories. While we were able to estimate serotype-specific concentrations for 007sp through a bridge to 89SF (Table 1), the actual ELISA concentrations for the WHO calibration sera used in this study were unknown, so it was not possible to compare experimentally the concentrations' “true” values. The ANOVA mixed model provided a mechanism for estimating consensus values, which served as assigned values for these sera (Table 2). This panel of sera can be used to calibrate pneumococcal ELISAs, as previously described (4); 75% of the calibration sera should have a percent error of 40% or less compared to the assigned point estimate (see http://www.vaccine.uab.edu/qc3.pdf).
In this study, we describe the assignment of weight-based units for 13 serotypes to a new pneumococcal reference serum (007sp) that has been established to replace dwindling stocks of 89SF. Having accepted concentration values for an existing standard has significantly simplified the assignment process. Using established laboratories and a well-characterized ELISA procedure (4) that was followed by all participating laboratories, we were able to assign weight-based units to 007sp by running 007sp alongside a standard curve of 89SF and treating 007sp as the unknown. Very high levels of agreement between the participating laboratories for the weight-based units of IgG specific for 13 serotypes in 007sp were achieved. These assignments were made using double absorptions (CPS and 22F polysaccharide) which will help in the day-to-day running of the pneumococcal ELISA, as both sera and standard will be double adsorbed. Previously, 89SF had values assigned with single adsorption only with the utility of 22F adsorption postdating the assignments (2), thus requiring the standard and unknown sera to be treated differently during each assay.
We were then able to further validate the values obtained and the performance of 007sp as a standard during the process of assigning serotype-specific IgG values (in μg/ml) to a panel of 12 calibration sera previously prepared from the sera of pneumococcal polysaccharide-vaccinated adults. Our statistical analyses indicated a high level of agreement among the five laboratories participating in this study. Concordance was high among laboratories (Table 3) and between results for laboratories and consensus ELISA concentrations. Concordance (rc) is a combined measurement of agreement: i.e., it combines accuracy and precision and in this study was calculated across all serotypes and samples by laboratory. When each laboratory was compared to the consensus for the 12 WHO calibration sera, rc was >0.93 (Table 3). This indicates that with the adherence to the uniform application of the WHO ELISA (4) in the present study, we were able to achieve a level of precision and accuracy that should inspire confidence in the values assigned to the 13 serotypes of 007sp and the WHO calibration sera.
Establishing a new reference serum for the pneumococcus is essential for ongoing efforts to evaluate new pneumococcal vaccines while maintaining the link with the original serology performed as part of the pivotal efficacy studies conducted prior to licensure. The new standard, 007sp, is available in large quantities and should provide continuity for the foreseeable future. Assignment of weight-based units for a further 11 serotypes contained in the 23-valent pneumococcal vaccine but not in PCV13 will be undertaken in the future. Functional pneumococcal assays are assuming increasing importance in the evaluation of new vaccines, and the new set of FDA OPA calibration sera should prove as valuable as the WHO calibration sera have for the ELISA. A multilaboratory OPA study is currently being planned to assign OPA titers in due course, and these sera will be available for distribution, as for 007sp, via CBER.
This paper is dedicated to the memory of Dr. Milan Blake who initiated the development of 007sp in 2006, but tragically died unexpectedly prior to the project's completion.
This study (study protocol no. 06-0093) was sponsored by CBER, USFDA, and NIAID and supported in part by Basic and Clinical Approaches to Controlling Human Respiratory Pathogens (N01-AI-30040 to J.S.).
Published ahead of print on 18 August 2011.