Statistical Approach To Estimate Vaccinia-Specific Neutralizing Antibody Titers Using a High-Throughput Assay

doi:10.1128/CVI.00109-09

Clin Vaccine Immunol. 2009 August; 16(8): 1105–1112.

Published online 2009 June 17. doi: 10.1128/CVI.00109-09

PMCID: PMC2725542

Statistical Approach To Estimate Vaccinia-Specific Neutralizing Antibody Titers Using a High-Throughput Assay

Richard Kennedy,¹ V. Shane Pankratz,² Eric Swanson,¹ David Watson,² Hana Golding,³ and Gregory A. Poland^1,⁴^*

Mayo Vaccine Research Group, Mayo Clinic, Rochester, Minnesota 55905,¹ Division of Biostatistics, Mayo Clinic, Rochester, Minnesota 55905,² Division of Viral Products, CBER, Food and Drug Administration, Washington, DC,³ Program in Translational Immunovirology and Biodefense, Mayo Clinic, Rochester, Minnesota 55905⁴

^*Corresponding author. Mailing address: Mayo Vaccine Research Group, Mayo Clinic, Guggenheim 611C, 200 First Street SW, Rochester, MN 55905. Phone: (507) 284-4968. Fax: (507) 266-4716. E-mail: Poland.gregory/at/mayo.edu

Received March 10, 2009; Revised May 19, 2009; Accepted June 8, 2009.

This article has been cited by other articles in PMC.

Abstract

Because of the bioterrorism threat posed by agents such as variola virus, considerable time, resources, and effort have been devoted to biodefense preparation. One avenue of this research has been the development of rapid, sensitive, high-throughput assays to validate immune responses to poxviruses. Here we describe the adaptation of a β-galactosidase reporter-based vaccinia virus neutralization assay to large-scale use in a study that included over 1,000 subjects. We also describe the statistical methods involved in analyzing the large quantity of data generated. The assay and its associated methods should prove useful tools in monitoring immune responses to next-generation smallpox vaccines, studying poxvirus immunity, and evaluating therapeutic agents such as vaccinia virus immune globulin.

Smallpox was one of the most feared diseases known, killing hundreds of millions of individuals in the last several centuries alone. Following a massive surveillance and vaccination campaign led by the WHO, the disease was eradicated in 1980 (17). Recent concerns about bioterrorism have renewed scientific interest in this disease, especially in the areas of pathogen biology, host response, vaccines, and therapeutic agents (3, 6, 21). As eradication occurred before the advent of modern immunologic techniques, the fine details regarding poxvirus immunology and vaccine efficacy are only now coming to light. Historically, humoral responses were thought to be primarily responsible for vaccine-induced protection against smallpox disease (17). Although there was some evidence for a role of cellular immunity in disease protection, modern tools to accurately measure these responses were not developed until after the eradication effort (52). Measurements of antibody (Ab) titer play a central role in most vaccine efficacy studies. Classically, plaque reduction neutralization tests (PRNT) were used to determine serum neutralizing Ab titers in smallpox vaccine recipients (15, 34, 39, 42). While effective, these assays suffer from a number of disadvantages, including the use of large volumes of clinical test material, 3 to 4 days of incubation for plaque development, manual counting of plaques, subjective readout interpretations, and high labor costs. Although a number of improvements have been made to increase the efficiency of PRNTs (5, 41), they remain extremely time and labor intensive and do not lend themselves well to large-scale applications, and comparing results directly between laboratories is difficult. Given the renewed interest in variola virus as a potential biological weapon (21) and documented monkeypox outbreaks in the United States and Africa (11, 27, 31), there is a considerable impetus to develop next-generation vaccines against smallpox and myriad other select agents and emerging pathogens. As a result, substantial effort has been expended in the development of new and improved assays to circumvent the limitations of PRNTs (4, 44, 46). Many of these assays utilize high-throughput methodologies to detect levels of viral activity in biological samples (16, 26). Several assays have been developed which rely on recombinant vaccinia viruses expressing reporter genes such as green fluorescent protein or β-galactosidase (β-Gal), with subsequent fluorometric or colorimetric detection of the reporter (12, 28, 35). The β-Gal-based assay was initially described in 2003 and is as sensitive as PRNTs but is more rapid and easily adapted to high throughput and has an objective readout conducive to large-scale statistical analyses (35). Since its initial description, it has been used for a number of studies, including a comparison of Dryvax (Wyeth Laboratories, Inc.) and modified vaccinia virus Ankara immunization in a mouse study (37), the detection of vaccinia virus-specific Ab titers in samples from human immunodeficiency virus-positive adults (29), the evaluation of different vaccination sites for human subjects (53), a nonhuman-primate study showing that Ab responses are sufficient for protection against monkeypox (13, 14), and the evaluation of a subunit recombinant vaccine for use in protection against monkeypox (22). While each of these studies illustrated the utility of the β-Gal-based assay, the number of samples tested in each study was relatively small. We have now optimized and employed this assay in a large population-based study examining immune responses in over 1,000 human subjects. The assay proved to be readily adaptable to large-scale use. However, the analytical methods for this assay suffered from the need to manually inspect the data and identify assay outliers prior to estimating the 50% infectious dose (ID₅₀). We now report the development of a robust statistical analysis plan that requires relatively little user intervention.

MATERIALS AND METHODS

Subject recruitment.

Eligible subjects were healthy adults between 18 and 40 years of age who had received a single dose of Dryvax within 4 years of recruitment and at least 1 month before the blood draw. Olmsted county residents who were vaccinated as part of the Department of Health and Human Services civilian health care worker program were recruited by the Mayo Clinic (49), while eligible armed forces personnel were recruited by the Naval Health Research Center in San Diego, CA. Institutional review board approval for all study procedures was granted, and written, informed consent was obtained from each subject. Serum samples were obtained, aliquoted into sterile microcentrifuge tubes, and stored at −70°C until use. Prior to the first use, each serum sample was heat inactivated at 56°C for 30 min and then kept at 4°C until the assays for that individual were completed.

Viruses and cell lines.

HeLa, HeLa S3, and Vero cells were all obtained from the ATCC and grown in Dulbecco's modified Eagle's medium (Vero) or Eagle's minimal essential medium (HeLa, HeLa S3) supplemented with 10% fetal bovine serum, 2 mmol/liter l-glutamine, 0.1 mmol of nonessential amino acids, 1.0 mmol/liter sodium pyruvate, 1.5 g/liter sodium bicarbonate, and penicillin and streptomycin at 500 IU/ml and 500 μg/ml, respectively. All cell lines were used as originally described (35). A single lot of fetal bovine serum was used for all assays. This serum, while not directly screened for neutralizing activity against vaccinia virus, was included in both the positive and negative controls and any background neutralizing activity would be adjusted for during the normalization procedures for this assay. It is possible that lot-specific differences in background vaccinia virus neutralizing activity contribute to the small differences seen between laboratories with respect to optimal virus activity.

Vaccinia virus strain vSC56, expressing β-Gal under the control of a synthetic early/late promoter, was kindly provided by B. Moss (NIAID). The virus was grown in HeLa S3 cells and subjected to titration in Vero cells. The virus stock used in the assay was partially purified by ultracentrifugation (15,800 rpm for 80 min at 4°C in a SW28 rotor) over a 36% sucrose cushion, resuspended in 1 mM Tris-HCl (pH 9.0), aliquoted, and stored in single-use vials at −80°C.

Reagents.

Vaccinia virus immune globulin (VIG) (MBL/Dynport VIG intravenous, lot 1) was kindly provided by Christine Anderson (CBER/FDA). VIG was diluted 1:10 in phosphate-buffered saline-0.1% bovine serum albumin, aliquoted, and stored at −70°C until use. CPRG (chlorophenol red-β-d-galactopyranoside) substrate was obtained from Calbiochem (San Diego, CA), diluted in phosphate buffer to 4 mg/ml, and stored at −20°C until use. Recombinant β-Gal enzyme was purchased from Roche (Indianapolis, IN). Igepal CA630 and the other chemicals necessary for the various buffers were obtained from Sigma (St. Louis, MO). The cell lysis buffer containing cell medium and Igepal CA630 was made fresh for each assay, and all other buffers were made in bulk and stored appropriately until needed.

Neutralization assay.

The assay procedure was performed as initially reported (35) with several modifications as described in this report. Briefly, serum samples were thawed, heat inactivated, briefly spun at 10,000 rpm in a table-top microcentrifuge, and serially diluted. These dilutions (five replicate samples per dilution) were incubated with a known quantity of vSC56 for 1 h at 37°C and added to wells containing 50,000 Vero cells. Positive and negative controls consisted of serially diluted VIG (five replicates per dilution) and virus-only samples (eight replicates per 96-well plate), respectively. One set of VIG dilutions was performed for each assay, while eight virus-only samples were included on each plate in every assay. The plates containing Ab, virus, and cells were incubated overnight and the cells lysed with detergent (Igepal CA630). We did not perform the freeze-thaw step outlined in the original protocol. An aliquot of each lysate was then incubated with a colorimetric substrate (CPRG) in an appropriate buffer system. The enzymatic reaction was quenched using Na2CO₃, and the plates were read using a ThermoMax plate reader (Molecular Devices, Sunnyvale, CA). Data were collected and transferred to a Microsoft Excel template for standard curve calibration and initial analysis.

Statistical analysis.

Each standard curve was assessed, verified to have an r² ≥ 0.99, and used to convert the values of the optical density readings on that plate to the levels of β-Gal activity. The levels of β-Gal activity in the virus-only wells, as well as the ID₅₀ of the VIG standard, were then estimated and verified to be within appropriate ranges (virus-only β-Gal activity, 1,000 to 6,000; VIG ID₅₀, 5 μg/ml to 30 μg/ml). All well-specific results were retained for analysis. Assays where both the virus-only and VIG averages fell outside of these ranges were discarded, and the subject's samples were tested again.

The basis of the innovation in the estimation of ID₅₀ values was the use of robust regression techniques. A bottleneck in the use of the assay in a high-throughput situation has been the need to inspect the data, identify assay outliers, and make a determination to accept or reject each outlier prior to the analysis. Our intent was to remove the need to identify and eliminate aberrant measures of β-Gal activity required by least-squares approaches to curve fitting. To accomplish this, we employed the M estimation approach introduced by Huber (25). This approach is a generalization of standard least-squares estimations; while the least-squares approach minimizes the sum of squared residuals, M estimators minimize the sum of the functions of the residuals that are increasing less rapidly. This has the effect of reducing the influence of outliers on the estimates of the regression coefficients. We relied on the iteratively reweighted least-squares approach in performing M estimations, with a bisquare weighting function, that is implemented in the ROBUSTREG procedure of the SAS software package (Cary, NC). Raw optical density readings were converted to units of enzyme activity based on standard curves for each plate and then scaled to percentages of control activity. These percentages of control values were then log transformed and all analyses carried out using SAS software. We developed a web-based tool that estimated the ID₅₀ value for each individual whose samples had been assayed on each plate. Together with the assay results, this tool returned a series of model fitting and other assessments that allowed the evaluation of data quality.

We fit a quadratic trend to the data to accommodate dilution readings which fell outside of the linear range of the assay. The resulting regression coefficients from the M estimation procedure were extracted and used with reference to the virus-only levels to estimate the ID₅₀ for each individual. Our web tool returned this estimate along with a collection of assessments of data quality for each assay, including (i) a plot of the observed data, with the fitted line superimposed, as a visual check of the agreement between the model and the data; (ii) the model deviance, a measurement for which large values are indicative of a poor agreement between the observed data and the fitted model; and (iii) indicators of whether the ID₅₀ estimates were markedly greater than 100% of the percentages of control values. Once all replicated ID₅₀ estimates were obtained in this way, a linear mixed model, with a per-subject random effect, was applied to the replicated log (ID₅₀) values on the logarithmic scale. This enabled additional assessments of the overall performance of the assay and provided an estimate of the intraclass correlation coefficient, which measures assay repeatability across all subjects, as well as measurements of the residuals, which assess the degree to which the ID₅₀ values deviated from their within-individual averages. Experiments with values more than 3.0 standard deviations from the per-subject means estimated from the linear mixed-model fit to all the data were repeated.

RESULTS

Assay optimization.

Our study population consisted of 1,076 healthy adults ranging from 18 to 40 years of age. Study subjects were military personnel or civilians involved in the U.S. Smallpox Preparedness and Response Program with documented “takes” (i.e., development of the characteristic pock at the vaccination site) after receipt of a single dose of smallpox vaccine (45, 49). Table Table11 lists additional demographic information. Ab responses peak 4 weeks after receipt of the smallpox vaccine (1, 17, 32); consequently, serum samples from each subject were obtained at least 6 weeks after smallpox vaccination, allowing sufficient time for the development of humoral responses. Each sample was tested in at least three separate experiments run on different days. Our protocol had several notable changes from the originally described procedure. These changes, briefly mentioned above in Materials and Methods, are reiterated here along with their effect on assay utility. (i) The addition of a partial purification step performed by ultracentrifugation of the infected HeLa lysate over a 36% sucrose cushion followed by sonication greatly increased the consistency of viral infection and the enzyme activity levels in our assays. (ii) Heat-inactivated serum samples were spun briefly at 10,000 rpm in a table-top microcentrifuge before use to remove Ab complexes and reduce the variability seen in our assays. (iii) The freeze-thaw step before the development phase of the assay was omitted. Given the volume of samples assayed at one time, this step was not necessary. Additionally, immediate development of the plates allowed real-time analysis of results and earlier detection of handling errors. (iv) Virus-only samples were included on every plate. This increased the numbers of replicates and the accuracy of the value used to normalize all β-Gal activity readings. This also allowed us to determine that the plate-to-plate variations within a given assay were minimal. (v) We made minor adjustments to the acceptable ranges for virus activity, multiplicity of infection (MOI), and VIG activity. In the original study, virus activity was kept below 1,000 units. In our hands, we found that lower levels of virus activity (<1,000) resulted in substantially increased variability of results among replicates. Conversely, very high virus-only values overwhelmed the neutralization activity of the serum samples tested. Our median virus activity value was 3,612.54. Our range of VIG ID₅₀ values was slightly lower than that originally published, with a median of 13.8 μg/ml (interquartile range [IQR] = 5.5 to 25.9). Given that the acceptable ranges for the MOI and virus activity in the assay differed between our laboratory and that of the FDA, these parameters may differ from laboratory to laboratory.

TABLE 1.

Subject demographics

Neutralizing Ab titers in study population.

Figure Figure11 shows the distribution of neutralizing Ab responses found in our study population, as measured by at least three separate experiments (each performed after optimization and assay modification). The median values of the individual assay ID₅₀ measurements are plotted. These ID₅₀ values ranged from 15.7 to 1,314.2, had a median value of 132.2, and had an interquartile range of 126.8. The 25th percentile value was 78.8, and the 75th percentile value was 205.6. The average ID₅₀ value for samples from unvaccinated subjects (n = 9) was 1.11 ± 2.72 (standard deviation). Because of differences between study populations and assays used to determine Ab titers, it is difficult to directly compare results among published reports. Nevertheless, our results are broadly similar to the values obtained for neutralizing Ab titers following smallpox vaccination that have been reported by other groups (2, 9, 10, 19, 20, 50).

FIG. 1.

Histogram of the distribution of the ID₅₀ values obtained from the sample of 1,076 study participants.

Initial testing of the robust modeling approach.

In order to assess the quality of the estimates provided by our high-throughput, robust modeling approach, we identified a sample of 30 subjects whose data were run in a short time interval and assessed the agreement between the ID₅₀ estimates calculated using both the original and the modified analytical methods. Using all data from the 30 assays resulted in a correlation of 0.94 between the log-transformed estimates from the two methods. However, differences between these estimates ranged from −0.199 to 0.153, with a mean ± standard deviation of −0.005 ± 0.080 (See Fig. Fig.2).2). Of these assays, a total of 17 (57%) had outliers; as recommended by the original method, these were removed. Following the re-estimation of the ID₅₀ values, the correlation value climbed to 0.97 and the differences between the results obtained for the original and the robust methods ranged from −0.107 to 0.091, with a mean ± standard deviation of −0.007 ± 0.061, on the logarithmic scale. The ID₅₀ values from our modified approach were on average within 3% of the ID₅₀ values obtained by the original approach without outliers.

FIG. 2.

Agreement between the original ID₅₀ estimation procedure and the modified robust estimate. Solid circles represent data from assays that met the original quality control metrics. Open circles represent data from assays that had outliers flagged for removal (more ...)

The approach to controlling the quality of the individual-specific assay readings described by Manischewitz et al. requires hands-on monitoring of each assay and outlines criteria for the removal of outlying observations (35). In our study, a single operator performed the assay on 38 samples at once. The hands-on time was ~6 h of effort, after which the operator was required to monitor each of approximately 1,000 data points to determine whether there were outliers in any set of replicates and which outliers were to be removed. This step takes additional time and effort. The approach described here employed statistical methods that are robust (i.e., that result in the presence of only a small number of outliers) and eliminate the time-consuming process of defining and then finding and removing outliers. In Fig. Fig.3,3, one can see the success of this approach in estimating a value of the ID₅₀ in the presence of suboptimal data (i.e., data containing outliers).

FIG. 3.

Example of ID₅₀ estimations calculated from data with an outlier that would have been removed using the original protocol.

For the bulk of the subjects (94.1% [1,013/1,076]), all of the replicated ID₅₀ values fell within the 1:20 to 1:540 range. Assays where a subject sample had an ID₅₀ value greater than 540 were repeated using an extended dilution range (1:60 to 1:1,620). After these assays were performed, using the extended range and fitting the linear mixed model, the intraclass correlation was found to be 0.70. While this value suggests a high degree of reproducibility of the results, 0.95% (31/3,260) of the assays had residuals that were more than 3.0 standard deviations from zero). Each of these 31 subjects was retested an additional three times. All of these second assays conformed to our quality control constraints, and a repeated residual analysis verified that all samples demonstrated acceptable degrees of variation among the replicate experiments. The value for intraclass correlation for the revised data set was 0.74.

The coefficient of variation (CV) was obtained for each subject in this final data set of log-transformed ID₅₀ values. The average CV was 6.9%, with a standard deviation of 4.1%. The largest CV was 26.1%, and the 25th and 75th percentiles of the CV distribution were 3.7% and 9.2%, respectively. The CV for more than 99% of the subjects was less than 20%.

Additional variables which affect data analysis.

As we found it necessary to establish a different range of VIG and virus-only values from those originally reported, we examined the effects of several different parameters on ID₅₀ results. These parameters included virus activity, VIG ID₅₀, date of assay, and interpretation of data by operator. With respect to virus activity, we found that higher levels of virus were associated with lower ID₅₀ measurements (P < 0.0001) and that doubling the level of virus activity resulted in a decrease in the average ID₅₀ measurements by a factor of about 0.88, a 12% decrease. Given that sufficiently high levels of virus are likely to overcome the neutralizing activity of most serum samples, this correlation is not surprising. Likewise, with VIG activity, there was a significant association with ID₅₀ values (P < 0.0001), and a doubling of the level of VIG activity was associated with a 7% increase in the average ID₅₀ values. We did not expect VIG ID₅₀ values to be directly related to those of serum ID₅₀; however, the VIG value is an indirect measure of assay performance, and differences in VIG activity may reflect systematic differences in serum neutralization. Finally, we analyzed VIG ID₅₀ values for evidence of assay drift. While VIG ID₅₀ values remained within a narrow range, there was no obvious trend in the observed values (See Fig. Fig.4).4). Interestingly, while the amount of variability of our VIG ID₅₀ values was nearly identical to that originally described, the actual range was somewhat lower and may reflect laboratory-specific differences in reagent usage and/or technique. The average month-to-month difference in VIG ID₅₀ values was 3.8%. We observed no significant association between operator identity and ID₅₀ estimates (P = 0.6184). Determinations of average ID₅₀ values differed among operators by less than 3%.

FIG. 4.

VIG quality control chart. This figure shows all of the VIG ID₅₀ values (in micrograms/milliliter) from the assays performed by a single operator. Dotted lines represent the upper and lower 95% confidence intervals (3.8 and 27.0, respectively). (more ...)

DISCUSSION

With any assay or technique, scaling up of sample sizes can pose multiple problems and necessitate numerous procedural changes and refinements. Fortunately, the original protocol was highly amenable to being scaled up. Most of the assay materials can be manipulated in a 96-well format with multichannel pipettes. One of the largest bottlenecks in the procedure was the transfer of serum from individual microcentrifuge storage tubes to the 96-well plate for dilution. Initial serum storage in 96-well plates would result in considerable time savings. Without any automation, we were able to readily test up to 20 96-well plates per operator per assay while remaining within the time constraints of the various protocol steps. The use of robotic workstations would dramatically increase this number. Another of the larger bottlenecks was the time and effort required for growing, maintaining, and expanding the HeLa cell cultures (we required ~4 × 10⁹ cells each week). Laboratories utilizing this assay for a large numbers of samples are encouraged to invest in appropriate bulk cell culture systems.

We also found that acceptable levels of virus activity, MOI, and duration of infection may differ between laboratories. Care should be taken to optimize these variables within each laboratory before large-scale testing is performed. In spite of these differences, the inclusion of the VIG standards should still allow results to be compared between laboratories. The development of comprehensive sets of assay standards would allow direct comparisons between laboratories and serve to address this concern.

Our results showed that the range of ID₅₀ values was quite wide (15.7 to 1,314.2), with 100% of our subjects demonstrating detectable levels of humoral immunity higher than those found with a small panel of vaccinia virus-naïve subjects (average ID₅₀ = 1.11). Our study inclusion criteria included a documented “take.” Historically, this take has been considered evidence of successful vaccination and protection from disease (17). Recent studies have confirmed that the vaccine take is associated with seroconversion and positive vaccinia virus-specific T- and B-cell responses (2, 8, 36), although in the absence of endemic disease, we can only estimate correlates of protection. Historically, titers of 1:20 or 1:32 were assumed to represent protective levels (34, 48), and while a direct comparison of our results with historical estimates of protection must be interpreted with caution, it is interesting that, in spite of the presence of a documented vaccine “take,” a small number of individuals (22/1,076 [2.0%]) had ID₅₀ values of <32. Further investigation of the immune responses (both humoral and cellular) is warranted to understand this apparent disconnect between the presence of a vaccine take and the neutralizing Ab titer seen with these individuals.

In our study cohort, the assay was easily able to accurately and reproducibly detect serum titers as low as 1:15 and as high as 1:1,300, and VIG titers were routinely between 1:2,735 and 1:5,537 (IQR). The range of threefold dilutions routinely used in the assay (1:20 to 1:540) sufficed for most subjects, and the use of a quadratic trend fit to the data allowed us to accurately determine ID₅₀ values even when the dilutions did not all fall within the linear range of the assay. For those individuals with ID₅₀ values well above 540, it was necessary use a higher dilution range (1:60 to 1:1,620). The need for repetition could be easily remedied by expanding the range of dilutions initially tested, for example (to a range of 1:20 to 1:1,620). Recent studies examining long-term persistence of smallpox immunity have found that humoral responses initially decline in the 5 to 10 years following vaccination and then remain stable for decades. For individuals vaccinated within 1 decade of serologic testing, these reports cite 50% neutralization titers of between 10 and 100 (10, 20), 64 and 256 (19), and 64 and 128 (50), values that are quite similar to the range of titers (median ID₅₀, 132.2; IQR, 78.8 to 205.6) seen in our population tested within 4 years of vaccination.

Vaccinia virus produces two main infectious forms, intracellular mature virus (IMV) and extracellular enveloped virus (EEV) (33, 40). EEV contains an additional membrane and several unique viral proteins and mediates long-range dissemination in vivo (43). Full protection against poxvirus infection or disease is believed to require antibodies against both IMV and EEV (7, 18, 23, 24, 30). The vast majority of viral particles produced are IMV, and EEV particles are extremely fragile; consequently, most poxvirus-specific serological tests, including the assay discussed here, measure Ab responses to IMV. Comet reduction assays are typically used to measure anti-EEV activity; however, these assays suffer from the same limitations as PRNTs. Alternatively, one can modify PRNT assays to focus on EEV-specific antibodies by using freshly made stocks of EEV supplemented by anti-IMV antibodies to reduce IMV activity and to remove EEV virions with disrupted membranes (30, 33, 38, 47, 51). Collection of vSC56 EEV virions from the supernatant of infected cells and the inclusion of anti-IMV Abs would be relatively straightforward modifications to the protocol and would allow investigators to measure EEV-specific neutralizing Ab titers. The need to continually prepare fresh EEV stocks, the differing viral titers of these stocks, and the need to ensure a steady supply of anti-IMV antibodies may limit the high-throughput nature of the assay, but the trade-off may well be worth it for certain study designs.

The reproducibility of this method, as well as its ability to provide reliable estimates of each individual's ID₅₀, is further highlighted by the final results. The within-person CVs of the log (ID₅₀) values were small, with the mean CV being equal to 6.9% and with 95% of the CVs falling between 1.0% and 16.7%. Likewise, the intraclass correlation coefficient of 0.74 is quite high, indicating that the measurement error of the assay is quite small relative to the person-to-person differences.

In conclusion, this assay has many benefits over traditional PRNTs. The β-Gal assay requires a low volume of clinical sample, is easily adapted to a 96-well format or even automated handling for high-throughput applications, is rapid, and provides an objective, automated readout, with high quality control confidence. This assay performed extremely well in our hands, and even lacking robotic workstations for automated sample handling, we were able to assay over 3,000 specimens. This assay should prove useful in clinical trials of next-generation smallpox vaccines or to monitor changes in immune status over time. It could also be used as a screen to determine which individuals within a given population should be revaccinated to boost waning immunity. Given the large number of recombinant viruses expressing β-Gal or other reporter genes, this assay could also be easily adapted to other viral systems.

Acknowledgments

We acknowledge Megan Ryan and Kevin Russell and their Naval Health Research Center team as well as the nurses and study coordinators from the Mayo Vaccine Research Group for their tremendous efforts in recruiting the large number of subjects needed for the study. We thank Cheri A. Hart for her editorial assistance.

This work was supported by the NIAID though NIH Population Genetics Analysis Program grant AI40065.

Disclosures and conflicts of interest: G.A.P. has provided consultant advice to Acambis. The other authors do not have any conflicts of interest.

Footnotes

Published ahead of print on 17 June 2009.

REFERENCES

1. Amanna, I. J., M. K. Slifka, and S. Crotty. 2006. Immunity and immunological memory following smallpox vaccination. Immunol. Rev. 211320-337. [PubMed]

2. Belshe, R. B., F. K. Newman, S. E. Frey, R. B. Couch, J. J. Treanor, C. O. Tacket, and L. Yan. 2004. Dose-dependent neutralizing-antibody responses to vaccinia. J. Infect. Dis. 189493-497. [PubMed]

3. Bhalla, D. K., and D. B. Warheit. 2004. Biological agents with potential for misuse: a historical perspective and defensive measures. Toxicol. Appl. Pharmacol. 19971-84. [PubMed]

4. Bonilla-Guerrero, R., and G. A. Poland. 2003. Smallpox vaccines: current and future. J. Lab. Clin. Med. 142252-257. [PubMed]

5. Borges, M. B., S. E. Kato, C. R. Damaso, N. Moussatche, M. da Silva Freire, S. R. Lambert Passos, and J. P. do Nascimento. 2008. Accuracy and repeatability of a micro plaque reduction neutralization test for vaccinia antibodies. Biologicals 36105-110. [PubMed]

6. Bossi, P., D. Garin, A. Guihot, F. Gay, J. M. Crance, T. Debord, B. Autran, and F. Bricaire. 2006. Bioterrorism: management of major biological agents. Cell. Mol. Life Sci. 632196-2212. [PubMed]

7. Boulter, E. A., and G. Appleyard. 1973. Differences between extracellular and intracellular forms of poxvirus and their implications. Prog. Med. Virol. 1686-108. [PubMed]

8. Cockburn, W. C., R. M. Cross, A. W. Downie, K. R. Dumbell, C. Kaplan, D. McClean, and A. M. Payne. 1957. Laboratory and vaccination studies with dried smallpox vaccines. Bull. W. H. O. 1663-77. [PubMed]

9. Combadiere, B., A. Boissonnas, G. Carcelain, E. Lefranc, A. Samri, F. Bricaire, P. Debre, and B. Autran. 2004. Distinct time effects of vaccination on long-term proliferative and IFN-gamma-producing T cell memory to smallpox in humans. J. Exp. Med. 1991585-1593. [PMC free article] [PubMed]

10. Crotty, S., P. Felgner, H. Davies, J. Glidewell, L. Villarreal, and R. Ahmed. 2003. Cutting edge: long-term B cell memory in humans after smallpox vaccination. J. Immunol. 1714969-4973. [PubMed]

11. Di Giulio, D. B., and P. B. Eckburg. 2004. Human monkeypox: an emerging zoonosis. Lancet Infect. Dis. 415-25. [PubMed]

12. Earl, P. L., J. L. Americo, and B. Moss. 2003. Development and use of a vaccinia virus neutralization assay based on flow cytometric detection of green fluorescent protein. J. Virol. 7710684-10688. [PMC free article] [PubMed]

13. Edghill-Smith, Y., M. Bray, C. A. Whitehouse, D. Miller, E. Mucker, J. Manischewitz, L. R. King, M. Robert-Guroff, A. Hryniewicz, D. Venzon, C. Meseda, J. Weir, A. Nalca, V. Livingston, J. Wells, M. G. Lewis, J. Huggins, S. H. Zwiers, H. Golding, and G. Franchini. 2005. Smallpox vaccine does not protect macaques with AIDS from a lethal monkeypox virus challenge. J. Infect. Dis. 191372-381. [PubMed]

14. Edghill-Smith, Y., H. Golding, J. Manischewitz, L. R. King, D. Scott, M. Bray, A. Nalca, J. W. Hooper, C. A. Whitehouse, J. E. Schmitz, K. A. Reimann, and G. Franchini. 2005. Smallpox vaccine-induced antibodies are necessary and sufficient for protection against monkeypox virus. Nat. Med. 11740-747. [PubMed]

15. el-Ad, B., Y. Roth, A. Winder, Z. Tochner, T. Lublin-Tennenbaum, E. Katz, and T. Schwartz. 1990. The persistence of neutralizing antibodies after revaccination against smallpox. J. Infect. Dis. 161446-448. [PubMed]

16. Eyal, O., U. Olshevsky, S. Lustig, N. Paran, M. Halevy, P. Schneider, G. Zomber, and P. Fuchs. 2005. Development of a tissue-culture-based enzyme-immunoassay method for the quantitation of anti-vaccinia-neutralizing antibodies in human sera. J. Virol. Methods 13015-21. [PubMed]

17. Fenner, F. 1988. Smallpox and its eradication. World Health Organization, Geneva, Switzerland.

18. Fogg, C., S. Lustig, J. C. Whitbeck, R. J. Eisenberg, G. H. Cohen, and B. Moss. 2004. Protective immunity to vaccinia virus induced by vaccination with multiple recombinant outer membrane proteins of intracellular and extracellular virions. J. Virol. 7810230-10237. [PMC free article] [PubMed]

19. Frey, S. E., F. K. Newman, L. Yan, K. R. Lottenbach, and R. B. Belshe. 2003. Response to smallpox vaccine in persons immunized in the distant past. JAMA 2893295-3299. [PubMed]

20. Hammarlund, E., M. W. Lewis, S. G. Hansen, L. I. Strelow, J. A. Nelson, G. J. Sexton, J. M. Hanifin, and M. K. Slifka. 2003. Duration of antiviral immunity after smallpox vaccination. Nat. Med. 91131-1137. [PubMed]

21. Henderson, D. A., T. V. Inglesby, J. G. Bartlett, M. S. Ascher, E. Eitzen, P. B. Jahrling, J. Hauer, M. Layton, J. McDade, M. T. Osterholm, T. O'Toole, G. Parker, T. Perl, P. K. Russell, K. Tonat, and the Working Group on Civilian Biodefense. 1999. Smallpox as a biological weapon: medical and public health management. JAMA 2812127-2137. [PubMed]

22. Heraud, J. M., Y. Edghill-Smith, V. Ayala, I. Kalisz, J. Parrino, V. S. Kalyanaraman, J. Manischewitz, L. R. King, A. Hryniewicz, C. J. Trindade, M. Hassett, W. P. Tsai, D. Venzon, A. Nalca, M. Vaccari, P. Silvera, M. Bray, B. S. Graham, H. Golding, J. W. Hooper, and G. Franchini. 2006. Subunit recombinant vaccine protects against monkeypox. J. Immunol. 1772552-2564. [PubMed]

23. Hooper, J. W., D. M. Custer, and E. Thompson. 2003. Four-gene-combination DNA vaccine protects mice against a lethal vaccinia virus challenge and elicits appropriate antibody responses in nonhuman primates. Virology 306181-195. [PubMed]

24. Hooper, J. W., E. Thompson, C. Wilhelmsen, M. Zimmerman, M. A. Ichou, S. E. Steffen, C. S. Schmaljohn, A. L. Schmaljohn, and P. B. Jahrling. 2004. Smallpox DNA vaccine protects nonhuman primates against lethal monkeypox. J. Virol. 784433-4443. [PMC free article] [PubMed]

25. Huber, P. J. 1973. Robust regression: asymptotics, conjectures and Monte Carlo. Ann. Stat. 1799-821.

26. Ifrah, M., S. Stienlauf, M. Shoresh, and E. Katz. 1998. A tetrazolium-based colorimetric assay for titration of neutralizing antibodies against vaccinia virus. Viral Immunol. 1149-54. [PubMed]

27. Jezek, Z., I. Arita, M. Mutombo, C. Dunn, J. H. Nakano, and M. Szczeniowski. 1986. Four generations of probable person-to-person transmission of human monkeypox. Am. J. Epidemiol. 1231004-1012. [PubMed]

28. Johnson, M. C., I. K. Damon, and K. L. Karem. 2008. A rapid, high-throughput vaccinia virus neutralization assay for testing smallpox vaccine efficacy based on detection of green fluorescent protein. J. Virol. Methods 15014-20. [PubMed]

29. Kan, V. L., J. Manischewitz, L. R. King, and H. Golding. 2007. Durable neutralizing antibodies after remote smallpox vaccination among adults with and without HIV infection. AIDS 21521-524. [PubMed]

30. Kaufman, D. R., J. Goudsmit, L. Holterman, B. A. Ewald, M. Denholtz, C. Devoy, A. Giri, L. E. Grandpre, J. M. Heraud, G. Franchini, M. S. Seaman, M. J. Havenga, and D. H. Barouch. 2008. Differential antigen requirements for protection against systemic and intranasal vaccinia virus challenges in mice. J. Virol. 826829-6837. [PMC free article] [PubMed]

31. Kile, J. C., A. T. Fleischauer, B. Beard, M. J. Kuehnert, R. S. Kanwal, P. Pontones, H. J. Messersmith, R. Teclaw, K. L. Karem, Z. H. Braden, I. Damon, A. S. Khan, and M. Fischer. 2005. Transmission of monkeypox among persons exposed to infected prairie dogs in Indiana in 2003. Arch. Pediatr. Adolesc. Med. 1591022-1025. [PubMed]

32. Kim, S. H., S. J. Choi, W. B. Park, H. B. Kim, N. J. Kim, M. D. Oh, and K. W. Choe. 2007. Detailed kinetics of immune responses to a new cell culture-derived smallpox vaccine in vaccinia-naive adults. Vaccine 256287-6291. [PubMed]

33. Law, M., and G. L. Smith. 2001. Antibody neutralization of the extracellular enveloped form of vaccinia virus. Virology 280132-142. [PubMed]

34. Mack, T. M., J. Noble, Jr., and D. B. Thomas. 1972. A prospective study of serum antibody and protection against smallpox. Am. J. Trop. Med. Hyg. 21214-218. [PubMed]

35. Manischewitz, J., L. R. King, N. A. Bleckwenn, J. Shiloach, R. Taffs, M. Merchlinsky, N. Eller, M. G. Mikolajczyk, D. J. Clanton, T. Monath, R. A. Weltzin, D. E. Scott, and H. Golding. 2003. Development of a novel vaccinia-neutralization assay based on reporter-gene expression. J. Infect. Dis. 188440-448. [PubMed]

36. McClain, D. J., S. Harrison, C. L. Yeager, J. Cruz, F. A. Ennis, P. Gibbs, M. S. Wright, P. L. Summers, J. D. Arthur, and J. A. Graham. 1997. Immunologic responses to vaccinia vaccines administered by different parenteral routes. J. Infect. Dis. 175756-763. [PubMed]

37. Meseda, C. A., A. D. Garcia, A. Kumar, A. E. Mayer, J. Manischewitz, L. R. King, H. Golding, M. Merchlinsky, and J. P. Weir. 2005. Enhanced immunogenicity and protective effect conferred by vaccination with combinations of modified vaccinia virus Ankara and licensed smallpox vaccine Dryvax in a mouse model. Virology 339164-175. [PubMed]

38. Midgley, C. M., M. M. Putz, J. N. Weber, and G. L. Smith. 2008. Vaccinia virus strain NYVAC induces substantially lower and qualitatively different human antibody responses compared with strains Lister and Dryvax. J. Gen. Virol. 892992-2997. [PMC free article] [PubMed]

39. Møller-Larsen, A., and S. Haahr. 1978. Humoral and cell-mediated immune responses in humans before and after revaccination with vaccinia virus. Infect. Immun. 1934-39. [PMC free article] [PubMed]

40. Moss, B. 2006. Poxvirus entry and membrane fusion. Virology 34448-54. [PubMed]

41. Newman, F. K., S. E. Frey, T. P. Blevins, M. Mandava, A. Bonifacio, Jr., L. Yan, and R. B. Belshe. 2003. Improved assay to detect neutralizing antibody following vaccination with diluted or undiluted vaccinia (Dryvax) vaccine. J. Clin. Microbiol. 413154-3157. [PMC free article] [PubMed]

42. Pattanayak, S., C. L. Sehgal, S. Choudhury, and P. Vijayan. 1977. Revaccination studies in children with graded potency of smallpox vaccines. Indian J. Med. Res. 65756-759. [PubMed]

43. Payne, L. G. 1980. Significance of extracellular enveloped virus in the in vitro and in vivo dissemination of vaccinia. J. Gen. Virol. 5089-100. [PubMed]

44. Poland, G. A. 2005. Smallpox vaccines: from first to second to third generation. Lancet 365362-363. [PubMed]

45. Poland, G. A., J. D. Grabenstein, and J. M. Neff. 2005. The US smallpox vaccination program: a review of a large modern era smallpox vaccination implementation program. Vaccine 232078-2081. [PubMed]

46. Poland, G. A., and J. M. Neff. 2003. Smallpox vaccine: problems and prospects. Immunol. Allergy Clin. N. Am. 23731-743. [PubMed]

47. Pütz, M. M., C. M. Midgley, M. Law, and G. L. Smith. 2006. Quantification of antibody responses against multiple antigens of the two infectious forms of vaccinia virus provides a benchmark for smallpox vaccination. Nat. Med. 121310-1315. [PubMed]

48. Sarkar, J. K., A. C. Mitra, and M. K. Mukherjee. 1975. The minimum protective level of antibodies in smallpox. Bull. W. H. O. 52307-311. [PubMed]

49. Strikas, R. A., L. J. Neff, L. Rotz, J. Cono, D. Knutson, J. Henderson, and W. A. Orenstein. 2008. US civilian smallpox preparedness and response program, 2003. Clin. Infect. Dis. 46(Suppl. 3)S157-S167. [PubMed]

50. Taub, D. D., W. B. Ershler, M. Janowski, A. Artz, M. L. Key, J. McKelvey, D. Muller, B. Moss, L. Ferrucci, P. L. Duffey, and D. L. Longo. 2008. Immunity from smallpox vaccine persists for decades: a longitudinal study. Am. J. Med. 1211058-1064. [PMC free article] [PubMed]

51. Viner, K. M., and S. N. Isaacs. 2005. Activity of vaccinia virus-neutralizing antibody in the sera of smallpox vaccinees. Microbes Infect. 7579-583. [PubMed]

52. Virelizier, J. L., M. Hamet, J. J. Ballet, P. Reinert, and C. Griscelli. 1978. Impaired defense against vaccinia in a child with T-lymphocyte deficiency associated with inosine phosphorylase defect. J. Pediatr. 92358-362. [PubMed]

53. Waibel, K. H., H. Golding, J. Manischewitz, L. R. King, M. Tuchscherer, R. L. Topolski, and D. S. Walsh. 2006. Clinical and immunological comparison of smallpox vaccine administered to the outer versus the inner upper arms of vaccinia-naive adults. Clin. Infect. Dis. 42e16-e20. [PubMed]

Articles from Clinical and Vaccine Immunology : CVI are provided here courtesy of
American Society for Microbiology (ASM)