|Home | About | Journals | Submit | Contact Us | Français|
In May 2009 the National Institute of Allergy and Infectious Diseases hosted a workshop on serologic assays that support vaccine efficacy evaluations. The meeting promoted exchange of ideas among investigators from varying disciplines who are working on anti-infectious agent vaccines at different stages of development. The presentations and discussions at the workshop illustrated the challenges common across various pathogens with recurring themes: 1) A thorough understanding of the science regarding the pathogen and the host response to disease and immunization is fundamental to assay selection. 2) The intended use of the immunoassay data must be clearly defined to ensure appropriate specificity, accuracy, and precision; a laboratory must also commit resources to assure data quality and reliability. 3) During vaccine development, an immunoassay may evolve with respect to quality, purpose, and degree of standardization, and, in some cases, must be changed or replaced as data are accumulated. 4) Collaboration on standardized reagents and methods, harmonization efforts, and multidisciplinary teams facilitates consistent generation of quality data. This report provides guidance for effective development and utilization of immunoassays based on the lessons learned from currently licensed vaccines. Investigators are encouraged to create additional opportunities for scientific exchange, noting that the discussed themes are relevant for immunoassays used for other purposes such as therapeutics and diagnostics.
On May 5–6, 2009, in Bethesda MD, the Division of Microbiology and Infectious Diseases, National Institute of Allergy and Infectious Diseases (NIAID) hosted a workshop on the use of serologic assays to support vaccine efficacy in clinical studies. The purpose of this meeting was to provide a forum for the exchange of ideas among researchers and clinicians and to identify some common themes across pathogens and technologies regarding the development and use of immunoassays. Participants in the meeting were those involved in vaccine or immunotherapeutic development programs that have been evaluated in clinical trials as well as those that are expected to enter Phase I clinical trials in the near future. Also attending were regulatory reviewers involved in evaluation of clinical and non-clinical assay data.
Generation of clinical data demonstrating efficacy and safety remain the gold standard for assessing candidate vaccines protecting against infectious diseases; however, efficacy studies for some products may be unfeasible because of statistical limitations (e.g., disease incidence is too low or variable) or because of ethical considerations (e.g., alternative vaccines or therapeutics are available). The primary purpose of this meeting was to review some vaccines that have been licensed recently and to extract some of the lessons learned which could facilitate the use of serologic assays to evaluate current and future candidate vaccines. The serologic data and experiences associated with vaccine evaluations were presented in three sessions (Table 1): (1) general concepts when using immunoassays to evaluate efficacy, (2) specific vaccines for which humoral correlates have been used for vaccine licensure, and (3) evolving issues related to humoral correlates of protective immunity with respect to new vaccine evaluations. The final session was a panel discussion with participation from the audience that addressed issues and questions raised during the meeting. This summary represents a distillation of the presentations and discussions, uniting the common themes and issues that appeared throughout the meeting.
One issue raised early in the meeting was the lack of clarity of terminology. For many terms, no definitions are universally accepted; this lack of clarity can itself hinder effective communication. In order to facilitate interpretation, we have defined our usage; however, these should not be considered consensus definitions. “Correlate” is a variable that is statistically related to a clinical endpoint, while a “protective correlate” or “predictive correlate” is a correlated variable that, based on additional evidence, is reasonably likely to predict the clinical endpoint[1–3]. We avoid using the term “surrogate”, as its definition indicates that the measured variable (serologic data, in our case) can totally explain and replace a defined clinical endpoint. “Immunoassays” is a broad term representing measurement of any number of biologic processes associated with immune function. The scope of this meeting was limited to serologic assays which identify and quantify antibodies, rather than assays that assess the cellular activities and functions. Serologic assays include both binding assays (e.g., Enzyme Linked Immunosorbent Assays (ELISAs) and Hemagglutination Inhibition (HI) assays) and functional assays (e.g., bactericidal, opsonophagocytic, neutralization assays). Assay “standardization” is defined as the establishment of an assay methodology within and across laboratories that utilizes a common standard operating protocol (SOP), defining reagent requirements as well as procedures, and generates comparable data sets. Assay “harmonization” is defined as the identification, through an iterative experimental process, of critical parameters that affect assay performance, and the development of related SOPs that conform in those critical variables, leading to the generation of comparable data sets within and across laboratories.
Several common themes emerged during the meeting and are the focus of this report.
Ideally, vaccine developers formulate a product that elicits a protective immune response specific for the target pathogen. Rational vaccine design requires the best possible understanding of the nature of the immune response to the pathogen and the cascade of immunological events that culminate in protection. Identification of the important types of response (e.g., humoral or cellular, specific isotype or subclass of immunoglobulin) and the kinetics of response to the initial and subsequent exposures (days, weeks, months) to the pathogen may play key roles in characterizing protective immunity. These kinds of data provide the foundation for the development of candidate vaccines and enable the development of immunoassays that can be used to evaluate relevant parameters post-immunization.
Studies of the protective humoral responses associated with exposure to Haemophilus influenzae type b (Hib), Streptococcus pneumoniae, and Neisseria meningitidis C and A were instrumental to the development of polysaccharide based vaccines against these pathogens. As respectively reviewed by Frasch, Nahm, Bash, and Stephens, the development of vaccines against these pathogens was based on observations that antibodies to the bacterial capsules provide protection from natural disease in immunocompetent individuals[5–10]. Furthermore, population studies measuring the concentration of anti-capsular polysaccharide antibodies associated with protection from Hib and pneumococcal disease by age as well as after passive immunoglobulin administration established correlates of protection[11–13]. These observations provided scientifically and epidemiologically sound data regarding the type of antibody that would be most protective (serum antibodies that were bactericidal or opsonic)[14, 15]. Frasch reviewed how serum bactericidal titers provided a biomarker for in vivo protective activity against Hib and that as the knowledge base grew identifying the most effective immunoglobulins in sera, more precise antibody binding assays were developed and could be used to facilitate evaluation of different vaccine formulations.
Serologic correlates of protection associated with meningococcal group C disease were based on a clinical study conducted in American soldiers in 1968, which established a 1:4 titer in the serum bactericidal assay (SBA) as predictive of protection from meningococcal C disease. Bash noted that in the original clinical study, sera were tested only at one dilution in the SBA (1:4) and that exogenous complement was not added. This epidemiological study provided the foundation for evaluation and introduction of capsular polysaccharide-based vaccines against four groups of Neisseria meningitidis (A, C, Y, W-135) despite significant procedural changes in the SBA used subsequent to the Goldschneider study. However, data now show that the complement source, bacteria strain, and other assay conditions affect the resultant SBA titers. Recent licensure of polysaccharide conjugate vaccines effective against these same pathogens was based on demonstration that SBA titers in subjects receiving the candidate vaccine were not inferior to the titers elicited by previously licensed polysaccharide vaccines, circumventing the uncertainty of the minimal protective threshold using current assays. To date, ELISA outcomes measuring serum IgG specific to the polysaccharide antigens appear not to correlate with SBA titers and thus, are not a focus for re-establishing a minimal protective serum antibody concentration[20, 21].
While further characterization of the immune responses with the goal of establishing a more accurate predictive correlate of protection may be explored, Bash pointed out that identification and definition of the immune competencies and environmental exposures that mold the responsiveness of populations are difficult to determine. Infants and adolescents appear to generate antibodies with unique protective qualities (e.g. avidity or epitope targets) to meningococcal vaccines confounding the search for a single minimal protective threshold. Additionally, Stephens noted some SBA negative individuals do not appear to be susceptible to meningococcal Group B disease[23, 24]. Thus a serologic correlate may provide incomplete information, in that a positive result may predict protection, but a negative result may have indeterminate predictive value. Thus, even in situations where a correlate has been widely used, inconsistent data may cloud interpretation and serve as a reminder that a correlative assay may not provide a complete picture of the mechanism of protection.
While developing sound predictive correlates has been possible for some vaccines, correlates for many other vaccines have not been straightforward. Meade reviewed how immunoassay data have been used effectively for the newer generation pertussis vaccines (acellular) even when the immune mechanisms responsible for protection are poorly understood. For acellular pertussis vaccines, licensing was based on a demonstration of clinical efficacy in the target population. For these vaccines, immunoassay data have been used in a limited way, namely to demonstrate consistent responses to a given product. To date, a number of different products containing an acellular pertussis component have been approved; however, these products differ substantially in antigen content and composition. Although a relationship between immune responses and clinical protection has been reported for some vaccines[27–30], no simple immune correlates or universally applicable protective thresholds have been determined.
Correspondingly, a variety of assays and reagents have been used by vaccine manufacturers and clinical investigators to evaluate these products. Thus, comparisons have been made only within any one product to compare different manufacturing batches, study populations, age groups, and immunization schedules.
The pathogens with vaccines in the development phase that were highlighted for discussion in the meeting were Group A and Group B streptococci (GAS, GBS), Neisseria meningitidis Group B, Respiratory Syncytial Virus (RSV), and influenza (flu). From these presentations (Table 1), many salient points were raised and several common themes arose.
Protective correlates will be affected by cross-reactivity of antigens found in the pathogen, in the vaccine formulation, and in the immunoassay. Dale cited the complexity associated with Group A streptococcus (GAS) where as many as 170 serotypes have been identified. Naturally acquired serum bactericidal antibodies could provide protection from infection with homologous strains, but provide little protection against heterologous strains[33, 34]. Subbarao noted that immunity to influenza infection by antigenic variants is reduced with a greater degree of antigenic variation[35–37]. Walsh indicated that the implications of strain variation on vaccine design for RSV remain unclear. The identification of protective antigens that show little or no strain variation could simplify both the development and evaluation of candidate vaccines.
The route of administration can affect the immunologic responses and determine the sampling strategy for collecting clinical specimens to be used for determination of correlates. In the case of GAS, Dale noted that while serum antibodies do protect against challenge with the homologous strain, intranasal immunization with M protein-based vaccine can prevent acquisition and clinical illness apparently without stimulating measurable serum antibodies. The need for mucosal assays to evaluate responses to vaccines against RSV was also raised by Walsh[40–42]. If mucosal antibodies correlate to protection, the appropriate assays and reagents may raise unique assay development and validation issues.
Protective correlates that apply for one vaccine formulation may not be universal for all vaccines against that same pathogen. Subbarao noted that the best correlate for influenza is the serum hemagglutination inhibition (HI) titer following immunization with inactivated vaccine formulations[43, 44]. However, following immunization with live attenuated influenza vaccines, the serum HI assay does not appear to provide the best correlative data and investigators are still searching for a predictive biomarker for protection[44, 45]. As Subbarao noted, one may need to understand the interplay of both cell mediated and humoral immunity to be able select an immunoassay predictive of vaccine effectiveness.
Animal models may be used to explore pathogenicity and responses to vaccines but the data may not be completely relevant to the human case. Walsh concluded that the micro-neutralization assay may provide the best correlative data for protection and severity from RSV disease in cotton rats; however, data from this assay is not predictive in humans[46, 47]. Dale indicated that a great deal of information on GAS candidate vaccines has been generated in mice, but the relevance to humans has not been demonstrated. Despite these limitations, much has been learned using animal models about the mechanisms and the interplay of immune responses associated with protection from infection and/or disease.
Correlative data between a serologic outcome and disease incidence may be difficult to identify because either inappropriate or insufficient laboratory data and/or clinical outcome data have been obtained. RSV is a pathogen with a high attack rate in children and in the elderly, but to date, serologic and mucosal antibody titers have not been able to discriminate between protected and susceptible populations. However, studies have shown that passively administered serum immunoglobulins as well as IgG against F protein can be protective[50–52] and the absence of neutralizing titers against RSV in sera is a risk factor for hospitalization of adults.
Even for pathogens where immunoassays and vaccines have been developed, the ability to collect clinical experience may be hampered[54, 55]. Currently, GBS vaccines are difficult to study in the United States as the disease load has been reduced because the current standard of care utilizes anti-infective drugs .
Understanding the quality and dynamics of the immune responses to a pathogen as well as to candidate vaccines are important contributors to the utility of any data generated by laboratory assays for a specific subject population. Kohberger noted that various statistical models are used for predicting protection, but most require a pairing of cases and immune response which are difficult data to acquire in clinical trials with relatively low disease incidence rates. As emphasized by both Kohberger and Kalos, without an understanding of the most appropriate laboratory assays, one must collect as much data as possible (e.g., different biological specimens and multiple time points) to enable sufficient statistical robustness to develop potential correlates which could provide insights into the biological outcomes associated with protection.
In the open discussion, panelists emphasized the importance of epidemiological and seroepidemiological studies of disease, the basic understanding of pathogenesis and the protective immune responses. For example, several talks mentioned the critical role played by landmark studies of Goldschneider and Gotschlich, as well as Fothergill and Wright whose studies helped to define the serological responses correlated with clinical protection for Group C meningococcal and Hib diseases, respectively. Such data are essential for defining the immune response to be monitored in a clinical trial, to lay the groundwork for future immunoassay development, and provide invaluable guidance for candidate vaccine development decisions.
At the initiation of a vaccine development program, data may not be available to determine which assays will be most informative or could be predictive of protection. However, even when correlates have not been established, meaningful immunoassay data can provide data critical for vaccine evaluation and development. In all cases, the immunoassays should measure the most relevant clinical outcome and should be developed in collaboration with clinicians and with statisticians to ensure that the data will have sufficient quality to address their intended purpose. The assay design considerations include pre-defining the methodological and statistical rigor needed to address the decisions that will be made based on the data.
Significant effort is required to develop each immunoassay. Unfortunately, no universal guides are available and, as noted by Burns and other presenters, the selection of assays should be based on the best science available. Jansen noted in the discussion session that one might want to explore as wide a range of immunoassays as possible in early development so that data can facilitate the selection of the preferred assay. However even as many different assays are often considered, practical considerations require investigators to identify and focus on a few limited responses that are specific to the disease/vaccine target and most likely to reflect a protective immune response. No matter which immunoassay is ultimately selected, useful data can only be generated by assays of adequate quality. As emphasized by Kalos, assay development requires a commitment on the part of the laboratory, a quality-focused mindset, an appropriate laboratory infrastructure, and an adequate investment of time and resources. Lynn pointed out that many less experienced investigators are tempted to utilize assays just because they are readily available or because robust responses are observed. She encouraged investigators to assess whether the relevant question has been defined and whether a particular assay is able to provide the data that address this question.
The initial design of the immunoassay requires consideration of many issues. For example, investigators must decide if all types of antibodies are to be measured, or just those of a specific isotype, avidity, functionality, or epitope recognition. Similarly, the clinical study population should be defined, as sera from different populations often have different behavioral characteristics and the immunoassays may need to be optimized for the common characteristics of that population (e.g., subjects age, environmental exposure to cross-reactive antigens, or presence of maternal antibodies). The specificity of the assay for detecting the antigen/infectious agent of interest is particularly important; all assay reagents should be tested to verify that they only detect the intended component (e.g., antigen-specific antibodies) in the specimen milieu (e.g., sera, saliva). A prerequisite for good assays is a thorough characterization of the critical reagents and materials and verification of their adequacy for the procedure.
After determining the best immunoassay design strategy, the assays are optimized and evaluated to demonstrate that they are acceptable for the intended use. Because an assay must perform in its entirety, Callahan recommended a statistically-based experimental design for assay optimization, called design of experiments (DOE). In this approach, the most important variables are run at 2 or 3 levels (e.g., high and low reagent concentrations, two time intervals per step) in the same assay rather than optimizing each step in sequential assays[58, 59]. Based on statistical analyses of the multiple variables, the laboratory analyst can prioritize the multiple parameters and fine-tune those that have the greatest influence on assay precision. In addition to improved efficiency, this multipronged approach allows potential reagent interactions (interfering or enhancing) to be identified more readily.
A recurring challenge is to understand the relative merits of assays that quantify antibody binding to an antigen (e.g., ELISA, HI) versus assays that measure functional activity of antibodies (e.g., neutralization, bactericidal or opsonic activity, or in vivo protection). Assays that measure a functional antibody response are often preferred because they are speculated to more closely reflect or predict the relevant protective immune response in vivo. However, several speakers pointed out that in predicting protection, there is no guarantee that any specific functional assay will yield better data than a simple binding assay. Burns noted that the anthrax toxin neutralization assay can give different results with different cell lines and with sera from different species[60, 61]. Functional assays can be potentially biased by choices of the pathogen strain, detector system cell line, complement source, and challenge dose. Investigators must keep in mind that all assays have limitations. The choice between binding and functional assays also provides an excellent example of the practical considerations and compromises that must be considered when making decisions regarding assay selection. Typically, functional assays require a larger volume of clinical specimens (e.g., serum), are more costly, are more complex and labor intensive, and provide less precise data. In some cases, laboratories have elected to perform a simpler, high-throughput binding assay on all samples and the more complex functional assay on a subset of samples. Because of these limitations, the balance between value and practicality becomes very critical; careful statistical analyses can help investigators evaluate the costs and benefits of expending the extra resources needed to perform the functional assay. Also, statistical input can help to control costs and effort by identifying the most appropriate sample size, allowing resources to be conserved by testing only the number of specimens required to answer a specific question.
Several investigators presented data comparing results generated by different assay methods. The x and y variables can represent values measured by different laboratories or different operators, and the goal of the analysis is to evaluate the level of agreement between the two sets of measurements. Plikaytis and Lynn stated that simple linear regression of x and y is not appropriate for assessing agreement between data from two different methods or laboratories because the assumptions for performing an Ordinary Least Squares regression (OLS) are not met[62, 63]. As a result, the estimated slope and intercept will not provide accurate indicators of agreement between the two methods or laboratories. One of the assumptions is that the independent variable (x) is measured without error and the dependent variable (y) is a random variable. When comparing methods, both the x and y are measured with error, thus violating this basic assumption necessary for using OLS regression. Additionally, with OLS, the x variable is considered to be the reference method and is regarded as the ‘correct’ value which can be used to ‘predict’ the y value. In a method comparisons study both x and y are random variables. There is no reason to assume that either x or y should serve as the method of reference because the two variables are interchangeable. Appropriate alternatives are available, with Plikaytis recommending a Deming regression model[62–65], and Meade and Lynn suggesting approaches described by Bland and Altman[66, 67].
Initially, immunoassay selection is based on the existing state of biology (e.g., the pathogenesis of the disease and the immune mechanisms involved in protection) and available testing technologies. A logical strategy to narrow the focus from multiple assays to the most relevant and predictive of the clinical outcome should be based on scientific data. However, practical issues must also be considered, including the ease and cost of assay performance and the availability of resources, such as equipment, reagents, and personnel. Importantly, once a candidate vaccine enters into clinical trials, new data become available to help guide subsequent decisions.
The development of a vaccine from concept through clinical testing is a multi-year endeavor and, throughout this development process, immunoassays are utilized to address different questions in order to progress to the next stage. Original licensing is usually based on a direct demonstration of clinical efficacy. Immunogenicity is important to initial licensing, but usually has greater importance and relevance for the studies and decisions that follow initial licensing. Subsequent activities would include, for example, evaluations of new candidate vaccines or changes in existing vaccine formulation. As outlined by Burns, the various studies that use immunogenicity data may cover a broad scope, including proof of concept (immune response) evaluations in animals and in humans, sero-epidemiological studies, clinical trials to select optimal vaccine formulations and preferred doses and schedules, and assessments of vaccine manufacturing and clinical performance consistency. Immunoassays may be used to evaluate the response in different age and risk groups, for new clinical indications, or among vaccine formulations from different manufacturers. The laboratory assays and expectations for the assays evolve in concert with the development of a new vaccine and the quality should be commensurate with their use at each stage.
Although assays used in early stages of product development are unlikely to be fully validated, laboratories should aim for a well-defined method based on solid science, supported by well-characterized reagents, adequate documentation, appropriate controls and pre-defined acceptance criteria. Adherence to standards of quality at all stages of clinical research process is important since data collected at all stages of the process are used to guide both assay and product development. Investigators and regulators expect that these assays will evolve and improve over time with increasingly tighter control. An assay used to support regulatory decisions or to define a correlate is held to high standards, and such assays need to be validated in a statistically robust manner for parameters such as specificity, accuracy, precision, detection and quantitation limits, and robustness as appropriate. A guidance document is available for the validation of bioanalytical assays through the FDA website. Kalos presented an overview and summarized the differences between assay qualification and validation (Tables 2 and and33).
Many research laboratories have limited experience in assay control and standardization; however, laboratories gain this experience by testing quality control specimens within each assay and monitoring the resultant data, which enables them to refine and establish acceptable assay performance criteria. Ongoing statistical analyses monitor the acceptability of assay performance and provide a mechanism to verify consistent day-to-day assay performance as well as to assess the impact of intentional changes (e.g., new reagents, personnel). Through the careful design of clinical studies and sample analysis, investigators can minimize the performance parameters that must be evaluated in validation studies. For example, Meade pointed out for the specific situation in which samples from comparison groups are assayed in the same laboratory within a narrow window of time with stringent internal monitoring and performance control, then a demonstration of comparability among laboratories or long-term stability within a laboratory is not required. However, such data can be valuable because they enable comparisons to data generated at a future date.
Assays supporting late stages of vaccine development generally require a “web of SOPs” in addition to the assay SOP. These supporting SOPs prescribe aspects of assay performance not directly described in the immunoassay SOP including equipment operations and maintenance, reagent receipt and qualification, clinical specimen access and handling, facility conditions, and personnel training. At every stage of vaccine development, investigators must define the analytical performance that is required to address a specific question, and then demonstrate with data that the assay possesses those characteristics.
As more diverse data become available, investigators gain an increased understanding of the attributes and limitations of the assays, the vaccines, the human antibody response to the vaccine, and potential protective mechanisms. These data help to identify and to optimize the most informative assays. Substantial changes may occur when investigators recognize that the assay itself either must be replaced by a new assay or significantly modified. Given the investment required to make a major modification, change often is met with understandable reluctance. Nahm described the recent progression of different generations of ELISA for quantifying IgG antibodies specific for multiple serotypes of S. pneumoniae. The development and use of fully validated assays for the measurement of antibodies in the sera of South African and American infants (including Native Americans) enabled establishment of a threshold protective antibody level[72, 73]. However, these serotype specific ELISAs required modification (i.e., an additional absorption step) when evaluating the sera from other populations, including adults and elderly, where the sera may have naturally occurring antibodies that bind to identified and/or unknown contaminating antigens. Also, as newer formulations of pneumococcal vaccines may include additional serotypes that generate cross-reactive antibodies, further assay modifications may be required. Alternative methods (e.g., opsonophagocytic) are being utilized for confirmation of pneumococcal vaccine effectiveness in clinical trials evaluating different target populations and vaccine formulations. Additionally new methods are being developed to improve the efficiency of testing due to limited volume of sera from infants and increased testing requirements for evaluating responses to more pneumococcal serotypes and concomitant vaccines. Nahm described a potential next generation immunoassay where functional antibodies to multiple pneumococcal serotypes can be tested simultaneously in the same assay well. Transitioning to more efficient assays requires bridging to previously validated methods. The use of multiple statistical analyses, across the full magnitude of the antibody responses, is required to ensure that biases are not introduced when a new method replaces an older one. Importantly, a thorough knowledge is required of what is being measured/detected in the established method as well as with the new replacement method. This understanding is critical for deciding whether a pre-established serological correlate for protection still can be applied.
The process of developing and applying an assay is a resource intensive process; resources invariably are limiting. In the panel discussion, Hildreth and Jansen emphasized the fact that large manufacturers typically have a mixture of specialty laboratories enabling different disciplines to work on the same project and for resources to be available to facilitate efficiency and timely decision making within a single organization. Academic investigators and smaller companies often face greater challenges when specific expertise and/or resources may not available or readily accessible.
A multi-disciplinary team approach to assay development was recommended by several panelists. Such an approach can lead to more efficient use of time and resources, accelerating immunoassay and candidate vaccine development. Teams commonly include clinical investigators, immunologists, microbiologists, immunoassay specialists, other laboratory scientists, and, importantly, statisticians. Individuals trained in different disciplines bring different talents and approaches to solving problems, as well as experience in addressing similar problems with different pathogens.
A recurring theme concerned the need for ongoing statistical support; a primary reason is that statistical methods applicable to immunoassays are not routine. Participants emphasized that collaboration with a statistician is invaluable for assay development, validation, and monitoring, as well as for clinical application and interpretation. While statistical support is considered essential, many laboratories find it challenging to obtain the needed statistical support. Two impediments were discussed. First, many investigators have limited knowledge in statistical analyses, and often find it difficult to communicate effectively with statisticians. Secondly, access to experienced statisticians can be difficult, in large part because few statisticians have been trained in the biological field and many are unfamiliar with the immunoassay terms and mechanisms. To overcome these challenges, Kohberger, Plikaytis, and others encouraged mutual training and communication as well as participation in multi-disciplinary meetings that bring together statisticians and laboratory scientists. Professional societies for Quality Control and for Clinical Pathology may provide a resource, as these groups face similar issues which may readily translate into the immunoassay field.
Within a laboratory, in-house reference and quality control sera are an essential resource during the assay development process. Commonly, as vaccine development proceeds, an assay may be transferred to other laboratories or the results from different laboratories must be compared. In order for laboratories to generate comparable data, efforts must be undertaken to share protocols, reagents, and control samples, and for laboratories to participate in inter-laboratory studies. Such comparability requires efforts for harmonization and standardization. While a fully standardized assay procedure might be advantageous, harmonization may be more achievable and reasonable than requiring all laboratories to utilize a common standardized method with the same source of reagents and materials. Specifically, use of the same method by different laboratories may not be necessary as long as results are demonstrated to be comparable by statistical analyses of data generated by testing appropriately representative control samples and proficiency panels (i.e., previously characterized specimens).
An important first step to the harmonization or standardization of any assay is the creation and characterization of reference sera that are used to quantify antibodies to critical antigens and can be shared among laboratories. Such reagents enable the establishment of minimum protective thresholds and to aid in the evaluation of new vaccine formulations, as in the case of vaccines for Hib and S. pneumoniae[16, 73, 76]. Subbarao cited recent observations evaluating the HI and neutralization assay outcomes from eleven laboratories testing sera from subjects immunized with influenza H3N2 vaccine. The inclusion of a standard reference serum greatly reduced the data variability. For all pathogens for which assays were presented, the speakers expressed the need for standardization, in particular for reference sera and critical reagents, such as well characterized antigens and cell strains. Several reference sera have been prepared and characterized to facilitate inter-laboratory quantitation of antibodies to RSV and to GBS, but they have not been used extensively[78, 79].
When additional antigens or different antigens are included inw a vaccine formulation to stimulate immune responses via multiple pathways while still targeting a single pathogen, the identification of protective correlates in any single reference serum presents challenges. One example, cited by Edwards was that some GBS candidate vaccines may utilize carrier proteins as well as capsular polysaccharides from the same pathogen, potentially augmenting the protective response[80–82]. All relevant immunologic responses to the vaccine formulation need to be measured. Meade pointed out that similar issues apply to acellular pertussis vaccines which are clinically effective and yet are heterogeneous with respect to protein antigen composition.
Several examples of the importance of other critical reagents were provided during the meeting. Subbarao presented issues regarding the HI assay for determining titers against influenza virus. Red blood cells (RBC) are the target indicator used to quantify antibodies to assess immunogenicity of influenza vaccines. However, the cell surface sialic acid residues linkages in RBCs differ by species source, with α2–3 and α2–6 linkages in RBCs from turkeys and a α2–3 linkage in RBCs from horses or geese. Avian influenza viruses generally bind to α2–3 linked sialic acids and human influenza viruses bind to α2–6 linked sialic acids. The source of the RBC used in the HI assay can influence the observed titer because the vaccine antigen may not bind RBCs with different sialic acid binding specificities equally well. Likewise, the titer assigned to clinical sera in the serum bactericidal assay for assessing antibodies to meningococcal strains is affected by the source of complement used in the assay. As described by Bash, rabbit complement generally enhances the serum bactericidal assay titers when compared to the same assay run using human complement that is endogenous or from an agammaglobulinemic donor. While correlations between titers and protection can be developed, the predictive threshold associated with protection may differ based on the immunoassay components.
The use of standardized reagents in conjunction with inter-laboratory collaborations can facilitate the characterization of an immunoassay and reduce duplication of effort and data. Lynn emphasized that a statistical plan, identifying the number and selection of specimens as well as the number of replicates, is needed to generate data to confirm that assay results from different laboratories can be compared. Such interactions by immunoassay experts have led to excellent degree of quantitative agreement in several vaccine fields, including S. pneumoniae[86, 87] and cancer[88, 89].
While often not considered during discussions of critical reagents, the panel emphasized the importance of the quality of the collected clinical specimen. The biological integrity of the specimens must be maintained throughout the collection, testing, and storage conditions. Unrecognized problems (e.g., inactivation methods, additives) can potentially yield false data regarding a vaccine formulation, confound the discovery of clinical correlates, as well as delay the potential licensure of an effective vaccine.
Although the benefits of standardization and harmonization efforts are clear, questions were raised during the panel discussion regarding the strategy for accomplishing this, with emphasis on clarifying the responsibility for management of these activities. Discussion revealed that no one approach has been applied for all candidate or licensed vaccines. For less mature vaccines, such activities are initiated by interested and involved research laboratories that create and share assays and reagents, hold meetings, and create working groups to promote standardization and harmonization efforts. These efforts often coalesce into a more coordinated effort as the candidate vaccine matures. In many cases, a single laboratory (or group of laboratories) will assume the position of a reference laboratory, and will take the lead role in preparing, evaluating, and distributing reference methods and reagents. Different types of laboratories have served this purpose including academic laboratories (for GBS and pneumococcus), government agencies (for Hib, pertussis, and meningococcus), The World Health Organization (for pneumococcus and pertussis), or a vaccine developer (for pneumococcus and RSV). Finding and obtaining the resources for these activities remains a challenge, particularly for academic laboratories.
A potential obstacle to harmonization is that for some products a specific vaccine developer may have proprietary assays or reagents that are used for assessment of their vaccine; in such cases, collaborative agreements may be required. Another practical challenge has arisen out of the need for clinical studies to evaluate potential interactions when a new vaccine is given concurrently with other licensed vaccines. In most studies, a vaccine developer evaluates the interactions with licensed vaccines from other manufacturers; however, a developer may not have access to all the assays needed for a concurrent immunization study involving multiple ccines. While there are no requirements that assay reagents and methods be made public, regulatory agencies do have access to proprietary information which they may use to verify assay outcomes. To overcome this obstacle in the research community, one vaccine developer described efforts to establish a “sustainable commercial home” for assays needed for such concomitant immunization studies and encouraged industry, government, and academic groups to utilize and support such a specialty laboratory.
Lynn encouraged the meeting participants to share their interests, activities, and challenges with regard to their respective immunoassay work. Collaborations with experienced investigators and publishing positive as well as negative outcomes can facilitate assay development by minimizing duplicate efforts as well as providing a forum for exchange of concepts, data, and reagents.
Although the meeting focused on antibody assays, immunological assays to measure cell-mediated immune (CMI) responses are also critically important to the assessment of many vaccines. Elkins noted that the state of knowledge related to CMI assays is less mature than for serum antibody assays. Analogous to the discussion above comparing functional and binding assays, many investigators believe that CMI-related assays are inherently better than antibody assays for many products; however, direct data supporting this conclusion are minimal in most systems. For example, ELISPOT assays have become very popular; however, their value for driving decisions on vaccine development and licensing remains to be demonstrated. Recommendations were made to hold workshops for sharing experiences with CMI assays analogous to this meeting for antibody assays. Some consortia are actively meeting to harmonize and improve the quality and consistency of CMI assays for cancer and HIV vaccines through sharing of proficiency panels[88, 89]. Non-governmental organizations and foundations have provided substantial financial support for such activities. These types of meetings help investigators expand their knowledge base as well as maintain a realistic understanding of the value and the limitations of each assay.
This meeting was organized to bring together investigators from diverse disciplines who are working with different pathogens at various stages of vaccine development. The goal was to encourage communication and explore common experiences with the development, implementation, and potential interpretation of data from serologic assays. One of the overarching goals of the meeting was to facilitate efficient immunoassay development for candidate vaccines by learning lessons from more mature products. The presentations and discussions at the workshop illustrated that many challenges are common across various pathogens and revealed several recurring themes that merit consideration by those who develop or use immunoassays. Throughout the meeting, the need to clearly define the purpose and intended use of the assays before assay selection and development was stressed. The discussions also underscored that the most useful assays were based on a solid scientific foundation provided by a thorough understanding of the pathogen and the host response to disease and immunization. A practical consideration raised was that assays progress most efficiently when individuals from different disciplines are brought together as part of the assay development team. Additionally, immunoassay development should be viewed as a dynamic process; specifically, the assay and the purposes for which it is used evolve during phases of product development.
While the meeting focused on assays used for vaccine studies, the lessons learned are relevant for immunoassays used for other purposes such as therapeutics and diagnostics. The meeting provided a valuable forum for facilitating communication among laboratories and disciplines, and many participants encouraged the creation of additional opportunities for exchange of scientific experiences.
This meeting was funded by the Division of Microbiology and Infectious Diseases, National Institute of Allergy and Infectious Diseases, National Institutes of Health.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.