This report summarizes the proceedings of the second workshop of the ‘Minimum Information for Biological and Biomedical Investigations’ (MIBBI) consortium held on Dec 1-2, 2010 in Rüdesheim, Germany through the sponsorship of the Beilstein-Institute. MIBBI is an umbrella organization uniting communities developing Minimum Information (MI) checklists to standardize the description of data sets, the workflows by which they were generated and the scientific context for the work. This workshop brought together representatives of more than twenty communities to present the status of their MI checklists and plans for future development. Shared challenges and solutions were identified and the role of MIBBI in MI checklist development was discussed. The meeting featured some thirty presentations, wide-ranging discussions and breakout groups. The top outcomes of the two-day workshop as defined by the participants were: 1) the chance to share best practices and to identify areas of synergy; 2) defining a series of tasks for updating the MIBBI Portal; 3) reemphasizing the need to maintain independent MI checklists for various communities while leveraging common terms and workflow elements contained in multiple checklists; and 4) revision of the concept of the MIBBI Foundry to focus on the creation of a core set of MIBBI modules intended for reuse by individual MI checklist projects while maintaining the integrity of each MI project. Further information about MIBBI and its range of activities can be found at http://mibbi.org/.
Ex vivo ELISPOT and multimer staining are well-established tests for the assessment of antigen-specific T cells. Many laboratories are now using a period of in vitro stimulation (IVS) to enhance detection. Here, we report the findings of a multi-centre panel organised by the Association for Cancer Immunotherapy Immunoguiding Program to investigate the impact of IVS protocols on the detection of antigen-specific T cells of varying ex vivo frequency. Five centres performed ELISPOT and multimer staining on centrally prepared PBMCs from 3 donors, both ex vivo and following IVS. A harmonised IVS protocol was designed based on the best-performing protocol(s), which was then evaluated in a second phase on 2 donors by 6 centres. All centres were able to reliably detect antigen-specific T cells of high/intermediate frequency both ex vivo (Phase I) and post-IVS (Phase I and II). The highest frequencies of antigen-specific T cells ex vivo were mirrored in the frequencies following IVS and in the detection rates. However, antigen-specific T cells of a low/undetectable frequency ex vivo were not reproducibly detected post-IVS. Harmonisation of the IVS protocol reduced the inter-laboratory variation observed for ELISPOT and multimer analyses by approximately 20 %. We further demonstrate that results from ELISPOT and multimer staining correlated after (P < 0.0001 and R2 = 0.5113), but not before IVS. In summary, IVS was shown to be a reproducible method that benefitted from method harmonisation.
Electronic supplementary material
The online version of this article (doi:10.1007/s00262-014-1593-0) contains supplementary material, which is available to authorized users.
T cell; In vitro stimulation; ELISPOT; Multimer; Harmonisation; Inter-laboratory testing
MIATA; T cell assay
Flow cytometry is the prototypical assay for multi-parameter single cell analysis, and is essential in vaccine and biomarker research for the enumeration of antigen-specific lymphocytes that are often found in extremely low frequencies (0.1% or less). Standard analysis of flow cytometry data relies on visual identification of cell subsets by experts, a process that is subjective and often difficult to reproduce. An alternative and more objective approach is the use of statistical models to identify cell subsets of interest in an automated fashion. Two specific challenges for automated analysis are to detect extremely low frequency event subsets without biasing the estimate by pre-processing enrichment, and the ability to align cell subsets across multiple data samples for comparative analysis. In this manuscript, we develop hierarchical modeling extensions to the Dirichlet Process Gaussian Mixture Model (DPGMM) approach we have previously described for cell subset identification, and show that the hierarchical DPGMM (HDPGMM) naturally generates an aligned data model that captures both commonalities and variations across multiple samples. HDPGMM also increases the sensitivity to extremely low frequency events by sharing information across multiple samples analyzed simultaneously. We validate the accuracy and reproducibility of HDPGMM estimates of antigen-specific T cells on clinically relevant reference peripheral blood mononuclear cell (PBMC) samples with known frequencies of antigen-specific T cells. These cell samples take advantage of retrovirally TCR-transduced T cells spiked into autologous PBMC samples to give a defined number of antigen-specific T cells detectable by HLA-peptide multimer binding. We provide open source software that can take advantage of both multiple processors and GPU-acceleration to perform the numerically-demanding computations. We show that hierarchical modeling is a useful probabilistic approach that can provide a consistent labeling of cell subsets and increase the sensitivity of rare event detection in the context of quantifying antigen-specific immune responses.
The use of flow cytometry to count antigen-specific T cells is essential for vaccine development, monitoring of immune-based therapies and immune biomarker discovery. Analysis of such data is challenging because antigen-specific cells are often present in frequencies of less than 1 in 1,000 peripheral blood mononuclear cells (PBMC). Standard analysis of flow cytometry data relies on visual identification of cell subsets by experts, a process that is subjective and often difficult to reproduce. Consequently, there is intense interest in automated approaches for cell subset identification. One popular class of such automated approaches is the use of statistical mixture models. We propose a hierarchical extension of statistical mixture models that has two advantages over standard mixture models. First, it increases the ability to detect extremely rare event clusters that are present in multiple samples. Second, it enables direct comparison of cell subsets by aligning clusters across multiple samples in a natural way arising from the hierarchical formulation. We demonstrate the algorithm on clinically relevant reference PBMC samples with known frequencies of CD8 T cells engineered to express T cell receptors specific for the cancer-testis antigen (NY-ESO-1) and compare its performance with other popular automated analysis approaches.
Robust and sensitive ELISPOT protocols are commonly applied concomitant with the development of new immunotherapeutics. Despite the knowledge that individual serum batches differ in their composition and may change properties over time, serum is still commonly used in immunologic assays. Commercially available serum batches are expensive, limited in quantity and need to be pretested for suitability in immunologic assays, which is a laborious process. The aim of this study was to test whether serum-free freezing media can lead to high cell viability and favorable performance across multiple ELISPOT assay protocols. Thirty-one laboratories from ten countries participated in a proficiency panel organized by the Cancer Immunotherapy Immunoguiding Program to test the influence of different freezing media on cell quality and immunologic function. Each center received peripheral blood mononuclear cells which were frozen in three different media. The participants were asked to quantify antigen-specific CD8+ T-cell responses against model antigens using their locally established IFN-gamma ELISPOT protocols. Self-made and commercially available serum-free freezing media led to higher cell viability and similar cell recovery after thawing and resting compared to freezing media supplemented with human serum. Furthermore, the test performance as determined by (1) background spot production, (2) replicate variation, (3) frequency of detected antigen-specific spots and (4) response detection rate was similar for serum and serum-free conditions. We conclude that defined and accessible serum-free freezing media should be recommended for freezing cells stored for subsequent ELISPOT analysis.
Electronic supplementary material
The online version of this article (doi:10.1007/s00262-012-1359-5) contains supplementary material, which is available to authorized users.
ELISPOT; Cryopreservation; Serum-free; Assay harmonization
Prediction of clinical outcome in cancer is usually achieved by histopathological evaluation of tissue samples obtained during surgical resection of the primary tumor. Traditional tumor staging (AJCC/UICC-TNM classification) summarizes data on tumor burden (T), presence of cancer cells in draining and regional lymph nodes (N) and evidence for metastases (M). However, it is now recognized that clinical outcome can significantly vary among patients within the same stage. The current classification provides limited prognostic information, and does not predict response to therapy. Recent literature has alluded to the importance of the host immune system in controlling tumor progression. Thus, evidence supports the notion to include immunological biomarkers, implemented as a tool for the prediction of prognosis and response to therapy. Accumulating data, collected from large cohorts of human cancers, has demonstrated the impact of immune-classification, which has a prognostic value that may add to the significance of the AJCC/UICC TNM-classification. It is therefore imperative to begin to incorporate the ‘Immunoscore’ into traditional classification, thus providing an essential prognostic and potentially predictive tool. Introduction of this parameter as a biomarker to classify cancers, as part of routine diagnostic and prognostic assessment of tumors, will facilitate clinical decision-making including rational stratification of patient treatment. Equally, the inherent complexity of quantitative immunohistochemistry, in conjunction with protocol variation across laboratories, analysis of different immune cell types, inconsistent region selection criteria, and variable ways to quantify immune infiltration, all underline the urgent requirement to reach assay harmonization. In an effort to promote the Immunoscore in routine clinical settings, an international task force was initiated. This review represents a follow-up of the announcement of this initiative, and of the J Transl Med. editorial from January 2012. Immunophenotyping of tumors may provide crucial novel prognostic information. The results of this international validation may result in the implementation of the Immunoscore as a new component for the classification of cancer, designated TNM-I (TNM-Immune).
The validation of assays that quantify antigen-specific T cell responses is critically dependent on cell samples that contain clearly defined measurable numbers of antigen-specific T cells. An important requirement is that such cell samples are handled and analyzed in a comparable fashion to peripheral blood mononuclear cells (PBMC). We performed a proof-of-principle study to show that retrovirally TCR-transduced T cells spiked at defined numbers in autologous PBMC can be used as standard samples for HLA/peptide multimer staining. NY-ESO-1157–165-specific, TCR-transduced CD8+ T cell batches were successfully generated from PBMC of several HLA-A*0201 healthy donors, purified by magnetic cell sorting on the basis of HLA tetramer (TM) staining and expanded with specific antigen in vitro. When subsequently spiked into autologous PBMC, the detection of these CD3+CD8+TM+ T cells was highly accurate with a mean accuracy of 91.6 %. The standard cells can be preserved for a substantial period of time in liquid nitrogen. Furthermore, TM staining of fresh and cryopreserved standard samples diluted at decreasing concentrations into autologous cryopreserved unspiked PBMC revealed that the spiked CD3+CD8+TM+ T cells could be accurately detected at all dilutions in a linear fashion with a goodness-of-fit of over 0.99 at a frequency of at least 0.02 % among the CD3+CD8+ T cell population. Notably, the CD3+CD8+TM+ cells of the standard samples were located exactly within the gates used to analyze patient samples and displayed a similar scatter pattern. The performance of the cryopreserved standard samples in the hands of 5 external investigators was good with an inter-laboratory variation of 32.9 % and the doubtless identification of one outlier.
Electronic supplementary material
The online version of this article (doi:10.1007/s00262-012-1351-0) contains supplementary material, which is available to authorized users.
Immunomonitoring; HLA/peptide multimer staining; T cells; Immunotherapy; Assay controls
Active immunotherapy for cancer is an accepted treatment modality aiming to reinforce the T-cell response to cancer. T-cell reactivity is measured by various assays and used to guide the clinical development of immunotherapeutics. However, data obtained across different institutions may vary substantially making comparative conclusions difficult. The Cancer Immunotherapy Immunoguiding Program organizes proficiency panels to identify key parameters influencing the outcome of commonly used T-cell assays followed by harmonization. Our successes with IFNγ-ELISPOT and peptide HLA multimer analysis have led to the current study on intracellular cytokine staining (ICS). We report the results of three successive panels evaluating this assay. At the beginning, 3 out of 9 participants (33 %) were able to detect >6 out of 8 known virus-specific T-cell responses in peripheral blood of healthy individuals. This increased to 50 % of the laboratories in the second phase. The reported percentages of cytokine-producing T cells by the different laboratories were highly variable with coefficients of variation well over 60 %. Variability could partially be explained by protocol-related differences in background cytokine production leading to sub-optimal signal-to-noise ratios. The large number of protocol variables prohibited identification of prime guidelines to harmonize the assays. In addition, the gating strategy used to identify reactive T cells had a major impact on assay outcome. Subsequent harmonization of the gating strategy considerably reduced the variability within the group of participants. In conclusion, we propose that first basic guidelines should be applied for gating in ICS experiments before harmonizing assay protocol variables.
Electronic supplementary material
The online version of this article (doi:10.1007/s00262-012-1282-9) contains supplementary material, which is available to authorized users.
T cells; Intracellular cytokine staining; Flow cytometry; Proficiency panel; Harmonization
To facilitate development of innovative immunotherapy approaches, especially for treatment concepts exploiting the potential benefits of personalized therapy, there is a need to develop and validate tools to identify patients who can benefit from immunotherapy. Despite substantial effort, we do not yet know which parameters of anti-tumor immunity to measure and which assays are optimal for those measurements.
The iSBTc-SITC, FDA and NCI partnered to address these issues for immunotherapy of cancer. Here, we review the major challenges, give examples of approaches and solutions and present our recommendations.
Results and Conclusions
While specific immune parameters and assays are not yet validated, we recommend following standardized (accurate, precise and reproducible) protocols and use of functional assays for the primary immunologic readouts of a trial; consideration of central laboratories for immune monitoring of large, multi-institutional trials; and standardized testing of several phenotypic and functional potential potency assays specific to any cellular product. When reporting results, the full QA/QC performed, selected examples of truly representative raw data and assay performance characteristics should be included. Lastly, to promote broader analysis of multiple aspects of immunity, and gather data on variability, we recommend that in addition to cells and serum, that RNA and DNA samples be banked (under standardized conditions) for later testing. We also recommend that sufficient blood be drawn to allow for planned testing of the primary hypothesis being addressed in the trial, and that additional baseline and post-treatment blood is banked for testing novel hypotheses (or generating new hypotheses) that arise in the field.
Immunotherapy; Immune Monitoring; Vaccines; iSBTc; SITC
Scientific discoveries that provide strong evidence of antitumor effects in preclinical models often encounter significant delays before being tested in patients with cancer. While some of these delays have a scientific basis, others do not. We need to do better. Innovative strategies need to move into early stage clinical trials as quickly as it is safe, and if successful, these therapies should efficiently obtain regulatory approval and widespread clinical application. In late 2009 and 2010 the Society for Immunotherapy of Cancer (SITC), convened an "Immunotherapy Summit" with representatives from immunotherapy organizations representing Europe, Japan, China and North America to discuss collaborations to improve development and delivery of cancer immunotherapy. One of the concepts raised by SITC and defined as critical by all parties was the need to identify hurdles that impede effective translation of cancer immunotherapy. With consensus on these hurdles, international working groups could be developed to make recommendations vetted by the participating organizations. These recommendations could then be considered by regulatory bodies, governmental and private funding agencies, pharmaceutical companies and academic institutions to facilitate changes necessary to accelerate clinical translation of novel immune-based cancer therapies. The critical hurdles identified by representatives of the collaborating organizations, now organized as the World Immunotherapy Council, are presented and discussed in this report. Some of the identified hurdles impede all investigators; others hinder investigators only in certain regions or institutions or are more relevant to specific types of immunotherapy or first-in-humans studies. Each of these hurdles can significantly delay clinical translation of promising advances in immunotherapy yet if overcome, have the potential to improve outcomes of patients with cancer.
Recent positive clinical results in cancer immunotherapy point to the potential of immune-based strategies to provide effective treatment of a variety of cancers. In some patients, the responses to cancer immunotherapy are durable, dramatically extending survival. Extensive research efforts are being made to identify and validate biomarkers that can help identify subsets of cancer patients that will benefit most from these novel immunotherapies. In addition to the clear advantage of such predictive biomarkers, immune biomarkers are playing an important role in the development, clinical evaluation and monitoring of cancer immunotherapies. This Cancer Immunotherapy Resource Document, prepared by the Society for Immunotherapy of Cancer (SITC, formerly the International Society for Biological Therapy of Cancer, iSBTc), provides key references and online resources relevant to the discovery, evaluation and clinical application of immune biomarkers. These key resources were identified by experts in the field who are actively pursuing research in biomarker identification and validation. This organized collection of the most useful references, online resources and tools serves as a compass to guide discovery of biomarkers essential to advancing novel cancer immunotherapies.
The introduction of antibody markers to identify undesired cell populations in flow-cytometry based assays, so called DUMP channel markers, has become a practice in an increasing number of labs performing HLA-peptide multimer assays. However, the impact of the introduction of a DUMP channel in multimer assays has so far not been systematically investigated across a broad variety of protocols.
The Cancer Research Institute's Cancer Immunotherapy Consortium (CRI-CIC) conducted a multimer proficiency panel with a specific focus on the impact of DUMP channel use. The panel design allowed individual laboratories to use their own protocol for thawing, staining, gating, and data analysis. Each experiment was performed twice and in parallel, with and without the application of a dump channel strategy.
The introduction of a DUMP channel is an effective measure to reduce the amount of non-specific MULTIMER binding to T cells. Beneficial effects for the use of a DUMP channel were observed across a wide range of individual laboratories and for all tested donor-antigen combinations. In 48% of experiments we observed a reduction of the background MULTIMER-binding. In this subgroup of experiments the median background reduction observed after introduction of a DUMP channel was 0.053%.
We conclude that appropriate use of a DUMP channel can significantly reduce background staining across a large fraction of protocols and improve the ability to accurately detect and quantify the frequency of antigen-specific T cells by multimer reagents. Thus, use of a DUMP channel may become crucial for detecting low frequency antigen-specific immune responses. Further recommendations on assay performance and data presentation guidelines for publication of MULTIMER experimental data are provided.
No consensus has been reached on how to determine if an immune response has been detected based on raw data from an ELISPOT assay. The goal of this paper is to enable investigators to understand and readily implement currently available methods for response determination. We describe empirical and statistical approaches, identifying the strengths and limitations of each approach to allow readers to rationally select and apply a scientifically sound method appropriate to their specific laboratory setting. Five representative approaches were applied to data sets from the CIMT Immunoguiding Program and the response detection and false positive rates were compared. Simulation studies were also performed to compare empirical and statistical approaches. Based on these, we recommend the use of a non-parametric statistical test. Further, we recommend that six medium control wells or four wells each for both medium control and experimental conditions be performed to increase the sensitivity in detecting a response, that replicates with large variation in spot counts be filtered out, and that positive responses arising from experimental spot counts below the estimated limit of detection be interpreted with caution. Moreover, a web-based user interface was developed to allow easy access to the recommended statistical methods. This interface allows the user to upload data from an ELISPOT assay and obtain an output file of the binary responses.
Electronic supplementary material
The online version of this article (doi:10.1007/s00262-010-0875-4) contains supplementary material, which is available to authorized users.
ELISPOT assay; Replicate variation; Background spot production; Positive response criteria; Harmonization
The Cancer Vaccine Consortium of the Sabin Vaccine Institute (CVC/SVI) is conducting an ongoing large-scale immune monitoring harmonization program through its members and affiliated associations. This effort was brought to life as an external validation program by conducting an international Elispot proficiency panel with 36 laboratories in 2005, and was followed by a second panel with 29 participating laboratories in 2006 allowing for application of learnings from the first panel. Critical protocol choices, as well as standardization and validation practices among laboratories were assessed through detailed surveys. Although panel participants had to follow general guidelines in order to allow comparison of results, each laboratory was able to use its own protocols, materials and reagents. The second panel recorded an overall significantly improved performance, as measured by the ability to detect all predefined responses correctly. Protocol choices and laboratory practices, which can have a dramatic effect on the overall assay outcome, were identified and lead to the following recommendations: (A) Establish a laboratory SOP for Elispot testing procedures including (A1) a counting method for apoptotic cells for determining adequate cell dilution for plating, and (A2) overnight rest of cells prior to plating and incubation, (B) Use only pre-tested serum optimized for low background: high signal ratio, (C) Establish a laboratory SOP for plate reading including (C1) human auditing during the reading process and (C2) adequate adjustments for technical artifacts, and (D) Only allow trained personnel, which is certified per laboratory SOPs to conduct assays. Recommendations described under (A) were found to make a statistically significant difference in assay performance, while the remaining recommendations are based on practical experiences confirmed by the panel results, which could not be statistically tested. These results provide initial harmonization guidelines to optimize Elispot assay performance to the immunotherapy community. Further optimization is in process with ongoing panels.
Elispot; Proficiency panel; Validation; Harmonization; Immune monitoring
Fluorescence-labeled peptide-MHC class I multimers serve as ideal tools for the detection of antigen-specific T cells by flow cytometry, enabling functional and phenotypical characterization of specific T cells at the single cell level. While this technique offers a number of unique advantages, MHC multimer reagents can be difficult to handle in terms of stability and quality assurance. The stability of a given fluorescence-labeled MHC multimer complex depends on both the stability of the peptide-MHC complex itself and the stability of the fluorochrome. Consequently, stability is difficult to predict and long-term storage is generally not recommended. We investigated here the possibility of cryopreserving MHC multimers, both in-house produced and commercially available, using a wide range of peptide-MHC class I multimers comprising virus and cancer-associated epitopes of different affinities presented by various HLA-class I molecules. Cryopreservation of MHC multimers was feasible for at least 6 months, when they were dissolved in buffer containing 5–16% glycerol (v/v) and 0.5% serum albumin (w/v). The addition of cryoprotectants was tolerated across three different T-cell staining protocols for all fluorescence labels tested (PE, APC, PE-Cy7 and Quantum dots). We propose cryopreservation as an easily implementable method for stable storage of MHC multimers and recommend the use of cryopreservation in long-term immunomonitoring projects, thereby eliminating the variability introduced by different batches and inconsistent stability. © 2014 International Society for Advancement of Cytometry
MHC multimer; cryopreservation; cryoprotectant; recommendations for MHC multimer storage; quality assurance; glycerol in T cell staining