Statistical methodologies used in this investigation have been applied previously in similar studies. In characterizing a new human pneumococcal standard reference serum, Goldblatt et al. (12
) performed a study bridging the new reference sera (007sp) with the one in current use (89SF). Five laboratories performed the WHO reference pneumococcal ELISAs on a panel of 12 WHO calibration sera for 13 pneumococcal serotypes. Kapasi et al. (17
) enrolled four laboratories to perform a comparative study of four different sources of pertussis toxin (PT) in an IgG anti-PT ELISA. Rose et al. (22
) examined the level of agreement among six laboratories measuring antibody-mediated killing of Streptococcus pneumoniae
(pneumococcus) by phagocytes using each laboratory's own optimized opsonophagocytic assay (OPA) on a panel of 16 WHO calibration reference sera for 13 pneumococcal serotypes. In previous studies (12
), concordance correlation results were very good since the laboratory-to-laboratory rc
values exceeded 0.96 and 0.97, respectively, among all laboratories. In the study by Kapasi et al. (17
), due to the lack of assay standardization and the inherent variability of OPA, concordance was somewhat reduced: laboratory-to-laboratory rc
values ranged from 0.67 to 0.99 for all six laboratories with five of the six laboratories exhibiting an rc
The laboratory-to-laboratory rc values measured during the present interlaboratory standardization of the MATS ELISA exceeded 0.99 for all labs, indicating an excellent agreement within and across all participating laboratories for all three antigens, which was equal to or higher than that observed in the similar studies mentioned above.
Of note, no predefined gold-standard RP values were used for the MenB strains examined in this study, and the ANOVA mixed model provided a mechanism for estimating consensus values that served as assigned values for each strain for the duration of the study. The choice to use a consensus value rather than establishing a definitive RP in the laboratory where MATS was initially developed reflects the desire to develop a real-world assay whose standardization and qualification criteria are co-owned by the laboratories that perform it.
Mixed-model analysis of variance allowed us to partition the total variance to measure reproducibility and repeatability. The within-laboratory variation was 2 to 2.5 times higher than the between-laboratory variation for each antigen, indicating a very good level of assay standardization across laboratories but also a significant level of intrinsic variability in the assay. Within-strain CVs measured for each antigen show that variability is slightly increased for low RPs and that CVs varied significantly from strain to strain, but also that, for each antigen, the less variable strain had a CV higher than the between-laboratory CV. Taken together, these results suggest that, even though strain-specific characteristics and the biochemistry of the assay at low concentrations may have an impact on MATS variability, a major source of assay variation is intrinsic to the experimental procedure and may be associated with the quantification of the bacterial suspension that is processed in the assay due to the biological variability of bacterial growth.
One interesting and unanticipated finding observed here was that a small number of bacteria survived Empigen lysis at room temperature in some collaborating laboratories. Exposure to live meningococci is an unacceptable health risk to laboratory workers; therefore, the lysate preparation temperature was adjusted to ensure the safety of MATS operators. This effect is probably strain dependent and indicates that safety precautions need to be evaluated for larger panels of meningococcal strains. Subsequent analysis showed that changing the lysate preparation temperature to 45°C had no significant effect on RPs and supported combining data from all of the assays in this study.
One important goal of the present study was to provide additional national laboratories with an opportunity to obtain the MATS ELISA without additional interlaboratory studies. Therefore, a set of qualification criteria were established to permit a laboratory to test its proficiency in MATS relative to the laboratories (A to G) standardized here. A volunteer national laboratory (laboratory H) followed the qualification procedure and obtained MATS ELISA results that were highly consistent with those in the seven standardized laboratories. As additional laboratories continue to provide assay data, the qualification criteria presented here may be refined.
Strain coverage is a critical component in estimating the potential clinical effects of vaccines against MenB, a pathogen characterized by dynamic mutability and strain epidemiology, as well as a propensity among its more virulent encapsulated strains to cause prolonged epidemic disease. Due to this diversity, the amount of serum required for testing using the existing correlate of protection, the hSBA, is prohibitively high, particularly in trials of infants. In a previous study (7
), a minimum MATS RP, the PBT, was established to indicate that a given MenB strain was susceptible to killing in the SBA by antibodies induced by 4CMenB. In the present study, a heuristic method based on interlaboratory variation was developed to derive 95% CIs for strain coverage estimates. The results presented here demonstrate that the MATS RPs produced in each standardized or qualified laboratory can be compared to the PBTs defined in one of the participating laboratories to obtain estimates of vaccine strain coverage that are consistent among them, within the 95% intervals defined.
The PBTs used to define vaccine strain coverage were derived by comparing MATS to pooled hSBA titers on a panel of 57 MenB strains. The use of a broader strain panel and comparison to hSBA data from individual subjects may further support the use of MATS to assess MenB strain coverage independent of clinical sera. This might allow clinical trials to predict vaccine efficacy using a limited number of strains. In addition, the extremely good agreement observed here among different laboratories suggests that the technology adopted in the MATS ELISA could be successfully used to perform similar testing with different antigens or pathogens, providing a general platform for bacterial antigen phenotyping.
A possible area for future investigation with MATS is as a means of postimplementation surveillance of the genetic profiles of fHbp, NadA, and NHBA on circulating meningococcal strains, allowing reference laboratories to monitor the antigenic profiles of pathogenic isolates in real time (24
). In addition, MATS might be used to assess the actual nature of potential vaccine failures, which has added importance for outer membrane protein vaccines against MenB, given that not all circulating strains will necessarily be covered. Postimplementation surveillance data based on a standardized assay could allow an indirect comparison of immunization policies across countries and regions, providing a valuable basis for rapid adaptation of public health policies based on worldwide quantitative data.
In summary, the results reported indicate that MATS is a standardized, reproducible antigen typing system that robustly predicts 4CMenB strain coverage in different geographical regions. These results suggest that MATS may have utility in epidemiologic surveillance of meningococci and could be adapted for assessing other pathogens.