|Home | About | Journals | Submit | Contact Us | Français|
To develop a clinically feasible stratification strategy for pediatric septic shock using gene expression mosaics and a 100-gene signature representing the first 24 hours of admission to the pediatric intensive care unit.
Prospective observational study involving microarray-based bioinformatics.
Multiple pediatric intensive care units in the United States.
Ninety-eight children with septic shock.
None other than standard care.
Patients were classified into 3 previously published, genome-wide, expression-based sub-classes (sub-classes A, B, and C) having clinically relevant phenotypic differences. The class-defining 100-gene signature was depicted for each individual patient using mosaics generated by the Gene Expression Dynamics Inspector (GEDI). Composite mosaics were generated representing the average expression patterns for each of the 3 sub-classes. Nine individual clinicians served as blinded evaluators. Each evaluator was shown the 98 individual patient mosaics and asked to classify each patient into one of the three sub-classes using the composite mosaics as the reference point. The respective sensitivities, specificities, positive predictive values, and negative predictive values of the sub-classification strategy were ≥84% across the 3 sub-classes. The classification strategy also generated positive likelihood ratios ≥16.8 and negative likelihood ratios ≤0.2 across the three sub-classes. The kappa coefficient across all possible inter-evaluator comparisons was 0.81.
We have provided initial evidence (proof of concept) for a clinically feasible and robust stratification strategy for pediatric septic shock based on a 100 gene signature and gene expression mosaics.
Human septic shock is a complex syndrome displaying a high degree of heterogeneity [1–3]. Our inability to manage and account for this heterogeneity is a major barrier to the conduct of more effective clinical trials. Indeed, multiple interventional clinical trials have been attempted in human septic shock and the majority of these trials have failed to demonstrate efficacy despite being founded on quality pre-clinical data and sound biological principles. It has been proposed that these repeated failures reflect our relative inability to address the profound heterogeneity of septic shock, rather than intrinsic flaws of the biological principles being tested [1, 4]. Robust illness severity scores are currently available for critically ill pediatric and adult populations, but it is clear that these scores are not appropriate tools to stratify individual patients for the purposes of clinical trials [5, 6]. Thus, there remains a need to develop more effective, clinical stratification/staging strategies for septic shock to meet the needs of clinical research and individual patient management. Successful achievement of this goal holds the promise of improving our therapeutic approach to human septic shock in an analogous manner to what has been achieved in the oncology field [7–11].
We recently reported the identification of three sub-classes of children with septic shock as a means to more effectively address the intrinsic heterogeneity of this syndrome . The three sub-classes were initially identified based exclusively on genome-wide expression patterns using microarray analysis and unsupervised hierarchical clustering. Importantly, the gene expression patterns represented the first 24 hours of meeting criteria for septic shock, which is an ideal time frame for clinically relevant stratification. Post-hoc phenotype analyses revealed that patients in one of the three sub-classes had significantly higher illness severity, organ failure rates, and mortality, compared to the other two subclasses, thus indicating that the expression-based sub-classes are clinically relevant . We further refined our sub-classification strategy to a 100-gene signature. Herein we report our initial attempt to develop a “clinician-friendly” stratification strategy based on these 100 class-predictor genes and the use of gene expression “mosaics”.
The 98 patients in the septic shock study cohort and their respective microarray studies have been previously reported with detailed clinical data and confirmation of Institutional Review Board approval . Briefly, we conducted genome-wide expression studies using the Human Genome U133 Plus 2.0 GeneChip (Affymetrix, Santa Clara, CA) and whole blood-derived RNA from 98 individual children meeting published criteria for septic shock [13–16]. All microarray data used in the current report represent the first 24 hours of meeting criteria for septic shock. Using discovery-oriented expression/statistical gene filters and unsupervised hierarchical clustering techniques, we identified 3 sub-classes of children with septic shock based exclusively on their respective genome-wide expression patterns (heretofore referred to as “sub-class A,” “sub-class B,” and “sub-class C”). Sub-class A patients had significantly higher illness severity, organ failure rates, and mortality, compared to sub-class B and C patients . Using K-means clustering and class prediction modeling, we refined the class-defining signature to 100 genes (see Supplementary Data for the 100 gene probe list). The microarray-derived expression patterns of these 100 class-defining genes are the basis of the current report.
Expression mosaics representing the 100 class-defining genes were generated using the Gene Expression Dynamics Inspector (GEDI) platform. GEDI is a publicly available gene expression analysis program developed by the Ingber Laboratory at Harvard University [17, 18]. The signature graphical outputs of GEDI are gene expression mosaics that give microarray data a “face” that is intuitively recognizable via human pattern recognition [19–22]. The underlying algorithm for creating the mosaics is a self-organizing map (SOM). Conventional applications of SOMs involve classifying genes or samples into predetermined numbers of discrete clusters. In contrast, GEDI creates an SOM-based mosaic for every sample. Thus, GEDI is said to be “sample oriented” rather than “gene oriented.” Extensive technical details regarding GEDI can be found at: http://www.childrenshospital.org/research/ingber/GEDI/gedihome.htm
The individual patients' microarray expression data for the 100 class-defining genes were uploaded to the GEDI platform and individual gene expression mosaics were generated for each patient. Subsequently, we generated composite mosaics for each of the 3 sub-classes. The respective composite mosaics represent the average expression patterns of the individual patients within a given subclass.
We next conducted internal cross validation procedures to test the ability of the GEDI mosaics to accurately identify the three sub-classes of children with septic shock. Nine individual clinicians participated as evaluators in the cross validation procedures. The nine evaluators are all practicing pediatric critical care medicine specialists. Only one of the nine evaluators has experience interpreting microarray data. Each evaluator was shown the 98 individual patient mosaics, and asked to classify each patient as subclass “A”, “B”, or “C”, based on the individual expression patterns/mosaics and using the composite mosaics as the reference point. The individual evaluators were not provided with any additional prompting or instructions, and were blinded to the original patient sub-classifications.
Individual evaluator responses were catalogued to determine the ability of the individual mosaics to accurately sub-classify patients based on the originally published sub-classifications. When discrepancies occurred for a given patient (i.e. different evaluators classified a given patient into more than one class) the “majority vote” (i.e. ≥5 votes for a given class) was considered as the final/consensus classification for that given patient. For example, if 6 of the evaluators classified an individual patient as “subclass B”, and the other 3 evaluators classified the same patient as “subclass C”, then that patient was assigned to subclass B (i.e. the consensus classification). The performance characteristics of the classification strategy (i.e. specificity, sensitivity, etc.) and 95% confidence intervals were based on the consensus classifications of the 9 individual evaluators. We also calculated performance characteristics for each individual evaluator. Performance characteristics were calculated using a web-based clinical calculator (http://faculty.vassar.edu/lowry/clin1.html) and the originally published sub-classifications as the gold standard. To further measure inter-rater/evaluator agreement across the 9 evaluators, a kappa coefficient for multiple evaluators was calculated using a web-based calculator (http://cosmion.net/jeroen/software/kappa/).
The composite mosaics for the 3 septic shock sub-classes are shown in Figure 1. Examples of individual patient mosaics are provided in Figures 2A through 2C. There was 100% concordance across the 9 evaluators for 62 of the 98 patients (63%). That is, for 63% of the patients all 9 evaluators classified them to the same sub-class. More specifically, there was 100% concordance across the 9 evaluators for 82% of patients in sub-class A, 69% of patients in sub-class B, and 32% of patients in sub-class C. For 80% of the patients in the entire cohort, at least 8 of the 9 evaluators classified them to the same sub-class. The average kappa coefficient across all possible interrater/evaluator comparisons was 0.81 (0 = no agreement; 1 = perfect agreement).
The performance characteristics of the sub-classification strategy, based on consensus classification, are provided in Table 1. Twenty seven of the 28 original sub-class A patients (96%) were correctly classified as sub-class A. Thirty eight of the 45 original sub-class B patients (84%) were correctly classified as sub-class B. Twenty three of the 25 original sub-class C patients (92%) were correctly classified as sub-class C. The respective sensitivities, specificities, positive predictive values, and negative predictive values of the sub-classification strategy were ≥84% across the three sub-classes. In addition, the sub-classification strategy yielded positive likelihood ratios ≥16.8 and negative likelihood ratios ≤0.2 across the three sub-classes.
Table 1 also provides the number of patients incorrectly classified. Two original sub-class B and 2 original sub-class C patients were incorrectly classified as sub-class A patients. Figures 3A and 3B provide the individual mosaics for these incorrectly classified patients. Four original sub-class B patients were incorrectly classified as sub-class C patients. Figure 4 provides the individual mosaics for these incorrectly classified patients. No patient was incorrectly classified as sub-class B. All of the incorrectly classified patients were survivors.
There were 2 patients that did not receive at least 5 “votes” (i.e. the majority vote) for any of the 3 sub-classes and therefore remained unclassified. Figure 5 provides the individual mosaics for these two patients. The patient represented by example 1 was an original sub-class A patient, but received 3 votes for each of the sub-classes. The patient represented by example 2 was an original sub-class B patient, but received 3 votes for sub-class A, 4 votes for sub-class B, and 2 votes for sub-class C. Both of these unclassified patients were survivors.
The performance characteristics for each one of the nine individual evaluators are provided in Tables 2a through through2i,2i, respectively. In addition, the median and intra-quartile performance characteristics of the 9 individual evaluators are provided in Table 3. These data demonstrate similar performance characteristics between the consensus classifications and the individual evaluator classifications.
We have provided initial evidence for a clinically feasible and robust stratification strategy for children with septic shock. The assertion of feasibility is supported at two levels. First, the stratification strategy is based on a 100 gene signature using whole blood-derived RNA. With current multiplex technology (e.g. quantitative polymerase chain reaction or digital quantification of mRNA abundance) and a dedicated molecular diagnostics laboratory, it is now feasible to quantify the expression levels of 100 genes within a timeframe that is clinically acceptable . Second, the sub-classification strategy relies on intuitive pattern recognition of GEDI-generated gene expression mosaics. We have demonstrated that clinicians without formal training in bioinformatics and complex gene expression analysis can accurately interpret these mosaics in a relatively reproducible and reliable manner, as objectively measured by a kappa statistic. We would expect that with additional training and experience in the interpretation of these mosaics, that correct response rates could be improved further.
The assertion of robustness is also supported at two levels. First, the data used for sub-classification represent the first 24 hours of meeting criteria for septic shock in the pediatric intensive care unit. This time frame is ideal for the goals of patient stratification, and identifies a sub-class of children (i.e. sub-class A patients) having a substantially higher illness severity and mortality (36% mortality vs. 11% mortality) than the other two sub-classes, with sensitivity and specificity ≥94% . The ability to reliably stratify patients within this timeframe could allow for the more selective enrollment of patients into clinical trials and for the selection of individual patients for currently available high risk therapies (e.g. ECMO). In the case of clinical trial stratification, the strategy is potentially most useful when the test intervention carries more than minimal risk. For example, patients with a low risk of mortality, who otherwise meet clinical entry criteria, could potentially be excluded and thereby not be exposed to the risks of the test agent. Alternatively, patients with a high risk of mortality could be more reliably selected for enrollment. In either case, the net result of this type of approach would be to optimize the risk to benefit ratio of a given experimental therapy.
Second, the performance characteristics of the stratification strategy demonstrate the ability to reliably classify patients into the correct categories with a high level of sensitivity and specificity, and high positive and negative predictive values. In addition, the positive and negative likelihood ratios are well above or below cutoffs, respectively, that are considered to be clinically relevant .
This initial proof-of-concept study has important limitations from the standpoint of validation. We have not prospectively validated the existence of these three sub-classes with regard to the expression patterns and the related clinical phenotypes. Our original study that led to the identification of the 3 putative classes was conducted in a rigorous manner, but has not yet been prospectively validated in an independent cohort of patients. Accordingly, the veracity of these sub-classes, while biologically plausible, remains to be formally demonstrated. Related to this limitation, the current study was conducted using an internal cross validation approach based on the same patients as the original study. This 100 gene- and mosaic-based approach also requires formal prospective validation in a separate cohort of patients.
These limitations notwithstanding, we have taken an important first step in bringing genomic-based data closer to the bedside of critically ill children for the purpose of clinically relevant stratification: proof-of-concept. A major barrier to the application of genomic data in the field of critical care medicine has been the translation, interpretation, and acceptance of these complex data by clinicians. We propose that the mosaic-based strategy described in this report may potentially allow for the use of genomic data by clinicians at the bedside of critically ill children. Validation studies to directly test this proposal are currently in progress.
Contributing investigators and centers for genomic database that generated the original sub-classification report : Thomas P. Shanley (C.S. Mott Children's Hospital at the University of Michigan, Ann Arbor, Michigan); Natalie Cvijanovich (Children's Hospital and Research Center Oakland, Oakland, CA); Richard Lin (The Children's Hospital of Philadelphia, Philadelphia, PA); Geoffrey L. Allen (Children's Mercy Hospital, Kansas City, MO); Neal J. Thomas (Penn State Children's Hospital, Hershey, PA); Douglas F. Willson (University of Virginia, Charlottesville, VA); Robert J. Freishtat (Children's National Medical Center, Washington, D.C.); Nick Anas (Children's Hospital of Orange County, Orange, CA); Keith Meyer (Miami Children's Hospital, Miami, FL); and Paul Checchia (St. Louis Children's Hospital, St. Louis, MO)
Supported by grants from the National Institutes of Health: R01GM064619 and RC1HL100474