|Home | About | Journals | Submit | Contact Us | Français|
The implementation of uniform nomenclature and classification in adult arterial ischemic stroke (AIS) has been critical for defining outcomes and recurrence risks according to etiology and in developing risk-stratified treatments. In contrast, current classification and nomenclature in childhood AIS are often overlapping or contradictory. Our purpose was to develop a comprehensive consensus-based classification system for childhood AIS.
Utilizing a modified-Delphi method, members of the International Pediatric Stroke Study (IPSS) developed the Childhood AIS Standardized Classification and Diagnostic Evaluation (CASCADE) criteria. Two groups of pediatric stroke specialists from the IPSS classified seven test cases using two methods each: 1) classification typical of the individual clinician’s current clinical practice, 2) classification based on the CASCADE criteria. Group 1 underwent in-person training in the utilization of the CASCADE criteria. Group 2 classified the same cases via an online survey including definitions but without training. Inter-rater reliability (IRR) was assessed via multi-rater unweighted kappa statistic.
In Group 1 (with training), IRR was improved using CASCADE criteria (κ=0.78, 95% CI=[0.49, 0.94]) compared to typical clinical practice (κ=0.40, 95%CI=[0.11, 0.60]). In Group 2 (without training), IRR was lower than among trained raters, (κ=0.61, 95%CI=[0.29,0.77]); but higher than current practice (κ=0.23, 95%CI=[0.03,0.36]).
A new, consensus-based classification system for childhood AIS, the CASCADE criteria, can be used to classify cases with good IRR. These preliminary findings suggest that the CASCADE criteria may be particularity useful in the setting of prospective multicenter studies in childhood-onset AIS, where standardized training of investigators is feasible.
The adoption of uniform nomenclature and a validated classification system in arterial ischemic stroke (AIS) in adults has been critical to the study of neurological outcomes, risk stratified therapies and recurrence risk.1;2 Multiple classification systems exist for adult AIS that have been validated and utilized for the purposes of multi-center collaborative efforts. As an example, the TOAST classification system has been utilized in over 180 peer- reviewed publications in adult stroke, facilitating the study of differences in adult stroke subtypes with respect to etiology, risk factors (including genetic polymorphisms), treatment, and outcomes.3-5 Utilization of the TOAST criteria has, in turn, led to the development and successful conduct of clinical trials evaluating risk-stratified approaches to treatment in adult AIS.
Over the past decade, collaborative groups have made considerable progress in understanding the epidemiology, etiology, pathophysiology, outcomes, and prognostic factors in childhood AIS.6-9 Nevertheless, current research efforts are hampered by the lack of standardized nomenclature and classification in the childhood form of this disease.10-12 Initial attempts at classification in childhood AIS utilizing adult criteria found that the majority of children with AIS do not meet criteria for atherothrombotic, cardio-embolic or small-vessel disease, as seen in adults.13;14 Subsequent efforts toward childhood-specific nomenclature and classification in AIS have been proposed (The Pediatric Stroke Classification [PSC] and the Sebire Criteria), but no one system has been widely adopted.15;16
The PSC was created by adding subtypes that were identified in the literature to selected subtypes from the TOAST criteria.15 Although the system has good IRR, with ICC=0.92, it has not been widely adopted in the childhood stroke literature, possibly due to the lack of consensus development, overlap with the Sebire criteria nomenclature and inability to classify changes in arteriopathy over time. Although the Sebire criteria are consensus-based, the authors did not attempt to categorize strokes of non-vascular etiology, making it an incomplete system for classification.16 In addition, these two systems also have overlapping and sometimes contradictory nomenclature, especially in regard to arteriopathies which are common in childhood AIS. The pediatric rheumatology literature provides additional definitions for inflammatory arteriopathies by adapting the adult Calabrese criteria for primary angiitis of the nervous system (PACNS), which have even further potential overlap with the Sebire and PSC definitions for arteriopathy.17;18
In order to devise a consensus-based classification system and uniform nomenclature for childhood AIS, the International Pediatric Stroke Study (IPSS) established a working group in childhood AIS classification in November 2007. The objectives of this report are to describe the methodology utilized to develop the Childhood AIS Standardized Classification and Diagnostic Evaluation (CASCADE) criteria, and to provide evidence of inter-rater reliability using this new, comprehensive classification system in childhood AIS.
The IPSS is an international research network that created a registry of patients with pediatric cerebrovascular disease, established in 2003 with the long-term goal of developing multinational clinical trials in pediatric ischemic stroke.7;19 Investigators of the IPSS created a working group to construct a consensus-based classification system for childhood AIS. The working group consisted of fourteen volunteers from the IPSS: ten pediatric neurologists, one adult neurologist, two pediatric hematologists and one pediatric rheumatologist. The initial version of the CASCADE criteria represented a compilation of criteria from three published systems: The Pediatric Stroke Classification (PSC),15 the Sebire definitions for cerebral arteriopathy16 and Calabrese criteria for childhood primary CNS vasculitis.17;20;21 In subsequent versions, the working group established a unifying theme for the new classification system: an anatomically based system. In order to minimize subjectivity, the primary classification is largely based upon objective anatomic evidence of any identifiable disease contributing to stroke causation, including disorders of the heart, cervical and intracranial vasculature.
The CASCADE criteria was revised utilizing a modified-Delphi method (Figure 1).22;23 Each revision incorporated critiques solicited from members of the working group regarding the clinical relevance and conceptual validity of the overall system as well as each subtype. The co-chairs (TB & RI) evaluated all responses and incorporated the critiques and suggestions into the next revision. Reviewers were asked to specify their assessments as: 1) Agreement, 2) Agreement with minor modifications, or 3) Disagreement. In addition, there was a section provided for additional comments and suggested revisions. The proposal for the CASCADE criteria underwent four revisions. When disagreements arose, the working group was polled regarding possible solutions, and the majority opinion was accepted.
Based upon this consensus-driven process, the 4th revision was presented to the IPSS members attending the IPSS investigators’ meeting in February, 2009. After a 20 minute training session, seven childhood stroke vignettes were presented to the membership by the working group chairs, and the IPSS members in attendance were asked to classify the cases using two methods: 1) the approach typical of the clinician’s current practice, via entry of free text (Individual Classification), and 2) the CASCADE criteria. Group 1 was comprised of the six members who volunteered to classify all seven cases. Each vignette was based on real cases selected from stroke registries at The Children’s Hospital of Philadelphia or The Children’s Hospital Colorado and contained a synopsis of clinical history, selected neurovascular imaging, echocardiogram results and thrombophilia testing. Real-life cases were selected by TB & RI from Children’s Hospital of Philadelphia or Children’s Hospital Colorado to represent a broad range of stroke etiologies including cardio-embolic, intracranial arteriopathy (2 cases), dissection, post-infectious, multiple causes (moyamoya and cardiac catheterization procedure) and idiopathic. Group 2 was comprised of members of IPSS who did not participate in Group 1, and responded to a call for volunteers. They reviewed the same cases via an e-mail survey. Group 2 raters classified the cases in the same manner as Group 1, via Individual Classification and the CASCADE criteria, but were not given a training session.
The primary CASCADE criteria are presented in Figure 2 and Table 1. The IPSS defines childhood AIS as follows: (1) neurological deficit of acute onset; (2) radiographic image(s) (MRI or CT) showing cerebral parenchymal infarct(s) conforming to known arterial territory(ies) and corresponding to clinical manifestations; and (3) occurring in children 29 days to 18 years of age. The CASCADE criteria are not designed to classify perinatal stroke, defined as stroke occurring before 29 days of life, which has unique risk factors.27
The primary CASCADE criteria is based upon anatomic site of disease (Table 1), which provides the capacity to classify any patient with childhood AIS into a single acute primary category at the time of initial diagnosis (within 1 month of presentation). Standardized definitions for vascular lesions are provided in Online Table 1a.
The classification at presentation is based upon the location and nature of any identifiable anatomic abnormality involving the heart, the great vessels in the neck, or the intracranial vessels that could explain AIS in an individual patient (Figure 2). It is based upon the investigator’s assessment of the anatomic site of disease after interpreting the results of the following clinical data: echocardiogram, vascular imaging (MRA, CTA and/or conventional angiogram), and pattern of anatomic distribution of infarct as demonstrated by MRI or CT, and clinical history.
There are wide variations in vascular imaging capabilities, algorithms, and interpretation across medical centers -- and these can change over time. We designed this system to be applicable regardless of the specific imaging modality or techniques used in the evaluation of an individual patient. For the purposes of this classification system, CTA and MRA are considered an adequate evaluation, while conventional angiogram (CA), when performed, is considered the gold standard.
In cases with multiple anatomic locations of disease potentially contributing to stroke mechanism (for example, dissection in a patient with congenital heart disease), the stroke is classified as multifactorial (and the multiple categories will be recorded). Finally, strokes in patients with normal cardiac and vascular imaging (that might be viewed as “unknown”), or strokes with an atypical anatomic etiology (i.e. AIS during an endovascular procedure), are classified as “other”.
The inter-rater reliability was measured among raters for each of the major seven classification categories represented by the cases. Agreement for Individualized Classification was defined as those cases which utilized the same or similar nomenclature within their response, and was inclusive of multiple spellings and/or tenses. If agreement was uncertain, the responses were reviewed by TB and MT, with agreement being awarded as a default in uncertain situations. We generated a Kappa-statistic for each of these categories, utilizing the Fleiss unweighted kappa statistic for multiple raters.28 In addition 95% confidence intervals were determined using the Wald method. Employing the parameters proposed by Landis and Koch,29 we considered a value above 0.8 to represent near perfect agreement, a value between 0.61–0.80 substantial agreement, a value 0.41–0.60 moderate agreement, a value 0.21–.40 fair agreement, and a value less than 0.20 slight agreement.
In Group 1 (with training), six raters provided complete data on all test cases (i.e. raters who classified all seven subjects for both the CASCADE criteria and individual system). Using these data from the six raters, we calculated an unweighted Kappa statistic as a measure of inter-rater reliability. In the seven test cases presented to Group 1, inter-observer agreement (average over subjects of the % of raters that give the majority score for each subject) and kappa-statistics are better when using CASCADE criteria (90%, range=4/6 to 6/6 and κ=0.78, 95% CI=[0.49, 0.94]), as compared to using Individual Classification (64%, range=1/6 to 6/6 and κ=0.40, 95%CI=[0.11, 0.60]). The pairwise rater kappas for the CASCADE criteria (i.e. each rater vs. all others) ranged from 0.61 to 1 indicating moderate to excellent agreement across all raters. In Group 2, 17 untrained raters evaluated the seven test cases. inter-observer agreement and kappa-statistics are better when using CASCADE criteria (82%, range=10/17 to 17/17 and κ=0.61 [0.29, 0.77]), as compared to using Individual Classification (52%, range=6/17 to 15/17 and κ=0.23[0.03, 0.36]; Table). The pairwise kappas for the CASCADE criteria ranged from -0.04 to 0.86.
The CASCADE criteria offer a novel and unique approach to classification of stroke subtype in childhood AIS compared to previous efforts. The criteria are derived from a consensus-based process involving pediatric AIS experts and are comprehensive with respect to childhood-onset AIS etiologies, and therefore seem more likely to be widely adopted than previously proposed schemes. In addition, the CASCADE criteria unifies previously published classification systems with the aim of eliminating terms with overlapping and/or nonspecific definitions, such as focal cerebral arteriopathy, primary angiitis of the central nervous system of childhood (cPACNS), transient cerebral arteriopathy and steno-occlusive arteriopathy.6;15-17 As arteriopathy is associated with increased recurrence risk and adverse outcomes,8;30 standardizing the nomenclature employed in describing vascular disorders will be essential to developing risk stratified therapies in childhood AIS.
While the initial kappa-statistic of the CASCADE criteria is encouraging (κ=0.61-0.78), it is based upon a limited number of raters in an unpowered analysis of few cases. Future efforts to establish inter-rater reliability and validity are warranted, utilizing larger cohorts, prospectively collected cases and consistently trained raters. The decreased Kappa found in Group 2 highlights the need for training to maximize the reliability of the CASCADE criteria. In addition, the discrepancy between groups may have resulted from Group 1 sharing their answers with each other (although they were asked not to do so) or a discrepancy between the imaging qualities presented to Group 1 and Group 2. The importance of imaging quality and the need for training (possibly web-based) prior to classification should be addressed by future studies of reliability. Finally, construct validity needs testing, possibly by determining the predictive utility of the CASCADE criteria upon recurrence risk and clinical outcomes.
A limitation of the system is its reliance upon consensus expert opinion when there is a lack of evidence-based data. In addition, the primary classification is anatomically based, but has subcategories which are not purely anatomic. This compromise was reached within the modified Delphi method in order to include established and accepted clinical definitions- such as dissection or Takayasu arteritis- within the seven major anatomic categories. Similar to the initial TOAST classification, the CASCADE criteria provide a starting point for consensus-based classification in childhood AIS; and has the potential for ongoing modification as new information about childhood-onset AIS is uncovered.
Additionally, there are many important risk factors that are not related to structural disease of the heart or blood vessels - such as inflammation, thrombophilias, genetic syndromes, hemoglobinopathies and infections. Future modifications of the CASCADE criteria will need to further unify and elaborate classification of these factors in a Secondary Classification System using the same modified-delphi methods, as well as further test our ability to revise classification beyond the acute period of childhood AIS. This secondary classification is under development (Online Table 1b), and will be further defined and validated in future studies. In addition, a temporal dimension to the criteria (Online Table 1c) is also under development and will classify the natural history of the patient’s vascular disease as stable, reversible, or progressive, based on follow-up imaging.
Despite these limitations, this represents the first consensus based classification system, developed by a large consortium of childhood stroke investigators that is inclusive of all childhood AIS. These findings suggest that the CASCADE criteria may be particularity useful in the setting of prospective multicenter studies in childhood-onset AIS, where standardized training of investigators is feasible.
We would like to acknowledge all of the IPSS members who participated as raters in the cases:
Ruth Nass, Veronica Gonzalez Alvarez, Steven Pavlakis, Paola Pergami, Neil Friedman, Mubeen Rafay, Marta Isabel Hernandez Chavez, Mahendra Moharir, Lisa Abraham, Ilona Kopyta, Clarisa Maxit, Virginia Wong, Lisa M. Nassif and Anneli Kolk
Sources of Funding
National Institutes of Health, National Heart, Lung, and Blood Institute; 1K23HL096895-01A1 (T.J.B)
Centers for Disease Control and Prevention UR6/CCU820552 (M.J.M-J.)
National Institutes of Health, National Heart Lung and Blood Institute 1K23HL084055-01A1 (N.A.G.)
National Institutes of Health, North and Central Texas Clinical and Translational Science Initiative KL2RR024983 (M.M.D)
First American Real Estate Information Services, Inc. (M.M.D)
National Institutes of Health, National Institute of Neurological Disease and Stroke R01NS050488 (R.I.)