|Home | About | Journals | Submit | Contact Us | Français|
Recent studies have demonstrated that gradient echo (GRE) MRI sequences are as accurate as CT for the detection of intracerebral hemorrhage (ICH) in the context of acute stroke. However, many physicians who currently read acute stroke imaging studies may be unfamiliar with interpretation of GRE images.
An NIH Web-based training program was developed including a pretest, tutorial, and posttest. Physicians involved in the care of acute stroke patients were encouraged to participate. The tutorial covered acute, chronic, and mimic hemorrhages as they appear on CT, diffusion-weighted imaging, and GRE sequences. Ability of users to identify ICH presence, type, and age on GRE was compared from the pretest to posttest timepoint.
A total of 104 users completed the tutorial. Specialties represented included general radiology (42%), general neurology (16%), neuroradiology (15%), stroke neurology (14%), emergency medicine (1%), and other (12%). Median overall score improved pretest to posttest from 66.7% to 83.3%, p < 0.001. Improvement by category was as follows: acute ICH, 66.7%–100%, p < 0.001; chronic ICH, 33.3%–66.7%, p < 0.001; ICH negatives/mimics, 100%–100%, p = 0.787. Sensitivity for identification of acute hemorrhage improved from 68.2% to 96.4%.
Physicians involved in acute stroke care achieved significant improvement in gradient echo (GRE) hemorrhage interpretation after completing the NIH GRE MRI tutorial. This indicates that a Web-based tutorial may be a viable option for the widespread education of physicians to achieve an acceptable level of diagnostic accuracy at reading GRE MRI, thus enabling confident acute stroke treatment decisions.
In the setting of acute stroke, accurate early detection of blood is crucial since a history of intracranial hemorrhage is a contraindication to the use of thrombolytic agents. There is now sufficient evidence that gradient echo MRI (GRE MRI) is equally sensitive a measure of acute intracerebral hemorrhage (ICH) as CT.1–4 In light of these recent studies, the current guidelines from the American Heart Association/American Stroke Association (AHA/ASA) state that either imaging modality may be used in the setting of suspected acute ischemic stroke to rule out hemorrhage.5 In many centers, CT remains the standard imaging modality of choice for the initial evaluation of patients presenting with acute stroke symptoms. A major disadvantage of CT within the first few hours of symptom onset, however, is limited sensitivity to early evidence of cerebral ischemia. Conversely, multimodal MRI with diffusion-weighted imaging (DWI) has excellent capacity to delineate the location and extent of early ischemia.6
In some centers, particularly academic tertiary stroke centers, MRI has already replaced CT as the primary imaging modality for acute stroke patients, including those who are thrombolytic candidates.3,4,7,8 However, the application of GRE MRI to screen for hemorrhage in routine practice at all hospitals may be a challenge. GRE sequences detect the paramagnetic effects of the breakdown products of hemoglobin. In the hyperacute and acute phase, hemorrhage appears as a rim of hypointense signal around an isodense core on these sequences.9,10
Many physicians who currently read imaging studies in acute stroke patients may be unfamiliar with interpretation of GRE magnetic resonance images. Therefore an intensive and validated reading skill training program is required to facilitate the widespread adoption of multimodal MRI as the primary imaging modality in acute stroke patients. This educational need is analogous to that required for interpretation of CT for early infarct signs when IV tissue plasminogen activator (tPA) was approved.11,12
The objective of this study was to determine how well physicians involved in acute stroke care could evaluate GRE MRI sequences for intracerebral hemorrhage at baseline, without specific training. Further, this study aimed to create, validate, and assess the utility of a Web-based tutorial designed to increase physician skill at interpreting GRE images for hemorrhage in the acute stroke setting.
An intramural NIH/National Institute of Neurological Disorders and Stroke Web-based training program was developed in collaboration with the University of California at Los Angeles. The National Institute of Neurological Disorders and Stroke GRE MRI tutorial Web site (http://gre.ninds.nih.gov) consisted of a pretest, tutorial, and posttest. The entire training program was designed to be completed within 1 hour, although participants were allowed to complete the tutorial at their own pace. One credit of continuing medical education (CME) was provided to participants who completed the full tutorial including both pretests and posttests. The study was approved by the NIH Office of Human Subjects Research and was granted exempt status from the NIH Institutional Review Board.
The tutorial section of the Web site described the appearance of various findings on GRE MRI. Descriptions were supplemented by one or more case examples of each finding. A total of 22 case examples were provided; each included full DWI and GRE MRI sequences as well as corresponding CT images for comparison. Findings were identified on the images with colored arrows for clarity.
Images for case examples and test cases (used in the pretests and posttests) were drawn from the databases of two prior ICH research protocols: the Hemorrhage and Early MRI Evaluation study1 and the NIH ICH natural history study. Images were used for case examples or test cases only if there was consensus diagnostic agreement among the authors.
The tutorial began by describing hemorrhage mimics and other findings, such as normal vessels visualized “end-on,” intravascular clots, venous sinus thromboses, calcifications, and artifacts. Acute hemorrhage topics covered in detail included intraparenchymal hematomas, hemorrhagic transformation of ischemic infarcts, intraventricular blood, subdural hematomas, and subarachnoid hemorrhage. The appearance of chronic hemorrhage on GRE was then described and compared to acute findings, with emphasis placed on chronic hematomas and microbleeds. The tutorial ended with a recommended general approach to interpreting GRE MRI sequences.
A total of 34 cases (independent of those used in the tutorial) were compiled for use in the pretests and posttests. Cases chosen for the testing pool were selected to capture varied and difficult hemorrhage types, and are on average more challenging to interpret than those that would be encountered consecutively in routine acute stroke practice. The cases were divided into three categories: acute ICH (11), chronic ICH (10), and ICH negative or mimics (13). Acute cases included primary intraparenchymal hemorrhage, hemorrhagic transformation of ischemic infarct, intraventricular hemorrhage, subarachnoid hemorrhage, and subdural hemorrhage. Nine of the 11 acute MRIs were within 3 hours of symptom onset. Chronic cases included hemorrhagic transformation of ischemic infarct, primary intraparenchymal hemorrhage, and microbleeds. Mimics included hypointense arterial vessel signs, basal ganglia calcifications, and venous sinus thromboses (figure). Each test case contained a full series of GRE and DWI images side by side. Participants were required to evaluate the GRE and DWI images alone; neither clinical information nor CT images were provided.
The pretests and posttests both consisted of six cases. The Web site generated each test by choosing two cases at random from each of the three categories described above (acute, chronic, and negative/mimic), and made sure that participants did not receive any overlapping cases in the pretest and posttest. In each test case, the participant was asked the following: 1) Is hemorrhage present? (yes/no). If the participant answered yes, he or she was then instructed as follows: 2) Indicate hemorrhage type; and 3) Indicate hemorrhage age. For the acute and chronic ICH cases, each of these three questions was worth one point, giving a maximum of three points possible per case. For the negative/mimic cases, three points were awarded for a correct answer to question 1 (Is hemorrhage present?) since questions 2 and 3 were irrelevant in these cases. Test scores were determined based on the answers to the above questions, with each case worth three points for a total of 18 points possible in both the pretest and posttest.
Targeted participants were those potentially involved in the care of acute stroke patients, specifically physicians in the fields of radiology, stroke neurology, neurology, and emergency medicine. The tutorial was advertised on websites and newsletters of professional societies such as the American Academy of Neurology, the American Society of Neuroimaging, and the Radiologic Society of North America, as well as on Medscape. In addition, emails were sent to stroke neurology program directors to make them aware of the opportunity for their staff. Participant data were collected from January 2005 to June 2008.
Before completing the tutorial, the following demographic data were collected from each participant: age, gender, medical specialty, and professional status. Participants were also asked if they are involved in the care of acute stroke patients, and how many patients they have treated with IV tPA in the past 12 months.
Participants’ overall scores out of 18 possible points for each test (pre and post) were calculated as a percentage. Differences from pretest to posttest were compared using the Wilcoxon signed rank test. Subgroups were analyzed based on participants’ level of training (attendings and fellows/residents) as well as by specialty. In the specialty subgroups, general neurology and stroke neurology were combined into one group (general/stroke neurology), while general radiology and neuroradiology were kept as individual groups. Sample size calculations demonstrated that we would have greater than 83% power (alpha = 0.05) to demonstrate a mean difference in score of at least 10 from pretest to posttest and an effect size of 0.29 with a sample size of 100 subjects using a paired t test. All statistical analyses were performed using the Statistical Package for the Social Sciences version 15.0 (SPSS, Chicago, IL).
In the time period from January 2005 to June 2008, 104 participants completed the pretest, tutorial, and posttest. Participants’ demographic data are presented in table 1. Specialties represented included general radiology (44), general neurology (17), neuroradiology (16), stroke neurology (14), emergency medicine (1), and other fields (12). Participants’ level of training was 62 attendings, 13 fellows, 11 residents, and 18 others. Eighty-two percent of participants reported being involved in the care of acute stroke patients.
The median total correct score for all participants improved from 66.7% on the pretest to 83.3% on the posttest, a score improvement of 16.6%, p < 0.001. Improvement by category from pretest to posttest for all participants was as follows: acute ICH, 66.7%–100%, p < 0.001; chronic ICH, 33.3%–66.7%, p < 0.001; and ICH negatives/mimics, 100%–100%, p = 0.787 (table 2, figure). The subgroups of attending physicians and fellows/residents were analyzed separately, and showed score improvements similar to those of the whole group (table 2).
When analyzed by specialty subgroups, scores for general/stroke neurology and general radiology participants were similar to the whole group, with significant improvement in total, acute ICH, and chronic ICH scores (table 3). The neuroradiology participants also demonstrated significant improvement in acute ICH scores; however, they did not demonstrate significant improvements in total (83.3%–83.3%, p = 0.381) or in chronic ICH scores (75.0%–66.7%, p = 0.357).
The sensitivity of all participants for the presence of any hemorrhage (acute or chronic) improved from 69.5% on the pretest to 92.5% on the posttest (table 4). Even higher posttest sensitivity for any hemorrhage was achieved by the attendings (94.6%), as well as by the general/stroke neurology (95.0%) and general radiology (93.5%) subgroups. Posttest sensitivity in the neuroradiology subgroup was 90.7%. Specificity for the presence of any hemorrhage stayed roughly the same from pretest (82.1%) to posttest (82.6%) for all participants (table 4). For acute hemorrhage alone, sensitivity improved from 68.2% to 96.4% on the posttest, and specificity had a higher baseline value of 89.1%, which again remained roughly unchanged posttest at 87.5% (table 4).
At baseline, our sample of healthcare professionals involved in the care of acute stroke patients demonstrated suboptimal skills in interpreting GRE sequences for hemorrhage. The median correct score for all participants on the pretest was only 66.7%. Sensitivity, the most important measure of a screening test, was only 68.2% for the presence of acute hemorrhage on the pretest, and was 69.5% for the presence of any hemorrhage, acute or chronic.
The National Institute of Neurological Disorders and Stroke GRE MRI tutorial significantly improved these skills from the pretest to the posttest timepoint. Participants improved to a median overall score of 83.3%. In the category of acute ICH, diagnostically the most relevant category of hemorrhage for acute treatment decisions, sensitivity improved to 96.4%. Sensitivity for the presence of any type of hemorrhage improved to 92.5%. Given that the hemorrhages used in the pretests and posttests may have been more difficult to interpret than those routinely encountered by most physicians, it is possible that sensitivity in clinical practice will be higher.
Significant improvement was seen from pretest to posttest, but this overall improvement was based only on increased identification of acute and chronic hemorrhages, not of hemorrhage negatives or mimics. In other words, participants improved their skill at identifying the presence, type, and age of intracranial hemorrhage on GRE, but did not improve their skill at determining the absence of hemorrhage (improved sensitivity, but not specificity). This may be due in part to a relatively high pretest specificity rate, particularly for acute hemorrhage. In addition, this may reflect the greater potential for identifying false positives on GRE sequences. When screening for hemorrhage in the setting of acute stroke, physicians are more likely to err on the side of making a false positive judgment than they are to make a false negative judgment. The consequences of missing a hemorrhage in an acute stroke patient could result in catastrophic administration of IV tPA, while the consequences of a false positive read of hemorrhage, though also detrimental to patient care, are much less severe for both the patient and the physician. Future improvements to the tutorial should specifically target improving specificity in order to minimize the identification of false positives. This issue is important to avoid denying appropriate thrombolytic treatment to eligible candidates.
The high level of sensitivity for identification of ICH on GRE achieved after training with the GRE MRI tutorial is impressive, but the question remains: How sensitive is sensitive enough? The most recent AHA/ASA guidelines for the treatment of acute ischemic stroke have added a Class I recommendation that “the brain imaging study should be interpreted by a physician with expertise in reading CT or MRI studies.”5 The definition of expertise, however, is not further specified. Some have suggested a sensitivity level of greater than 95% in recognizing hemorrhage is required, due to the potential consequences of a false negative ICH diagnosis.12
In our study, after completing the GRE MRI tutorial, a sensitivity level of greater than 95% was achieved for the detection of acute hemorrhage. Additionally, sensitivity levels around 95% were achieved by the subgroups of attendings (94.6%), general/stroke neurology (95.0%), and general radiology (93.5%) for the detection of any hemorrhage, both acute and chronic. Neuroradiologists are the gold standard for a physician with expertise in reading intracranial MRI studies, but the availability of a neuroradiologist to read an acute MRI in time to make IV tPA and alternative treatment decisions is often not possible, especially at smaller community hospitals. After completing the GRE MRI tutorial, our sample of stroke and general neurologists, as well as general radiologists, were able to achieve comparable sensitivity levels at identifying the presence of ICH as the neuroradiologists. This suggests that equivalent expertise at reading intracranial GRE MRI studies for hemorrhage in the context of acute stroke may be attained by these specialties through Web-based training, thus making reliable acute stroke treatment decisions possible in the absence of a trained neuroradiologist.
One limitation of this study is the relatively small sample size. Insufficient numbers of stroke and general neurologists participated to analyze them as separate groups, and the number of neuroradiology participants was smaller than expected. Therefore, these results may not be fully generalizable to all acute stroke settings. Only one emergency medicine physician participated, so this important specialty involved in acute stroke care could not be included in the analyses. There were also insufficient numbers of residents and fellows to allow a thorough analysis of improvement by level of training. However, since attending physicians are generally involved in acute stroke treatment decisions, this limitation does not have a large impact on the results of the study. While the subgroup analyses in this study suggest promising results, the small sample size of subgroups requires future larger studies to validate these findings.
The convenience sampling method of participant recruitment employed in this study may have produced a biased estimate of nationwide physician capabilities at interpreting GRE MRI for hemorrhage. The tutorial was advertised on lists of CME opportunities, and those physicians who chose to do this MRI-based tutorial are likely to have been more comfortable with MRI at baseline than those who chose not to participate. The tutorial was also advertised by emails to stroke fellowship program directors, which tend to be at larger academic institutions where physicians are more exposed to MRI than physicians at smaller hospitals. Thus we cannot conclude that our sample is representative of the national population of physicians and that similar results should be expected in all physicians who complete the tutorial. Future studies specifically designed to avoid sample bias are needed to validate these results.
The method of image presentation is another possible source of bias in this study. Images were viewed online and participants were not able to adjust the contrast level. The images were preset at what the authors believed was the most optimal level of contrast, but the inability to adjust this level may have affected performance. Future studies should more closely replicate real-world situations by requiring the participant to adjust the level of contrast. Only the DWI and GRE sequences were presented, and it is plausible that participants would have benefited from the full set of multimodal MRI sequences that are usually acquired. Further, no information about patient history or clinical presentation was given, both of which can aid in the interpretation of images to improve diagnostic accuracy.
There are additional factors that may have positively affected physician performance in this study. The GRE and DWI images came from centers with considerable expertise in MRI, and thus the image quality may have been better than that routinely achieved at other centers around the country. This may have led to better diagnostic accuracy than would be seen in practice. Additionally, since there was no required lag time between the tutorial and the posttest, it is possible that the score improvements seen were a product of recent exposure to the tutorial rather than of permanent acquisition of skills at reading GRE MRI for hemorrhage. Further studies need to implement a posttest weeks or months after the tutorial to investigate the long-term benefits of this intervention.
These findings suggest that a Web-based training tutorial may be a viable option for the widespread education of physicians involved in acute stroke care in the interpretation of GRE MRI for hemorrhage. However, further studies are needed to validate the use of this tool and its generalizability to all physicians involved in acute stroke care. The nationwide transition to MRI as the standard imaging modality for acute stroke will represent an improvement in patient care, and should not be deterred due to lack of physician skill at evaluating GRE images for hemorrhage. A Web-based training module, as well as the establishment of a target level of accuracy in reading GRE images for hemorrhage, may aid physicians in incorporating MRI into standard acute stroke care. Ultimately, definitive studies will be needed to address the knowledge/cost benefit of CT vs MRI in order to determine the utility of the additional information obtained with MRI compared with CT in the acute setting.
Statistical analyses were conducted by C.S. Kidwell and B.R. Copenhaver.
Address correspondence and reprint requests to Dr. Chelsea S. Kidwell, 4000 Reservoir Road, Bldg. D, Suite 150, Washington, DC 20007 ude.nwotegroeg@652kc
Supported by the Intramural Research Program of the National Institutes of Health, National Institute of Neurological Disorders and Stroke.
Disclosure: The authors report no disclosures.
Received August 13, 2008. Accepted in final form February 6, 2009.