PMCCPMCCPMCC

Search tips
Search criteria 

Advanced


 
Formats
Article sections
Authors
Related links
Logo of bmcmiBioMed Centralsearchsubmit a manuscriptregisterthis articleBMC Medical Imaging
 
BMC Med Imaging. 2012; 12: 16.
Published online 2012 July 18. doi:  10.1186/1471-2342-12-16
PMCID: PMC3423015
Copyright ©2012 Galinovic et al.; licensee BioMed Central Ltd.
Automated vs manual delineations of regions of interest- a comparison in commercially available perfusion MRI software
Ivana Galinovic,corresponding author1 Ann-Christin Ostwaldt,2 Carina Soemmer,3 Helena Bros,1 Benjamin Hotter,1 Peter Brunecker,1 and Jochen B Fiebach1
1Center for Stroke Research Berlin, Charité-Universitätsmedizin Berlin, Campus Benjamin Franklin, Hindenburgdamm 30, 12200, Berlin, Germany
2International Graduate Program Medical Neurosciences, Charite-Universitätsmedizin, Berlin, Luisenstrasse 56, 10117, Berlin, Germany
3Charité-Universitätsmedizin, Berlin, Charitéplatz 1, 10117, Berlin, Germany
corresponding authorCorresponding author.
Ivana Galinovic: ivana.galinovic/at/charite.de; Ann-Christin Ostwaldt: ac.ostwaldt/at/googlemail.com; Carina Soemmer: carina.soemmer/at/charite.de; Helena Bros: hnbros/at/gmail.com; Benjamin Hotter: benjamin.hotter/at/charite.de; Peter Brunecker: peter.brunecker/at/charite.de; Jochen B Fiebach: peter.brunecker/at/charite.de
Received August 24, 2011; Accepted May 21, 2012.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Abstract
Background
In perfusion magnetic resonance imaging a manual approach to delineation of regions of interest is, due to rater bias and time intensive operator input, clinically less favorable than an automated approach would be. The goal of our study was to compare the performances of these approaches.
Methods
Using Stroketool, PMA and Perfscape/Neuroscape perfusion maps of cerebral blood flow, mean transit time and Tmax were created for 145 patients with acute ischemic stroke. Volumes of hypoperfused tissue were calculated using both a manual and an automated protocol, and the results compared between methods.
Results
The median difference between the automatically and manually derived volumes was up to 210 ml in Perfscape/Neuroscape, 123 ml in PMA and 135 ml in Stroketool. Correlation coefficients between perfusion volumes and radiological and clinical outcome were much lower for the automatic volumes than for the manually derived ones.
Conclusions
The agreement of the two methods was very poor, with the automated use producing falsely exaggerated volumes of hypoperfused tissue. Software improvements are necessary to enable highly automated protocols to credibly assess perfusion deficits.
Keywords: Magnetic resonance imaging, Perfusion MRI, Acute ischemic stroke
Background
The increased use of the perfusion imaging (PI) – diffusion-weighted imaging (DWI) mismatch hypothesis [1] in studies of acute ischemic stroke (AIS) and clinical practice [1,2] is raising demands from software packages developed for volumetric calculations of hypoperfusion. Typically these programs offer the possibility to delineate a region of interest (ROI) manually but also, to a varying extent, automatically. Manual approaches to delineation are biased and require time intensive operator input. Therefore a mostly automated procedure, if accurate, would be preferred in clinical practice. The goal of our study was comparing the automated approach to the manual approach while using a number of different software packages.
Methods
The study design and cohort characteristics have been previously reported [3]. Briefly, using Stroketool (Digital Image Solutions, Germany, http://www.digitalimagesolutions.de), PMA (v3.2.0.4, ASIST, Japan, http://asist.umin.jp/index-e.htm) and Perfscape/Neuroscape (Olea Medical SAS, France, http://www.olea-medical.com) perfusion maps of cerebral blood flow (CBF), mean transit time (MTT) and Tmax were calculated for 145 patients imaged within 24 hours of AIS. The inclusion criteria were: clinically and radiologically confirmed AIS, hypoperfusion on the initial PI examination (assessed by the attending neuroradiologist) and availability of a follow-up MRI scan. For each parameter map, three thresholds were applied. The Tmax thresholds were 4, 6 and 8 seconds of delay [4] and the MTT thresholds were 5, 6 and 8 seconds. As no uniform CBF scale was available, the three CBF thresholds were different across software. Both MTT and CBF thresholds were chosen empirically using a random sample of acute stroke patients. ROI volumes were calculated using a manual and an automated protocol. In the automated protocol, once the thresholds have been applied, no further post-processing was done. Maps created in PMA and Stroketool also underwent a second post-processing step in SPM8 (Wellcome Trust Centre for Neuroimaging, UK) to cut away scalp and spaces filled with cerebrospinal fluid (CSF). This was not necessary for Perfscape/Neuroscape due to the program’s implemented filtering. In the manual protocol a human rater excluded, from the thresholded maps, areas unlikely to reflect credible hypoperfusion.
Radiological outcome was defined as the final lesion volume on follow-up FLAIR images. Clinical outcome was defined as the National Institute of Health Stroke Scale (NIHSS) score at the time of hospital discharge. All statistics were done in PASW Statistics 18. Analyses of correlations were performed using the Spearman signed-ranks correlation test.
Results
The median final lesion volume was 6.55 ml (IQR 0.8 – 31.6 ml). The median difference between the automatically and manually derived volumes was up to 210 ml (between 90% and 386%) in Perfscape/Neuroscape, 123 ml (between 192% and 1415%) in PMA and 135 ml (between 357% and 815%) in Stroketool. (Table 1). Bland-Altman plots for agreement of methods are shown in Figure ​Figure1.1. All correlation coefficients between volumes of perfusion deficit and radiological and clinical outcome were considerably lower for the automatic volumes than for the manually derived ones. The top performing automated map was Tmax in all three programs. With additional filtering applied to automated volumes calculated in PMA and Stroketool the median difference between the automatically and manually derived volumes dropped down to a maximum of 64.91 ml (between 44% and 238%) and 67.28 ml (between 33% and 369%), respectively (Table 1).
Table 1
Table 1
Difference of ROI volumes between the automated and the manual protocol
Figure 1
Figure 1
Bland-Altman plots for the top performing automatic map and threshold, without additional CSF filtering for PMA and Stroketool a- Perfscape/Neuroscape; b- PMA and c- Stroketool. The mean and the limits of agreement (mean +/−2 standard deviations) (more ...)
Discussion
The median volume of the perfusion deficit varied greatly based on the map and threshold and even across software for the same map and threshold. This discrepancy could in part be explained through the different choice of AIFs, the use of different deconvolution techniques, different implementations of the same calculation algorithm as well as differences in motion correction across different programs [5-7]. Our group had already conducted a study on a group of patients with no ischemia, using the same three software packages, and observed the presence of numerous artifacts; typically the cortex proximal to the skull and infratentorial cerebral and cerebellar tissue [8]. Programs without implemented CSF filtering also suffered from artifacts such as ventricles, eyeballs and scalp [8]. As expected, the same artifacts were present in our current patient cohort. This explains the weaker correlation coefficients and overshoot of the automated delineations as compared to the manual ones. Although most of the values on the Bland-Altman difference plot fall within the limits of agreement (Figure 1), these are much too broad with regards to the cohort’s median lesion volumes and the fact that, based on location, even a lesion of a few ml can be clinically significant. Additional CSF filtering greatly reduced the differences between the automatic and the manual volumes, pointing to a need for implemented filtering.
Conclusion
In conclusion, current automated use of the here evaluated programs would lead to falsely exaggerated volumes of hypoperfused tissue in patients with AIS. However a number of improvements, such as algorithms for judging perfusion asymmetry between hemispheres and allowing selection of the vessel territory of expected hypoperfusion, could aid automated protocols in credibly assessing perfusion deficits.
Competing interests
The authors have no conflicts of interest to declare. All vendors provided software free of charge, no author received grants or fees from any of the involved software right owners and companies.
Authors’ contributions
IG gathered the patient cohort, partly analyzed the imaging data, performed statistical analysis and drafted the manuscript. ACO participated in the analysis of the imaging data. CS participated in the analysis of the imaging data. HB participated in the analysis of the imaging data. BH participated in the gathering of the patient cohort and the drafting of the manuscript. PB performed the automated segment of data analysis. JBF was responsible for supervision and co-drafted the final version of the manuscript with IG. All authors read and approved the final manuscript.
Pre-publication history
The pre-publication history for this paper can be accessed here:
http://www.biomedcentral.com/1471-2342/12/16/prepub
Acknowledgements
No acknowledgements.
Sources of funding
The research leading to these results has received funding from the Federal Ministry of Education and Research via the grant Center for Stroke Research Berlin (01 EO 0801).
References
  • Hacke W, Albers G, Al-Rawi Y, Bogousslavsky J, Davalos A, Eliasziw M. et al. The Desmoteplase in Acute Ischemic Stroke Trial (DIAS): a phase II MRI-based 9-hour window acute stroke thrombolysis trial with intravenous desmoteplase. Stroke. 2005;36:66–73. doi: 10.1161/01.STR.0000149938.08731.2c. [PubMed] [Cross Ref]
  • Kohrmann M, Juttler E, Fiebach JB, Huttner HB, Siebert S, Schwark C. et al. MRI versus CT-based thrombolysis treatment within and beyond the 3 h time window after stroke onset: a cohort study. Lancet Neurol. 2006;5:661–667. doi: 10.1016/S1474-4422(06)70499-9. [PubMed] [Cross Ref]
  • Galinovic I, Ostwaldt AC, Soemmer C, Bros H, Hotter B, Brunecker P. et al. Search for a map and threshold in perfusion MRI to accurately predict tissue fate: a protocol for assessing lesion growth in patients with a persistent vessel occlusion. Cerebrovasc Dis. 2011;32:186–193. doi: 10.1159/000328663. [PubMed] [Cross Ref]
  • Olivot JM, Mlynash M, Thijs VN, Kemp S, Lansberg MG, Wechsler L. et al. Optimal Tmax threshold for predicting penumbral tissue in acute stroke. Stroke. 2009;40:469–475. doi: 10.1161/STROKEAHA.108.526954. [PMC free article] [PubMed] [Cross Ref]
  • Kosior RK, Kosior JC, Frayne R. Improved dynamic susceptibility contrast (DSC)-MR perfusion estimates by motion correction. J Magn Reson Imaging. 2007;26:1167–72. doi: 10.1002/jmri.21128. [PubMed] [Cross Ref]
  • Kim J, Leirab EC, Callison RC, Ludwiga B, Moritani T, Magnotta VA, Madsen MT. Toward fully automated processing of dynamic susceptibility contrast perfusion MRI for acute ischemic cerebral stroke. Comput Methods Programs Biomed. 2010;98:204–213. doi: 10.1016/j.cmpb.2009.12.005. [PubMed] [Cross Ref]
  • Yamada K, Wu O, Gonzalez RG, Bakker D, Copen WA, Weisskoff RM, Rosen BR, Yagi K, Nishimura T, Sorensen AG. Magnetic resonance perfusion-weighted imaging of acute cerebral infarction: effect of the calculation methods and underlying vasculopathy. Stroke. 2002;33(1):87–94. doi: 10.1161/hs0102.101893. [PubMed] [Cross Ref]
  • Galinovic I, Brunecker P, Ostwaldt AC, Soemmer C, Hotter B, Fiebach JB. Fully automated postprocessing carries a risk of substantial overestimation of perfusion deficits in acute stroke magnetic resonance imaging. Cerebrovasc Dis. 2011;31:408–413. doi: 10.1159/000323212. [PubMed] [Cross Ref]
Articles from BMC Medical Imaging are provided here courtesy of
BioMed Central