Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Hum Brain Mapp. Author manuscript; available in PMC 2010 May 1.
Published in final edited form as:
PMCID: PMC2736103

Patterns of Structural Complexity in Alzheimer’s Disease and Frontotemporal Dementia


The goal of this project was to utilize an information theoretic formalism for medical image analysis initially proposed in [Young, K., Chen, Y., Kornak, J., Matson G. B., Schuff, N., 2005. Summarizing Complexity in High Dimensions, Phys. Rev. Lett. 94 098701-1] to detect and quantify subtle global and regional differences in spatial patterns in patients suffering from Alzheimer’s disease and Frontotemporal dementia by estimating the structural complexity of anatomical brain MRI. The sensitivity and specificity of the results are compared with those of a recent analysis, currently considered state of the art for MR studies of neurodegeneration. The previous study used regional estimates of cortical thinning and/or volume loss to differentiate between normal aging, Alzheimer’s disease, and Frontotemporal dementia. The analysis illustrates that the structural complexity estimation method, a general multivariate approach to the study of variation in brain structure which does not depend on highly specialized volumetric and thickness estimates, is capable of providing sensitive and interpretable diagnostic information.


Changes in brain structure during aging are complex and it is often hard to distinguish the effects of normal aging from early changes resulting from neurodegenerative pathologies, such as Alzheimer’s disease (AD) and Frontotemporal dementia (FTD) [13]. Subtle changes in structure can nonetheless lead to severe consequences that can be highly specific for a given pathology. In particular AD and FTD are sometimes difficult to clinically differentiate because of overlapping symptoms. [46]. Definite diagnosis currently requires histopathological examination of brain tissue. Although structural MRI data depicts characteristic patterns of brain atrophy in AD and FTD, aiding a differentiating diagnosis between the dementias, [713], a complete separation based on MRI has not been accomplished. Histopathological studies have indicated that AD and FTD pathology are associated with damage to specific cortical layers, e.g. layer II of the entorhinal cortex and layer III of the neocortex in AD and layer III and V of frontal and temporal lobes in FTD[1417]. Though current MRI methods lack power to resolve individual cortical layers, these histological observations led to a MRI-based study of cortical thickness [18], in the hopes that such estimates would improve differential diagnosis between AD and FTD. However, since the cortex is a highly folded structure and its surface is rarely aligned with any of the cardinal axes in MRI, estimates of cortical thickness are difficult, especially in the presence of pathological alterations. Though elegant techniques have recently been developed for estimating cortical thickness from MRI [1923], they were not shown to be substantially more effective than brain volume estimates for distinguishing AD and FTD subjects [18]. This raises the possibility that specialized volumetric and thickness estimates may not be the most effective way to extract available information given current MRI resolution and geometry. In particular a number of combined effects such as variations in sulcal depth, cortical area, and cortical thickness are likely involved in different pathologies and such effects would appear in combination at current MRI resolution. This suggests the use of image markers that capture some degree of combined global and local variability in brain structures while also supplying some degree of interpretability. To this end we have explored the use of information theoretic quantities that provide quantitative measures of structural complexity as image markers [2428]. This approach seems particularly compelling in light of recent evidence that distributed changes in structural complexity of the brain have been shown to occur in specific forms in both normal and pathological age related loss of cognitive function [1].

Since its introduction as a means of studying communication over noisy channels [29, 30], information theory has been successfully applied in a variety of scientific settings, including neuroimaging [31] and modeling of brain function [3]. Information theoretic methods provide a convenient framework for summarizing information in the form of quantitative measures. The particular information theoretic measures chosen for the analysis presented in this paper were initially introduced via a graph theoretical, optimal prediction based formalism that provided estimates of dynamical complexity in time series data [24]. These methods were later generalized to allow for the study of structure in multidimensional, multivariate data sets, in particular multimodal medical images, such as structural and spectroscopic MRI [27, 28]. The fundamental hypothesis is that the complexity of spatial patterns in neuroimages, such as the convoluted spatial distribution of human cortex evident in MRI, can effectively be captured by structure based information theoretic measures that correspond well with visual impressions of image complexity. Hence the specific information theoretic quantities discussed in this paper will be referred to as complexity measures.

The specific objectives of this study were to: 1) determine the extent to which structural complexity measures capture the characteristic spatial patterns of tissue loss and cortical thinning in AD and FTD relative to cognitive normal (CN) subjects, 2) compare the sensitivity and specificity of complexity measures for differentiating between AD and FTD with that provided by current state of the art MRI estimates of cortical volume loss and thinning.



Twenty-three CN subjects, 24 patients diagnosed with AD and 19 patients diagnosed with FTD were included in the study (Table (1)). Patients with FTD and AD were recruited from the Memory and Aging Center of the University of California, San Francisco for a previously described study [33]. All patients were diagnosed based upon information obtained from an extensive clinical history and physical examination. MRI data were used to rule out other major neuropathologies such as tumor, stroke, severe white matter disease, or inflammation but not for diagnosis of dementia. Inclusion criteria were age between 35 and 80 years and no history of brain trauma, brain tumor, stroke, epilepsy, alcoholism, psychiatric illness, or systemic disease that affects brain function. FTD was diagnosed according to established consensus criteria. Patients with FTD, who had motor neuron disease–related symptoms, were excluded. Patients with AD were diagnosed according to the criteria of the National Institute of Neurological and Communication Disorders and Stroke/Alzheimer’s Disease and Related Disorders Association [5]. All subjects received a standard battery of neuropsychological tests. This included assessment of global cognitive impairment by Mini-Mental State Examination (MMSE) [34] and global functional impairment by Clinical Dementia Rating (CDR) Scale [35]. In addition the California Verbal Learning Test (CVLT)–Short Form was administered to assess episodic memory and a modified version of the Trail-Making Test (TMT) was administered to evaluate executive function. Two patients with FTD and 2 CN subjects, who had MRI of inferior quality, were eliminated as the MRI were not suitable for tissue segmentation and spatial normalization processing with FreeSurfer (

Table 1
Study Demographics.

Data acquisition and processing

Data acquisition has also been described in detail in [18]. MRI data were obtained on a 1.5 T Siemens Vision™ System (Siemens Inc., Iselin NJ), including coronal T1-weighted images using a volumetric magnetization-prepared rapid gradient echo sequence (MPRAGE, TR/TE/TI = 10/7/300 ms timing, 15 degree flip angle, 1.00 ×1.00 mm2 in-plane resolution, and 1.40 mm thick coronal partitions. The 3D MPRAGE images were segmented into gray matter, white matter, CSF and non brain tissue and then mapped to the Montreal Neuroimaging (MNI) brain atlas [36] for spatial normalization using the FreeSurfer software.

Assessments of Image Complexity

Three information theoretic measures, the statistical complexity (SC), the entropy (H), and the excess entropy (EE), introduced in [24, 26], described in more detail in the appendix, and applied to medical image analysis in [27, 28], were estimated from the spatially normalized, segmented images. As outlined in ref [28], SC measures the degree of spatially correlated structure in the image. It accomplishes this by quantifying the amount of information required for predicting the image values in a region given the image values in a neighboring region, averaged over the entire image or region under consideration. H measures the degree of apparent randomness in the image and thus corresponds to the usual notion of entropy in physics. It is a complimentary measure of complexity to SC in that it measures the number of patterns observed in an image or region without regard to their correlation structure. While H is a maximum for completely uncorrelated sets of patterns SC is a minimum for such sets. Lastly, EE provides a quantitative complexity measure of the spatial scaling properties of the image. In particular EE quantifies how long it takes the average of H over a volume to converge to a constant value as a function of increasing volume. As simple examples: 1) the ordered pattern of a black and white checkerboard (without noise) would have low H, low SC, and low EE; 2) in contrast, a completely random black and white pattern would have high H, low SC, and low EE. More complicated patterns, like an image of the cortex, would have intermediate values of H, and higher values of SC and EE relative to these two simpler patterns.


To determine the ability of H, SC, and EE to detect structural changes in the brain in AD and FTD, the measures were obtained for 13 brain regions that MRI studies reported, e.g. [37], to be affected by AD, FTD or both. These regions included the anterior cingulum, posterior cingulum, inferior frontal lobe, superior frontal lobe, Heschl gyrus, hippocampus, insula, inferior parietal lobe, superior parietal lobe, precentral gyrus, precuneus, putamen, and inferior temporal lobe.

H, SC, and EE estimates were obtained for the 13 brain regions using the methods described in the appendix at a scale of 2 mm for H and SC and a range of scales from the voxel size to the size of the region for EE. Classification accuracy between paired groups, i.e. AD or FTD versus CN and AD versus FTD was tested using Platt's sequential minimal optimization algorithm for training a support vector classifier. The algorithm used a linear kernel, cache size 250007, and complexity parameter 1.0. Sensitivity, specificity, and overall classification accuracy was assessed using 10-times 10-fold stratified cross validation [38].

In addition, to test how well structural complexity estimation performed when trying to separate all 3 classes at once, linear discriminant analysis (LDA) was performed for various sets of regions for a combined 3 group analysis using the structural complexity estimates and compared with a similar LDA analysis using cortical thickness estimates.


The variability of the three complexity measures in different brain regions is illustrated in Figure (1), separately for single, representative CN, AD, and FTD subjects. An additive red-green-blue (RGB) color space is used to represent simultaneous values of H, EE, and SC. In this color space the value of H is represented on the red axis, EE on the green axis and SC on the blue axis. In this representation, a higher saturation of red represents a higher value of H, implying lack of correlation of structural patterns in an image region. Similarly, a higher saturation of green represents a higher value of EE, implying increased long range correlations of structural patterns and a higher saturation of blue represents a higher value of SC, implying an increase of locally correlated patterns. Accordingly, a simultaneous increase/decrease of all three complexity measures results in brighter/darker levels of gray. The most prominent effects in the AD subject compared to the CN and FTD subjects as seen in this representation are decreased correlation in the hippocampus (faint red regions, yellow arrows in columns 1 and 2) and diminished long range correlations of structural patterns in superior parietal lobe regions (faint green regions, arrows in column 6). In contrast, the most prominent effect in the FTD subject compared with the CN and AD subjects is greater long range correlation in medial frontal lobe and anterior cingulum (intense green regions, arrows in columns 5 and 6). Somewhat surprisingly, AD and FTD show little change of the complexity measures in the posterior cingulum (gray regions, arrows in column 4) relative to CN. In addition to differences between the groups, the color scheme also illustrates anatomical differences in that inferior brain regions seem dominated by decreased correlation in spatial patterns (red, H) whereas superior brain regions are dominated by greater long range correlations of patterns (green, EE) and subcortical structures, such as the basal ganglia, by more locally correlated patterns (blue, SC). This representation was provided to convey a direct visual impression of how the complexity measures vary across individual subjects in different groups but to increase its utility, for example as it might be used for clinical assessment, a way to additionally provide a visual characterization of group variability for the various groups (and in particular normal subjects) would be necessary.

Figure 1
Simultaneous variability of entropy (H), excess entropy (EE) and statistical complexity (SC) of different brain regions in a single control subject, a single subject diagnosed with AD, and a single subject diagnosed with FTD, represented in an additive ...

Table (2) compares results using the structural complexity estimation against results on use of cortical thickness estimation using the FreeSurfer software on the same set of subjects. In the table, comparisons are between classification accuracy based on structural complexity estimation and classification accuracy based on tissue volume and cortical thickness estimation (the parietal lobes provided the best separation between AD and CN subjects and the only significant separation between AD and FTD subjects for the volume and thickness estimates). For each, complexity or FreeSurfer, the regions providing the best separation between the groups are listed, i.e. for complexity the hippocampus, parietal lobe, precuneus, and heschel gyrus taken together and for FreeSurfer the thickness of parietal lobe gray matter. This shows that structural complexity measures slightly outperformed volume and cortical thickness measures for the differential classification between AD and FTD as well as between FTD and CN. For the classification between AD and CN, volume and cortical thickness estimation achieved slightly higher classifications than structural complexity estimation. In addition to classification accuracy, used in the direct comparison with the cortical thickness and volume study, Table (3) lists the sensitivity and specificity obtained for structural complexity estimation for the 2 group comparisons.

Table 2
Comparison of classification accuracy (in percent of total subjects) between AD/CN groups, FTD/CN groups, and AD/FTD groups for complexity estimation methods and cortical thinning.
Table 3
Classification accuracy, sensitivity, and specificity (in percent of total subjects) between AD/CN groups, FTD/CN groups, and AD/FTD groups for structural complexity estimation.

Table (4) lists the prediction accuracy of structural complexity and FreeSurfer measures when trying to separate all three 3 groups at once, using LDA based on either complexity or FreeSurfer volume and thickness estimates. The prediction accuracy is also illustrated graphically in Figures (2a), (2b), and (2c) which depict the projections onto the first two linear discriminants (labeled LD1 and LD2 in the Figures) from the LDA corresponding to the region selections for complexity estimation. This shows first that group separation prominently increased from global measures, such as whole brain, to more focal measures, such as each of the 13 regions, as expected. Second, structural complexity measures outperformed cortical thickness and volume estimates when utilizing specific focal information. In detail, using the structural complexity estimates from all 13 brain regions produced an LDA prediction accuracy of 96 %, whereas a similar LDA for volume and cortical thickness (illustrated in Figure (3)) estimates in the same regions [18] achieved a prediction accuracy of 90 %.

Figure 2Figure 2Figure 2
Results of linear discriminant analysis (LDA) using structural complexity estimates with x and y axes representing projections of complexity estimates onto the 1st and 2nd linear discriminants. Figure (2a) shows complexity analysis results for only the ...
Figure 3
Results of linear discriminant analysis (LDA) using cortical thickness and volume estimates with x and y axes representing projections of thickness and volume estimates onto the 1st and 2nd linear discriminants using a set of regions similar to that used ...
Table 4
Linear Discriminant Analysis (LDA) results for application of structural complexity estimation to separation of AD/FTD/CN groups.

To provide some indication of the baseline complexity measures for normal subjects compared with those for AD and FTD subjects Table (5) shows comparisons of average H and EE values in the frontal lobe and hippocampus, respectively, for the various subject groups analyzed in the paper. Significance values for between group t-tests are provided in Table (5) as well.

Table 5
Means and standard deviations for Hippocampal EE and Frontal H, and p-values for between group t-tests.


This paper provides two main results: 1) it was shown that use of structural complexity estimates is effective at capturing systematic differences on brain MRIs, exhibiting a variety of effects such as cortical volume loss and thinning, and 2) it was demonstrated that use of structural complexity estimates can achieve similar classification results between dementia and controls as well as between AD and FTD as highly specialized measures of cortical thinning. It should be noted that complexity estimation achieved similar group classifications to cortical thickness estimates using FreeSurfer for the same brain regions (e.g. parietal lobe). In selected brain regions (i.e. the hippocampus), complexity estimates achieved even better classification than FreeSurfer thickness and volume estimates. The classification accuracy provided by both methods is in fact at the limit of the ability to reliably diagnose subjects and so further comparisons between classification methods will require improved clinical testing methods, much larger samples, or some other, more accurate means of classification to account for test variability. Nonetheless, the results suggest that the complexity based formalism for image analysis shows promise for classification of neurodegenerative diseases.

Similar to conventional image processing, the analysis involved tissue segmentation and spatial normalization to a brain atlas. It should be noted, however, that structural complexity estimation does not in principal require prior segmentation or spatial normalization of the images. The appendix provides a description of how continuous intensity images are processed by the complexity estimation method. The reason for performing the initial FreeSurfer gray/white/CSF tissue segmentation was to bring the computational dimensionality to a manageable size by using a few tissue classes rather than the continuum of image intensities. In addition it was felt that decoupling the segmentation step from the structural complexity estimation by using a method that is widely used for segmentation (FreeSurfer) in thickness and volume studies, provided a better basis for comparison of methods. Though various segmentation methods could well affect complexity estimation results, the fact that complexity estimation methods performed well at separating the classes when using segmented images generated with FreeSurfer suggests that, at least initially, the utility of the complexity estimation methods can be evaluated independently. But further exploration of the effects of segmentation algorithms on the complexity estimation methods is an important step, both to gain a better understanding of the interaction, as well as to attempt to improve the sensitivity of the complexity estimation methods in the face of neural abnormalities such as white matter lesions.

Similarly, spatial normalization was used for region based comparisons with conventional volume and thickness estimates but global (or arbitrary region) versions of the structural complexity estimates can be obtained without the step of spatial normalization. This offers the possibility of eliminating the difficult problem of choosing appropriate group specific atlases for the analysis [39, 40].

Despite their simplicity and fully automated application, the structural complexity estimates provided classification accuracy similar to that from volume and thickness estimates. As expected, the parietal lobe volume and thickness estimates already yielded excellent separation between CN and AD groups. That was difficult to match by structural complexity estimates, as specific local tissue volume changes are expected to provide a good image marker for AD related changes [11] and such prior information should certainly be taken into account if available. In contrast, structural complexity estimation yielded a better separation between the CN and FTD groups as well as the AD and FTD groups, especially in regions considered to be affected by both AD and FTD. Frontal and temporal lobe volume and thickness estimates both provided comparable separation of the CN and FTD groups to structural complexity estimation but didn’t do as well at separating the CN and AD groups and showed no significant separation between the AD and FTD groups. This suggests that structural complexity estimation potentially provides a more robust overall method of separating and classifying populations in a realistic setting where subjects generally exhibit a variety of conditions. The general nature of the method allows it to be applied with a minimum of assumptions about the locality of disease specific effects as well as at which anatomical locations the effects are expect to appear in the images. Thus, the method should be particularly effective for simultaneous examinations of spatially scalable effects that may occur across the wide spectrum of neurodegenerative diseases.

In both AD and FTD effects are expected in a number of regions and the structural complexity estimation analysis was effective at distinguishing those situations. However, discrimination between the 3 groups based on global structural complexity estimates was not particularly effective, as depicted in Figure (2a). The reason for this is that on the level of the whole brain, CN subjects are reasonably well separated from those suffering from neurodegeneration but as might be expected for a measure that doesn’t provide for any regional distinctions, AD and FTD yield similar values for the structural complexity estimates. That is, a whole brain analysis ignores the clear regional differences in neurodegeneration exhibited in AD and FTD patients; this would also be true of a whole brain cortical thickness and/or volume analysis. On the other hand if 3 regions, known to be strongly affected in one or both AD and FTD, are considered, i.e. the hippocampus, subiculum, and precuneus, the prediction accuracy is considerably improved, as seen in Table (3) and Figure (2b). Finally, including more predetermined regions that are considered strongly affected by FTD and/or AD achieves an excellent separation of the 3 groups, as shown Figure (2c) and Figure (3).

While it is a long term goal of the authors to establish baseline estimates of H, SC, and EE for normal subjects, given the that this work is in its early stages, the primary goal of the present paper is to determine useful image classification methods based on between group differences. This must be viewed as a limitation to clinical application of the complexity estimation methods but this situation is similar to other image based classification methods such as those based on thickness and volume estimates. Establishing baseline complexity estimates will involve identifying sources of variability such as changes in normal aging. Some indication of the differences between H and EE between baseline values for normal subjects and values for AD and FTD subjects in regions affected in AD and FTD, is provided by the data in Table (5). The measures and regions used for Table (5) were chosen for illustration based on the relatively well understood clinical effects in those regions. Hippocampal atrophy (represented as a slower decay of spatial entropy represented by higher EE) has been found in MRI studies of both AD and FTD subjects. Frontal lobe atrophy (represented as lower spatial entropy, H) has been found in MRI studies of FTD subjects and to a lesser degree in advanced stage AD subjects.

Though the complexity estimation results were promising in terms of providing image based classification of subjects with AD and FTD, a number of issues remain before the methods can provide a concrete, interpretable tool suitable for clinical use. Future work will extend structural complexity estimation to multimodal imaging, as demonstrated in [27], to studies of neurodegenerative disease. This approach is expected to be particularly effective as it doesn’t depend on spatially confined effects in the different modalities for its classification power as is the case for multivariate image analysis [41]. In addition it provides a more general and interpretable approach to understanding structural image properties than methods such as fractal [2] and texture analysis [32]. In contrast to methods such as that described in [42] which provide accurate classification via exploratory analysis, the methods described in this paper are fully automatic and independent of operator bias.

In conclusion, information theory based structural complexity estimation shows promise for use in the study and classification of neurodegenerative disease.


This work was supported in part by a Research Applications Grant of the Department of Veterans Affairs REAP, NIH grants AG10897, a Program Project Grant from the National Institute of Aging (PO1AG19724), ADRC (P50-AG023501) and a Network Grant from the Hillblom Foundation.


In this appendix a brief mathematical description of the quantities, H, SC, and EE and how they are obtained is provided. This discussion closely follows that contained in [28].


The components required for defining SC and H are an index set I representing a set of spatial or space/time coordinates and a feature space F defined over I , representing some set of variables defined at each index. The feature space used in this paper consists of the values at each voxel of a set of coregistered images in a standardized space, in particular the 1 dimensional space of intensity values from T1 images.

Definition of the complexity measures requires the conditional probability


defined in terms of the joint empirical distribution over observed patterns


where the template pair, Tj1 and Ti2, are ordered sets of indices forming distinct but possibly overlapping regions of voxels, in the underlying index set I, at positions j and i respectively. N is the total number of template pairs observed over the whole image or region. FTk is a product feature space, which is the product space of values in the image(s) at voxels in the templateTk. The mapping M: FTkZ , maps the set of all patterns observed in the image or region over the product feature space FTk , to the integers. That is M indexes the observed patterns, assuming that the set of patterns form a discrete set. The sums are taken over all particular instances, Tik, that is instances of template k “located” at index i . The functional δ (·)is the indicator “function”, yielding 1 if a particular pair of patterns is observed, 0 otherwise. Thus P(Z = M (FT1) | Z ' = M (FT2)) effectively constitutes a 2D histogram of counts of feature space patterns observed when parsing over the index space with the templates Tj1 and Ti2. Further restrictions can be imposed such as requiring that T1 and T2 be non-overlapping and/or contiguous. In the analysis presented in this paper T1 and T2 are linear sets of contiguous voxels of a given length L. Hence the analysis effectively studies local correlation structure at scale L. In the following we will simplify the notation using the conversion.Z = M(FTk) → Z The marginal distribution is then defined as:


To estimate SC the set of optimally predictive states must be determined from the images.

Determination of states

For this paper the states are determined after tissue segmentation of the univariate T1. Better segmentation algorithms provide some benefit in obtaining better complexity estimates but complexity estimation as a whole, given the global nature of the analysis, can be viewed as less sensitive to the accuracy of segmentation algorithms than techniques that rely on accurate “local” information such as cortical thickness estimation.

The next step in the algorithm is to choose templates and parse the image using those templates. The templates are moved over the image and the image values over the templates are recorded as counts in a joint histogram. The number of joint histogram bins sv1 × sv2 is determined by the number of segmentation values, s, in the image and the number of voxels, v1 and v2, in the templates. sv1 is the number of possible patterns that can be observed over template 1 and sv2 is the number of possible patterns that can be observed over template 2. For this paper the two template structures are identical, simple linear sequences of voxels, and sv1 and sv2 are identical.

From the joint histogram the conditional histograms defined in equation (1) are obtained. The conditional histograms are then grouped, based on a measure of similarity for probability distributions; there are a number of choices for similarity measure between distributions. In this paper a hard clustering algorithm, PAM [43], is used with the Euclidean distance between bin counts as the similarity measure. Note that the grouping of conditional histograms into states is the step that specifically characterizes structure in the image and distinguishes the complexity based method from simply estimating entropies from the joint histogram as a co-occurrence matrix [28]. This step is also what distinguishes SC from the joint entropy though in the special case where there is a one to one correspondence between joint histograms and states, SC reduces to the joint entropy. This can occur when there are a small number of joint histograms that are too distinct to be clustered into states.

The ability to specify arbitrary template structures also distinguishes the complexity estimation methods from standard co-occurrence analysis. In addition, while the joint histogram, analogous to a co-occurrence matrix, is a convenient representation of the results of the image parsing step, the underlying construction is based on a rigorously defined and general set of graph theoretic methods described in [24].

Statistical Complexity (SC)

Given the set of states s , accumulated as just described, SC can then be defined as:


This quantifies the information contained in the distribution of observed patterns, conditioned and summed over all states, s. Note that as mentioned above, via conditioning over the states s, SC is distinct from the conditional entropy of the histogram and measures something quite different. For example when the set of N conditional histograms making up the joint histogram are identical the joint entropy would equal N times the entropy of one of the conditional histograms. But SC would equal zero as there is only a single state.

Though not explicitly represented in equation (4) SC is dependent on the particular templates T1 and T2 over which the patterns are observed and implicitly on the scale of the template patterns. However, as is demonstrated by the results reported above, as long as the choice of template patterns is consistently applied, the results can be used to detect systematic group differences exposed in the images.

Entropy (H)

The standard measure of the number and distribution of observed patterns produced by a system is the entropy


where the joint probabilities were defined in equation (2). The entropy in the current case is over the joint distribution defined in equation (2), accumulated over the underlying template pairs and summed over the indices of the observed patterns.

Excess Entropy (EE)

EE measures the convergence rate of H as a function of increase in volume. Despite some subtleties in interpreting EE, discussed in [26], it provides a useful and complimentary measure to H and SC. EE was defined and initially discussed in [26]. We first define the metric entropy as the limit of entropy per unit volume as the volume is taken to infinity, initially without specifying explicitly how the volume is to be taken to infinity:


EE is then defined as:


where V is the volume of the templates in index space. Despite the subtleties involved with estimating EE in dimensions higher than 1, any particular choice, consistently applied, results in a useful method for comparing structure in images. In particular note that if the entropy per unit volume converges quickly to hμ as a function of scale then EE is small indicating a lack of large scale structures in the image.

EE provides an estimate of the scaling properties in image data that compliments information obtained using SC and H. Specifically H provides a measure of the number and distribution of structures observed at a given template scale, SC provides a measure of the complexity of the spatial correlation of those structures at a given template scale, and EE provides a measure of the variation in the number of observed structures as a function of template scale


For the particular case of classifying neurodegenerative disease via T1 MRI, the steps for obtaining sets of states and calculating SC, H, and EE are as follows:

  1. Feature space reduction - For the analysis in this paper, feature space reduction, had already been performed given that the complexity estimation steps were applied to the segmented T1 image data. That is, the feature space consists of a single value, the intensity of a T1 -weighted MR brain image, and the feature space reduction corresponds to standard tissue segmentation of the brain image into 3 classes, gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) via FreeSurfer
  2. Parsing - The index space is parsed according to the specified templates. The observed patterns over all instances of the templates in the index space are then catalogued via the joint histogram. In this study parsing is done for the 3 dimensional index space in the cardinal directions (x, y, z), and templates consist of linear sequences of voxels in those directions.
    To obtain the list of distinct states, as describe above, a robust hard clustering algorithm (PAM [43] in the current implementation) is used to group the conditional distributions. There are various similarity measures that could be used to group the conditional histograms into states but use of PAM for clustering based on Euclidean distance between histogram bin counts proved to be relatively robust in being transparent to numerical problems resulting from histogram bins with zero counts.
  3. Complexity estimation - The total probabilities of the states, that is the normalized sums of the histogram counts in the conditional histograms in that cluster, are obtained and used to estimate SC using equation (4). H is estimated over all histogram bins treated independently using equation (5).

The software implementation of the above methods is an open source package written in Python [44] and SciPy [45], and uses the Rpy [46] package to provide access to the statistical and graphical capabilities of the R statistical language [47] and supplemental libraries. The cluster and e1071 [43] R packages were used for clustering and the AnalyzeFMRI [48] package for MR image processing. Image analysis was performed using this package on a 46 processor Beowulf cluster using the PyPAR [49] Python wrapper for the message passing interface MPI. Complete (fully automated) processing of a single subject takes on the order of 40 minutes on a single 3 GHz processor.


1. Dickstein DL, Kabaso D, Rocher AB, Luebke JI, Wearne SL, Hof PR. Changes in the structural complexity of the aged brain. Aging Cell. 2007;6(3):275–284. [PMC free article] [PubMed]
2. Zhang L, Dean D, Liu JZ, Sahgal V, Wang X, Yue GH. Quantifying degeneration of white matter in normal aging using fractal dimension. Neurobiol. Aging. In Press. [PubMed]
3. Tononi G. Consciousness, information integration, and the brain. Prog Brain Res. 2005;150:109–126. [PubMed]
4. Neary D, Snowden JS, Gustafson L, Passant U, Stuss D, Black S, et al. Frontotemporal lobar degeneration: a consensus on clinical diagnostic criteria. Neurology. 1998;51(6):1546–1554. [PubMed]
5. McKhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM. Clinical diagnosis of Alzheimer's disease: report of the NINCDS- ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer's Disease. Neurology. 1984;34:939–944. [PubMed]
6. Siri S, Benaglio I, Frigerio A, Binetti G, Cappa SF. A brief neuropsychological assessment for the differential diagnosis between frontotemporal dementia and Alzheimer's disease. Eur J Neurol. 2001;8(2):125–132. [PubMed]
7. Rosen HJ, Gorno-Tempini ML, Goldman WP, Perry RJ, Schuff N, Weiner M, et al. Patterns of brain atrophy in frontotemporal dementia and semantic dementia. Neurology. 2002;58(2):198–208. [PubMed]
8. Grossman M, McMillan C, Moore P, Ding L, Glosser G, Work M, et al. What's in a name: voxel-based morphometric analyses of MRI and naming difficulty in Alzheimer's disease, frontotemporal dementia and corticobasal degeneration. Brain. 2004;127(Pt 3):628–649. [PubMed]
9. Kitagaki H, Mori E, Yamaji S, Ishii K, Hirono N, Kobashi S, et al. Frontotemporal dementia and Alzheimer disease: evaluation of cortical atrophy with automated hemispheric surface display generated with MR images. Radiology. 1998;208(2):431–439. [PubMed]
10. Gee J, Ding L, Xie Z, Lin M, DeVita C, Grossman M. Alzheimer's disease and frontotemporal dementia exhibit distinct atrophy-behavior correlates: a computer-assisted imaging study. Acad Radiol. 2003;10(12):1392–1401. [PubMed]
11. Laakso MP, Frisoni GB, Kononen M, Mikkonen M, Beltramello A, Geroldi C, et al. Hippocampus and entorhinal cortex in frontotemporal dementia and Alzheimer's disease: a morphometric MRI study. Biol Psychiatry. 2000;47(12):1056–1063. [PubMed]
12. Frisoni GB, Laakso MP, Beltramello A, Geroldi C, Bianchetti A, Soininen H, et al. Hippocampal and entorhinal cortex atrophy in frontotemporal dementia and Alzheimer's disease. Neurology. 1999;52(1):91–100. [PubMed]
13. Lipton AM, Benavides R, Hynan LS, Bonte FJ, Harris TS, White CL, III, et al. Lateralization on neuroimaging does not differentiate frontotemporal lobar degeneration from Alzheimer's disease. Dement Geriatr Cogn Disord. 2004;17(4):324–327. [PubMed]
14. Kersaitis C, Halliday GM, Kril JJ. Regional and cellular pathology in frontotemporal dementia: relationship to stage of disease in cases with and without Pick bodies. Acta Neuropathol (Berl) 2004;108(6):515–523. [PubMed]
15. Pearson RC, Esiri MM, Hiorns RW, Wilcock GK, Powell TP. Anatomical correlates of the distribution of the pathological changes in the neocortex in Alzheimer disease. Proc Natl Acad Sci U S A. 1985;82(13):4531–4534. [PubMed]
16. Lewis DA, Campbell MJ, Terry RD, Morrison JH. Laminar and regional distributions of neurofibrillary tangles and neuritic plaques in Alzheimer's disease: a quantitative study of visual and auditory cortices. J Neurosci. 1987;7(6):1799–1808. [PubMed]
17. Gomez-Isla T, Price JL, McKeel DW, Jr, Morris JC, Growdon JH, Hyman BT. Profound loss of layer II entorhinal cortex neurons occurs in very mild Alzheimer's disease. J Neurosci. 1996;16(14):4491–4500. [PubMed]
18. Du AT, Schuff N, Kramer JH, Rosen HJ, Gorno-Tempini ML, Rankin K, Miller BL, Weiner MW. Different regional patterns of cortical thinning in Alzheimer's disease and frontotemporal dementia. Brain. 2007 Apr;130(Pt 4):1159–1166. [PMC free article] [PubMed]
19. Lerch JP, Pruessner JC, Zijdenbos A, Hampel H, Teipel SJ, Evans AC. Focal decline of cortical thickness in Alzheimer's disease identified by computational neuroanatomy. Cereb Cortex. 2005;15(7):995–1001. [PubMed]
20. Fischl B, Sereno MI, Dale AM. Cortical surface-based analysis. II: Inflation, flattening, and a surface-based coordinate system. Neuroimage. 1999;9(2):195–207. [PubMed]
21. Dale AM, Fischl B, Sereno MI. Cortical surface-based analysis. I. Segmentation and surface reconstruction. Neuroimage. 1999;9(2):179–194. [PubMed]
22. Fischl B, Dale AM. Measuring the thickness of the human cerebral cortex from magnetic resonance images. Proc Natl Acad Sci U S A. 2000;97(20):11050–11055. [PubMed]
23. Lerch JP, Pruessner JC, Zijdenbos A, Hampel H, Teipel SJ, Evans AC. Focal decline of cortical thickness in Alzheimer's disease identified by computational neuroanatomy. Cereb Cortex. 2005;15(7):995–1001. [PubMed]
24. Crutchfield JP, Young K. Inferring Statistical Complexity. Physical Review Letters. 1989;63:105–107. [PubMed]
25. Hopcroft JE, Ullman JD. Introduction to Automata Theory, Languages, and Computation. 2nd Edition. Addison Wesley; 2000.
26. Feldman DP, Crutchfield JP. Structural Information in Two-Dimensional Patterns: Entropy Convergence and Excess Entropy. Physical Review E. 2003;67:051104. [PubMed]
27. Young K, Chen Y, Kornak J, Matson GB, Schuff N. Summarizing Complexity In High Dimensions. Physical Review Letters. 2005;94:098701. [PubMed]
28. Young K, Schuff N. Measuring Structural Complexity in Brain Images. Neuroimage. 2008;39(4):1721–1730. [PMC free article] [PubMed]
29. Shannon CE. A mathematical theory of communication. Bell Sys Tech Journal. 1948;27:379–423.
30. Cover T, Thomas J. Elements of Information Theory. Wiley-Interscience; 2006.
31. Pluim JP, Maintz JB, Viergever MA. F-information measures in medical image registration. IEEE Trans Med Imaging 2004. 2004;23(12):1508–1516. [PubMed]
32. Freeborough PA, Fox NC. MR Image Texture Analysis Applied to the Diagnosis and Tracking of Alzheimer's Disease. IEEE Transactions on Medical Imaging. 1988;17(3):475–479. [PubMed]
33. Du AT, Jahng GH, Hayasaka S, Kramer JH, Rosen HJ, Gorno-Tempini ML, et al. Hypoperfusion in Frontotemporal Dementia and Alzheimer's Disease by Arterial Spin Labeling MRI. Neurology. 2006 Oct 10;67(7):1215–1220. [PMC free article] [PubMed]
34. Folstein MF, Folstein SE, McHugh PR. "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12:189–198. [PubMed]
35. Morris JC. The Clinical Dementia Rating (CDR): current version and scoring rules. Neurology. 1993;43(11):2412–2414. [PubMed]
36. Tzourio-Mazoyer N, Landeau B, Papathanassiou D, Crivello F, Etard O, Delcroix N, Mazoyer B, Joliot M. Automated Anatomical Labeling of Activations In SPM Using a Macroscopic Anatomical Parcellation of the MNI MRI Single-subject Brain. Neuroimage. 2002;15(1):273–289. [PubMed]
37. Varma AR, Adams W, Lloyd JJ, Carson KJ, Snowden JS, Testa HJ, Jackson A, Neary D. Diagnostic patterns of regional atrophy on MRI and regional cerebral blood flow change on SPECT in young onset patients with Alzheimer's disease, frontotemporal dementia and vascular dementia. Acta Neurol Scand. 2002;105:261–269. [PubMed]
38. Hastie T, Tibshirani R, Friedman J. The elements of statistical learning: data mining, inference, and prediction. New York: Springer; 2001.
39. Mega MS, Dinov ID, Mazziotta JC, Manese M, Thompson PM, Lindshield C, Moussai J, Tran N, Olsen K, Zoumalan CI, Woods RP, Toga AW. Automated brain tissue assessment in the elderly and demented population: construction and validation of a sub-volume probabilistic brain atlas. Neuroimage. 2005;26(4):1009–1018. [PubMed]
40. Lorenzen P, Davis B, Joshi S. Unbiased atlas formation via large deformations metric mapping. Med Image Comput Comput Assist Interv. 2005;8(Pt 2):411–418. [PubMed]
41. Worsley KJ, Taylor JE, Tomaiuolo F, Lerch J. Unified univariate and multivariate random field theory. Neuroimage. 2004;23 Suppl 1:S189–S195. [PubMed]
42. Bocti C, Rockel C, Roy P, Gao F, Black SE. Topographical patterns of lobar atrophy in frontotemporal dementia and Alzheimer's disease. Dement Geriatr Cogn Disord. 2006;21(5–6):364–372. [PubMed]
43. Dimitriadou E, Hornik K, Leisch F, Meyer D, Weingessel A. R Package: e1071: Misc Functions of the Department of Statistics. 2004. Available at
44. van Rossum G, Drake FL, editors. Python Reference Manual. Virginia, USA: PythonLabs; 2001. Available at
45. Jones E, Oliphant T, Peterson P, et al. SciPy: Open Source Scientific Tools for Python. 2001. Available at
46. Moreira W. RPy Package. 2004. Available at
47. R Development Core Team. R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing ISBN 3-900051-00-3. 2004. Available at
48. Marchini JL. R Package: AnalyzeFMRI: Functions for Analysis of fMRI Datasets Stored in the ANALYZE Format. 2004. Availablae at
49. Nielsen O, Ciceri GP, Ramachandran P, Orr D, Kaukic M. PyPAR - Parallel Python, Efficient and Scalable Parallelism Using the Message Passing Interface (MPI) 2003. Avaialble at