|Home | About | Journals | Submit | Contact Us | Français|
The recent commentary by Derrfuss and Mar (2009) proposed a universal coordinate database to archive functional neuroimaging results. In this response, we discuss our strategy in developing the BrainMap database, which was created as a mechanism to promote coordinate-based meta-analysis methods.
Recent commentaries by Derrfuss and Mar (2009), Hamilton (2009), Nielsen (2009), and Van Essen (2009) all advocated the creation of a universal coordinate database as a means to aggregate functional neuroimaging results published in standardized coordinate form, which are growing at a rapidly accelerating pace. The BrainMap database (Fox and Lancaster, 2002) currently contains 1729 papers, 8007 experiments, and 64,940 foci (as of April 2009); Derrfuss and Mar estimate this to be approximately 20% of the relevant literature, making it the largest coordinate-based database in functional neuroimaging to date. BrainMap is a freely accessible community database in which reported activations can be searched for within user-defined ROI boundaries, thus offering the opportunity to relate behavioral functions to specific brain locations.
Reconciling new results to those previously published can be overwhelming, particularly when the relevant studies pertain to different research domains. Derrfuss and Mar proposed that a coordinate database be used to comprehensively identify published studies reporting activation in a given brain region, so that researchers can compare papers reporting foci proximate to their own results. Given the extremely large amount of neuroimaging results that have been reported thus far, the BrainMap project has elected to focus on coordinate-based meta-analysis methods to synthesize this data and provide a means to ascribe a set of functions to a given set of brain regions (Fox et al., 2005a). Derrfuss and Mar calculated that approximately 330 coordinates have been reported in the literature for every single cubic centimeter of gray matter, which is an impressive statistic that conveys the enormity of the task of results summation. Without the aid of meta-analysis, users of a universal coordinate database who query regions of interest will be left with long lists of published studies, the contents of which must be manually filtered and interpreted. BrainMap’s approach to investigating function-location correspondences has been to reduce this burden of labor by developing and promoting quantitative meta-analyses of peak coordinates and their associated metadata. The BrainMap database offers the ability to not only retrieve studies returned by regional searches without domain-specific biases, but also provides the means to synthesize the search results into coherent brain networks using the GingerALE meta-analysis application (Laird et al., 2005a).
Coordinate databases offer an opportunity to localize brain activation from a number of different studies that employed a wide array of tasks. The easiest and most rapid path to achieving a comprehensive coordinate database is to archive only coordinates and citation information; however, the range of potential inferences to be made from this type of database is limited. Systematically establishing function-location associations requires that function must somehow be defined in relation to the archived coordinates. To accomplish this, metadata for each focus must be extracted from the published studies. From 1992 to 1998, developers of the BrainMap database held a series of annual workshops in which leaders of the field debated the structure for a taxonomy of functional neuroimaging experiments. Much of the debate focused on determining the appropriate level of detail for what eventually evolved into the BrainMap coding scheme. These metadata allow each coordinate to be linked with how the observed activation was experimentally derived, a formulation that lends itself to rich data mining possibilities. BrainMap’s power to capture knowledge associated with function- location relationships is due to both the quantity and quality of metadata that is archived. But the ability to perform complex analyses of coordinate data in BrainMap comes at the cost of manually extracting metadata from each publication. Peer-reviewed publications can be submitted to BrainMap by the original authors (uncommon) or by investigators performing a meta-analysis (very common); two BrainMap research assistants also enter data on a full-time basis. All entries are reviewed for quality control by BrainMap staff and faculty before being entered into the database to ensure the accuracy and consistency of coding.
In addition to citation information, the current BrainMap coding scheme contains metadata descriptions on subjects, experimental conditions (stimulus, response, instructions), paradigms, and behavioral domains. Derrfuss and Mar suggest that a greater volume of the literature could be more effectively archived by a reduction of BrainMap’s required submission fields. However, their recommended list of necessary core fields is nearly comprehensive to BrainMap’s current structure. Reduction of the design and results of an entire neuroimaging experiment into a small set of metadata fields is a complex neuroinformatical dilemma, with agreement rarely observed across investigators as to which are the truly critical components. We agree that BrainMap’s data entry procedure can be time-consuming (Laird et al., 2005b). It takes a research assistant approximately 30–60 minutes to enter the details of a single publication into our data entry application, Scribe. However, we argue that the depth of the current coding strategy is what provides diverse data mining opportunities and hence increases the value of the database. Examination of published studies reveals that the BrainMap taxonomy performs well in matching to search filters applied by meta-analysis authors, thereby reducing the time needed for manual searches of the literature (Fox et al., 2005b). The current depth of the BrainMap coding scheme represents our instantiation of a compromise between a rapid data entry procedure and a sufficient level of metadata to yield useful data mining results.
In the BrainMap taxonomy, a structured keyword system has been favored over free text entry to reduce redundancy due to alternative or competitive terminology. Only an ontology for functional neuroimaging experiments will fully prevent the loss of information associated with alternative vocabularies; however, a complete ontology does not currently exist in this domain despite increasing acknowledgement that one is necessary (Toga, 2002; Price and Friston, 2005; Poldrack, 2006; Binder et al., 2009). Such an ontology would also enable classification of studies in BrainMap at a deeper level of detail. Poldrack (2006) argues that BrainMap’s divisional structure for behavioral domains is too coarse and does not allow for experiments to be coded at a sufficient level of detail to enable meaningful structure-function mappings. Yet until an ontology of cognitive processes is developed and adopted by the functional neuroimaging community, the use of alternative terminologies will result in the dilution of concepts over many domains, thereby interfering with both the data submission and retrieval procedures in coordinate-based databases. At this time, it is unwarranted to assume that functional neuroimaging results will reveal an organizational structure of the human mind that conforms to theoretical cognitive models, such as those associated with cognitive architectures (Langley et al., 2009) or conceptual primitives (Mandler, 2006). Until a more finely detailed behavioral domain hierarchy is tested and validated against neuroimaging data, BrainMap will continue to utilize a broader approach to behavioral classifications. This domain structure was designed to group like studies, rather than segregate them based on an unproven classification system.
A comprehensive coordinate database would undoubtedly be a welcome addition to the neuroimaging community, as this would allow researchers to expedite their literature searches and streamline the collation of relevant coordinates. Although not comprehensive, BrainMap contains a significant percentage of the literature and “provides a broad enough sample of different studies to provide a useful proof of concept” (Poldrack, 2006). While we aim for a database that contains 100% of eligible studies, funding restrictions dictate that the BrainMap project pursue a joint strategy of both data archival and tool development. We are unwilling to redistribute our efforts to focus solely on data entry, as this would undermine our ability to develop and improve meta-analysis tools. In this way, we favor an ideology in which scientific contributions are valued over the convenience provided by a comprehensive database. For example, we recently implemented several modifications of the activation likelihood estimation (ALE) method (Eickhoff et al., 2009), which is the coordinate-based meta-analysis method supported by BrainMap. ALE now includes: estimates of the between-subject and between-laboratory variability, to more explicitly model the spatial uncertainty associated; and weights each study by the number of included subjects. The method of testing for statistical significance in ALE was also modified, resulting in a transition from fixed-effects to random-effects meta-analyses. In addition, we have developed and validated coordinate conversion algorithms that reduce the disparity between MNI and Talairach coordinates (Lancaster et al., 2007). These corrections for varying spatial normalization techniques affect both data retrieval results for regional queries of coordinates, as well as meta-analytic results since more accurate coordinate corrections result in tighter, more coherent nodes of concordance. BrainMap automatically applies these corrections to incoming database entries as part of the data submission process, a feature that directly resulted from our commitment to the development of meta-analysis methods and neuroinformatics tools.
It is our aim that this commitment will allow further extension of how these methods are applied, such that coordinate-based meta-analyses are not limited to the simple pooling of studies utilizing the same experimental task. We strive for a wider scope of applications in line with the BrainMap’s intended goal of facilitating the creation of a functional brain atlas. BrainMap’s search capabilities can support various types of queries, such as “for a given function, what regions are typically engaged?”, “for a given region, what tasks elicit activation?”, or “for a given region, what other regions are coactivated?”. Using metadata archived in BrainMap these correspondences (function-to-regions, region-to-tasks, or region-to-network) can be constructed in a data-driven manner. For example, BrainMap is capable of generating function-to-regions associations by creating whole-brain meta-analytic maps for each behavioral domain category, which can then be decomposed into sub-networks based on different levels of the domain hierarchy. Region-to-network correspondences can be constructed by analyzing which foci coactivate with coordinates located in an anatomically defined region of interest, as a meta-analytic analogue of functional connectivity studies (Koski and Paus, 2002; Postuma and Dagher, 2006; Toro et al., 2008). Applying high-level filters from the entire BrainMap coding scheme to either the domain or coactivation meta-maps may be an effective strategy for refining their spatial specificity. Thus, while paradigm class and behavioral domain have been established as important metadata fields in the BrainMap coding scheme, other fields, such as stimulus modality and response type, also have the potential to assist in unraveling the brain’s systems and their interactions.
In conclusion, we agree with Derrfuss and Mar in their discussion of the value offered by a universal coordinate database, but caution database developers in designing neuroinformatics tools with limited applicability. A large-scale archive of foci is only as useful as the corresponding metadata that it contains. In our view, databases in functional neuroimaging have not yet fully realized their potential for knowledge discovery in mapping human brain function. The meta-analytic applications made possible through BrainMap will evolve and grow more powerful as development of the database continues, perhaps leading to a multi-layered probabilistic functional brain atlas of meaningful mappings between function and structure.
The authors would like to acknowledge support from the NIMH to PTF (R01-MH074457) and ARL (R01-MH084812).
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errorsmaybe discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.