|Home | About | Journals | Submit | Contact Us | Français|
Language consists of sequences of words, but comprehending phrases involves more than concatenating meanings: A boat house is a shelter for boats, whereas a summer house is a house used during summer, and a ghost house is typically uninhabited. Little is known about the brain bases of combinatorial semantic processes. We performed two fMRI experiments using familiar, highly meaningful phrases (LAKE HOUSE) and unfamiliar phrases with minimal meaning created by reversing the word order of the familiar items (HOUSE LAKE). The first experiment used a 1-back matching task to assess implicit semantic processing, and the second used a classification task to engage explicit semantic processing. These conditions required processing of the same words, but with more effective combinatorial processing in the meaningful condition. The contrast of meaningful versus reversed phrases revealed activation primarily during the classification task, to a greater extent in the right hemisphere, including right angular gyrus, dorsomedial prefrontal cortex, and bilateral posterior cingulate/precuneus, areas previously implicated in semantic processing. Positive correlations of fMRI signal with lexical (word-level) frequency occurred exclusively with the 1-back task and to a greater spatial extent on the left, including left posterior middle temporal gyrus and bilateral parahippocampus. These results reveal strong effects of task demands on engagement of lexical versus combinatorial processing and suggest a hemispheric dissociation between these levels of semantic representation.
Comprehending language involves more than just understanding individual words; the meanings of individual words are fluently combined to produce larger structures expressing relations between the constituent words. Although the neural structures that support the comprehension of isolated words have been studied extensively, less is known about the brain bases of combinatorial semantic processes. We investigated these processes using simple noun-noun phrases such as LAKE HOUSE and HOUSE LAKE. The meaning of LAKE HOUSE depends on the meanings of the two words but expresses a further relation between them: a lake house is a house located on or near a lake. HOUSE LAKE, however, does not express an easily interpretable relation between the same words. This difference arises from the underlying semantic structure of the constituent nouns, which determines how naturally or automatically their meanings are combined. For example, large, stationary objects like houses have a fixed location and thus can be felicitously combined as head nouns with a modifying noun describing a larger object on which the head noun is located (e.g., COUNTRY HOUSE, CITY HOUSE, BEACH HOUSE, MOUNTAIN HOUSE, PRAIRIE HOUSE, etc.). HOUSE LAKE violates this semantic constraint because lakes are larger than houses. This is not to say that HOUSE LAKE cannot be interpreted with some additional effort (a lake on which there are numerous houses?), but we assume that in such cases the combination is constructed less successfully, resulting in little meaning or one marked by considerable residual ambiguity. In such cases, the typicality of the relationship between words influences ease of comprehension. The modifier noun MOUNTAIN, for example, more often indicates a location relationship with the head noun (MOUNTAIN STREAM) than an “about” relationship (MOUNTAIN MAGAZINE), and the more typical relations are associated with faster sensibility judgments.
We examined these combinatorial semantic processes by contrasting highly meaningful noun-noun phrases with their reversed, minimally meaningful forms. The conditions differ in meaningfulness but are matched with respect to word-specific properties. The three aims of this study were (1) to identify the neural systems that support successful combinatorial processing, (2) to identify the neural correlates of lexical (word-level) processing as distinct from combinatorial processing, and (3) to compare activation in these neural systems for tasks that engage explicit compared to implicit semantic processing. In contrasting meaningful phrases with their reversed versions, we expected that the reversed phrases would elicit greater effort, attention, and working memory in searching for a viable interpretation, particularly during an explicit semantic judgment task. Our main focus, however, is on the neural signature of successful conceptual combination, as identified by higher levels of activation for items that participants judged to be meaningful than for items they judged to be meaningless. Our interest in this aspect of fluent semantic processing arises from its ubiquity and central importance in everyday language use.
In addition to the relation-based account described above, several other mechanisms have been proposed to underlie combinatorial semantic processing. These mechanisms are not mutually exclusive, and our study was not intended to distinguish among them. The inventory of relation-based interpretations derives from world knowledge (e.g., concerning properties and functions of objects, the contexts in which they are used, and so on), encoded by knowledge structures such as schemas. Murphy focuses on cases in which phrases must be interpreted with respect to relatively specific world knowledge. To take an example from the current study, the meaning of FLOWER GIRL does not derive in any obvious way from a relation between the head noun and modifier, nor do the properties of girl and flower appear to align in any useful way. While FLOWER GIRL refers to a girl who carries or scatters flowers, knowing that this is done at a wedding by a girl who is too young to be a bridesmaid is critical to understanding the phrase. A somewhat different proposal holds that noun-noun combinations are interpreted in terms of their shared properties. For example, in the relation approach, ROBIN HAWK could be a hawk that preys on robins. According to Wisniewski and Love this phrase could be interpreted in terms of the properties of the nouns, where a ROBIN HAWK could be a hawk with a red breast. Thus, interpretations in which “one or more properties of the modifier concept apply in some way to the head concept,” also play a role in conceptual combination, as demonstrated for about 30% of their noun-noun phrases. The current study focused on the processing of phrases for which a meaningful interpretation can be readily derived (e.g., FLOWER GIRL), by comparison to phrases such as GIRL FLOWER that lack conventional meanings and can only be interpreted with effort and in varying ways. We assume that activations for meaningful compared to reversed phrases reveal the neural systems used to derive phrase-level meaning through lexical semantic combination under typical conditions involving compatible semantic constraints provided by the head and modifier nouns.
The above theories share the assumption that determining the meanings of noun-noun phrases involves combinatorial processes. It is also possible, however, that many such combinations are stored as lexical entries. A phrase like LAKE HOUSE, for example, having been encountered in the past, may be stored as a lexical item, much like COTTAGE or FARMHOUSE. Numerous studies have examined how phrase-level frequency affects the comprehension of noun-noun constructions. At one extreme, very frequent combinations are often labeled compound words or collocations, although the boundary between “noun-noun phrase” and “compound word” is not well defined or reliably indicated by typography. For example, FRONT DOOR is almost always written with a space, whereas BACK DOOR is nearly equally written with and without the space. Familiar phrases and compounds might be stored as lexical entries, obviating the need for combinatorial processing. This hypothesis has been offered as an alternative interpretation of previous studies of conceptual combination. Recent evidence from lexical decision during online sentence comprehension suggests that the lexical constituents of familiar phrases are processed both individually and combinatorially, but the issue is not settled. In the present study, we addressed this issue empirically by including in the fMRI analyses a continuous regressor for whole-phrase frequency (see Methods section for details). This enabled the effects of combinatorial processing to be examined separately from any effects of phrase usage frequency.
Several areas of prior imaging research are relevant to the current study. Combinatorial semantic processing is presumably related to the process of semantic integration in sentence comprehension. Electrophysiological investigations of semantic integration often involve sentence stimuli in which the beginning of the sentence sets up a semantic context (e.g., I LIKE MY COFFEE WITH CREAM AND) that is violated by the final word (e.g., SOCKS). This manipulation typically results in a negative-going current peaking around 400 ms after the stimulus of interest (SOCKS). Although this result, referred to as the N400, is often taken to reflect attempts to semantically integrate the target word with its preceding context, an alternate interpretation is that the incongruent context leads to increased difficulty of lexical access for the target word. Relevant to the current fMRI study, recent reviews have tentatively localized N400 effects to primarily left-hemisphere (LH) regions within the temporoparietal and inferior frontal lobes. Regarding conceptual combination, Koester et al. presented German compounds for semantic judgment and found an increased N400 for less plausible head constituents. Similarly, El Yagoubi et al. presented Italian compounds (e.g., CAPOBANDA, band leader) for lexical decision, using nonword trials constructed by reversing the order of the constituents in the compound words (e.g., BANDACAPO). A significantly larger N400 was found for nonwords compared to compound words. Although relevant in the sense that they deal with compounds, these results stand in contrast to the goal of the current study, which is to reveal the neural correlates of successful combinatorial semantic processing, the conditions for which are maximized by presenting highly meaningful phrases and minimized by presenting phrases for which the reversed form has minimal meaning. Thus if N400 effects increase with difficulty of lexical processing, and occur primarily in left temporoparietal and inferior frontal areas, then we might expect to see activation related to lexical processing in the LH that is distinct from areas related to combinatorial semantic processing.
To distinguish “the amount of effort needed to perform semantic integration” from the “degree to which the target word is pre-activated by context”, Pylkkänen et al. used magnetoencephalography (MEG) to compare activation for sentences such as THE AUTHOR BEGAN THE BOOK, in which a meaning (in this case, writing) is implied but not stated, with both control (THE AUTHOR WROTE THE BOOK) and anomalous sentences (THE AUTHOR DISGUSTED THE BOOK). They found increased signal amplitude for implied-meaning phrases compared to anomalous and control phrases that peaked around 400 ms after presentation of the critical word (in this case, BOOK) and localized to the anterior midline region. This study and a follow-up that found similar results using a different task are instructive in that they reveal a neural correlate of N400-like effects that are distinct from those induced by semantic anomaly, and point to a possible candidate area for the kind of semantic integration that may also take place in conceptual combination.
Regarding lexical-level processing, several recent studies have demonstrated LH activation associated with increased levels of lexical-semantic information, as indexed by high word frequency and imageability. Thus, if lexical processing occurs in parallel with or just prior to combinatorial processing, LH systems that support lexical-semantic processing should be activated to the extent that a phrase contains familiar lexical units. The lexical constituents of these combinations are presumably processed prior to computing the phrase-level concept, as suggested, for example, by the results of the Swinney et al. study discussed above. In the present study we investigated the neural correlates of lexical processing by performing an fMRI analysis using the sum of the frequencies of the lexical items in each phrase. This analysis, performed for both of the experiments reported here, also included terms for meaningful and reversed phrases, thereby potentially revealing separate neural correlates for lexical compared to phrase-level semantic processing for the same stimuli.
The first experiment used a 1-back task that required monitoring for repetition of single words across phrases. Although this task does not require conceptual combination, it had the advantage of placing similar performance demands on meaningful and reversed phrases, thereby allowing any activation differences between meaningful and reversed phrases to be attributable to implicit (i.e., obligatory) combinatorial semantic processing. A small preliminary behavioral study (N = 10) was performed outside the scanner to confirm that these conditions were equated in terms of difficulty as measured by reaction time and error rate. Because of the possibility that conceptual combination might not occur during the 1-back task, a second fMRI experiment was performed using the same stimuli but with a classification task that required phrases to be judged for meaningfulness, a more explicit task. If similar processes are engaged for implicit and explicit semantic processing, similar activation patterns for the conditions of interest should obtain across the two experiments, but to a somewhat greater extent for the second experiment due to the more extensive processing needed to perform the semantic judgment task. Alternatively, if the 1-back task primarily engages lexical processing and the semantic judgment task primarily engages combinatorial processing, then there may be little or no overlap across the two tasks.
The same 400 stimuli of interest were used in both experiments, and a complete list is provided in the supplemental material (Table S2). Stimulus selection began by compiling a list of all English words in the CELEX database that have a higher noun than verb or adjective frequency. The 500 most highly imageable words in this list were selected using a database of imageability ratings compiled from six sources, the last three available through the MRC Psycholinguistic Database. All possible non-identical pairs were created for these 500 nouns, resulting in 249,500 candidate noun-noun combinations. A large corpus of human-generated text was then searched to find potentially meaningful pairs, resulting in 1475 items. This set was then filtered so that only pairs appearing in the corpus in one direction but not the other were included, resulting in a list of 1351 noun-noun pairs. These candidate pairs were then manually filtered to exclude potentially problematic items such as taboo words or phrases, resulting in a final list of 1080 noun-noun phrases.
Ratings were obtained from a sample of healthy adults to verify the meaningfulness of the original and reversed pairs. Two lists were prepared, with the pairs in original and reversed orders, respectively. Each list was then split into five sublists of 216 phrases each. Each participant in the rating study saw one meaningful sublist and one reversed sublist (for a total of 432 phrases), with the restriction that no word pair was seen in both orders.
Subjects in the rating study (N = 150) were recruited from the psychology student subject pool at the University of Wisconsin – Madison and received course credit. For each noun-noun phrase, they were asked to “judge how meaningful it is as a single concept, using a scale from 0 to 4.” Each phrase was preceded by the definite article to encourage subjects to treat the phrase as a noun. Subjects were given the following examples as anchor points: THE GOAT SKY, 0 (makes no sense). THE FOX MASK, 2 (makes some sense). THE COMPUTER PROGRAMMER, 4 (makes complete sense).
One subject failed to complete a majority of the ratings and was removed from analysis. For each of the five lists, the mean ratings for all items across subjects was calculated and correlated with each subject's ratings. Eight subjects whose correlations were more than 2 standard deviations from the mean were excluded from further analyses. Final ratings were calculated in the absence of these outliers, with each phrase rated by an average of 28.2 subjects (min: 27, max: 29).
From these ratings 200 word pairs were selected that had been judged to be very meaningful when presented in the original order and to have very little meaning in the reverse order. For example, THE SKI JACKET received a mean rating of 4.0, while THE JACKET SKI received a 0.7. The mean ratings were 3.91 (SD: 0.08) and 1.08 (SD: 0.25) for meaningful and reversed stimuli.
To ensure that the meaningfulness of the stimuli is due to combinatorial semantic processing rather than simple association between the two words, we examined whether the constituent words in the meaningful stimuli were associated, as measured by association norms. If the first word calls to mind the second word by association, that process would be different than computing the meaningfulness of the phrase. Association statistics were obtained from two independent databases by presenting the first word in a stimulus pair and recording the probability that the second word was produced as an associate. AUTUMN LEAF, for example, had a mean association value of 0.02 (i.e., LEAF was produced as an associate of AUTUMN by 2% of participants) across the two databases, while LEAF AUTUMN had a mean association value of 0.01. By comparison, the meaningfulness rating for THE AUTUMN LEAF was 3.93, whereas THE LEAF AUTUMN was 1.07. Overall, the correlation between association and meaningfulness was quite small, though reliable (r = 0.10, p < 0.05); thus, association ratings were included as a covariable in the fMRI analysis.
We also examined phrase-level and lexical-level frequency. Phrase-level frequency was estimated by how often each phrase appeared in a large text corpus, a 518,339,522 word download of Wikipedia articles in March 2006. Mean frequency for the stimulus phrases was 36.34 (min: 0, max: 1182). Although meaningful phrases are often high in frequency (e.g., MOUNTAIN BIKE has a meaningfulness rating of 4 and a frequency count of 690), this is not always true (e.g., PILL BOTTLE: meaningfulness 3.90, frequency 1). For our stimuli the correlation of frequency and meaningfulness was modest but reliable (r = 0.33, p < 0.0001). Lexical-level frequency was obtained for each phrase by log-transforming the per million frequency of each word form in CELEX and summing this figure across the two words. Because the same words were used to create the meaningful and reversed phrases, lexical-level frequency is orthogonal to meaningfulness.
The 80 phrases used to elicit 1-back responses in the first experiment were taken from the larger group of 1080 normed phrases but did not overlap with the 400 phrases of interest. Like the phrases of interest, 40 were selected that were meaningful in the forward direction and not meaningful when reversed. Responses to these phrases were modeled separately from the phrases of interest.
Twenty-five participants underwent the scanning procedure. One who did not receive all four runs of the task was excluded. A second participant was excluded as an outlier after analysis of his data showed activation across the entire brain for the reversed compared to the meaningful condition that was more than two standard deviations from the mean. We checked to ensure that excluding this participant did not bias our results by re-analyzing the data from Experiment 1 with the outlier included. The results were nearly identical to those with the outlier included, except that activation in the left inferior frontal cortex for reversed compared to meaningful phrases extended somewhat more ventrally and medially to include the junction between the pars triangularis of the inferior frontal gyrus (IFG) and the anterior insula. Thus, analyses were based on data from 23 remaining participants (13 females), all of whom were healthy, literate adults, had normal or corrected-to-normal vision, were right handed on the Edinburgh Handedness Inventory, and spoke English as a first language. All participants provided written informed consent according to local Institutional Review Board protocols and were paid an hourly stipend. The mean age of the participants was 24.2 (SD: 3.0), and mean years of education was 17.0 (SD: 1.9). A verbal IQ estimate from the Wechsler Test of Adult Reading was also available for 16 participants, with a mean standard score of 114.1 (SD: 6.8).
The fMRI experiment used a fast event-related design and a 1-back task. On each trial, a phrase was displayed for 1000 ms then replaced with a fixation cross. Participants were instructed to press the button under the right index finger if either word in the current phrase matched a word in the same position in the previous phrase. The scanning session was split into four runs. Each run consisted of 50 meaningful phrases, 50 reversed phrases, and 20 1-back targets; these trials were randomly intermixed with 100 baseline (fixation) trials to produce randomly varying inter-trial intervals (mean: 3.6 s, SD: 2.4). Stimuli always subtended less than six degrees of horizontal visual angle. Stimuli were presented and reaction times recorded using E-prime (Psychology Software Tools, Inc.; http://www.pstnet.com/eprime).
MRI data were acquired using a 3.0 Tesla GE Excite system with an 8-channel array head RF receive coil. High resolution, T1-weighted anatomical reference images were acquired as a set of 134 contiguous axial slices (0.938 × 0.938 × 1.000 mm) using a spoiled-gradient-echo sequence (SPGR, GE Healthcare, Waukesha, WI). Functional scans were acquired using a gradient-echo echoplanar sequence with the following parameters: 25 ms TE, 2 s TR, 224 mm field of view, 64 × 64 pixel matrix, in-plane voxel dimensions 3.5 × 3.5 mm, and slice thickness 3.0 mm with a 0.5 mm gap. Thirty-three interleaved axial slices were acquired, and each of the four functional runs consisted of 232 whole-brain image volumes.
Image analysis was performed using AFNI (http://afni.nimh.nih.gov/afni). For each subject, the first six images in the time series were discarded prior to regression analysis to avoid saturation effects; images were slice timing corrected and spatially co-registered. Estimates of the three translation and three rotation movements at each point in each time series computed during registration were saved for use as noise covariates. Image volumes containing artifact were identified using an automated voxel-wise regression analysis and censored from subsequent analyses. Voxelwise multiple linear regression was then performed using the AFNI program 3dDeconvolve. This analysis included the following covariables of no interest: a fourth-order polynomial to model low-frequency trends, the six previously calculated motion parameters, and a term for signal in the ventricles used to model noise. Covariables of interest were modeled as impulse functions occurring at stimulus onset and convolved with a gamma variate function approximating the hemodynamic response. They consisted of the following: (1) an indicator variable with a value of 1 for each of the 200 phrases that were meaningful and did not require a 1-back response, otherwise 0; (2) a 1 for the 200 phrases that were not meaningful and did not require a 1-back response, otherwise 0; (3) an indicator of 1 for each of the 80 phrases requiring a button press (1-back responses), otherwise 0; (4) mean-centered word association values for phrases indicated in covariables 1 and 2; (5) mean-centered phrase-frequency values; (6 and 7) summed word frequency values for meaningful and reversed phrases, respectively. The effect of summed word frequency across meaningful and reversed phrases was obtained by testing for the combined effects of 6 and 7, while the interaction of summed word frequency and phrase type was obtained by contrasting 6 and 7.
The resulting contrast coefficient maps for each participant were linearly resampled in standard stereotaxic space to a voxel size of 1 mm3 and spatially smoothed with a 5 mm full-width-half-maximum Gaussian kernel. These smoothed coefficient maps were then passed to a random effects analysis comparing the coefficient values to a null hypothesis mean of zero across participants. The resulting group activation maps were thresholded at a voxelwise p < 0.005, uncorrected. A cluster extent threshold was then calculated using the AFNI program alphasim to perform Monte Carlo simulations estimating the chance probability of spatially contiguous voxels passing this threshold. This standard method capitalizes on the fact that activated voxels tend to occur in clusters, and the larger the cluster, the less likely it is to occur by chance. Clusters smaller than 600 μl were removed, resulting in a whole-brain corrected probability threshold of p < 0.05.
Experiment 2 was identical to Experiment 1 in terms of image acquisition, data analysis, and stimuli of interest, differing only with respect to the task. Participants were instructed to press one button if the phrase being displayed was meaningful, another if it was not meaningful, and a third if it was made of “nonwords”. Button order was counter-balanced across subjects. Phrases composed of pseudowords were included as a low-level control condition. Results for the pseudoword condition are not relevant to the present hypotheses and will not be discussed further.
Twenty-seven participants underwent scanning. Participants were excluded if they performed the task with an overall error rate > 45% (> 1.1 SDs from the group mean). This resulted in exclusion of 5 participants, and analyses were based on data from the 22 remaining participants (15 females). Inclusion criteria and informed consent were as described for Experiment 1. Two participants performed both experiments, with the first occurring approximately eight months before the second. Mean age was 24.7 (SD: 5.4), and mean years of education was 15.8 (SD: 2.3). Verbal IQ was estimated as in Experiment 1 for 17 participants, with a mean standard score of 116.0 (SD: 7.2).
For image analysis the covariables of interest were: (1) an indicator variable with a value of 1 for each of the phrases correctly judged to be meaningful, otherwise 0; (2) a 1 for the phrases correctly judged to be not meaningful, otherwise 0; (3) a 1 for each correctly classified pseudoword phrase, otherwise 0; (4) mean-centered RT values for all correct classification responses; (5) mean-centered association values for phrases indicated in covariables 1 and 2; (6) mean-centered phrase-frequency values; (7 and 8) summed word frequency values for meaningful and reversed phrases, respectively. The effect of summed word frequency across meaningful and reversed phrases was obtained by testing for the combined effects of 7 and 8, while the interaction of summed word frequency and phrase type was obtained by contrasting 7 and 8. Erroneous responses were modeled as a covariable of no interest.
Response times (RTs) for correct 1-back responses following meaningful and reversed phrases were compared, as were error rates across subjects. No reliable performance differences (either RT or error) were observed for 1-back responses to meaningful (mean RT: 915 ms, SD: 271, mean subject-wise percent error rate: 8.5, SD: 8.2) compared to reversed (RT: 929 ms, SD: 270, error rate: 9.1, SD: 7.5) phrases.
For Experiment 1 using the 1-back task, the contrast of meaningful (forward) compared to reversed phrases revealed a general hemispheric dissociation (Figure 1, upper row), with greater activation of the right supramarginal gyrus (SMG) for meaningful phrases, and greater activation of left inferior frontal junction (IFJ, a region at the intersection of the inferior frontal and precentral sulci) for the same words in reversed order (Table 1 and Figure 1). A single area, the left fusiform gyrus, was modulated by degree of word association between the nouns (see the Methods section for how association was operationalized). BOLD signal in this area showed a negative correlation with association values; no areas showed positive correlations (Supplement Table S1). No areas showed significant correlations between BOLD signal and phrase frequency.
To examine lexical effects as distinct from phrase-level effects, data from both experiments were analyzed in terms of the correlation of the BOLD signal with the sum of the frequency of the words comprising each phrase. For Experiment 1 all correlations with word frequency were in the positive direction, indicating increased activity with increasing word frequency (upper row of Figure 2). These effects were found in posterior left middle temporal gyrus and adjacent angular gyrus, bilateral parahippocampal gyrus (PHG), bilateral posterior cingulate gyrus, and left precuneus (Table 2).
In contrast to the previous experiment, there was a significant effect of phrase type on RT, with semantic decisions made more rapidly to meaningful than to reversed phrases (meaningful = 978.0 ms, SD = 233.7; reversed = 1158.3 ms, SD = 261.2; t = 30.8, p < 0.001). Percent error rates showed the same general pattern as seen for RTs, in that reversed phrases elicited a reliably higher percent error rate than meaningful phrases (meaningful = 14.7, SD = 8.2; reversed = 22.4, SD = 11.6; t = 2.6, p < 0.05). Note that trials on which errors occurred were coded as a separate error condition in the image analysis. Thus all trials included in the meaningful condition received a “meaningful” response, and all trials included in the reversed condition received a “not meaningful” response.
For Experiment 2 using the semantic decision task, the contrast of meaningful compared to reversed phrases, similar to Experiment 1, also yielded right hemisphere (RH) inferior parietal activation (Figure 1, lower row), involving the angular gyrus (AG) and adjacent SMG. Other activated areas included bilateral middle and posterior cingulate gyri and bilateral dorsomedial prefrontal cortex. Activation for reversed compared to meaningful phrases, on the other hand, showed predominantly LH activation (similar to Experiment 1), including left IFJ, precentral gyrus, and IFG; bilateral intraparietal sulcus (IPS); left supplementary motor area (SMA) and pre-SMA; right pre-SMA; left posterior inferior temporal gyrus; and left fusiform gyrus (Figure 1, lower row, and Table 1). Terms for word association and for whole-phrase frequency were also included in the analysis. There were no reliable effects of whole-phrase frequency. The only effects related to word association were negative correlations between BOLD signal and association values (Table S1).
The pattern of activity associated with word frequency in Experiment 2 (lower row of Figure 2) was very different from that found in Experiment 1, showing exclusively negative correlations between BOLD signal and word frequency. These were in bilateral fusiform and inferior temporal gyri, right superior temporal sulcus, left orbitofrontal cortex, left putamen, left insula, and right amygdala (Table 2).
This study examined the neural correlates of combinatorial semantic processing, as distinct from lexical-level processing, during processing of noun-noun combinations in which words were presented in either meaningful or reversed order. We assumed that the process of successfully combining two concepts to form a third concept produces a neural signature detectable by fMRI. In contrast, when two concepts do not combine in a clearly meaningful way, this neural signature representing the successful activation of a combined meaning should be weaker. Thus, the contrast of meaningful versus reversed phrases should reveal areas engaged in successfully combining concepts. We also expected activations in the opposite direction, reflecting greater effort, attention, and working memory demands for noun pairs that could not be successfully combined. Results from both a task eliciting implicit and a task eliciting explicit combinatorial semantic processing suggested a role for RH temporoparietal regions in successful combinatorial semantic processing. Differences between tasks were also found, with the implicit task eliciting more extensive lexical-level activation, and the explicit task eliciting more extensive activation related to combinatorial processing.
One goal of this study was to reveal the neural correlates of lexical processing in the context of, but distinct from, combinatorial processing. Data from both experiments were analyzed in terms of the summed lexical frequencies of the words comprising each phrase. In contrast to the largely right-sided activations for the combinatorial comparisons, for the 1-back task the lexical frequency analysis yielded positive correlations between frequency and BOLD signal in left posterior middle temporal gyrus, bilateral PHG, and bilateral posterior cingulate/precuneus. These areas were among those implicated in lexical semantic processing in a recent large-scale meta-analysis. These results also echo findings from a recent study of single-word reading aloud, in which overlapping positive correlations of word frequency and imageability were found in left angular, posterior middle temporal, and posterior cingulate gyri. For the semantic decision task, on the other hand, no areas showed a positive correlation of BOLD signal with lexical frequency. Together with the relative paucity of activation for meaningful phrases in the 1-back task, these results suggest that the 1-back task primarily engaged lexical processing, while the semantic decision task primarily engaged processing at the whole-phrase level.
The behavioral data in Experiment 1 showed the expected lack of performance differences across the two types of phrases, suggesting that they were treated similarly in terms of extra-linguistic factors such as attention and time-on-task. Experiment 2, in contrast, showed performance differences across conditions, as expected for a task requiring explicit semantic judgments. The task in Experiment 2 was used to elicit explicit semantic processing, with the trade-off that reversed phrases were associated with longer response times than meaningful phrases. Although this led to the concern that the forward compared to reversed contrast would be dominated by activation for the reversed condition, it turned out that in Experiment 2 several areas were indeed activated for the forward compared to reversed condition.
The contrast of reversed compared to meaningful phrases revealed greater activation for reversed phrases in the left lateral prefrontal cortex for the 1-back task. Similar but more spatially extended activations for reversed phrases were found in the semantic decision task. Additional activation for this contrast was also found for the semantic decision task in bilateral IPS and SMA, and left mid-fusiform gyrus. With the possible exception of the left mid-fusiform gyrus, all of these areas are typically associated with increased demands on attention, cognitive control, and working memory, suggesting that the greater extent of activation in these areas for the reversed compared to meaningful phrases represents greater demands on these general cognitive processes as participants searched (unsuccessfully) for a viable interpretation for the reversed phrases. Also consistent with this interpretation is the more extensive activation in these regions during the semantic decision compared to the 1-back task, as only the semantic decision task showed a difference in behavioral performance across conditions.
Imaging results from Experiment 1 for the meaningful compared to reversed phrase contrast showed activation exclusively in the right SMG. The same contrast in Experiment 2 revealed a much larger set of areas that included the right AG adjacent to the SMG activation seen in Experiment 1, but also several areas not seen in Experiment 1, such as bilateral posterior cingulate gyri and dorsomedial prefrontal cortex (DMPFC). Thus, both tasks led to RH greater than LH activation for the contrast of meaningful compared to reversed phrases. The pattern of more extensive activation for this contrast in the semantic decision task, along with the fact that positive correlations between BOLD signal and summed word frequency were only found in the 1-back task, suggests that the semantic decision task was more successful at eliciting combinatorial processing. This is consistent with a recent fMRI study by Kuperberg et al. in which effects of semantic priming in a lexical decision task were compared to those from a semantic judgment task in which the participants judged meaning relatedness between primes and targets. A priming-by-task interaction was reported such that enhanced neural responses to priming were found for the semantic judgment task in a set of areas that included the left AG. The location of activations in the current study for the semantic decision task in bilateral posterior cingulate/precuneus, dorsomedial prefrontal cortex, and AG, corresponds to areas reliably implicated in semantic processing. The rightward asymmetry of the AG activations, however, is novel. Together with the right SMG activation in the 1-back task, these results suggest a role for right inferior parietal cortex in combining concepts.
As mentioned in the Introduction, an alternative account of noun-noun phrase processing is that meaningful phrases already exist as stored complex lexical items, in which case combinatorial processing is not necessary. This account seems especially plausible for frequent collocations (including frequent adjective-noun phrases, such as LITTLE BOY or USED CAR, in addition to noun-noun phrases). The apparently limitless productive capacity of human language, however, makes it somewhat unlikely that each meaningful phrase should be stored as a lexical unit. Two aspects of our results argue against such an interpretation. First, in both experiments the full regression model revealed no areas in which BOLD signal was significantly correlated with whole-phrase frequency. Second, if the meaningful phrases were being treated as whole words, then they should elicit activation in areas related to word-level processing. This is also true of the reversed phrases, because individual words clearly have to be processed before any attempt to make sense of the phrase. If both types of phrase elicit only lexical processing, one would expect no activation differences related to successful lexical access. Instead, greater activation would be expected for the reversed phrases, the condition that elicited longer RTs, in areas associated with cognitive control, working memory, and attention. While the latter results were obtained, we also found activation for meaningful compared to reversed phrases in RH analogs of language areas, such as AG and SMG, along with other areas in the semantic processing network (posterior cingulate/precuneus and DMPFC). Thus this pattern of results is counter to what would be predicted if the meaningful phrases were being processed purely as lexical units, and suggest instead that they elicited a distinct neural activation pattern reflecting successful combinatorial semantic processing.
The exact nature of the role of the RH in combinatorial processing is not entirely clear. Evidence supporting an interpretation based on the RH coarse semantic coding hypothesis comes from a study of healthy participants using lateralized visual presentation. Triplets of words were presented that, when considered together, semantically primed a target word. The summation advantage was greater for the RH, suggesting the presence of larger semantic fields with greater potential for overlap in priming the target concept. Applied to combinatorial semantic processing, the larger fields for concept representations in the RH may provide more opportunity for constructive linkage among compatible concepts. For example, restrictive semantic fields for the concepts MOUNTAIN and BIKE containing only immediately relevant information such as shape, motion, etc. would be unlikely to overlap. A wider semantic field for MOUNTAIN that also includes things used on mountains would, on the other hand, be more likely to overlap with a similarly wide semantic field for BIKE that included types of use.
Another account of the contrast between meaningful and reversed phrases, which may be complimentary to the fine-coarse coding hypothesis, involves attractor spaces that are built up during training of recurrent connectionist networks. A network is said to “settle” into an attractor basin over time as it finds a region of error space for which the mapping between inputs and outputs is most accurate. Our tentative proposal is as follows. Temporoparietal areas of the LH, such as the posterior middle temporal and angular gyri, contain relatively narrow attractor basins for representing individual word meanings. In contrast, temporoparietal areas of the RH, along with midline structures such as the posterior cingulate and DMPFC, may contain attractor basins that, like those in the LH, code verbal meaning, but differ in that they are wider. These wider basins would enable the RH to represent partial overlap of concepts in a meaningful combination. For example, interpretation of the phrase ROCK STAR may rely on overlapping attractor basins for ROCK and STAR, both of which must be relatively wide to accommodate the sense of ROCK as a style of popular music and STAR as a celebrity. In fact, the interpretation of words with multiple senses has previously been modeled using an attractor network, where the presence of wider attractor basins aided in the recognition of polysemous words. In a priming study using MEG, Pylkkänen et al. showed bilateral effects of polysemy, along with evidence for competition among senses arising specifically from RH temporoparietal sources at approximately 400 ms after target stimulus presentation. We propose that the constructive overlap of the relevant aspects of, for example, the concepts ROCK and STAR into a combined concept relies on the overlap of relatively wide attractor basins instantiated in the RH areas discussed above. The alternative of a single attractor basin representing a stored, “overlearned” concept is not supported by the results of the current study, as no areas showed a significant correlation with whole-phrase frequency independent from the contrast between meaningful and reversed phrases.
As noted in the Introduction, we could not be sure that combinatorial processing would occur in the 1-back task. In addition, some combinatorial processing may have occurred for the reversed phrases. These two concerns are similar in that they highlight the possibility of not detecting activation for successful combinatorial semantic processing with meaningful phrases over and above activation for reversed phrases and frequency of whole phrase usage. Given these potential risks to sensitivity, the fact that activation was detected for meaningful compared to reversed phrases is a clear indication that there are detectable neural correlates of successfully combining concepts.
The current design does not distinguish among relational, feature-based, or world-knowledge-based accounts of conceptual combination. There are two points to note in this regard. One is that our interpretation of the findings is neutral with respect to mechanisms underlying conceptual combination, in that this study concerned successful combinatorial semantic processing in general. The second is that, while multiple mechanisms could likely be brought to bear in interpreting the phrases, based on inspection of the stimuli (see Table S2 of the supplementary material) it appears that most of the meaningful phrases could be easily interpreted in terms of a thematic relation between the head noun and its modifier. This is in line with the study by Wisniewski and Love, in which roughly 70% of their noun-noun phrases were interpreted using a thematic relation strategy.
This study focused on combinatorial semantic processes that occur in fluent language comprehension, using simple phrases comprised of noun-noun combinations. The results are consistent with an account in which coarse semantic coding by RH temporoparietal structures supports the combining of individual lexical concepts into a whole. At the same time, or perhaps just prior to combinatorial processing, medial and posterior left temporal regions process the lexical constituents of the phrases. The results also revealed strong effects of task demands in eliciting lexical compared to combinatorial processing and support accounts of hemisphere-level dissociations, offering new neuroanatomical detail regarding the brain areas supporting lexical and combinatorial semantic processing.
We thank Jon Willits for help obtaining phrase frequencies, David A. Medler, Ph.D., for providing the composite imageability database, and Edward Possing for help with data collection. This work was supported by NIH grants from the NINDS to author JRB (R01 NS033576) and the NICHD to author WWG (F32 HD056767).
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.