|Home | About | Journals | Submit | Contact Us | Français|
Event-related brain potentials (ERPs) were used to examine the role of grammatical gender in written sentence comprehension. Native Spanish speakers read sentences in which a drawing depicting a target noun was either congruent or incongruent with sentence meaning, and either agreed or disagreed in gender with that of the preceding article. The gender-agreement violation at the drawing was associated with an enhanced negativity between 500 and 700 msec post-stimulus onset. Semantically incongruent drawings elicited a larger N400 than congruent drawings regardless of gender (dis)agreement, indicating little effect of grammatical gender agreement on contextual integration of a picture into a written sentence context. We also observed an enhanced negativity for articles with unexpected relative to expected gender based on prior sentence context indicating that readers generate expectations for specific nouns and their articles.
Approximately half of the world’s languages have a grammatical gender system, according to which all nouns are sorted into classes that are not necessarily correlated with semantic gender (based on biological sex). These grammatical gender classes determine the behavior of other words in a sentence (verbs, adjectives and articles) that are associated with nouns in that class. Hence, a ‘gender’ is essentially a set of nouns that take the same form of agreement with respect to other words (Corbett, 1991).
Languages vary in the number of gender classes they have. Bantu languages of Southern Africa have numerous genders, up to 22 if singular and plural parallel genders are counted separately (Corbett, 1991); Spanish has only 2. Depending on the language, nouns can belong to a particular gender based on semantic (meaning), morphological (word structure) or phonological (sound) features. Swahili, for example, divides nouns into gender classes based on simple semantics and formal properties of words; the word for “big snake” belongs to genders 5/6 while that for “small snake” belongs to genders 7/8, and words beginning with ki- belong to gender 7, while words beginning with mi- belong to gender 4, and so on. Spanish nouns belong to a gender based either on biological sex of the referent or according to their sound. Hence, males are masculine and females are feminine, and nouns ending in “-o” tend to be masculine (with few exceptions) while nouns ending in “-a” tend to be feminine (with several exceptions of masculine words ending in “-a”).
Though grammatical gender is a pervasive phenomenon across the world’s languages, we have little idea of exactly what role(s) it plays in sentence processing in real time. Speakers of a gender-marked language reliably use gender-agreement information, even at an early age (e.g., Corbett, 1991), suggesting that it is likely to serve some, if not several purposes in natural language processing. And, indeed, empirical results from a variety of tasks suggest a number of ways in which gender information might facilitate decisions about, or processing of, pictures and words. Gender markings on a noun have been shown to expedite its recognition (Grosjean et al., 1994; Bates et al., 1995; Radeau and van Berkum, 1996). Lexical decisions for nouns are speeded by the prior occurrence of an article or adjective that agrees with it in gender (Gurjanov, 1985; Lukatela et al., 1987; Jakubowicz and Faussart, 1998). The presence of gender-marked words in the preceding context can reduce the time it takes to say a word (e.g., for nouns in Italian, see Bates et al., 1996; for verbs in Hebrew, see Deutsch et al., 1999), name a picture (Jacobsen, 1999; Bentrovato et al., 1999), read a text (Cacciari et al., 1997; Deutsch and Bentin, 2001; Deutsch et al., 1999) or identify the object of an utterance among words sharing an initial phoneme (e.g., based on gender agreement in French, the article le can be followed by bouton but not by bouteille, Dahan et al., 2000). Gender information can also contribute to anaphor resolution within a sentence by aiding in determining co-reference relations between a noun and pronoun, or for a single pronoun that may have multiple antecedents of different genders (e.g., Carreiras et al., 1993; Garnham et al., 1995). Likewise, semantic and grammatical gender can each contribute to the resolution of epicenes, where a word of a single grammatical gender can refer to a being of either sex (e.g., la vittima, a feminine noun can refer to both female and male entities, Cacciari et al., 1997). Overall, these results suggest that grammatical gender is important in natural language comprehension and production (see Friederici and Jacobsen, 1999; and Schriefers and Jescheniak, 1999 for reviews on the role of gender in comprehension and production, respectively).
It remains to be determined, however, exactly how and when gender information becomes available and to what end it is used during language processing in real time. On the one hand, gender agreement may be a passive comparison or matching process, in which the gender of each incoming word is compared to that of all prior words to which it is associated. On this possibility, gender information may serve to verify grammatical and thematic relationships between words, among its other functions; i.e., gender cues would contribute to some sort of post-lexical matching process. A gender mismatch thus could not occur until at least the second of two related gender-marked items appeared. On the other hand, gender information may take on a more active role in the unfolding of sentence meaning; for example, gender cues may be (consciously or unconsciously) anticipated and in turn used not just for agreement verification, but also to predict and thereby ease integration of upcoming gender-matching words. On this more predictive view of sentence processing (Kutas and Federmeier, 2000), we might expect to see some sign of a gender mismatch at any gender-marked item in a sentence, even the first, if its markings are inconsistent with the concept expected based on prior context (and thus the gender markings of the associated lexical item).
One aim of this study was to determine whether, and if so to what extent, processing a noun (depicted as a line drawing) is affected when there is a mismatch between its grammatical gender and that of the preceding article, that is, when the two do not agree in gender (e.g., a gender matching effect). This was assessed by comparing the pattern of electrical response activity from the scalp, in young Spanish speaking adults, to nouns that were either semantically congruent or incongruent with a written sentence context, and preceded by an article that either matched or mismatched it in grammatical gender. In this way we could also assess possible interactions between semantic processing and grammatical gender agreement at the target noun (depicted by a picture). In addition, this design allowed us to test for on-line expectation of specific nouns (e.g., a predictive effect). We assessed this hypothesis by comparing the pattern of brain electrical activity to gender-marked articles, which either did or did not agree in gender with that of an upcoming noun expected on the basis of the preceding context up to that point.
To these two ends, we recorded event-related brain potentials (ERPs) to nouns, depicted via line drawings, and the immediately preceding gender-marked articles in each of a series of Spanish sentence pairs such as: “Caperucita Roja llevaba la comida para su abuela en una … muy bonita. Pero el lobo llegó antes que ella.” (English translation: Red Riding Hood carried the food for her grandmother in a … [very pretty]. But the wolf arrived before her.). Young adults were asked to read such pairs of sentences for comprehension. Each pair had embedded, somewhere within one of these sentences, a line drawing of either the expected continuation (in this case, canasta; basket) or a semantically incongruent continuation of the same grammatical gender (corona; crown). To determine whether, and if so to what extent, grammatical gender information would affect the integration of the picture into the sentence context, half the target items, both the semantically congruent and incongruent ones, were preceded by an article of the wrong grammatical gender (e.g., for the example above un canasta and un corona, respectively).
It is well known that when a written or spoken word or a line drawing does not make sense within a sentence context, the ERP to the eliciting item is characterized by a widely-distributed negativity between 200–500 msec post-stimulus called the N400 (Kutas and Hillyard, 1980; see also Kutas and Federmeier, 2000; Holcomb and Neville, 1991; Ganis et al., 1996; Federmeier and Kutas, 2001; Wicha et al., 2000). N400 amplitude is inversely correlated with an item’s predictability in a context and positively correlated with the difficulty of integrating the item into a context, be it a single word, a scene, a sentence, or a discourse. Accordingly, we expected semantically incongruent line drawings to elicit larger N400s relative to semantically congruent ones.
Insofar as readers attend to articles and their gender markings and use these, at all, to help integrate the line drawing into the sentence context, we would expect to see this primarily in the region of the N400 component of the ERP. Perhaps gender information will ease integration of meaningful pictures (decrease N400 amplitude), add to the incongruity of anomalous drawings (increase N400 amplitude) or have no apparent effect on the processing of anomalous pictures. Alternatively, a gender mismatch may stop semantic integration altogether, such that both congruent and incongruent pictures will elicit large, equal-sized N400s.
Prior research with agreement violations would lead us to expect a gender mismatch to elicit a late positive component variously called the P600 or syntactic positive shift (SPS). Such positivities are presumed to index syntactic processing or reprocessing (e.g., Osterhout et al., 1996) or in a more domain-general view the recognition of a task-related anomaly (e.g., Coulson et al., 1998; Patel et al., 1998). Another possible outcome would be the elicitation of a greater negativity to gender mismatches over frontal sites (i.e., a so-called left anterior negativity or LAN). The LAN, like the P600 has been variously argued to index either a sintax-specific process (Neville et al., 1991) or a non-specific process reflecting some aspect of working memory demands (Kluender and Kutas, 1993; King and Kutas, 1995).
In Spanish, the combination of a gender-marked article followed immediately by its associated noun is relatively a very frequent event. This statistic taken together with the fact that our sentences are designed to lead to an expectation for a specific noun (and thus an article with a specific gender marking) allowed us to use the ERP pattern elicited by articles to test the hypothesis that readers naturally predict upcoming concepts, and in some cases even word referents, at least when such concepts are represented as line drawings. If an individual uses context to anticipate an item and its associated article, we should see this in the brain’s response to the article when its gender violates that expected on the basis of the context as opposed to when it does not. The nature of the ERP effect at the article – enhanced negativity or enhanced positivity – will give us a sense of whether the mismatch is being appreciated primarily at a semantic level (as indexed by a negativity) or some other level.
Twenty-eight right-handed, native speakers of Spanish (18 women, mean age 20.8 yrs, ranging between 18–29 yrs; 10 men, mean age 21.2 yrs, ranging between 18–27 yrs), residents of Baja California, Mexico and students at the Universidad Autónoma de Baja California (UABC) in Tijuana, received $ 24 for one 3-hour session of testing at the University of California at San Diego. Handedness and language dominance were assessed via a self-rating questionnaire. All participants were exposed to Spanish from birth and used Spanish on a daily basis, had normal or corrected-to-normal vision and reported no physical or cognitive disabilities that would interfere with the reading comprehension task.
The electroencephalogram (EEG) was recorded from 26 electrode sites in a standard electro-cap, each referenced on-line to the left mastoid. The approximate scalp distribution for the 26 recording sites is illustrated in Figures 3 and and5,5, for the following recording sites: Midline Prefrontal (MiPf); Left Lateral Prefrontal (LLPf); Right Lateral Prefrontal (RLPf); Left Medial Prefrontal (LMPf); Right Medial Prefrontal (RMPf); Left Dorsal Frontal (LDFr); Right Dorsal Frontal (RDFr); Left Lateral Frontal (LLFr); Right Lateral Frontal (RLFr); Left Medial Frontal (LMFr); Right Medial Frontal (RMFr); Left Dorsal Central (LDCe); Right Dorsal Central (RDCe); Midline Central (MiCe); Left Medial Central (LMCe); Right Medial Central (RMCe); Left Lateral Temporal (LLTe); Right Lateral Temporal (RLTe); Left Dorsal Parietal (LDPa); Right Dorsal Parietal (RDPa); Midline Parietal (MiPa); Left Medial Occipital (LMOc); Right Medial Occipital (RMOc); Left Lateral Occipital (LLOc); Right Lateral Occipital (RLOc); Midline Occipital (MiOc).
Blinks and eye movements were monitored through a bipolar recording from electrodes placed on the outer canthus of each eye and under each eye (referenced to the left mastoid). Electrode impedances were maintained below 5Kohms. The EEG was amplified with Grass amplifiers with band pass set from 0.01 to 100 Hz, and sampled at a rate of 250 Hz.
Trials with artifacts due to eye movements, excessive muscle activity or amplifier blocking were eliminated off-line before averaging – approximately 4.2% of the data time locked to the picture and 3.3% of data time locked to the article. Data with excessive blinks were corrected using a spatial filter algorithm (Dale, 1994), and a digital band-pass filter set from 0.1 to 20 Hz was used on all the data prior to running analyses to reduce the contribution of high frequency noise. Data were re-referenced to the algebraic sum of the left and right mastoids, and averaged for each experimental condition time locked to the onset of the critical determiner (article) and to the line drawing, respectively.
The stimuli consisted of sentence pairs in Spanish presented visually in black type on a white background. One noun in each sentence pair was replaced during presentation by a black on white line drawing (300 by 300 dpi). This picture target always appeared immediately after a gender-marked determiner. The picture target could appear in either the first or the second sentence, and in any position of the sentence. The picture target was never sentence final, though it could appear at clause boundaries, and was in some cases only followed by an adjective before the sentence terminated.
The sentence containing the picture target was presented one word at a time in the center of the computer monitor, and was preceded by a fixation marker (“++++”) for 1 second to focus the participant’s attention. A custom font was used to display the appropriate accents in Spanish. Each word remained on the screen for 300 msec, with 200 msec ISI. The picture target remained on the screen for 500 msec, with 200 msec ISI. The other sentence in the pair was always presented in its entirety, and remained on the screen until the participant pressed a button to advance to the next stimulus event. Approximately half of the trials began with the whole sentence followed by the target sentence; the other half of the trials began with the target sentence, followed by the whole sentence. The sentences were blocked by sentence type, target-first separate from target-second sentence pairs, so the reader could know when to blink without interfering with the recording. There was a 1 second ISI between sentence pairs.
Participants were seated in a dimly lit, sound-insulated booth approximately 3 feet from a color computer monitor. There was a brief practice session in which 2 examples of each experimental condition were presented. Participants were instructed to read the sentences silently for comprehension, and to interpret the picture as part of the sentence. Participants were told that they would be tested with a brief recognition questionnaire at the end of the experiment. They were asked to remain still during the experimental blocks, and to avoid blinking during the presentation of the word-by-word sentence. They could read the other sentence at their own pace, and advance to the next stimulus event by pressing a hand-held button. Stimuli were presented in several blocks of 33 or fewer sentence pairs. The participants were given a short break after each block, in addition to a longer break after half of the blocks were presented. At the end of the recording session the participants were given a brief recognition test with 50 line drawings, approximately one third of these were new and two thirds were items they actually saw during the recording session. They were asked to circle the drawings that they recalled seeing.
Stimuli were adapted from the original stimulus set of Wicha et al. (2000; submitted). Sentences from this original set for which the target item had a cloze probability of less than 0.65 in a norming study with a different group of participants were either eliminated or used as filler sentences. This left 59 experimental sentences for use from the original set. A new set of 117 sentences was added for a total of 176 experimental items. All the sentences were normed for cloze probability by a separate sample of native Spanish speaking students at the UABC (who were asked to complete sentence fragments, up to but not including the target article and noun, with the first word or words that came to mind), and had a cloze probability of 0.65 or greater (mean cloze of 0.80). Each of the 176 experimental items appeared in 4 experimental conditions (i.e., gender matching and semantically congruous, gender matching and semantically incongruous, gender mismatching and semantically congruous and gender mismatching and semantically incongruous) across 4 experimental lists for a total of 704 unique experimental events. A total of 88 sentences, each with a cloze probability of less than 0.65 for the critical word, were used as filler items. Hence, each of the 4 experimental lists consisted of a total of 264 sentences (i.e., 176 experimental and 88 filler sentences per list).
The picture targets were taken from the International Picture Naming Database (CRL-IPNP, 2000; Bates et al., in press). All the picture targets were tested on a separate sample of native Spanish speaking students of the UABC, and had a mean agreement of 0.91 for the name they elicited. To avoid confounding the effect of grammatical gender with semantic gender, there were no images of humans in the experimental set. In addition, only a small set of images depicting animals for which the name remains constant across genders was used (e.g., cebra – zebra; pulpo – octopus). Images depicting feminine objects that take a masculine article because of their initial stressed phoneme [a] were not used as experimental items (e.g., el hacha – the[masc] ax[fem]), nor were images that had a strong lexical competitor or a synonym of opposite gender (e.g., ball – balón[masc]/pelota[fem]). However, a small number of images with synonyms of the same gender were included (e.g., snake – víbora/serpiente/culebra[fem]).
Each of the 176 experimental sentence pairs and each of the 176 picture targets appeared once per experimental list (i.e., each participant saw 176 experimental sentences and picture targets only once in one of 4 experimental conditions), but were counterbalanced to appear in every experimental condition across four lists (i.e., the 704 total experimental events were seen across 4 participants/experimental lists). The stimuli were manipulated to create 4 experimental conditions: 1) gender congruent and semantically congruent, 2) gender congruent and semantically incongruent, 3) gender incongruent and semantically congruent and 4) gender incongruent and semantically incongruent. Two sentence pairs shared two line drawings of the same gender, one was the congruent and the other the incongruent continuation of the target sentence. An attempt was made to control for visual complexity, size and animacy between the matched images. The following is an example of two matched pictures embedded in their corresponding sentence pairs across the 4 experimental conditions. In 1a and 1b the pictures are semantically congruent with the sentence context and match in gender with the preceding article. In 2a and 2b the pictures are swapped to create the semantically incongruent sentences, but still agree in gender with the preceding article.
1a) Gender Match – Semantically Congruous
El príncipe soñaba con tener el trono de su padre. El sabía que cuando su padre muriera podría al fin ponerse la CORONA por el resto de su vida.
The prince dreamt about having the throne of his father. He knew that when his father died he would finally be able to wear the [fem] CROWN[fem] for the rest of his life.
1b) Gender Match – Semantically Congruous
Caperucita Roja llevaba la comida para su abuela en una CANASTA muy bonita. Pero el lobo llegó antes que ella.
Little Red Riding Hood carried the food for her grandmother in a [fem] BASKET[fem] [very pretty]. But the wolf arrived before her.
2a) Gender Match – Semantically Incongruous
El sabía que cuando su padre muriera podría al fin ponerse la CANASTA…
He knew that when his father died he would finally be able to wear the [fem] BASKET[fem]…
2b) Gender Match – Semantically Incongruous
Caperucita Roja llevaba la comida para su abuela en una CORONA…
Little Red Riding Hood carried the food for her grandmother in a [fem] CROWN[fem]…
To create the gender-agreement violations, an article of the opposite gender replaced the article of correct gender immediately preceding the target picture (3a and 3b). For example, the feminine items crown and basket in the gender agreement violation were each preceded by a masculine article (el and un, respectively). The fourth experimental condition was a combination of both types of violations (4a and 4b).
3a) Gender Mismatch – Semantically Congruous
El sabía que cuando su padre muriera podría al fin ponerse el CORONA…
He knew that when his father died he would finally be able to wear the [masc] CROWN[fem]…
3b) Gender Mismatch – Semantically Congruous
Caperucita Roja llevaba la comida para su abuela en un CANASTA…
Little Red Riding Hood carried the food for her grandmother in a [masc] BASKET[fem]…
4a) Gender Mismatch – Semantically Incongruous
El sabía que cuando su padre muriera podría al fin ponerse el CANASTA…
He knew that when his father died he would finally be able to wear the [masc] BASKET[fem]…
4b) Gender Mismatch – Semantically Incongruous
Caperucita Roja llevaba la comida para su abuela en un CORONA…
Little Red Riding Hood carried the food for her grandmother in a [masc] CROWN[fem]…
It is important to note that in addition to the manipulated target article, the sentence pairs contained articles that did not precede a picture and were not manipulated to violate gender agreement or semantic fit to the sentence context. Hence a reader could not link the appearance of any given correct article to an upcoming violation or predict with high certainty that a violation would occur at a particular article.
Sentences were randomly assigned to one of four lists. However, in order to avoid a sentence or picture appearing more than once per list, each sentence/picture pair appeared in the same condition as its counter sentence/picture pair within the same list (e.g., 1a and 1b in list X; 2a and 2b in list Y; etc.). The sentences were then blocked by target-first and target-second sentence pairs for each list, and randomized for presentation.
Figure 1 illustrates the average visual ERPs (N = 28) for a 3-word time window, including the word prior to the article of interest (onset at 0 msec), the article (onset at 500 msec) and the picture target (onset at 1000 msec). Consistent with previous reports, the ERPs to visually presented words are characterized by early sensory components – N1, P2 – over frontal sites, followed by a slower negative component (N400 region). For the articles, this negativity is modulated by gender expectancy. The ERP to the picture target is likewise characterized by early sensory components, followed by a slow negative-going component and then a later positivity. For the picture target both these late slow components are modulated by semantic congruity; the latter half (positive-going) of this late region is modulated by both semantic congruity and gender agreement. Analyses are reported for the picture target and article, analyzed separately, each relative to their own 100 msec pre-stimulus baseline.
Mean amplitudes of ERPs to the line drawings were subjected to ANOVAs with 2 levels of semantic congruity (congruent and incongruent), 2 levels of gender agreement (article-picture referent match and mismatch), and 26 levels of electrode for two time windows; the first between 200–500 msec encompasses the region of the N400, while the second between 500–700 msec post stimulus onset encompasses the region of the P600. Effects for repeated measures with greater than one degree of freedom are reported after Huynh-Feldt correction. Performance on the off-line picture recall test was high, with a mean accuracy for recalling the pictures presented in the experiment of 0.85 and only 2 subjects whose accuracy was less than 0.70. This measure confirms that the subjects were attending to the stimuli at an appropriate level. No subjects’ data were eliminated based on their memory scores.
Representative ERPs to the semantically congruent and incongruent target line drawings collapsed across gender agreement, from electrodes going from front to back along the middle of the head are shown in the left half of Figure 2. As expected, semantic congruity had a large effect, with a relatively greater negativity (i.e., larger N400) to incongruent than congruent line drawings between 200 and 500 msec [main effect of congruity, F (1, 27) = 63.10; p < 0.00001)]. A significant congruity by electrode interaction effect [F (25, 675) = 11.57; p < 0.00001] confirms the typical distribution of an N400 effect for pictures being broadly distributed and largest over medial frontal electrode sites on the right. Additional analyses indicate that both the main effect of semantic congruity and its interaction with electrode site are reliable as early as 175 msec post picture onset and are maintained at least throughout the recording epoch (~ 900 msec).
Representative ERPs to the line drawing at these same electrode locations as a function of gender match or mismatch of the line drawing’s referent with the preceding article, collapsed across semantic congruity, are shown in the right half of Figure 2. First, notice that the effect of gender mismatch on the picture is much smaller and starts later than the effect of semantic congruity, although it is in the same direction. Gender mismatches had no reliable effect on the ERP between 200 and 500 msec [F (1, 27) = 0.34, p = 0.57], but were significantly more negative between 500 and 700 msec post-stimulus onset than gender matches [F (1, 27) = 9.96, p < 0.004]. Moreover, a significant gender by electrode interaction between 500 and 700 msec [F (25, 675) = 7.25, p < 0.0001] indicates that the distribution of this gender mismatch effect was most pronounced over medial frontal electrode sites over the left hemisphere. Additional analyses of 100 msec intervals indicate that there were significant interactions with electrode between 0 and 200 msec, as well as from 500 msec to the end of the recording epoch (through ~ 900 msec). Of the early effects, in the first 100 msec the response to gender mismatches had a larger negativity than that to gender matches over frontal, prefrontal and medial central sites but not parietal, temporal or occipital electrode sites (p < 0.003). By contrast, between 100 and 200 msec the ERPs to gender mismatches were very slightly more positive than those to matches, but only over occipital sites (p < 0.002).
Perhaps surprisingly, there was no significant interaction between semantic congruity and gender agreement during either the N400 or P600 measurement window [200–500 msec: F (1, 27) = 0.84, p = 0.37 and 500–700 msec: F (1, 27) = 0.57, p < 0.46]. In fact, finer-grained analyses using 100 msec intervals indicate that there is never a significant interaction between semantic congruity and gender agreement throughout the recording epoch (Figure 3). Nor were there any significant three-way interactions between semantic congruity, gender agreement and electrode site in any time window [200–500 msec: F (25, 675) = 0.85, p = 0.54 and 500–700 msec: F (25, 675) = 1.52, p = 0.20].
This lack of interaction between semantic congruity and gender agreement can be seen in the difference ERPs in Figure 4. The left half of the figure shows that the effect of semantic congruity (incongruous minus congruous ERPs) is the same whether the picture’s referent matches or does not match the preceding article in gender. The right half of the figure shows that the effect of gender agreement (mismatch minus match) is the same whether the target picture fit or did not fit with the meaning of the sentence context. None of the small differences in these difference ERPs is reliable.
Post-hoc analyses were conducted to further elaborate the distribution of the effects of Gender Agreement and Semantic Congruity. The electrode sites were divided by hemisphere, thereby eliminating midline electrode sites (MiPf, MiCe, MiPa, MiOc), then according to lateral and medial locations within each hemisphere, thereby eliminating dorsal electrode sites (LDFr, RDFr, LDPa, RDPa). The remaining 16 electrodes were then divided into 4 regions from front to back of the scalp and were subjected to an ANOVA on 5 repeated measures with 2 levels of semantic congruity, 2 levels of gender agreement, 2 levels of hemisphere (left and right), 2 levels of laterality (lateral and medial) and 4 levels of anterior/posterior (prefrontal, frontal, central-temporal and occipital). These were used to examine the distribution of the effects in the two time windows of interest (200–500 and 500–700 msec) for the picture targets. The distributional analysis between 200–500 msec showed main effects of Semantic Congruity [F (1, 27) = 59.31; p < 0.00001], Hemisphere [F (1, 27) = 7.87, p < 0.01], Laterality [F (1, 27) = 58.48, p < 0.00001] and Anteriority [F (1, 27) = 7.30, p < 0.01], but no main effect of Gender Agreement [F (1, 27) = 0.18, p = 0.68]. Semantic Congruity interacted with Hemisphere [F (1, 27) = 18.48, p < 0.0002], Laterality [F (1, 27) = 26.97, p < 0.00001], and marginally with Anteriority [F (1, 27) = 4.54, p < 0.04]. In brief, the N400 effect was larger over right than left hemispheres, larger at medial than lateral sites, and marginally larger over anterior than posterior sites. Significant three–way interactions between Semantic Congruity, Hemisphere and Laterality [F (1, 27) = 26.38, p < 0.0001] and Semantic Congruity, Laterality and Anteriority [F (1, 27) = 16.93, p < 0.0001] indicate that though the N400 effect is larger at medial sites than lateral sites, the medial-lateral difference is more pronounced over the left than right hemisphere and posteriorly more so than anteriorly.
Between 500–700 msec post-picture onset, there were significant main effects of Semantic Congruity [F (1, 27) = 29.6, p < 0.0001], Gender Agreement [F (1, 27) = 9.1, p < 0.006], Laterality [F (1, 27) = 125.34, p < 0.00001] and Anteriority [F (1, 27) = 9.61, p < 0.004], but no main effect of Hemisphere [F (1, 27) = 0.94, p = 0.34]. Semantic Congruity interacted with Laterality [F (1, 27) = 18.98, p < 0.0002], Anteriority [F (1, 27) = 7.23, p < 0.01] and marginally with Hemisphere [F (1, 27) = 6.29, p < 0.02], indicating that the effect of Semantic Congruity was again greater at medial than lateral sites, larger over anterior than posterior sites and marginally larger over the right than left hemisphere. Significant three-way interactions between Semantic Congruity, Hemisphere and Laterality [F (1, 27) = 11.52, p < 0.002] and Semantic Congruity, Laterality and Anteriority [F (1, 27) = 12.56, p < 0.003] indicate that the effect of Semantic Congruity is symmetric over right and left hemispheres at medial sites, but slightly greater over the right hemisphere at lateral sites, and greater at medial sites especially over central–temporal electrode sites.
Gender agreement also interacted with Laterality [F (1, 27) = 13.18, p < 0.001] and Anteriority [F (1, 27) = 6.73, p < 0.003], but not with Hemisphere [F (1, 27) = 0.25, p = 0.62]. The effect of Gender Agreement was greater at medial than lateral electrodes and was larger at frontal sites with little to no effect at occipital sites. A significant three–way interaction between Gender Agreement, Hemisphere and Anteriority [F (3, 81) = 6.43, p < 0.003] indicated that the effect of gender was symmetric across right and left hemispheres except at frontal electrode sites, where the effect was larger over the left than right hemisphere.
In sum, though widely distributed across the scalp, the effect of Gender Agreement and Semantic Congruity both appear to be greater at medial than lateral and anterior than posterior sites. The effect of Semantic Congruity shows a tendency toward the right hemisphere, particularly at lateral sites, whereas the effect of Gender Agreement had a slight tendency toward the left hemisphere, particularly at frontal electrode sites.
To determine if the article was expected based on the meaning of the sentence context up to that point, we subjected the mean amplitude of the ERPs between 300–600 msec after the onset of the article to an ANOVA with 2 levels of gender expectancy (expected and unexpected gender) and 26 levels of electrode. Expected gender was defined as the gender corresponding to that of the target noun (depicted by a line drawing) with the highest cloze probability for each sentence context, based on the cloze procedures described in the method section. In the example above, canasta is the expected noun; hence an article of feminine gender would be expected (una) while an article of masculine gender would be unexpected (un).
The main effect of expectancy (Figure 5) was marginally significant [F (1, 27) = 6.80; p < 0.015] between 300–600 msec with a larger negativity for articles with unexpected than expected gender. Additional analyses of 100 msec intervals to track the timing of the effect reveal that the gender expectation effect is significant between 300–400 msec [F (1, 27) = 10.58, p < 0.003], marginally significant as an interaction with electrode site between 400–500 msec over central and posterior, but not at prefrontal and frontal sites (p < 0.038), and marginally significant between 500 and 700 msec (p < 0.02). The effect between 300 and 400 msec was widely distributed, with a slightly larger amplitude over left occipital sites, though the gender expectancy by electrode interaction was not significant [F (25, 675) = 1.51, p = 0.19].
In brief, the semantic fit of the picture with the prior sentence context modulated the amplitude of the N400 to the target picture between 200–500 msec, maximally over medial-frontal electrode sites on the right. Gender agreement modulated the response to the drawing as a main effect between 500–700 msec, maximally over medial-frontal electrode sites on the left, but also slightly in an interaction with recording site between 0–200 msec and 500–900 msec. In addition, gender expectation modulated the ERP to the article with a widely distributed enhanced negativity to unexpected relative to expected articles between 300–600 msec post-article onset.
Our results are clear in demonstrating that readers are sensitive to grammatical gender and gender agreement between syntactically related items (an article and picture referent) during sentence reading. The brain’s electrical response to a picture was only slightly, but significantly, different when the grammatical gender of its referent mismatched that of the preceding article. Moreover, we found that readers do seem to use accumulating sentence context to anticipate upcoming concepts at a fairly specific level. The brain’s electrical response to the preceding article was different when its grammatical gender differed from that of the noun expected, or preferred, on the basis of sentence context. Both for gender mismatches at the picture and gender violations at the article, the response to the unexpected item was characterized by a greater negativity relative to the response to the expected item.
The presence of a classic N400 effect for semantically congruent versus incongruent pictures indicates that readers were analyzing the stimulus materials for meaning and did attempt to integrate a visual depiction of a concept into the written utterance. When the picture did not fit, there was an enhanced negativity (including the N400 component) starting as early as 175 msec, lasting for at least another 700 msec throughout the recording epoch. This N400 effect is similar to that previously reported for pictures appearing at the end of written sentence contexts (Ganis et al., 1996; Federmeier and Kutas, 2001), thereby confirming previous reports stating that at least some aspects of semantic integration in a sentence context are neither modality-dependent (i.e., word versus picture) nor rely on explicit presence of a lexical item (word).
Our results go beyond these studies in showing that gender-agreement between the preceding article and the picture’s referent had a reliable effect, although quite small relative to that of semantic fit, on the picture’s analysis. Pictures that were associated with a gender-agreement violation elicited greater negativity between 500–700 msec maximal over medial frontal electrodes on the left, relative to those whose referents agreed with the preceding article in gender. This effect of gender agreement at the picture appears to be relatively independent of semantic fit, since it affected the ERP response to the picture similarly whether or not it was a meaningful continuation in the sentence. Inspection of the topographical distribution of the effect (Figure 6) shows a left frontal focus for this gender agreement effect between 500–700 msec, suggesting that it recruits different neural processes than those involved in generating semantic congruity N400 effects, which are more prominent over right frontal locations for these pictures.
Our results indicate that semantic processing clearly was not halted when the picture’s referent mismatched in gender with the preceding article. In fact, a gender-agreement violation between the article and picture did not significantly affect either the timing or amplitude of the N400 between 200–500 msec elicited by the picture when it was semantically inappropriate. Thus, overall our results show only a very small, relatively late effect of grammatical gender agreement violations on picture processing. Moreover, this effect bears no resemblance to any of the ERP effects previously observed in association with a violation of grammatical gender, which have typically included a large P600 sometimes preceded by an earlier anterior negativity (e.g., Hagoort and Brown, 1999; Gunter et al., 2000).
The effect of gender mismatch on the processing of the target may be relatively small because the gender mismatch was perfectly predictable once the unexpected article appeared. Following an article of unexpected gender, the line drawing always depicted a word with the gender that was expected based on the sentence context. For example, if the sentence context set up an expectation for the feminine word canasta, it would either continue congruously at a semantic level with canasta, or incongruously with a word of the same gender, such as corona. Thus the gender of the picture’s referent was always correct with respect to the prior sentence context. The article-noun gender mismatches were created by altering the gender markings on the article just prior to the picture. As a consequence, when participants read an article with a gender marking that was inconsistent with the noun expected, s/he might guess that the following item would be a drawing and that its referent would then have a gender different from that of the article. It may not be too surprising therefore that the gender effect was quite small.
We observed two interactions between gender agreement and electrode site immediately after picture onset. In the first 100 msec gender mismatches were associated with a slightly larger negative component (N1 component) over medial-central, frontal and prefrontal sites. In the following 100 msec interval between 100–200 msec the response to gender mismatches was slightly more positive than those to matches, but only over occipital electrode sites. The first effect seems to occur too early to be driven by activity related to processing the grammatical gender of the picture’s referent, unless the preceding article is providing this information. The N1 component is generally linked to early sensory processing and is modulated by attentional manipulations (e.g., Hillyard et al., 1973, 1995). Hence, this may be an attention-driven effect. If readers attend to the gender feature of the article, the brain may have used this information to direct attention to the gender information of the upcoming target, especially given that it is a picture whose referent is implicit, i.e., requires lexical look up. Whenever the article has unexpected gender markings based on the prior sentence context, the greater attention directed to the upcoming picture may be reflected in an enhancement of the N1 relative to when the article has the expected gender. Another possibility is that the activity reflects residual processing of the unexpected gender-marked article, which may have required more effort than the expected article. Further studies are needed to adjudicate between these and other alternatives.
Interestingly, as mentioned previously, the effect of gender agreement between a written article and a picture was different from what has been reported by others for agreement violations (gender, number or other types of agreement) between two words – written or spoken. Several studies examining the effect of verb agreement (tense or number) and gender agreement on pronouns all have reported a larger P600 – a late positive going potential over posterior electrodes – to the violations (Hagoort et al., 1993; Van Berkum et al., 1999; Brown et al., 2000), at times accompanied by a left anterior negativity (Friederici et al., 1993; Osterhout and Mobley, 1995; Coulson et al., 1998). By contrast, the gender agreement effect in the current study is neither positive, nor restricted to anterior electrode sites over the left hemisphere.
The absence of a P600 response to gender agreement violations in our data could be due to the fact that in our stimuli a gender mismatch is engendered by a word that does not actually appear – i.e., the target picture’s referent. Other studies that have reported a P600 to gender mismatches used only word stimuli. In a distantly related study of Spanish-sentence processing, Demestre et al. (1999) examined ERPs to gender-marked adjectives, which either did or did not match their antecedents (a proper noun) in naturally spoken Spanish sentences (e.g., “Pedro quiere ser rico/rica…” – Pedromasc wants to be richmasc/fem…). The ERP of interest was time-locked to the last syllable of the gender-marked adjective. The ERP to the violation was characterized by a biphasic response comprising a negativity (150 to 250 msec) over anterior and central sites, followed by a widely distributed positivity between 250 and 500 milliseconds (P600) relative to the gender correct adjectives. These materials differed from ours in that (1) the gender violation occurred on an adjective rather than a noun, (2) the antecedent and adjective were separated rather than adjacent, and (3) the contextual constraint leading up to the adjective was relatively weak and thus did not lead to a strong semantic expectation, though the expected gender (i.e., the semantic gender of the referent) was clear.
In a study more closely related to ours, Hagoort and Brown (1999) presented sentences visually in Dutch one word at a time, and similar to the current study, manipulated the gender agreement between an article-noun pair, with an intervening adjective. They reported that the ERP to the target noun, which could appear either at the beginning (e.g., “De/Het Kapotte paraplu staat in de Garage.” – Thecommon/Theneuter broken umbrellacommon is in the garage.) or the end of the sentence (“Cindy sliep slecht vanwege de/het griezelige Droom”. – Cindy slept badly due to thecommon/theneuter scary dreamcommon.), showed an increased positivity maximal over posterior sites (500 msec post onset) to gender agreement violations as compared to nouns with correct agreement markings. These materials, unlike ours, had little to no context to set up an expectation for the noun at the semantic level. In a similar study, Gunter et al. (2000) investigated the interaction of grammatical gender and semantic information by manipulating gender agreement between an article-noun pair and the cloze probability of the noun based on its relation to the preceding verb in written German sentences presented one word at a time (e.g., “Sie bereist/befährt das/den Land … – She travels/drives the neuter/masculine land neuter …”). Like us, they found no effect of gender agreement on the amplitude or timing of the N400 to the noun. They did, however, observe an increased late positivity over centro-parietal sites (P600) to high (but not to low) probability nouns with a gender-agreement violation, as well as a left anterior negativity (LAN) to both high and low probability nouns with a gender-agreement violation. They did not report an analysis of the ERP to the article itself.
Our findings are consistent with these studies in that we find that gender agreement does have an effect on processing a target in a sentence context. The response to target pictures, however, was associated with an increased negativity to gender agreement violations, whereas the response for words in prior experiments was characterized by an increased late positivity. This negativity occurred within approximately the same time window as the P600. This suggests that processing gender agreement between a word and a picture is different from that between two words. The ease of appreciation, relevance or use of the information provided at the mismatch may differ considerably when integrating a picture or a word into a written sentence context. As mentioned previously, some researchers maintain that the P600 to the target word indicates that an individual is forced to reprocess the sentence in order to make the target fit the incorrect structure (e.g., Osterhout et al., 1996). Others in contrast have argued that it is the recognition or surprise value of the error itself, or the inability to fully integrate the target into the context, that leads to the P600 (e.g., Coulson et al., 1998; Patel et al., 1998). Our findings at the picture cannot adjudicate between these two alternatives, as we do not observe a P600. Similar to word processing, integrating a picture into a sentence context is sensitive to gender agreement. Therefore, it seems that readers are using gender agreement information in real time during sentence comprehension.
The most novel finding in the current study was obtained is when the reader came across an article with gender markings that were unexpected based on the context up to that point. Here we observed a greater negativity to articles of unexpected gender relative to expected gender. It is important to remember that an article of either gender is syntactically, semantically and pragmatically correct at this point in the sentence. Red Riding Hood in the example above could just as well have been carrying food in a masculine gender sack (un costal) as in a feminine gender basket (una canasta), though one of these continuations is clearly more expected according to the offline cloze ratings. There is no overt syntactic violation at the article, regardless of which gender marking it bears. Thus, the ERP effect at the article as a function of grammatical gender must be due to a “violation” between the gender of the article-noun pair expected on the basis of the preceding context and that of the article actually presented. Our observation of an enhanced negativity at the article thus suggests that our participants were in fact expecting, if not a specific noun, a noun of a specific gender and its associated article of the same gender, and moreover that they considered it a type of semantic violation when an article of the opposite (unexpected) gender appeared instead.
Most ERP studies of gender agreement were unable to see such an effect as they focused on the response to the second of a pair of items that either did or did not agree with each other in gender, since this is the locus at which the gender (dis)agreement is obvious. For example, Hagoort and Brown (1999) analyzed the effect of gender agreement on umbrella in “Thecommon/Theneuter broken umbrellacommon …”; there was no prior context that supported any particular expectation at the article. Similarly, Gunter et al. (2000) used short sentence contexts that relied on the lexical-semantic relationship between a noun and a preceding verb (“Sie bereist/befährt das/den Land … – She travels/drives the neuter/masculine land neuter …”). Their sentences always included neuter nouns, since cloze probability was defined in terms of the context and neuter article das. The P600 and/or LAN effects observed in these studies presumably reflect the agreement violations per se at the level of syntax.
By contrast, we took advantage of the fact that people generally concurred about what target item was expected given the sentence context and the fact that Spanish has a frequently used and reliable gender-agreement system, and found that individuals do use context to create semantic expectations about upcoming concepts and their referents, at least when these are represented in line drawings. When these expectations are violated, the ERPs are characterized by a larger N400; not a LAN or a greater P600. The presence of a differential ERP effect at the article as a function of expected gender indicates that readers do pay attention to articles and their gender, and do use the meaning of the sentence context to expect articles of specific gender. Van Berkum and colleagues (Van Berkum et al., 2001) recently observed an effect that supports this conclusion. They conducted an auditory sentence comprehension study in Dutch – a language that also has two gender classes, neuter and common genders – where they examined a gender-marked adjective preceding a target noun. Preliminary analyses revealed a greater positivity between 50 to 250 msec to adjectives unexpected based on the prior sentence context, when the activity was time-locked to the onset of the gender-marked syllable of the adjective, as compared to expected adjectives. The difference in the polarity and timing of the effects across our studies may be due to the differences in the stimuli – spoken adjectives as opposed to visually–presented articles. Nevertheless, their findings and ours converge in demonstrating that language comprehenders have expectations for particular targets based on prior sentence context, as the effect at the article or adjective could only be due to the violation of such expectations.
In summary, the presence of an effect of semantic fit at the picture indicates that readers attempted to integrate these pictures into the sentence context, and were successful in so doing when they were semantically appropriate. Similarly the presence of an effect of gender agreement at the picture indicates that readers do pay attention to articles and to gender agreement information during sentence comprehension, even for referents of non-lexical items, such as line drawings. This effect of gender agreement seems in large part to be independent of semantic fit, at least for picture stimuli. Most importantly, the differential response to an unexpected versus expected article indicates that readers attend to gender markings of articles. This is important because it shows that grammatical gender functions not only to indicate agreement between items in a sentence, but also as part of the greater context in leading to comprehension. Our data thus clearly show that context can lead to very specific expectations (lexical, conceptual and/or morphological), and that various sources of information, including grammatical gender of an article, are used in comprehension in real time as sentence meaning unfolds.
This study was supported by NIH grants HD22614 and AG08313 to M.K. and NIH/NIDCD DC00351 and APA-MFP MH18882 to N.Y.Y.W. We thank Ron Ohst, Paul Krewski, Robert Buffington and Araceli Orozco for outstanding technical support, Kara Federmeier and Elizabeth Bates for comments, and Lourdes Gavaldón de Barreto, Guadalupe Bejarle Pano and the Universidad Autónoma de Baja California – Tijuana for their collaboration.
Each row consists of two sentence pairs (and their corresponding English gloss) across which two corresponding pictures were counterbalanced. A sentence-pair took the corresponding picture for the semantically congruent conditions (i.e., sentence-pair A with picture A; sentence-pair B with picture B) and the opposite picture for the semantically incongruent conditions (i.e., sentence-pair A with picture B; sentence-pair B with picture A). The blank space in a sentence indicates where the picture appeared during presentation. In addition, each sentence appeared with the expected article (e.g., la for first sentence A) and unexpected article, indicated with an *, with each of the pictures to create the 4 experimental conditions.
|sentence-pair A||picture A||sentence-pair B||picture B|
|El príncipe soñaba con tener el trono de su padre. El sabía que cuando su padre muriera podría al fin ponerse la/*el _____ por el resto de su vida.||Caperucita Roja llevaba la comida para su abuela en una/*un _____ muy bonita. Pero el lobo llegó antes que ella.|
|The prince dreamt about having the throne of his father. He knew that when his father died he would be able to finally wear the[fem]/the[masc] _____ for the rest of his life.||Red Riding Hood carried the food for her grandmother in a[fem]/a[masc] _____ [very pretty]. But the wolf arrived before her.|
|Cada año mueren miles de animales porque su piel se usa para fabricar artículos de invierno. Al descubrir que mi tía se acababa de comprar un/*una _____ de piel me quedé muy desilusionado.||Cuando Guillermo le propuso matrimonio a Brenda, se hincó en una rodilla y le dio un/*una _____ de compromiso. No fue gran sorpresa que la novia le dijera que sí.|
|Each year thousands of animals die because their fur is used to make winter articles. When I discovered that my aunt had just purchased a[masc]/a[fem] _____ I was very disappointed.||When Guillermo proposed matrimony to Brenda, he kneeled down on one knee and gave her a[masc]/a[fem] _____ of engagement. It was not a big surprise that the bride-to-be-said yes.|
|El aire acondicionado estaba descompuesto y hacía tanto calor que tuve que abrir la/*el _____ para refrescar el cuarto. Pero como era el fin del verano decidí no arreglarlo hasta el próximo año.||Cada vez que mi tía Elena cocina sopa de verduras parece que está llorando. Ella dice que desde su niñez, siempre que corta una/*un _____ no puede evitar que le salgan lágrimas.|
|The air conditioning was broken and it was so hot that I had to open the[fem]/the[masc] _____ to refresh the room. But since it was the end of the summer I decided not to fix it until the next year.||Each time my aunt Elena cooks vegetable soup it appears that she is crying. She says that since her childhood, every time she cuts an[fem]/an[masc] _____ she cannot stop the tears from coming.|
|Para escarbar un hoyo afuera, necesitaré la/*el _____ que está en la cochera. También debería ponerme unas botas para que no se ensucien mis pies.||Un monociclo es como una bicicleta con una/*un _____ en vez de dos. La primera vez que vi a alguien andando en monociclo fue en el circo.|
|To dig a hole outside, I will need the[fem]/the[masc] _____ that is in the garage. I should also put on some boots so that I don’t dirty my feet.||A unicycle is like a bicycle with a[fem]/a[masc] _____ instead of two. The first time I saw someone riding a unicycle was at the circus.|
|Mi hermano siempre se anda metiendo en pleitos con los otros chicos del vecindario. La última vez que se peleó llegó a la casa sangrando de la/*el _____ y todavía le quedaban ganas de pelearse más.||Cuando estoy solo y alguien toca la/*el _____ me gusta preguntar quien es antes de abrirla. Fue mi abuelo quien me aconsejó que siempre preguntara.|
|My brother is always getting into fights with the other boys in the neighborhood. The last time he fought he arrived at the house bleeding from the[fem]/the[masc] _____ and he still felt like fighting.||When I am alone and someone knocks on the[fem]/the[masc] _____ I like to ask who it is before opening it. It was my grandfather who recommended that I always ask.|
|El apartamento donde vive mi hermana es muy chico. Mi sobrina tendrá más espacio donde jugar ahora que compren la/*el _____ nueva y se cambien.||Esta mañana mi madre y yo fuimos al supermercado a comprar comida para la fiesta. Como teníamos prisa, le ayudé a la cajera a meter las cosas en la/*el _____ para que nos pudiéramos ir pronto.|
|The apartment where my sister lives is very small. My niece will have more space to play now that they will buy the[fem]/the[masc] _____ [new] and they move.||This morning my mother and I went to the supermarket to buy the food for the party. Since we were in a hurry, I helped the cashier put the things in the[fem]/the[masc] _____ so that we could leave quickly.|
|En el zoológico le dimos de comer a varios animales. El pichón se comió las semillas y el elefante se comió los/*las _____ que Paco le dio.||Antes de que pudiera ponerme los zapatos, tuve que encontrar los/*las _____ que estaban en el cuarto. Finalmente, los encontré debajo de la cama.|
|At the zoo we gave food to various animals. The pigeon ate the seeds and the elephant ate the[masc]/ the][fem] _____ that Paco gave it.||Before I could put on [my] shoes, I had to find the[masc]/the[fem] _____ that were in the room. Finally, I found them under the bed.|