|Home | About | Journals | Submit | Contact Us | Français|
Although phonological representations have been a primary focus of verbal working memory research, lexical-semantic manipulations also influence performance. In the present study, the authors investigated whether a classic phenomenon in verbal working memory, the phonological similarity effect (PSE), is modulated by a lexical-semantic variable, word concreteness. Phonological overlap and concreteness were factorially manipulated in each of four experiments across which presentation modality (Experiments 1 and 2: visual presentation; Experiments 3 and 4: auditory presentation) and concurrent articulation (present in Experiments 2 and 4) were manipulated. In addition to main effects of each variable, results show a Phonological Overlap × Concreteness interaction whereby the magnitude of the PSE is greater for concrete word lists relative to abstract word lists. This effect is driven by superior item memory for nonoverlapping, concrete lists and is robust to the modality of presentation and concurrent articulation. These results demonstrate that in verbal working memory tasks, there are multiple routes to the phonological form of a word and that maintenance and retrieval occur over more than just a phonological level.
The ability to maintain a representation of information that is no longer in the environment is critical to many aspects of human behavior. The nature of the system underlying this ability, termed short-term or working memory, has been a source of investigation for over a century (Hebb, 1949; James, 1890; Miller, 1956). In the case of verbal working memory, a vast amount of research has been dedicated to examining how people maintain phonological and/or acoustic representations, with relatively less research dedicated to examining memory for lexical-semantic representations. Phonological and lexical-semantic representations have almost always been considered in isolation, both experimentally and theoretically. In fact, within the most prevalent model of working memory, the multicomponent model (Baddeley, 1986; Baddeley & Hitch, 1974), lexical-semantic representations plays no role in maintenance. Instead, maintenance is thought to occur within a phonological loop composed of a phonological store whose contents decay over time unless refreshed via articulation. A growing body of research has led to questioning of the assumption that maintenance is strictly phonological (e.g., Haarman & Usher, 2001; R. C. Martin, Shelton, & Yaffee, 1994), with some suggesting that maintenance occurs by virtue of the interaction of phonological and lexical-semantic representations (e.g., N. Martin & Saffran, 1997; Patterson, Graham, & Hodges, 1994). The goal of the present investigation is to examine this interaction through a manipulation of phonological, articulatory, and lexical-semantic factors that have been shown to influence working memory performance: phonological similarity, concurrent articulation, and word concreteness. We first briefly review the independent contributions of each of these factors, followed by a more extensive review of research in which researchers have simultaneously manipulated or observed lexical-semantic and phonological interactions.
Phonological and articulatory influences on working memory performance have been extensively studied and have yielded a set of replicable phenomena. For instance, individuals have a harder time remembering lists of multisyllabic words relative to single-syllable ones (the word length effect; Baddeley, Thomson, & Buchanan, 1975). Memory for lists containing phonological overlap is worse than for those that do not (the phonological similarity effect [PSE]; Baddeley, 1966; Conrad & Hull, 1964; Wickelgren, 1965). Listening to irrelevant sound impairs memory (Colle & Welsh, 1976), most notably when the irrelevant sound is in a changing state, such as in speech (Salame & Baddeley, 1982) or in a sequence of tones (Jones & Macken, 1993). Finally, concurrent articulation of an irrelevant syllable (e.g., “the, the, the”) abolishes the phonological similarity and word length effects when information is presented visually but not auditorily (Levy, 1971; Murray, 1968). Of particular relevance to the present investigation is the interaction of phonological similarity, concurrent articulation, and presentation modality. Within the phonological loop framework, the classic interpretation of this interaction is that the PSE is due to interference among elements in a phonological store dedicated to short-term maintenance. Information that is presented auditorily has direct access to this store, whereas visual information must be recoded via articulation, hence the PSE is abolished when information is presented visually. However, there is considerable evidence to suggest that articulation is not the only means by which the phonological form of a visually presented word is accessed (see Experiment 2 below and Besner, 1987). Furthermore, Jones and colleagues (Jones, Macken, & Nicholls, 2004) showed that the survival of the PSE under concurrent articulation may be limited to the recency portion of serial position curves, which, when combined with the irrelevant tone effect described above, suggests an auditory-perceptual basis to the PSE (Jones, Hughes, & Macken, 2006). Coupled with evidence for lexical-semantic influences on working memory performance, these findings necessarily lead to consideration of alternative architectures underlying maintenance in verbal working memory.
Important insight into the systems responsible for verbal working memory maintenance is provided by research showing that long-term, lexical-semantic representations influence performance. Words are recalled better than nonwords (Besner & Develaar, 1982; Brener, 1940; Crowder, 1976); high frequency words are recalled better than low frequency words (Roodenrys, Hulme, Alban, Ellis, & Brown, 1994; Tehan & Humphreys, 1988; Watkins, 1977); and concrete or imageable words are recalled better than abstract or nonimageable ones (Bourassa & Besner, 1994; Walker & Hulme, 1999). Semantic similarity can also influence working memory performance, with some studies showing impairment (Baddeley, 1966; Dale & Gregory, 1966; Kintsch & Buschke, 1969; cf. Baddeley & Dale, 1966) and others showing improvement (Levy & Murdock, 1968; Murdock & Vom Saal, 1967; Poirier & Saint-Aubin, 1995; Saint-Aubin & Poirier, 2000; Wickens & Eckler, 1968). Finally, neuropsychological investigations have shown that impairments affecting lexical-semantic representations also affect working memory performance. Patients with deep dysphasia (a disorder associated with semantic errors in word repetition and an inability to repeat nonwords) do not exhibit typical effects of lexical frequency or imageability (N. Martin, Saffran, & Dell, 1996). Furthermore, double-dissociations have been observed between semantic and phonological impairments in patients with damage due to cerebral infarct (R. C. Martin, et al., 1994). Although this latter research suggests that lexical-semantic and phonological influences on working memory may be independent, there is also substantial evidence demonstrating that these two levels of representation interact in working memory and that this interaction may be critical to maintenance.
Evidence for independent lexical-semantic and phonological influences on performance has been provided from a number of studies. Some patient populations appear to have selective deficits to maintaining phonological representations with relatively spared language processing (i.e., short-term memory patients; Shallice & Warrington, 1970; Vallar & Baddeley, 1984; although, see Allport, 1984). In addition to patient studies above showing double dissociations (R. C. Martin et al., 1994) or continuous influences (N. Martin & Saffran, 1997) of lexical-semantic and phonological impairments on working memory tasks, behavioral research has shown dissociations between lexical-semantic and phonological influences on item and order memory, respectively. For instance, phonological similarity impairs memory for the order in which items appear and not for the items themselves (Fallon, Groves, & Tehan, 1999) and, in some cases, can even improve item memory (Lian, Karlsen, & Eriksen, 2004). Semantic similarity effects facilitate item memory, with no effect on order memory (although see Baddeley, 1966), and are robust to concurrent articulation, which suggests that the effects may be independent of those observed for phonological manipulations (Poirier & Saint-Aubin, 1995). Walker and Hulme (1999) observed superior item memory for concrete word lists relative to abstract word lists in spoken recall, but not a matching span procedure, the latter of which isolates order memory. However, a more recent study using an open set of words showed concreteness effects not only for matching-span but also for an order reconstruction task (Romani, McAlpine, & Martin, 2008). Furthermore, although Murdock and Vom Saal (1967) found superior item memory for same-category lists, they also observed more item transpositions. Thus, although much of the evidence cited above suggests that lexical-semantic factors influence item memory and that phonological factors influence order memory, the studies by Romani et al. (2008) and Murdock and Vom Saal (1967) call into question this clear dissociation.
Some of the most compelling evidence for the interaction of lexical-semantic and phonological representations comes from investigations of patients suffering from semantic impairments. Patients with semantic dementia or primary progressive fluent aphasia, a degenerative disorder associated with the loss of semantic information, show better recall for words that they can name from a picture relative to those words that they cannot name (Knott, Patterson, & Hodges, 2000; Patterson, Graham, & Hodges, 1994). Insight into the interaction of lexical-semantic and phonological representations comes from errors on words that patients cannot name, which often corresponded to movements among sublexical, phonological elements comprising the words in a list (e.g., recalling “hat cow” as “cat how,” in which the /h/ and /k/ phonemes have exchanged for each other). Similar errors were observed in an anomic patient (MS) who did not show the typical advantage for words over nonwords in recall (R. C. Martin, Lesch, & Bartha, 1999), and in patient HP, whose semantic impairment emerged from herpes encephalitis (Caza, Belleville, & Gilbert, 2002). These results are difficult to reconcile with the view that maintenance occurs strictly over a phonological level of representation in working memory. Instead, they suggest a system in which the maintenance and/or retrieval of a phonological sequence is directly affected by the integrity of a person’s lexical-semantic representations.
Phonological errors resulting from lexical-semantic impairments have been interpreted in two ways. The pattern of performance in the case of the anomic patient MS suggests impairment in activating phonological output representations in production from a semantic representation (R. C. Martin, Lesch, & Bartha., 1999). Patterson and colleagues (Patterson et al., 1994) have hypothesized something similar in proposing that semantic representation plays an important role in binding phonological elements together (i.e., the semantic binding hypothesis). According to this latter account, two types of associations bind speech sounds together: First is the co-occurrence of the speech sounds themselves; second is that words consistently activate their respective semantics. Thus, damage to the lexical-semantic system or in the connections between lexical-semantic representations and systems responsible for the serial ordering of speech sounds in language production will impair the integrity of the phonological form, resulting in both item-level and phonological ordering errors in recall and language production.
Despite this neuropsychological evidence, few behavioral studies in nonpatient populations have been directed at examining how lexical-semantic and phonological factors interact in verbal working memory. Although Baddeley (1966) manipulated semantic and phonological similarity within a single experiment, the manipulations were not done factorially. Studies by Hulme and colleagues (Walker & Hulme, 1999) have simultaneously manipulated concreteness with word length or phonological neighborhood density (Allen & Hulme, 2006), with results supporting independent influences of lexical-semantic and phonological factors. Although this research seems to further support the independence of these two factors, there is evidence for the lexical-semantic binding of the phonological form of words in nonpatient populations. Speech errors in normal language production provide one example.
When people substitute an inappropriate word for one they intended to say or when they blend two words together (e.g., saying “tab” when they intended to say taxi or cab), the substituted or blended words are phonologically related to the target word at a level far above chance (Dell & Reich, 1981). Similar errors can be induced experimentally. Jefferies, Frankish, and Lambon Ralph (2006) varied the amount of lexical-semantic binding within a serial recall paradigm by manipulating the ratio of words to nonwords in to-be-remembered list material. Jefferies et al. (2006) predicted that having a higher ratio of words to nonwords (e.g., three words and two nonwords) would result in fewer errors relative to lists with smaller ratios (e.g., two words and three nonwords), as the semantic binding of words should leave fewer sublexical positions over which speech sounds could move. Results showed that as the ratio of words to nonwords increased, people made fewer phoneme movement errors on the nonwords. Errors on words occurred as the ratio of words to nonwords decreased. Specifically, people made more phoneme intrusion errors, and words were more likely to move to different list positions. Finally, consistent with past research (Saint-Aubin & Poirier, 2000), words tended to cohere more than nonwords, thus there was a tendency for whole-item movements rather than the sublexical, phonological movements observed for the nonwords. These results support important components of the semantic binding hypothesis in showing that lexical-semantic binding of words leads their phonological units to stay together during recall, and furthermore, such binding influences the sublexical binding of the phonological elements of nonwords.
It is clear that there is ample evidence to suggest that lexical-semantic representations are fundamental to verbal working memory performance and that they interact with phonological representations. An important question to ask is what representational differences at the lexical-semantic level might account for the different patterns of behavior highlighted above. An understanding of these mechanisms is critical for a broad account of how verbal information is perceived, maintained, and produced over time. The focus of the present investigation is on one particular lexical-semantic property, word concreteness.
In addition to memory research above, processing advantages for concrete words relative to abstract words have been observed across many domains, including language production (Allen & Hulme, 2006; Bird, Howard, & Franklin, 2003), reading comprehension (Paivio & Begg, 1971; Schwanenflugel & Shoben, 1983), language acquisition (Brown, 1957), and word recognition (Davelaar & Besner, 1988). Two accounts have emerged to explain this processing advantage, the difference between which can be summarized according to whether the representational differences between concrete and abstract word differences are quantitative or qualitative.
On the quantitative side are accounts that suggest that concrete words may be associated with more propositions in long-term memory (De Groot, 1989) or that concrete words have both visual and contextual representations, while abstract words only have the latter (i.e., dual-coding theory; Paivio, 1971). Another possibility is that abstract and concrete words might simply differ in the number of features they possess, a perspective that lies at the heart of a computational model of deep dyslexia (Plaut & Shallice, 1993). Deep dyslexia is a reading disorder that is typified by semantic errors during reading (e.g., reading flower as tree), coupled with apparent visual errors (e.g., reading spoon as soon; Plaut & Shallice, 1993). Thus, similar to semantic dementia patients, those with deep dyslexia produce numerous semantic errors, and these semantic errors influence sublexical processing (although in this case, errors seem to be in translating orthography to semantics). Plaut and Shallice (1993) made the assumption that relative to abstract words, concrete words have more semantic features (e.g., they are more imageable, more predicates can be generated, etc.). The result of this very simple differentiation in representation led to profound differences in how the words were represented in a model that mapped from orthography to phonology via a recurrent semantic layer. After the model learned the mappings from orthography to semantics, concrete words established robust connections with the recurrent, cleanup layer, whereas abstract words did not. Consequently, concrete words formed a more robust semantic representation and were less susceptible to damage than were abstract words.
However, not all agree that quantitative differences are the means by which abstract and concrete words differ. There are individual patients, for instance, who show a reverse concreteness effect (i.e., better processing of abstract than concrete words; Breedin, Saffran, & Coslett, 1994; Sirigu, Duhamel, & Poncet, 1991; Warrington, 1975; Warrington & Shallice, 1984). Such findings are difficult to reconcile if the only difference between abstract and concrete words is due to different numbers of features or multiple levels of representation. Instead, the differences between abstract and concrete representation may be qualitative (Breedin et al., 1994). Support for this perspective comes from a study of a patient with semantic refractory access dysphasia (Crutch & Warrington, 2005). Refractoriness is associated with a “reduction in the ability to utilize the system for a period of time following activation” (Warrington & McCarthy, 1983; p. 874), and in the case of semantic refractory access dysphasia, patients exhibit difficulties processing semantically similar material. In testing such a patient, Crutch and Warrington (2005) showed a double dissociation between abstract and concrete words: Whereas processing difficulties for concrete words were induced by semantic similarity and not their association to each other, abstract words showed the opposite pattern. Although this pattern of performance was interpreted as indicating that concrete words are represented via a categorical structure, whereas abstract words are defined by their association to other words (Crutch & Warrington, 2005), a recent study failed to replicate these results. In studying another patient with a refractory access deficit, Hamilton and Coslett (2008) found comparable patterns of impairment on semantically related and associated words for both abstract and concrete word lists. Thus, although the representation of abstract and concrete words may be qualitatively different, these latter results question the categorical versus associative distinction for concrete and abstract representations made by Crutch and Warrington (2005).
Regardless of whether the differences between concrete and abstract words is quantitative or qualitative, it is safe to conclude that concrete words are associated with more sensory-motor properties, whereas abstract words are more dependent on the linguistic or associative context in which they appear. Although the research above shows that it is not uniformly the case the concrete words have a stronger or more robust semantic representation than do abstract words, in the absence of a supporting linguistic context (e.g., random word lists), this is precisely what would be predicted. In the present study, we take advantage of these differences to ask how word concreteness interacts with phonological similarity in verbal working memory. The theoretical motivation for the present study comes from perspectives suggesting that verbal working memory may be driven by language production processes (Acheson & MacDonald, 2009a; 2009b; Allen & Hulme, 2006; Page, Madge, Cumming, & Norris, 2007). In their recent review, Acheson and MacDonald (2009b) suggested that the PSE may occur during the process of phonological (i.e., positional) encoding in language production, although they noted that other levels of the production architecture are likely to influence performance. What is particularly germane about this perspective is that it offers predictions about the manipulations in the present study. Representations of conceptual concreteness occur at what has been termed the message level of production (a nonlinguistic representation of an idea to be uttered) or in the nonphonological representations of a word (i.e., a lemma; see Bock & Levelt, 1994). Following activation of a message, language production proceeds in two largely independent stages (Garrett, 1975). During functional encoding, the grammatical frame and lexical choices (i.e., words) are selected. In the second stage, positional encoding, the serial order of the words and their speech sounds is specified. This architecture thus successfully captures the distinction between item (i.e., word) and order that is emphasized in the working memory literature and predicts that manipulations of lexical-semantics will primarily influence item and phonological overlap order memory. However, interactive models of word production (e.g., Dell, 1986; Dell & O’Seaghdha, 1992; Foygel & Dell, 2000) also predict that there should be instances in which semantic representation influences phonological ordering.
In order to systematically investigate the interaction of concreteness and phonological overlap, we manipulated both the presentation modality of the stimuli (Experiments 1 and 2 have visual presentation; Experiments 3 and 4 have auditory presentation) and the presence of concurrent articulation (present in Experiments 2 and 4). On the basis of the language production architecture discussed above, we predict two ways in which a semantic-phonological interaction might present itself in performance on working memory tasks. First, the effect of concrete words may be robust enough to eliminate the deleterious effects of phonological similarity, hence the PSE will be reduced for concrete lists relative to abstract lists. Alternatively, phonological similarity may be too robust to allow for concreteness to influence performance in overlapping lists. Instead, concreteness effects may only be apparent only in nonoverlapping lists, hence the PSE will be larger for concrete relative to abstract lists. The first form of this interaction would demonstrate that lexical-semantic influences can supersede phonological ones regardless of the phonological environment, the latter would demonstrate that the combined effect of semantic and phonological representations will only occur when both simultaneously provide information that dissociates lexical representations in memory. Regardless of the direction of this interaction, we anticipate that it will be robust to concurrent articulation irrespective of the modality in which information is presented, as there are both visual and auditory routes to semantic representation (e.g., Plaut & Shallice, 1993).
We begin the examination of the interaction of phonological similarity and word concreteness using visual presentation of materials without any concurrent articulation. In addition to the interaction of concreteness and phonological overlap, we also anticipated observing main effects of phonological overlap and word concreteness individually, as both of these effects have been observed under comparable stimulus presentation conditions.
Participants (N = 21, 14 women, 7 men) participated in this study for course credit in an introductory psychology class. Mean age was 19.9 years (SD 2.9).
Ten lists of five words were generated for each phonological overlap and concreteness condition and are listed in the Appendix. Table 1 contains the descriptive statistics for each of the variables over which stimuli differed or were matched. Phonological overlap in this study was defined by items sharing a common rhyme unit (e.g., the æt sound of the word cat). We used this definition of phonological overlap relative to common onset and vowel (e.g., cat, cab, can) and similar offset consonants (e.g., tap, top, tip) as overlap in the rhyme unit more consistently produces the classic PSE, in which the other two manipulations do not (see Gupta, Lipinski, & Aktunc, 2005 for a review).
Given that concrete and abstract words differ on many psycholinguistic dimensions other than concreteness (Reilly & Kean, 2007), words were selected on the basis of semantic, phonological, orthographic, and lexical criteria. Concrete words were mostly nouns, although some of the words could be used as nouns or verbs; abstract words were a mixture of nouns, verbs, and adjectives. Although this mixture of word class between the two lists presents a potential confound in interpreting results, it would have been impossible to have a factorial manipulation of phonological overlap and concreteness while controlling for word class in addition to the other psycholinguistic variables across which stimuli were matched (see Table 1). This being said, although it is very likely that nouns, verbs, and adjectives are remembered differently, they are all content words, thus at a minimum, we avoided the differences that have been observed for function and content words in verbal working memory tasks (e.g., Bourassa & Besner, 1994).
In order to ensure that our concreteness manipulation was robust, we had a separate group of participants (N = 20, 12 women, 8 men; mean age 19.6 years, SD = 1.5) from those in the experiments below rate the overall concreteness of the words on a scale from 1 (abstract) to 7 (concrete), using the same procedure as Walker and Hulme (1999). A 2 (concreteness) × 2 (phonological overlap) repeated measures analysis of variance (ANOVA) revealed a significant main effect of concreteness, F(1, 50) = 388, p < .001, a main effect of phonological overlap, F(1, 49) = 4.65, p < .01, but no interaction between the two. Concrete words were rated a significantly more concrete (M = 6.05, SD = 0.76) than abstract words (M = 2.95, SD = 1.02). Unfortunately, we were not able to perfectly match the overlapping and nonoverlapping lists for concreteness, in that words in overlapping lists were rated as slightly less concrete (M = 4.35, SD = 0.94) than words in the nonoverlapping lists (M = 4.65, SD = 0.86). We did not view this difference in mean concreteness across overlapping lists as problematic for two reasons. First, we anticipated that the slight difference in concreteness across overlapping lists might only exacerbate the PSE while leaving the critical interaction intact. Second, no Phonological Overlap × Concreteness interaction was observed for the concreteness ratings; hence, any interaction effects should not be due to differences in the stimuli being used.
As shown in Table 1, we were successful at matching stimuli for overall length, number of syllables, and summed positional and biphone probability (Vitevitch & Luce, 2004). Given that many of the stimuli used in this study were not found in some of the smaller, older corpora (e.g., Kucera & Francis, 1967; Thorndike & Lorge, 1944), we used a larger and more recent corpus to get estimates of written frequency (Burgess & Livesay, 1998). The same ANOVA as above was conducted on the log frequency and revealed a main effect of phonological overlap F(1, 49) = 8.37, p < .01, but no main effect of concreteness and no interaction. In this case, items in the overlapping lists were slightly more frequent (M = 3.82, SD = 0.62) relative to nonoverlapping lists (M = 3.62, SD = 0.65). As before, we did not consider these differences to be problematic as there was no interaction across our manipulations in the frequency with which words occurred.
Each experiment participant’s working memory span was assessed in a pretest composed of an immediate serial recall task with visual presentation. Pretest words were not used in the actual experiment and did not repeat during the pretesting procedure. Words were presented at a rate of one every 1.5 s.
Participants were instructed to read each word they saw on the screen as it appeared. After presentation of a list finished, a yellow question mark appeared. This served as the participants’ cue to recall the items they had just seen in the order in which they were presented. Participants were instructed to say the word “blank” for any item they could not remember. List lengths started at two words, increasing by a word every two trials until a list length of seven words was achieved. Participants continued performing the task until they no longer correctly recalled either of the two lists in correct serial order. A participant’s span was defined as the last list length in which they had correctly recalled one of the lists in correct serial order.
Participants were run at their predetermined span in this experiment. Participants with a span greater than five were run with lists composed of five words. Stimulus presentation was the same as pretesting. Presentation of the lists was randomized across both phonological overlap and concreteness conditions, and two different list orders were used across participants.
Although attempts were made to encourage people to do their best, there were times when participants were clearly not engaged in the task. For instance, despite being tested at their predetermined span, some participants indicated that they would only try to remember the first or last few items in any given list. As such, we adopted a performance criterion in which participants must have recalled at least 75% of the items in a list regardless of serial order to have their data included in analyses. In all instances below, this criterion reduced error variance while leaving the overall pattern of the data intact.
In order to determine the extent to which phonological similarity and concreteness were affecting order and item memory, we adopted a two-step approach to scoring the data. In the first step, participant performance was scored using strict serial scoring criteria, in which an item was scored as correct only if it occurred in the correct serial position. Such scoring, however, combines item and ordering errors and is thus incapable of demonstrating the extent to which the manipulations in the present study were affecting each. As such, errors reflected in strict serial scoring were characterized as item ordering and omission errors. Item ordering errors reflect a situation in which individuals recall an item in the wrong serial position and have been referred to as substitution and/or exchange errors by language production researchers (Shattuck-Hufnagel, 1979). Item omissions, on the other hand, reflect an instance in which an individual failed to recall an item and thus reflect a pure item error. For instance, if a person was asked to recall cat–dog–mouse and recalled it as mouse–dog–blank, the data would have been scored as an item omission for the first item (cat) and as a one item ordering error for the third item (mouse).
As a final analysis, we examined the extent to which the interaction of phonological similarity and concreteness varied as a function of list position. Although participants were run at different list lengths, we included this analysis for three reasons. First, it allows for comparison with other memory research (e.g., Allen & Hulme, 2006; Jones et al., 2006; 2004; Romani et al., 2008; Walker & Hulme, 1999). Second, it allows us to examine the extent to which the interaction of phonological similarity and concreteness may driven by certain list positions. Third, it allows us to examine the extent to which the survival or removal of the PSE under concurrent articulation may be limited to certain list positions (e.g., recency; Jones et al., 2004). In order to compare patterns across experiments and to maximize the number of serial positions across which any patterns might emerge, we chose to analyze only participants who had a predetermined word span of five items.
Prior to analysis, one participant’s data were removed due to item accuracy less than 75%. Mean memory span determined by pretesting was 4.6 (SD = 0.86).
Given that participants were run at different list lengths, participant performance was initially analyzed by averaging their recall performance across all list positions. In order to ensure that participant performance did not vary as a function of the randomized list condition to which they were assigned, we conducted a 2 (list condition) × 2 (phonological overlap) × 2 (concreteness) mixeddesign ANOVA. List condition did not interact with either of our main factors of interest.
Examination of Figure 1, which contains graphs of the mean error rate across both phonological overlap and concreteness manipulations, reveals the expected effects of each of these factors in isolation and an interaction. Error bars in this and all subsequent figures represent the 95% confidence intervals using pooled estimates of standard errors for each error type (see Masson & Loftus, 2003). The differences reported in this and subsequent follow-up tests will be between error rates on overlapping and nonoverlapping lists, thus positive values will indicate poorer recall of overlapping lists. In this particular experiment, this is precisely what was observed across all concreteness conditions. Results of a 2 (phonological overlap) × 2 (concreteness) repeated measures ANOVA using the strict serial scoring criteria showed significant main effects of phonological overlap, F(1, 18) = 41.64, MSE = 0.29, p < .01 and concreteness, F(1, 18) = 20.33, MSE = 0.11, p < .001 as well as an interaction between the two, F(1, 18) = 4.73, MSE = 0.03, p < .05. Examination of Figure 1 shows that the main effect of phonological overlap is understood by the poorer recall of overlapping lists relative to nonoverlapping lists. The main effect of concreteness is explained by poorer recall of abstract lists relative to concrete lists. Finally, planned comparisons showed that the magnitude of the PSE was larger for concrete lists, t(19) = 5.25, p < .001; μD = 0.16, SD = 0.13, relative to abstract lists, t(19) = 4.35, p < .001; μD = 0.10, SD = 0.09.
In order to determine whether the locus of the effects above were in item or order memory, people’s errors were broken down into item ordering and omission errors (Figure 1). An analysis of item ordering errors revealed a main effect of phonological overlap, F(1, 18) = 45.60, MSE = 0.18, p < .001, and a main effect of concreteness, F(1, 18) = 9.85, MSE = 0.02, p < .001, but no interaction between these variables, F(1, 18) = 0.18, MSE = 0.0001, p > .65. The direction of these effects was commensurate with those above. Omissions showed a main effect of concreteness, F(1, 18) = 13.38, MSE = 0.01, p < .01, and an interaction between phonological overlap and concreteness, F(1, 18) = 10.73, MSE = 0.02, p < .01, but no main effect of phonological overlap, F(1.18) = 3.74, MSE = 0.01, p > .05. Follow-up t tests for the interaction revealed that although there was a PSE on omissions for concrete lists, t(19) = 2.84, p < .02; μD = 0.08, SD = 0.05, no such effect was observed for abstract lists, t(19) = −0.58, p > .05; μD = −0.01, SD = 0.05. This error analysis thus points to the fact that the interaction between phonological overlap and concreteness is primarily driven by differences in item and not order errors.
Figure 2 contains the mean error proportion at each list position for each of level of concreteness and phonological overlap for the 9 participants in this experiment who had a word span of five. Examination of this figure reveals that the magnitude of the interaction between phonological overlap and concreteness varied as a function of list position. This was confirmed in a 2 (concreteness) × 2 (phonological overlap) × 5 (list position) ANOVA, the results of which are summarized in Table 2. Results from this analysis showed main effects of phonological overlap and list position, two-way interactions between Concreteness × Phonological Overlap and Phonological Overlap × List Position, as well as a significant three-way interaction among all of the factors.
Given that the primary emphasis of this analysis is on the three-way interaction, follow-up tests were conducted accordingly. Table 3 contains a summary of the mean difference in overlapping and nonoverlapping lists for concrete and abstract word lists at each list position. As before, positive values indicate that error rates were higher for overlapping lists relative to nonoverlapping lists. Examination of Figure 2 and Table 3 reveals that a significant effect of phonological overlap was observed across all serial positions except Position 2 for concrete lists, whereas for abstract lists, a significant effect of phonological overlap was only observed at Position 5. This latter result is consistent with the findings in Jones et al. (2004), although it should be noted that there was a trend toward an effect of phonological overlap at list Positions 1 and 2 that likely failed to reach significance due to the small number of participants included in this analysis.
The results of this study replicated previous research in showing main effects of phonological similarity and word concreteness in the expected directions. It is, however, more important that there was an interaction between these two variables. The data point to the second of the two potential forms of this interaction discussed in the introduction, namely, that the magnitude of the PSE was enhanced for concrete lists relative to abstract lists. The mechanism by which concreteness was interacting with phonological similarity was revealed by looking at the pattern of item ordering and omission errors. Whereas phonological similarity was clearly having its effect on order memory, concreteness affected people’s memory for the items independent of order. Finally, analyses of the serial position data showed that differences in error rates due to phonological similarity were more consistently observed for concrete lists relative to abstract lists.
This pattern in the data demonstrates two theoretically important results. First, in corroboration of past research findings, phonological and lexical-semantic effects do seem to have differential effects of serial order and item memory, respectively. Critically, results show that these levels of representation are unquestionably interacting. It is clear from the pattern here that semantic information alone is not sufficient to dissociate lexical representations when they share similar phonological features. Instead, the interacting effects of semantic and phonological representations only appear when items do not have phonological overlap. This suggests that input from semantic representations may provide an additional means of maintaining and dissociating lexical representations in memory.
One of the potential limitations of this study is that we used a stimulus timing (one word per 1.5 s) that is not entirely typical of immediate serial recall tasks used by others. This choice was designed to maximize the likelihood that participants would access lexical-semantic representations. It is, however, possible that our results may somehow be driven by this relatively slow rate of presentation. Although we do not report the results here, the same experiment as above was run at a more typical rate of one word per second, replicating all of the results from above. As such, all the experiments below use the same presentation rate as Experiment 1.
Having established that phonological overlap and word concreteness do interact when items are presented visually and that this interaction is robust to the rate at which stimuli were presented, in the following experiment we examine whether this interaction is robust to the deleterious effects of concurrent articulation.
One of the classic findings in verbal working memory research is that the effects of phonological similarity are abolished under conditions of concurrent articulation when information is presented visually but not auditorily (Levy, 1971). Such results have been interpreted to indicate that information in working memory is stored in a phonological form and that visual information must be recoded into this form via articulatory processes (Murray, 1967; Sperling, 1967). Whether this interpretation is correct or not (see below), it is safe to conclude that concurrent articulation (i.e., articulatory suppression) disrupts articulatory rehearsal as a means of refreshing representations in verbal working memory (Baddeley, 1986).
Both behavioral research and computational modeling efforts suggest that articulation is not the only route by which phonological representations can be accessed (see Besner, 1987). For instance, concurrent articulation does not affect judgments of phonological acceptability in sentence comprehension (Coltheart, Avons, & Trollope, 1990). Similarly, articulatory suppression does not prevent phonological recoding for lexical access (Besner, Davies, & Daniels, 1981). Computationally, these results can be understood within a well-known connectionist model of word reading (Harm & Seidenberg, 1999; Plaut, McClelland, Seidenberg, & Patterson, 1996; Seidenberg & McClelland, 1989), in which there are two routes from which the written form of a word (i.e., a word’s orthography) can access phonological representations: directly from the visual form or indirectly via the orthography–semantics–phonology pathway. Note that neither route from orthography to phonology requires articulation.
If phonological information can be activated without articulation then the extent to which concurrent articulation impairs phonological effects should vary as a function of the degree to which the material used relies on articulatory rehearsal to be maintained. Previous research that has shown that the PSE is abolished under concurrent articulation for visually presented items has had material that is likely very reliant on such rehearsal. The first research with these sorts of manipulations had letters as stimuli (Levy, 1971, Experiment 1; Murray, 1968), which do not have a rich semantic representation. Those studies with words have generally not controlled for the relative strength of the semantic representation of the words being used (e.g., Coltheart, 1993). One exception is work by Levy (1971, Experiment 2), who manipulated the semantic similarity of visually presented list material under concurrent articulation, and found very little effect for the semantic manipulation. However, Levy’s (1971) methodology may not have been optimal for observing lexical-semantic effects for several reasons. First, Levy (1971) used list lengths of 10 items, which is well beyond a typical individual’s span. Second, use of probed recall and recognition may not have been sensitive enough to observe robust effects of semantic similarity. Finally, although lists were semantically similar or distinct, it is difficult to determine the extent to which items may have been concrete or abstract. As a result, semantic effects were very small and were only observed in the last two serial positions. Thus, much of the research above likely minimized the use of semantic representations, which may serve as an additional means of maintenance in verbal working memory.
The present experiment is an important test for the interaction of lexical-semantic and phonological representations in working memory. As noted above, one of the major differences between abstract and concrete words is that the representation of the abstract words is highly reliant on the linguistic or associative context in which it appears. Although concrete words may not uniformly have a more robust semantic representation than do abstract words, such should be the case in random word lists, given that concrete words have access to sensory-motor properties that abstract words lack. Thus, abstract words may be more reliant on articulatory rehearsal mechanisms than are concrete words, whose phonological representations may be better supported by the semantic route described above. As such, we predict that concurrent articulation will abolish the PSE for abstract but not concrete word lists.
Participants (N = 26, 20 women, 6 men) participated in this study for course credit in an introductory psychology class. Mean age was 18.6 years (SD = 0.9).
Materials were the same as those used in Experiment 1.
The procedure was the same as that used in Experiment 1, with two exceptions. First, to ensure comparable recall performance with the previous experiment, participants were run at one item less than their predetermined span. Pilot testing revealed unacceptably low performance when participants were tested at their span, with many participants only trying to remember the first or last few items. Second, participants engaged in concurrent articulation throughout the encoding of the stimuli. Three seconds before each trial participants were shown an asterisk on the screen, at which point they were instructed to start saying the word the repeatedly and throughout presentation of the stimuli. After presentation of the stimuli, participants again saw a yellow question mark. Participants were instructed to stop saying the word the once they saw the yellow question mark and to try and recall the items from that trial in the order in which they were presented. As with Experiment 1, participants were instructed to say the word blank in places in which they could not remember a word.
Data scoring and exclusion criteria were the same as Experiment 1.
A total of 6 participants were removed due to performance below the 75% accuracy threshold. The mean span of the remaining participants was 4.75 items (SD = 0.63). Below, we perform the same analyses as in Experiment 1. None of the effects interacted with the two different list order conditions.
Figure 3 shows mean recall error rate across experimental conditions for this experiment, and the overall pattern is unquestionably different from Experiment 1; concurrent articulation clearly abolished any effect of phonological overlap for abstract lists, but not for concrete lists. A 2 (phonological overlap) × 2 (concreteness) ANOVA on strict-serial scoring revealed a main effect of concreteness, F(1, 18) = 37.47, MSE = 0.15, p < .001, and a Phonological Overlap × Concreteness interaction, F(1, 18) = 7.22, MSE = 0.05, p < .02, but no main effect of phonological overlap, F(1, 18) = 2.01, MSE = 0.02, p < .17. Examination of Figure 3 reveals that concrete words were again easier to recall than abstract ones. Although the lack of a significant main effect for phonological overlap may suggest that the PSE was abolished, the significant interaction suggests otherwise. Simple effects tests on the interaction revealed a significant PSE for concrete lists, t(19) = 3.45, p < .01; μD = 0.08, SD = 0.11, but not abstract lists, t(19) = 0.48, p > .6; μD = −0.02, SD = 0.16. As before, nonoverlapping lists were easier to recall than overlapping ones, but in this case, only for the concrete lists.
Errors from above were coded as either item ordering or item omission errors. Figure 3 contains the mean proportion of each of these types of errors, and results demonstrate that the interaction above was primarily driven by item ordering and not by omission errors. An analysis of item ordering errors revealed only a main effect of phonological overlap, F(1, 18) = 17.63, MSE = 0.05, p < .001, with the main effect of concreteness, F(1, 18) = 0.26, MSE = 0.0002, p > .60, and the interaction, F(1, 18) = 1.54, MSE = 0.006, p > .20, failing to reach significance. Planned comparisons revealed that there were differences between concrete and abstract word lists. Whereas a significant effect of phonological overlap was observed for concrete lists, t(19) = 4.44, p < .001; μD = 0.07, SD = 0.07, no such effect was observed for abstract lists, t(19)=1.39, p > .05; μD = 0.03, SD = 0.10.
Omission errors, on the other hand, revealed a very different pattern of results (see Figure 3). Analyses revealed main effect of concreteness, F(1, 18) = 28.15, MSE = 0.08, p < .001, and a Phonological Overlap × Concreteness interaction, F(1, 18) = 6.30, MSE = 0.02, p < .05, but no main effect of concreteness, F(1, 18) = 0.64, MSE = 0.002, p > .4. Examination of Figure 3 and planned comparisons show no PSE for concrete lists, t(19) = 1.0, μD = 0.02, SD = 0.07, and a tendency toward a reversal of the PSE for abstract lists, t(19) = −1.94, μD = −0.04, SD = 0.09. Unlike the previous study, the combination of results across ordering errors and omissions indicates that the locus of the effect of word concreteness on the magnitude of the PSE was on memory for the order in which items appeared, not simply on memory for the items themselves.
Finally, we examined the extent to which the interaction between phonological overlap and concreteness varied as a function of list position for participants who had a word span of five items (N = 12). Figure 4 contains the proportion of strict-serial scoring errors for each phonological overlap and concreteness condition at each of four list positions. Examination of this figure reveals that although the PSE was completely abolished for abstract lists at all list positions, it remained for concrete lists at Positions 2 and 3. This pattern was confirmed in a 2 (concreteness) × 2 (phonological overlap) × 4 (list position) ANOVA summarized in Table 2, which showed main effects of concreteness, phonological overlap and list position, a two-way interaction between phonological overlap and list position, and three-way interaction among all the factors. Follow-up t tests (Table 3) also confirm the patterns in Figure 4, showing effects of phonological overlap at list Positions 2 and 3 for concrete lists, with no such effects observed for any list position for abstract lists.
The results of this study replicate previous findings by researchers who have examined the effect of concurrent articulation on the PSE when information is presented visually and also demonstrate a critical role for lexical-semantic representations in serial recall performance. Past research has shown that concurrent articulation abolishes the PSE when items are presented visually (e.g., Levy, 1971), and this is what we observed for abstract lists across all list positions. It is, however, interesting to note that the PSE remained for concrete lists, most notably in the middle list positions. In this case, it appears that individuals are able to use the semantic content of concrete words to help maintain the order of words in working memory. Another way of reframing this in terms of language production is that concrete words provide a more coherent message from which lexical selection and phonological ordering can be generated. In this case, the place in which people benefit most from this message is in lists in which little phonological confusion exists (i.e., the nonoverlapping lists). This study thus demonstrates that it is neither semantic nor phonological representations alone that drive people’s superior performance for nonoverlapping, concrete lists, but the combination of these two factors.
What is particularly novel in these results is the demonstration that the maintenance of stimuli containing sensory-motor properties in semantics can be maintained when articulatory rehearsal is disrupted. Previous research has lacked the sorts of manipulations used here and was thus insensitive to observing these effects. As predicted from language-based accounts of verbal working memory maintenance, the finding that the PSE can survive concurrent articulation under visual presentation for concrete lists demonstrates that there are multiple routes to the phonological form of a word in working memory. Furthermore, the present study also demonstrates a scenario in which individuals use lexical-semantic representations to influence serial ordering processes as predicted by interactive activation accounts of production (e.g., Foygel & Dell, 2000). We return to these points in the General Discussion section.
The first two experiments in this study demonstrated that phonological and lexical-semantic factors interact when information is presented visually. What remains to be seen is whether these manipulations are robust to the modality in which the stimuli are presented.
It is well known that individuals use some form of phonological or auditory perceptual code to maintain information in working memory (Conrad & Hull, 1964; Wickelgren, 1965), and this appears to occur despite the modality in which information is presented. Still, individuals are generally superior at recalling information when it is presented auditorily rather than visually (Craik, 1969), and the PSE survives concurrent articulation under auditory presentation (Levy, 1971). These results have been interpreted by some to indicate that auditorily presented material has direct access to stores specifically dedicated to maintaining phonological representations (Baddeley, 1986). Alternatively, differences between visual and auditory presentation may simply reflect differential learning as individual have substantially more experience accessing the phonological form of a word via hearing relative to reading (Acheson & MacDonald, 2009b). What remains unclear, however, is whether individuals will continue to use lexical-semantic representations when the modality in which the stimuli are presented encourages them to use a phonological or acoustic code. To test this hypothesis, in the present experiment we use the same methods as in Experiment 1, but with auditory instead of visual presentation of list items.
Participants (N = 23, 12 women, 11 men) participated in this study for course credit in an introductory psychology class. Their mean age was 18.8 years (SD = 0.77).
Materials were the same as those used in the previous experiments. The items were digitally recorded by a male speaker at a sampling rate of 22,050 kHz and were then normed for loudness. Each word totaled 1 s in duration and began at the beginning of the 1 s window.
Pretesting to determine an individual’s memory span was the same as in Experiments 1 and 2, but presentation was auditory instead of visual.
Experimental testing was nearly identical to Experiments 1 and 2, except presentation of the stimuli was auditory. Participants heard a 1 s tone 2 s prior to the presentation of the stimuli. As with Experiment 1, each word was presented at a rate of one every 1.5 s. After presentation of the list, participants heard another tone in concert with seeing a yellow question mark on the screen. As before, they were instructed to recall the list in the order in which it was presented, saying the word blank for items that they could not recall. Participants were run at their predetermined span.
Data scoring and exclusion criteria were the same as Experiments 1 and 2.
Prior to data analysis, 3 participants were removed due to item accuracy less than 75%. The mean span for those participants included in this analysis was 5.3 (SD = 0.58). In order to determine whether phonological similarity is interacting with word concreteness when stimuli were presented auditorily, we performed the same analyses as in Experiments 1 and 2. As before, none of the effects interacted with list presentation order.
Figure 5 contains the mean recall error rate as a function of the experimental manipulations across both scoring types and reveals a similar pattern to that observed in Experiment 1. Analyses revealed a main effect of phonological overlap, F(1, 18) = 85.42, MSE = 0.28, p < .001, a main effect of concreteness, F(1, 18) = 42.21, MSE = 0.11, p < .001, and an interaction between the two, F(1, 18) = 4.42, MSE = 0.03, p < .05. As with Experiment 1, phonologically nonoverlapping lists were easier to recall than were overlapping lists, and concrete lists easier to recall than were abstract ones. The interaction between Phonological Overlap × Concreteness also parallels that which was observed in Experiment 1. Planned comparisons revealed that the magnitude of the PSE was larger for concrete lists, t(19) = 7.90, p < .001; μD = 0.18, SD = 0.10, relative to abstract lists, t(19) = 3.97, p < .01; μD = 0.10, SD = 0.11. In both instances, nonoverlapping lists were easier to recall than overlapping ones.
Similar to Experiment 1, the analysis on the locus of the interaction of phonological overlap and concreteness reveals that the interaction is primarily driven by item and not order memory (see Figure 5). Results of the analysis on item ordering errors revealed only a main effect of phonological overlap, F(1, 18) = 45.96, MSE = 0.11, p < .001, with the main effect of concreteness, F(1, 18) < 1, MSE = 0, p > .9, and the interaction, F(1, 18) = 1.30, MSE = 0.004, p > .25, failing to reach significance. Results of the analysis on omissions, however, showed main effects of phonological overlap, F(1, 18) = 29.48, MSE = 0.05, p < .001, and concreteness, F(1, 18) = 27.01, MSE = 0.06, p < .001, although the interaction failed to reach significance, F(1, 18) = 3.30, MSE = 0.01, p = .086. Examination of Figure 5 shows that for omission errors, the PSE appears to remain for concrete lists but not for abstract lists. Planned comparisons confirmed this result as overlapping lists produced more omission errors for concrete lists, t(19) = 6.68, μD = 0.08, SD = 0.05, but not abstract lists, t(19) = 1.39, p > .05, μD = 0.03, SD = 0.09. As with Experiment 1, the results of the error analysis demonstrate that the locus of the interaction of concreteness and phonological similarity is in superior item memory for concrete, nonoverlapping lists.
Figure 6 contains the mean error rates for each concreteness and phonological overlap condition at each list position for individuals with a span of five (N = 11). Although examination of the figure reveals differences in the magnitude of the PSE between abstract and concrete lists at different serial positions, these differences were not enough to lead to a significant three-way interaction between these factors (see Table 2). Results of a 2 (concreteness) × 2 (phonological overlap) × 5 (list position) ANOVA revealed main effects of concreteness, phonological overlap, and list position, as well as a significant Concreteness × List Position interaction. All other interactions failed to reach significance. Despite the nonsignificant interaction, we proceeded with paired t tests to examine whether the magnitude of the PSE varied between abstract and concrete lists at different list position (see Table 3). Results of this analysis revealed that whereas concrete lists showed a significant PSE at each list position, abstract lists only showed this effect at Position 1. Given the lack of a significant three-way interaction, however, these differences should be interpreted with caution.
The results of Experiment 3 generally replicated the findings of Experiment 1, but with auditory presentation. The only difference between the two is that the three-way interaction between phonological overlap, concreteness, and list position was significant in Experiment 1 but not in Experiment 3. These differences were likely driven by the greater variability in performance in Experiment 3 relative to Experiment 1, thus we may have had insufficient power. Despite these differences, the overall pattern of errors showed the same interaction of phonological overlap and concreteness as Experiment 1, with a greater PSE for concrete lists relative to abstract lists that was driven primarily by superior item memory for concrete, nonoverlapping lists. The interaction of phonological similarity and word concreteness thus appears to be robust across presentation modality. In addition to again demonstrating that multiple levels of linguistic representation are likely maintained in verbal working memory, the present results also point to the automaticity with which the meaning of a word is accessed from its phonological form. In this case, even when the presentation modality presumably encouraged individuals to use a phonological code to maintain information, the interaction of the PSE with concreteness shows that people continued to access the lexical-semantic properties of the words within the lists.
Although we do not anticipate that concurrent articulation will necessarily modify this present finding, for the purposes of completeness, the final experiment examines whether concurrent articulation modulates both the individual and interactive effects of phonological similarity and concreteness under auditory presentation.
Concurrent articulation abolishes the PSE when items are presented visually but not auditorily (Levy, 1971). Experiment 2, however, demonstrated an interesting pattern of results in which the PSE was abolished for abstract lists but not for concrete ones under concurrent articulation. In this final experiment, we examine the extent to which this pattern of results will differ when information is presented auditorily. Previous research with auditory presentation has demonstrated that concreteness effects are robust to manipulations of concurrent articulation (Romani et al., 2008). As such, we anticipate that concurrent articulation will not affect the advantage for concrete lists relative to abstract lists, nor will it abolish the PSE across these list types, thus replicating previous findings of studies using auditory presentation of list materials under concurrent articulation.
Participants (N = 26, 20 women, 6 men) participated for credit in an introductory psychology course at the University of Wisconsin—Madison.
Materials were the same as those used in Experiment 3.
The procedure was the same as that of Experiment 2, except that items were presented auditorily. Three seconds before the list was presented, participants heard a tone and saw an asterisk on the screen, which was their cue to begin saying the word the repeatedly. After presentation of the list, participants saw a question mark paired with another tone. Upon seeing the question mark, participants were instructed to stop saying the and to try to recall the list in the order in which it was presented.
As with Experiment 2, participants were run at one item less than their predetermined span.
Data scoring and exclusion criteria were the same as those used the previous experiments.
Prior to analyzing the data, 6 participants were removed due to poor performance. For those participants analyzed below, mean span was 5.4 items (SD = 0.62).
As with the previous studies, none of the effects interacted with list presentation. Figure 7 contains a graph of mean recall error rate across phonological overlap and concreteness conditions for both scoring types, and the qualitative patterns are very similar to Experiments 1 and 3. There was a main effect of phonological overlap, F(1, 18) = 22.44, MSE = 0.18, p < .001, and a main effect of concreteness, F(1, 18) = 17.29, MSE = 0.23, p < .01, although in this case the interaction failed to reach significance, F(1, 18) = 2.67, MSE = 0.02, p = .12. As before, nonoverlapping lists were easier to recall than were overlapping ones, and concrete lists easier to recall than were abstract ones. Planned comparisons showed that there was a numerically larger PSE for concrete lists, t(19) = 4.22, p < .001; μD = 0.14, SD = 0.15, relative to abstract lists, t(19) = 2.25, p < .05; μD = 0.08, SD = 0.15. As noted by the lack of a significant interaction, however, these differences were only numerical, although in the same direction as all of the previous studies.
Examination of Figure 7 shows a pattern of results similar to those of Experiments 1 and 3, in which the interaction of phonological overlap and concreteness is primarily driven by differences in item and not order memory. Analysis of item ordering errors revealed only a main effect of phonological overlap, F(1, 18) = 25.80, MSE = 0.12, p < .001, with the main effects of concreteness, F(1, 18) = 2.14, MSE = 0.008, p > .15, and the interaction, F(1.18) < 1, MSE < 0.0001, p > .75, failing to reach significance. Analysis of omission errors, on the other hand, revealed only a main effect of concreteness, F(1, 18) = 14.20, MSE = 0.04, p < .001, although the main effect of phonological overlap, F(1, 18) = 4.31, MSE = 0.02, p = .053, and the interaction between these two variables, F(1, 18) = 3.68, MSE = 0.004, p = .07, approached significance. Examination of Figure 7 and planned comparisons showed that although there was a PSE on omission errors for concrete lists, t(19) = 2.43, p < .03; μD = 0.05, SD = 0.09, no such effect was observed for abstract lists, t(19) = 1.10, p > .05; μD = 0.02, SD = 0.08. The combination of these two analyses thus confirms that the locus of the interaction of phonological overlap and concreteness in this study is driven by the superior item memory for concrete, nonoverlapping lists.
Figure 8 contains the mean error rate for each concreteness and phonological overlap condition at each list position for participants with a span of five items (N = 14) and reveals a different pattern than was observed in the previous experiments, most notably, Experiment 2. The results of a 2 (concreteness) × 2 (phonological overlap) × 4 (list position) ANOVA (summarized in Table 2) revealed main effects of Concreteness, phonological overlap, and list position, a significant two-way interaction between phonological overlap and list position, and a significant three-way interaction between the variables. Examination of Figure 8 and the results of paired t tests (Table 3) revealed that the three-way interaction came from the fact that a PSE was observed for list Positions 2, 3, and 4 for concrete lists, but only Positions 1 and 4 for abstract lists. The serial position analyses thus coincide with the overall error rates above in showing that the magnitude of the PSE was larger for concrete lists relative to abstract word lists.
The results of Experiment 4 replicated classic findings in the working memory literature in demonstrating that concurrent articulation does not abolish the PSE when items are presented auditorily. Although concreteness continued to have an effect on participant performance, it did not interact as strongly with phonological overlap as in the previous experiments. The overall patterns in the data were similar to those in Experiments 1 and 3 in that the magnitude of the PSE was numerically larger for concrete lists relative to abstract lists, but the interaction between these factors failed to reach significance. Also, similar to Experiments 1 and 3, the interactive effect of concreteness on the PSE was explained by the superior item memory for concrete lists relative to abstract lists. Although this trend failed to reach statistical significance in the present study, the qualitative pattern was the same as those seen in as Experiments 1 and 3. In sum, this study again demonstrates that lexical-semantic and phonological levels of representation interact in verbal working memory and that these effects are robust to the modality in which information is presented as well as to concurrent articulation.
Results of the serial position analysis both replicate and extend previous research that has shown that the survival of the PSE with auditory presentation under concurrent articulation is primarily driven by the recency portion of the serial position curve (Jones et al., 2004). This pattern is precisely what was observed for abstract word lists in which no PSE was observed for visual presentation under concurrent articulation (Experiment 2), whereas in the present study the PSE was primarily (although not entirely) driven by the recency portion of the serial position curve. Concrete word lists, on the other hand, did not show this same pattern. Under both visual and auditory presentation with concurrent articulation, the PSE survived concurrent articulation, and this effect was driven by middle and terminal list positions in both experiments. These results thus point to the fact that more than just phonological (or acoustic) coding is responsible for the interaction of phonological similarity and presentation modality under concurrent articulation. We return to these points in the General Discussion that follows.
The present study examined the interaction of word concreteness (a lexical-semantic factor) and phonological overlap on the recall of word lists. Within each experiment, word concreteness and phonological overlap were factorially manipulated, and across experiments, presentation modality (visual or auditory) and concurrent articulation (with and without) were manipulated. A consistent interaction between concreteness and phonological overlap was observed across studies in which the magnitude of the PSE was larger for concrete lists relative to abstract lists. In the following section, we discuss the effects of our phonological overlap and concreteness manipulations individually, emphasizing whether each primarily affected order or item memory. We then discuss the interaction effects we observed and theoretical accounts that might address our results. We conclude with an evaluation of these accounts along with suggestions for future research.
Phonological representations have long been known to influence serial-ordering processes in verbal working memory tasks (Conrad & Hull, 1964; Wickelgren, 1965), and the present study is consistent with these results. Main effects of phonological overlap were observed when data were scored with serial scoring criteria, and error analyses consistently revealed equivalent main effects of phonological overlap for item ordering errors across both levels of the concreteness manipulation, with one very important exception (see below). Although results from the present study again confirm that phonological representations are critically involved in serial ordering processes, they also speak to a somewhat less appreciated role for phonological representations in item memory.
Part of the reason that emphasis has been placed on the role of phonological representations in order memory is that the classic PSE appears when accuracy is scored according to some serial ordering criterion but not when data are scored for item accuracy alone. Some studies have even showed that under appropriate conditions (i.e., rhyming items in open word lists), item recall can be superior for phonologically overlapping lists relative to nonoverlapping ones (see Gupta et al., 2005 for a thorough review of these findings). Results such as these have been used to argue for the fact that the rhyme unit of a word can serve as a category cue during recall (Fallon et al., 1999; Lian, Karlsen, & Eriksen, 2004). Alternatively, such effects could also be accounted for within an interactive activation framework in language production (e.g., Dell, 1986) by assuming that phonological representations of a rhyme sound will feed back and enhance activation for all lexical entries that contain that rhyme sound.
Regardless of whether one interprets these effects as occurring at recall or during maintenance, beneficial effects of rhyming were observed in the present study, but for abstract lists only. In this case, across all the experiments, no PSE was observed in item omission errors for abstract lists, but a PSE remained for concrete lists. This result highlights an important point about the interactive nature of the language production system, as there are circumstances in which phonological representations will directly influence lexical ones. The present study indicates that these effects are likely to be observed in situations in which semantic input to lexical representation is impoverished.
The observation from this study that concreteness primarily affects item memory is not a new one (Poirier & Saint-Aubin, 1995; Saint-Aubin & Poirier, 2000; Romani et al., 2008; Walker & Hulme, 1999). When errors from the initial strict serial analysis were separated into ordering and omission errors, the main effect of concreteness could primarily be traced to differences in the number item omissions and not item ordering errors. These results thus corroborate past research that suggests that the majority of the effect of semantic manipulations is on the representation of whole lexical representation and is thus consistent with the perspective that semantic representation primarily influences the functional stage of language production (Garrett, 1975). Furthermore, the critical interaction between phonological overlap and concreteness is due to the superior item-memory for concrete, nonoverlapping lists. What is particularly striking about this pattern of results is that with the exception of auditory presentation under concurrent articulation, error rates for phonologically overlapping lists did not differ between abstract and concrete words. Rather, it was only under situations in which both phonological and semantic factors provided cues to distinguish the items that effects of concreteness were observed, hence the larger PSE for concrete lists relative to abstract lists.
Although we have emphasized that the majority of the interaction between phonological overlap and concreteness is explained by superior item memory for concrete lists, there is one very important exception: visually presented lists under concurrent articulation (Experiment 2). This experiment represents the one combination of conditions in which semantic representation seems to be having its effect primarily on order memory, as evidenced by the fact that a PSE was observed for ordering errors but not omission errors for concrete lists. The fact that no PSE was observed for abstract lists serves as a replication of previous studies that have demonstrated that concurrent articulation can abolish the PSE for visually presented lists (Levy, 1971; Murray, 1968). The classic interpretation of this effect is that concurrent articulation prevents visually presented information from being recoded into a phonological form (Baddeley, 1986). However, this account of articulatory suppression no longer seems tenable in light of the present results and previous research that has shown that the phonological form of a word can be accessed under articulatory suppression (Besner, 1987).
If individuals did access the phonological form of the visually presented material then the present results necessarily speak to the fact that articulation is not the only means by which information is maintained in working memory. Within interactive activation accounts of language production, for instance, the maintenance of lexical items is achieved via feedback connections from phonological encoding and/or articulation and feed forward input from semantics (e.g., Dell & O’Seaghdha, 1992; Foygel & Dell, 2000). The differences between abstract and concrete words in this study likely emerged from differences in the extent to which each type of material was able to use semantic input in maintenance. In particular, the semantic representation of abstract words in the present study was likely impoverished relative to concrete words, given the reliance of abstract words on linguistic context or association. As such, maintenance of abstract words was likely more reliant on articulatory rehearsal processes than were concrete words, whose semantic representation is defined via sensory-motor representations. We argue that previous research showing the concurrent articulation abolishes the PSE with visual presentation with material largely devoid of semantic content. Thus, rather than demonstrating that accessing the phonological form of a visually presented word relies on articulation, these studies may simply have been showing that maintenance of some material primarily relies on articulation.
This interpretation suggests two things. The first is that there should be situations under which the PSE can survive for abstract words under visual presentation and concurrent articulation, namely, when the list in which they are embedded provides enough context to activate abstract word semantics. The second and more general point is that given sufficiently robust semantic representation, repeated interaction between lexical-semantic and phonological representations can be used to maintain the phonological form of a word. This maintenance-based account, however, is not the only means of explaining the pattern of results, thus we explore an alternative interpretation presently.
Having established that phonological and lexical-semantic factors do indeed interact in verbal working memory tasks, the logical question to ask is what is the mechanism? We see two major accounts of verbal working memory performance that could accommodate these results: trace redintegration (e.g., Walker & Hulme, 1999) and language-based accounts, including lexical-semantic binding (e.g., Jefferies et al., 2006), multiple codes (Haarman & Usher, 2001; R. C. Martin et al., 1999), and the language production hypothesis (e.g., Acheson & MacDonald, 2009b; N. Martin & Saffran, 1997). Each is discussed in turn.
Trace redintegration accounts (Hulme, Maughan, & Brown, 1991; Schweickert, 1993) posit a clear delineation between short- and long-term representations and have been used to try and account for lexicality (Hulme et al., 1991), written-frequency (Hulme et al., 1997; Roodenrys et al., 1994) and concreteness (Walker & Hulme, 1999) effects on verbal working memory performance. This perspective adopts the assumption offered by the multicomponent model of working memory (Baddeley, 1986; Baddeley & Hitch, 1974) that maintenance relies solely on phonological representations that decays over time. At the time of recall, these decayed representations are subject to a reconstruction (or redintegration) process through an item-based comparison to phonological representations in long-term memory. This account is thus similar to models of language production that posit late-stage error monitoring mechanisms (see Levelt, Roelofs, & Meyer, 1999; Shattuck-Hufnagel, 1979). Although semantic effects have not been previously emphasized, one might assume that the effects of concreteness observed in this study can be accounted for by the fact that concrete words have a better representation in long-term memory than do abstract ones and hence are redintegrated more easily at the time of recall. The fact that the majority of the effect observed in the interaction effect seems to be for item memory and not for order memory would seem to support such an item-based account.
With language-based accounts, in contrast, one does not assume that semantic representation is incorporated only at the time of recall. They instead suggest that such representation is used during maintenance as well. For instance, the semantic-binding hypothesis (e.g., Patterson et al., 1994) would explain the current pattern by suggesting that the stronger semantic representation of concrete words in the present experiment led to better binding of the phonological elements during both maintenance and recall. A very similar account is offered by the multicodes account (R. C. Martin et al., 1999). Here, separate short-term buffers for input and output phonological representations feed into and receive input from shared lexical and semantic representation. Rather than being buffered, the lexical-semantic representations is assumed to be an activated subset of long-term memory (Cameron, Haarmann, Grafman, & Ruchkin, 2005; Romani et al., 2008). In exactly the way outlined above, this account suggests that relative to weak representations, stronger semantics will feed activation to a lexical level that in turn will lead to enhanced activation of buffered phonological representations. Finally, the language-production accounts (Acheson & MacDonald, 2009b; N. Martin & Saffran, 1997) suggest that working memory may be nothing more than the temporary activation of representations within the language production architecture. This account is thus a specific instantiation of a more general theory that states that working memory is emergent from the temporary activation of long-term representations within perception and action systems guided by the focus of attention (Cowan, 1995; Postle, 2006; Ruchkin, Grafman, Cameron, & Berndt, 2003). Within this language production account, the present findings can be explained by the fact that concrete lists offer a more coherent message than do abstract ones by virtue of the ease with which they evoke a mental image. As a result, all levels of production planning that follow this message retrieval will necessarily be improved. Such an account is broadly consistent with research showing that effects of concreteness often arise in the production system as individuals translate from a semantic to a phonological code (Allen & Hulme, 2006; Newton & Barry, 1997). Thus, although the exact mechanism by which lexical-semantic representations is affecting working memory performance differs slightly between these language-based accounts, all agree that such representation is involved in maintenance as well as recall.
One of the major advantages of the language-based models over trace redintegration accounts is that they offer a natural explanation of why lexical-semantic representations primarily have their effect on item memory and not on order memory. Perhaps because of the item-based focus of past working memory research, in trace redintegration accounts, it has simply been assumed that redintegration is occurring at the level of the whole item (Hulme et al., 1991; Schweickert, 1993). In models of production and comprehension, however, the nature of information flow is such that the mapping from semantics to phonology necessarily must go through a lexical (i.e., whole item) representation (Levelt, 1989; Plaut & Kello, 1999). As a result, manipulations of semantics will primarily affect lexical representation before affecting processes that follow (i.e., phonological encoding and articulation in production). Language production accounts would argue that in the absence of grammatical information (e.g., in serial recall tasks), phonological encoding is the stage at which serial ordering errors are being made (see Acheson & MacDonald, 2009b). However, evidence from the phonological similarity of word substitutions and mixed speech errors speaks to the fact that lexical-semantic and phonological levels of linguistic representation can and do interact under appropriate circumstances (Dell & Reich, 1981). Experiment 2 in the present study is one such instance. When concurrent articulation largely blocked articulatory rehearsal, the serial order of items was achieved via interaction between lexical-semantic and phonological representations, hence a PSE was observed for concrete lists. There are thus two critical points that these language-based accounts emphasize that classic perspectives on working memory do not. The first is that sustained interaction among semantic, lexical, and phonological representations drives performance, not simply a comparison to semantic representation at the time of recall. The second is that phonological representations alone are not the sole means by which information is maintained in working memory. Further evidence for this last point is provided by a recent study showing that when participants are asked to focus on the meaning of the words they are remembering, the PSE is abolished (Campoy & Baddeley, 2008). Thus, although it is often convenient to discuss the distinction between item and order memory, the reality is that at the level of the lexicon, these two levels of representation are interacting (see Dell & O’Seaghdha, 1991 for a similar discussion).
In sum, the discriminating factor that distinguishes redintegration from the other accounts is the stage at which semantic effects are taking place and the level of representation over which information is being maintained. Trace redintegration accounts (e.g., Walker & Hulme, 1999) hold that semantic effects occur during late-stage retrieval, whereas the other accounts suggest that semantic representations are maintained along with phonological and other language-based representations. The present results do not clearly distinguish between these accounts, but two other studies are informative about this point. First, the results of Jefferies et al. (2006) favor a semantic-binding account over a strong version of redintegration in that recall of words affected errors on nonwords occurring later in the list. This can only have happened if lexical-semantic binding was occurring over multiple items within the list during maintenance and not during a retrieval process that occurs on an item specific basis (e.g., Schweickert, 1993). Second, results from Romani et al. (2008) also speak against output-only versions of redintegration in that concreteness effects were observed in tasks that require no spoken output: matching-span and order reconstruction.
At stake in this debate is the very nature of how verbal working memory functions and what sorts of representations are maintained, thus future research should be directed at delineating between these accounts. An obvious place to start would be to devise testing situations that determine whether semantic and other long-term, linguistic effects are occurring during recall only, or during maintenance and encoding as well. Another possibility would be to use techniques from neuroscience. For instance, a recent neuroimaging study has shown that in the case of visual working memory, long-term, semantic representations are maintained throughout the delay period of a working memory task (Lewis-Peacock & Postle, 2008). In the verbal domain, another recent study has combined neuroimaging and repetitive transcranial magnetic stimulation to target language production regions as people maintain information in working memory. Results suggested that stimulation of brain regions associated with phonological encoding in production during the delay period of a serial recall task impairs working memory performance (Acheson, Hamidi, Binder, & Postle, 2008). Such techniques could easily be applied to the current question regarding when semantic representations are being used in the service of working memory.
In conclusion, the present study demonstrated that lexical-semantic and phonological factors interact in verbal working memory. In particular, the magnitude of a classic finding in verbal working memory, the PSE, was modulated by whether the word lists being recalled were abstract or concrete. This interaction highlights the fact that a complete account of verbal working memory performance cannot solely emphasize phonological or acoustic representation but must also take into consideration other levels of linguistic representation. Furthermore, although the present study largely confirmed that lexical-semantic and phonological effects influence item (i.e., functional) and order (i.e., positional) representations, respectively, it also shows that there are circumstances under which this clear dissociation does not hold. What remains to be determined is whether these other levels of linguistic representation have their impact primarily at the time of recall or whether they play a more central role in encoding and maintenance processes as well. More generally, understanding the nature of how different linguistic representations interact over time will not only provide important insight into maintenance processes in verbal working memory but also into the organization of language systems in normal, developing, and impaired individuals.
This research was supported by National Institutes of Health Grant MH064498 to Bradley R. Postle, National Institute of Child Health and Human Development Grant R01 HD047425 and the Wisconsin Alumni Research Fund awarded to Maryellen C. MacDonald, and an American Psychological Association Dissertation Award to Daniel J. Acheson. Further support was provided by research fellowships to Daniel J. Acheson from the Cognitive Science Cluster and the Department of Psychology at the University of Wisconsin—Madison. We thank Nelson Cowan, Art Glenberg, and Tim Rogers for helpful suggestions and comments.
|Phonological nonoverlap and abstract|
|Phonological nonoverlap and concrete|
|Phonological overlap and abstract|
|Phonological overlap and concrete|