Auditory, motor, and language tasks were performed over the course of 2–3 days in each patient, including the verb generation and picture naming tasks. The SI gives complementary results from additional auditory tasks. TF analysis of the ECoG used a Gaussian filter-bank and the Hilbert transform, yielding a time-series of analytic amplitude (AA, envelope of the filtered signal) for each frequency band. We focus on γhigh
(averaged over the range 70–160 Hz), since this band shows the most robust tracking of language activations. Significant γhigh
activations were in the range of ~10–100% greater than baseline levels (–), with a sufficient SNR to study single-trial averages with only a 3-trial moving-window average (–). Full TF results from 4–250 Hz are presented for comparison in the SI (Fig. S4
Verb Generation Task
The median RT was 1.25 s for Patient 1 (range: 0.87–3.45 s) and 1.48 s for Patient 2 (range: 0.85–3.1 s). For Patients 3 and 4 (SI), the median RTs were 2.05 s and 1.67 s, respectively. The γhigh results show three broad stages of cortical processing for the verb generation task: speech comprehension of the auditorily-presented noun, semantic processing for selecting an associated verb, and a speech production interval surrounding the verbal response. These broad stages, each of ~0.5–1 s duration, follow each other with only partial overlap (serial stages). Within each of these stages, highly overlapping activation time courses are seen (parallel or incremental processing). The results are organized by the three broad intervals of processing.
Several activation foci are seen during the speech perception interval (~50–600 ms following stimulus onset), including Wernicke’s area (mid-to-posterior STG and MTG, see for abbreviations), the precentral gyrus (sPMv), and regions of the central and parietal operculum just dorsal to the Sylvian fissure. All of the activation results are confirmed in detail during a single-word auditory task (Figs. S5, S6
In Patient 1 (), the activation focus shifts from the pSTG site (dark blue electrode, MNI: −65, −52, 18 mm) at ~100 ms to the pMTG site (turquoise electrode, MNI: −65, −53, 2 mm) at ~300 ms. The sPMv site (orange electrode, MNI: −55, −26, 49 mm) also exhibited significant auditory activations (~75% above baseline, P<0.02) peaking at ~310 ms. One electrode on the central operculum (red electrode, MNI: −65, −19, 9 mm) showed a similar activation profile as sPMv, except with a greater weighting for the response (post-stimulus activation ~40%, post-response activation ~120%). Additional auditory-responsive electrodes without any motor responses were found on the most anterior–inferior part of the SMG (parietal operculum, MNI: −64, −39, 17 mm) and on the mid-STG directly lateral from Heschl’s gyrus (MNI: −67, −26, 1 mm). It is noted (see Discussion on dual ventral streams) that the mid-STG electrode is located over 2 cm anterior to the pSTG focus, and exhibits no significant activations during the semantic or pre-response intervals of picture naming or verb generation.
In Patient 2 (), the early (~100 ms) auditory activations are focused on the mid-STG, with the peak at the mid-STG electrode directly lateral from Heschl’s gyrus (dark blue electrode, MNI: −68, −12, −3 mm). The posterior Sylvian morphology was considerably different in this patient compared to Patient 1, terminating at a relatively anterior level with both ascending and descending terminal rami (Ono et al., 1990
, p. 148). Two auditory responsive electrodes were found around these posterior Sylvian branches, but neither seems to be the typical pSTG site (as seen in Patients 1, 3 and probably 4). The first was on the most anterior–inferior SMG or parietal operculum (turquoise electrode, MNI: −67, −30, 17 mm) and it exhibited a purely auditory response profile with peak (55%, P
<0.02) at ~300 ms. The second was just posterior to the descending terminal ramus (MNI: −70, −44, 11 mm), but its responses were relatively weak (peak ~20%, ~300 ms) and so it is not studied further or shown separately in . It is possible this electrode was near the typical pSTG site located within the descending ramus or another nearby sulcus for this patient. Just superior to this electrode on the TPJ was the important electrode labeled ‘Spt/TPJ’ (red electrode in , MNI: −69, −41, 22 mm). Although its response was stronger during the motor interval, it also gave significant (30%, P
<0.02) auditory activations peaking at ~300 ms. A similar activation profile was obtained from the sPMv site (orange electrode, MNI: −54, −6, 52 mm), and from another precentral electrode (dark red, MNI: −62, −1, 29) that gave a weak (~15%) but significant (P
<0.02) auditory response at ~400 ms. In all electrodes, the peak latency is only a rough indication since a sustained plateau of 100–500 ms duration was the general rule.
Activations during the semantic processing interval (~600 ms post-stimulus to ~200 ms pre-response) are distributed over IFG, MFG, and SMG sites, generally considered higher, association areas of neocortex. The IFG and MFG activations are seen most clearly in Patient 1 () due to more anterior placement of the grid. In Patient 2 (), the more posterior placement allows imaging of posterior SMG sites (yellow electrodes), which exhibit an activity profile similar to the prefrontal pattern. Typical activations in association cortices were 10–40% above baseline during the semantic interval, with the highest peak level at 65% (Patient 2, dark green electrode in MTG). Although these were significant (P<0.02), they are generally weaker than sensory and motor activations. In Patient 1, the pMTG site shows significant activation (10–25%, P<0.02) during the semantic association interval, in contrast to the STG sites that return completely to baseline or below baseline levels (−10% to 10%) following stimulus processing (~800–1200 ms).
Speech production activations begin ~300 ms prior to verbal response in the peri-Rolandic cortices (pre- and post-central gyri) of both patients. In stark contrast to the peri-Rolandic time-courses, STG sites were active in the speech production interval following
response onset (peak at ~150–500 ms post-response). This is consistent with self-stimulatory auditory activations from the patient’s own voice. In both patients, STG and MTG sites were generally less active in the post-response interval than in the post-stimulus interval, consistent with notions that auditory feedback is suppressed during speech production (Houde et al., 2002
; Towle et al., 2008
). However, in Patient 1 the more anterior pSTG site (blue electrode, MNI: −66, −46, 10 mm) showed equal stimulus-related and response-related activation levels (~100%, P
<0.02), consistent with an active role in feedback control of speech production (Indefrey and Levelt, 2004
; Tourville et al., 2008
). This same pSTG site gave the peak auditory activation during a phoneme task (Fig. S5c–d
). In Patient 2, both mid-STG sites were less active during production than perception. However, it was the more anterior STG site (dark blue) and the parietal operculum site (turquoise) that showed the strongest activations during the post-response interval, and these were also the sites giving the peak auditory activation during the phoneme task (Fig. S6c–d
). This is consistent with participation in an external loop of self-monitoring during speech production, with a special focus at the phonological level of representation (Indefrey and Levelt, 2004
In Patient 2, a surprising activation was found at a posterior peri-Sylvian site that showed clear motor
-related activity (red electrode, MNI: −69, −41, 22 mm). This site is located just posterior to the branch point of the Sylvian fissure on the TPJ (, labeled ‘Stp/TPJ’), but its response profile was very similar to premotor sites (). Thus, this TPJ electrode represents a restricted focus of response-related activation surrounded by completely different activation profiles in the STG and SMG, emphasizing the spatial resolution of γhigh
effects. This TPJ activation is not understood in terms of classic models of anterior vs. posterior involvement in speech production vs. perception. However, it confirms more recent findings and models that implicate this posterior TPJ area in speech production (called ‘Tpt’ or ‘Spt’) (Wise et al., 2001
; Hickok and Poeppel, 2007
). Our findings suggest it extends onto the free surface of the TPJ in some individuals, particularly those with a short posterior extent of the Sylvian fissure as seen in Patient 2.
Picture Naming Task
Both patients performed this task without difficulty, and neither made any errors or late responses (>3.5 s that would have contaminated the baseline of the following trial). The γhigh results are organized according to the stages of visual perception, semantic processing, and speech production.
Visual areas were generally not covered and few visual responses were obtained. In Patient 1 (), only the most inferior–posterior electrode (MNI: −62, −59, 11 mm) showed an early visual processing response at ~100 ms (~40% increase, P<0.02). In Patient 2 (), only electrodes from an inferior temporal strip contacting the fusiform gyrus (not shown) exhibited visual responses.
Fig. 4 Picture naming results for Patient 1. Plotting conventions are the same as . Picture stimuli are black-and-white line drawings and remain displayed throughout each trial. The median RT (1026 ms) is faster for this task than for verb generation. (more ...)
Significant activations with time courses consistent with a role in semantic association (or, in any case, inconsistent with a role in either visual or motor processing) were observed in several of the prefrontal sites implicated in the verb generation task. Prefrontal activations were weaker and less extensive in picture naming relative to verb generation. For example, the MFG electrode of Patient 1 (, dark green electrode) peaked at 25% above baseline, compared to 65% during verb generation. In Patient 2 (), the SMG sites (e.g., the yellow electrode, MNI: −61, −43, 47 mm) again showed an activation time-course similar to prefrontal sites, peaking at ~400–500 ms post-stimulus (~20–40%, P<0.02). These were only slightly weaker than during verb generation, but of shorter duration.
In Patient 1 (), the pMTG site (turquoise electrode, MNI: −65, −53, 2 mm) exhibited significant elevation of activity (20–40%, P
<0.02) during the semantic association interval (~350 ms post-stimulus to ~200 ms pre-response) of the picture naming task. In fact, this was slightly stronger and of longer duration than its activation during the verb generation semantic interval. The pSTG sites (darker blue electrodes), by contrast, show no activation or weak suppressions
(−10 to 0%) during this interval, supporting the assertion that the pSTG plays a role primarily in acoustic–phonetic processing, whereas the pMTG plays an additional linguistic role at the word-form or semantic level (Binder et al., 2000
; Bates et al., 2003
; Dronkers et al., 2004
; Indefrey and Cutler, 2004
; Hickok and Poeppel, 2007
Precentral sites, including sPMv, exhibit robust (~100%, P<0.02) γhigh increases surrounding the speech production interval in both patients. The pattern is very similar to the speech production interval of the verb generation task. As in the verb generation task, STG sites exhibit robust (40–100%, P<0.02) γhigh increases beginning after speech onset, implicating them only in auditory feedback during speech production.
In Patient 1 (), the precentral electrodes attain significance (P
<0.02) as early as ~550 ms prior to response onset and rise steadily until just after response onset (peak levels at 50–200 ms). The postcentral electrode (dark red, MNI: −63, −26, 33 mm) shows a similar pattern but shifted somewhat later in latency. In the post-response interval, it was again the anterior of the two pSTG sites (blue electrode, MNI: −66, −46, 10 mm) that showed the strongest auditory feedback response (>100%, P
<0.02). This is the same site showing the peak activation to phoneme stimuli (Fig. S5
). These sites showing post-response activations may be involved in an external loop of monitoring during speech production (Levelt, 1999
; Indefrey and Levelt, 2004
In Patient 2 (), activations in ventral peri-Rolandic cortex began ~400 ms prior to verbal response onset, peaking ~50–200 ms after response onset. The same pattern is observed at the Spt/TPJ site (red electrode, MNI: −69, −41, 22 mm) implicated above in speech production. On the other hand, activations in STG and peri-Sylvian auditory-responsive sites begin strictly following response onset (peak ~300–500 ms post-response).
Overall, the results from picture naming confirm in every case the functional roles demonstrated in the verb generation task for speech production, including auditory feedback. The semantic activations exhibited a high degree of overlap, but with less intense and less widespread activation for naming than verb generation.