Imagine stepping into a jazz club, where you are met by the strains of a pianist negotiating a mesmerizing solo unlike anything that you have heard before. Would you be able to tell from the sounds alone whether the pianist is improvising or playing a rehearsed melody?
Spontaneity is a highly valued quality in many of the world's music performance traditions. Its appreciation by listeners presumably relies upon the interaction of objective auditory cues in a musical performance and the subjective experience and expertise of the listener. It is, therefore, quite likely that individuals differ in their ability to evaluate spontaneity in a performer's actions. This ability, broadly speaking, concerns the sensitivity of one individual to the degree of spontaneity in another's behavior. Such sensitivity is relevant not only to the esthetic appreciation of music and drama, but may be also relevant when inferring others’ intentions in everyday situations (e.g., when judging whether someone's behavior is calculated and intended to deceive). Improvised musical performance, however, presents a paradigmatic domain in which to study the perception of spontaneity in human behavior.
Musical improvisation is a creative process during which a performer aims to compose novel music by deciding what sounds to produce – and when, as well as how, to produce them – within the real-time constraints of the performance itself (Pressing, 1988
). Fluid melodic invention during improvisation is typically achieved through a mixture of spontaneous decision-making and skills honed through deliberate practice. Improvisers invest effort into developing vocabularies of musical patterns (including pitch and rhythmic sequences), which they then, during performance, freely combine and vary in a manner that is sensitive to the prevailing musical form and stylistic context (Ashley, 2009
). Improvisation thus differs from scripted and rehearsed performance, where the goal is to reproduce or re-interpret previously composed music, albeit often in a manner that is intended to sound spontaneous. Despite noteworthy attempts to elucidate the cognitive underpinnings of improvisation (Pressing, 1988
), the precise nature of the processes that enable a performer to invent melodic material in real-time remain “shrouded in mystery” (Ashley, 2009
Our own previous research (Keller et al., in press
) has sought to identify auditory cues to musical spontaneity through the analysis of performance timing and intensity in improvised and rehearsed jazz piano solos. In this work, jazz pianists first improvised melodies and later imitated excerpts from their own and others’ improvisations. The analysis of event timing (i.e., the duration of intervals between successive keystrokes) and intensity (i.e., the force of each keystroke, which determines loudness) indicated that the entropy of both measures was greater for improvised than imitated melodies (though, interestingly, the effect for timing was not reliable when imitating one's own improvisations in familiar jazz styles). Information theoretical approaches to psychology assume that entropy – a measure of the randomness of a probability distribution of values (Shannon, 1948
) – reflects uncertainty in human behavior (e.g., decision-making; see Berlyne, 1957
; Koechlin and Hyafil, 2007
). In view of this, the finding that timing and intensity are more variable in improvisations than in imitations may indicate irregularities in force control associated with fluctuations in certainty about upcoming actions – i.e., when spontaneously deciding which keys to strike – during improvised musical performance.
The current study extends this work to behavioral and brain processes associated with the perception of musical spontaneity by listeners. Specifically, in a functional magnetic resonance imaging (fMRI) experiment, we investigated the ability of musically trained listeners to differentiate between excerpts from the improvised and rehearsed jazz piano solos that were analyzed by Keller et al. (in press
). Our earlier finding that the entropy of timing and intensity is higher in improvised than imitated melodies suggests that these parameters could potentially provide listeners with reliable cues to musical spontaneity. The main research questions addressed here concern (a) whether skilled listeners would be able to judge accurately whether the performances were improvised or rehearsed, and (b) whether listening to improvisations is associated with patterns of brain activation that are distinct from those associated with listening to rehearsed performances. We expected that musically trained listeners would be able to evaluate spontaneity, and that their ability to do so would be grounded in neural mechanisms that allow them to mentally simulate – and thus predict to a certain degree – the performer's actions. Such simulation should allow the skilled listener to experience, at least partially, the brain states associated with improvised and rehearsed musical performance. Precise hypotheses about the behavioral and brain processes that enable the evaluation of musical spontaneity are formulated below.
With respect to listener behavior, we assumed that the perception of musical spontaneity is based on the detection of auditory cues reflecting uncertainty in the performer, and, therefore, that the ability to judge whether a performance is improvised or rehearsed depends on the listener's sensitivity to fluctuations in parameters such as event timing and intensity. Sensitivity to these fluctuations could potentially be influenced by factors that affect general responsiveness to uncertainty in other individuals’ behavior, as well as by factors related more specifically to uncertainty in the production of piano melodies. The former, general factors may include socio-cognitive variables like empathy, i.e., the ability to understand others’ feelings (of uncertainty, in the present case). Factors related more specifically to the perception of piano melodies include the listener's own experience at playing the piano. Through such experience, an individual has the opportunity to learn about the effects of uncertainty on the variability of timing and intensity in their own playing. Experienced listeners – especially if highly empathic – may therefore be able to recognize these hallmarks of spontaneity in another pianist's performance.
A distinction between domain general and music specific processes can, likewise, be hypothesized with respect to the brain mechanisms underlying the perception of musical spontaneity. At one level, brain areas that are generally sensitive to behavioral variability related to uncertainty may play a role in the detection of such spontaneity. Studies investigating the neural correlates of the perception of behavioral uncertainty point to the involvement of brain regions including anterior cingulate cortex, insula, and amygdala (Singer et al., 2009
; Sarinopoulos et al., 2010
). The amygdala is particularly interesting in this regard because it has been implicated in a range of processes that may be relevant to evaluating musical spontaneity, including novelty detection (Wright et al., 2003
; Blackford et al., 2010
) and the perception of emotionally neutral stimuli that are ambiguous or difficult to predict in terms of their timing (Hsu et al., 2005
; Herry et al., 2007
In addition to brain regions that are generally sensitive to behavioral uncertainty, neural mechanisms that enable a skilled listener to perceive uncertainty in a performer's actions on a specific instrument (in this case, the piano) may facilitate the evaluation of musical spontaneity. Functional links between perceptual and motor processes constitute such a mechanism. Considerable evidence for such perception–action links has accumulated in the auditory domain (Kohler et al., 2002
; Gazzola et al., 2006
), notably in the context of music, where auditory–motor associations develop with experience playing an instrument (Bangert et al., 2006
; Lahav et al., 2007
; Mutschler et al., 2007
; Zatorre et al., 2007
). Relevant studies have shown that listening to music that belongs to an individual's behavioral repertoire leads to the activation of sensory and motor-related brain areas, including the anterior insular cortex, the frontal operculum (Mutschler et al., 2007
), the inferior parietal lobe (IPL), and the ventral premotor area (vPM; Bangert et al., 2006
; Lahav et al., 2007
). The IPL and vPM (in addition to Brodmann's area 44) have also recently been discussed in connection with a postero-dorsal auditory pathway that subserves the processing of speech and music-related communicative signals (Rauschecker and Scott, 2009
; Rauschecker, 2011
The co-activation of sensory and motor areas (in the absence of overt movement) is consistent with the proposal that action perception recruits covert sensorimotor processes that internally simulate the observed action (Rizzolatti and Craighero, 2004
). Studies of individual differences suggest that activity in action simulation networks increases with increasing action-specific motor expertise (Bangert et al., 2006
; Lahav et al., 2007
; Mutschler et al., 2007
) and, furthermore, is relatively strong in individuals who score highly on self-report measures that assess dimensions of empathy such as “perspective taking” (Gazzola et al., 2006
). Researchers in the field of social cognition have argued that action simulation plays a role in generating online predictions about upcoming events in order to facilitate action perception, action understanding, and the coordination of one's own actions with those of others (Wilson and Knoblich, 2005
; Sebanz and Knoblich, 2009
; see also Gallese et al., 2004
; Schubotz, 2007
In the context of music listening, internal simulation processes may trigger anticipatory auditory images of upcoming sounds (Keller, 2008
; Leaver et al., 2009
). Action simulation may thus facilitate music perception by utilizing the skilled listener's motor system to enable the real-time prediction of acoustic parameters including pitch, rhythmic timing, and sound intensity (Schubotz, 2007
; Rauschecker, 2011
). The proportion of neural resources recruited by brain networks engaged in action simulation generally varies as a function of the degree to which prediction is challenging (Schubotz, 2007
; Stadler et al., 2011
). Fluctuations in performance timing and intensity related to a performer's uncertainty may therefore be associated with relativity strong activation of such simulation networks in experienced, empathic listeners.
Viewing action simulation in light of the broader claim that perception and action recruit common neural networks (Hommel et al., 2001
; Rizzolatti and Craighero, 2004
) suggests that listening to improvisations should be associated with patterns of brain activation that are partially distinct from those associated with listening to rehearsed performances, as studies examining the execution of these two varieties of action have highlighted their differential processing (Bengtsson et al., 2007
; Berkowitz and Ansari, 2008
; Limb and Braun, 2008
). One such study found stronger activation in a network comprising the dorsolateral prefrontal cortex (DLPFC), the pre-supplementary motor area (pre-SMA), and the dorsal premotor area (dPM) when pianists improvised variations on a visually displayed melody compared to when they reproduced their improvisations from memory (Bengtsson et al., 2007
). The authors interpret this result in terms of the involvement of these areas in tasks where participants can choose a response “freely.”
While research studies on musical spontaneity are small in number, a relatively large body of related work has been conducted in the broader field of voluntary action control. In this field, actions are classified along a continuum reflecting the degree to which they are controlled internally (i.e., endogenously) by the agent or externally (i.e., exogenously) by environmental cues (Waszak et al., 2005
; Haggard, 2008
). Musical improvisation is internally controlled to the extent that actions are chosen freely and spontaneously; imitating a melody is relatively externally controlled, as a pre-existing sequence of sounds provides an exogenous cue that constrains the actions of the imitating performer. Empirical investigations of internally and externally controlled actions outside the music domain – e.g., in arbitrary tasks involving freely selected vs. externally cued button presses – have revealed distinct behavioral (timing) and brain (electrophysiological) signatures for the two modes of action (Waszak et al., 2005
; Keller et al., 2006
). Further studies using fMRI have found that activity in the rostral cingulate zone (RCZ) of the anterior cingulate cortex is associated with action selection (deciding “what” to do) while activity in a sub-region of the superior medial frontal gyrus (SFG) is associated with action timing (deciding “when” to do it; Mueller et al., 2007
; Krieghoff et al., 2009
). To the extent that these regions subserve free response selection, they may also be differentially involved during the perception of internally controlled musical improvisations vs. externally guided imitations.
Based on the foregoing, we hypothesized that listeners with jazz piano experience would be sensitive to differences in the degree of musical spontaneity in improvised and imitated jazz piano solos. Specifically, the ability to discriminate between these modes of performance should vary as a function of the listener's amount of musical experience and empathy, to the extent that these factors affect the ability to simulate the performers’ actions (for both improvisation and imitation) and to recognize auditory cues to uncertainty in the timing and intensity of the performances. With respect to the neural correlates of evaluating musical spontaneity, we hypothesized that brain regions that have been implicated in the detection of behavioral uncertainty (anterior cingulate cortex, insula, and amygdala) would be sensitive to uncertainty related fluctuations in performance timing and intensity. Furthermore, we expected that the probability of judging a performance to be improvised would increase with increasing demands placed on brain regions involved in action simulation (vPM, IPL, anterior insula, frontal operculum) due to perceived unpredictability in the performer's actions. Finally, to the extent that simulations are high in fidelity, differences in brain activation when listening to improvisations vs. imitations should be observed in similar regions to those found for the production of improvised and rehearsed music (pre-SMA, dPM, DLPFC) and of internally and externally controlled actions (RCZ, SFG).
To take into account the possibility that some of the above-mentioned brain regions may differentiate between improvisations and imitations even when listeners fail to make accurate explicit judgments, we analyzed the fMRI data in accordance with a 2
2 factorial design that incorporated both the objective classification of stimulus melodies (real improvisations/real imitations) and subjective classifications based on listeners’ responses (judged improvised/judged imitated). The main effect contrast for the objective classification was expected to reveal differences in neural processing related to physical differences between improvised and imitated melodies. The main effect contrast for subjective classifications tested the neural bases of listeners’ beliefs, and was expected to be informative about experience-related processes such as action simulation.