|Home | About | Journals | Submit | Contact Us | Français|
Emotion is a cultural and psychobiological adaptation mechanism which allows each individual to react flexibly and dynamically to environmental contingencies. From this claim flows a description of the elements theoretically needed to construct a virtual agent with the ability to display human-like emotions and to respond appropriately to human emotional expression. This article offers a brief survey of the desirable features of emotion theories that make them ideal blueprints for agent models. In particular, the component process model of emotion is described, a theory which postulates emotion-antecedent appraisal on different levels of processing that drive response system patterning predictions. In conclusion, investing seriously in emergent computational modelling of emotion using a nonlinear dynamic systems approach is suggested.
If we want to compute emotions, we need to know what exactly we are going to compute. As more than a century of debate has shown, there is little agreement on what an emotion really is. In view of this stalemate, Frijda & Scherer (2009) have recently suggested that the following features of emotion are relatively uncontroversial and are generally seen as being of central importance to the understanding of the phenomenon.
Frijda & Scherer (2009) argue that it is these four central features that jointly define what is generally meant by emotion, both in lay and scientific terminology. These features also allow distinguishing emotions from other affective states such as preferences, moods, attitudes, interpersonal stances or affective dispositions or traits.
Based on what precedes, emotion will be considered here as a bounded episode in the life of a system that is characterized as an emergent pattern of component synchronization, preparing adaptive action tendencies to relevant events, as defined by their behavioural meaning and aiming at establishing control precedence over behaviour. In what follows, different theoretical models of emotion will be reviewed with respect to their use for a computational approach using the highlighted features.
An important issue also concerns the conceptual distinction between the term ‘emotion’ as defined above and related terms such as mood, affective dispositions or preferences. Scherer (2005b) proposed a design feature system to distinguish such terms and suggested that emotions are specific with respect to the following features: they (i) focused on specific events; (ii) involve the appraisal of intrinsic features of objects or events as well as of their motive consistency and conduciveness to specific motives; (iii) affect most or all bodily subsystems which may become to some extent synchronized; (iv) are subject to rapid change owing to the unfolding of events and reappraisals; and (v) have a strong impact on behaviour owing to the generation of action readiness and control precedence.
Scherer (2009) identified three major models which emerged out of different schools of thought over the centuries.
These theories assume an emotion architecture that is based on an individual's subjective evaluation or appraisal of the significance of events for their well-being and goal achievement, postulating a specific set of appraisal criteria (e.g. novelty, intrinsic pleasantness, goal conduciveness or motive consistency, agency, responsibility, coping, legitimacy and compatibility with self and societal standards). Detailed predictions are made about the emotional experiences generated by specific appraisal combinations. There are also a number of appraisal-related theories that differ in scope, focus or the underlying architecture (e.g. Weiner 1985; Ortony et al. 1988). In addition, there are several psychological theories of emotion that do not fit squarely into the three traditions outlined above, focusing on a specific aspect or component of emotion, such as motivation or action preparation, or combining features from the three major orientations (see Moors 2009).
To illustrate the differences between these three theoretical traditions, in a highly simplified form, figure 1 (adapted from Scherer 2009) synthesizes the three models graphically. In the following, these major theoretical traditions are systematically compared with respect to the features of the emotion construct outlined above (following Scherer 2009).
Comparison of three major traditions of emotion theories: (a) basic emotion theories (Ekman, Izard); (b) early constructivist theories—Schachter, Mandler; (c) current constructivist theorists—Russell, Barrett; and (d) appraisal theories ...
Both basic and appraisal theories consider emotions as bounded episodes in time, having a clear onset and a somewhat fuzzy offset. By contrast, constructivist theories suggest that core affect varies continuously and that this stream is segmented by the individual's constructive categorization and conceptualization.
Basic emotion theories postulate neuromotor affect programmes (even though Ekman and Izard have suggested that there may be a certain degree of flexibility in the execution of these programmes; see Scherer & Ellgring 2007). In this sense, there is no emergent pattern but the relatively rigid execution of a programme. Constructivist theories negate the existence of predictable patterns in the emotion process and see the only regularity in the categories or concepts applied by the individual in a post hoc fashion on the basis of a large number of factors (Russell 2003; Barrett 2006). By contrast, most appraisal theorists (Scherer 2001, in press a; Smith & Kirby 2001; Ellsworth & Scherer 2003) assume the emergence of patterns driven by appraisal results (which in Scherer's component process model (CPM) follows a lawful sequence).
The protagonists of all three theory traditions accept the notion of a componential architecture of emotions which constitutes a major advance over the earlier version of constructivist dimensional theories (Russell 1980). However, only some appraisal theories, in particular Scherer's (2001, 2004, in press a) model (CPM) of emotion strongly insist on a process of synchronization and desynchronization of components within the bounded episode, to the point of making the degree of coherence a central criterion for the existence of an emotion (Scherer 2005b; Dan Glauser & Scherer 2008).
All three theories assume some degree of functionality. In the case of basic theories, the affect programme is pre-programmed by evolution to deal with the eliciting event. Appraisal theories define the adaptive functions in terms of the efferent results of individual appraisal checks that add up cumulatively to prepare appropriate action tendencies (Scherer 2001; Ellsworth & Scherer 2003). Constructivist theories endorse the adaptive value of the processes they postulate but do not provide a justification for this claim in terms of the postulated architecture and the criteria to be applied (Barrett 2006).
Both basic and appraisal theories see events as elicitors of bounded emotion episodes, even though appraisal theorists assume that it is not the event itself, but the appraisal by the individual, which is decisive and which may change over time, in the course of reappraisal. Constructivist theories do not clearly specify how events affect continuous core affect. According to Russell (2003), individuals may attribute a certain core affect to an event in retrospect. In the case of Barrett (2006), the relation of events to core affect and conceptualization remain unspecified and the intentional object of the emotion has been discarded (Deonna & Scherer in press).
This is not a meaningful feature for basic or constructivist theories. The former take the type of event as the major discriminating factor, the latter see categorization and conceptualization of core affect as independent from event evaluation. For appraisal theorists, this is the essential feature, insisting on the fact that it is only through the specific behavioural meaning of an event for an individual that the action preparation following the appraisal process can have adaptive value.
Apart from these differences with respect to the central features of emotion, the three theory traditions also differ on several other counts (see Scherer 2009 for further detail). Apart from these differences with respect to definitional features, the three types of models have diverging views on the scope of an emotion definition, the number and type of emotions and the causal nature of the underlying mechanisms.
For basic emotion and appraisal theorists, the term emotion denotes all of the components of emotion: elicitation processes, physiological symptoms, motor expression, motivational changes and subjective feeling. By contrast, ever since James, constructivist theorists tend to redefine the concept ‘emotion’ exclusively in the sense of the subjective feeling component. Using these two terms synonymously has been, and continues to be, a major source of confusion and debate (Scherer 2005b; Deonna & Scherer in press).
On one extreme we find the notion of a limited number of evolutionarily continuous adaptive emotion systems (held by many basic emotion theorists) and on the other, that of fuzzy, unpredictable state changes that achieve coherence only by their place in a valence/arousal space and by conceptual classification, espoused by some constructivists. In this debate, appraisal theorists are somewhere in the middle—they accept neither the idea of a limited repertoire of basic, homogeneous emotions with highly prototypical characteristics, nor that of emotions being individually labelled points in two-dimensional affect space. Rather, while assuming that there are widely varying types of emotions, they postulate the existence of modal emotion families (Scherer 1994), with frequently occurring appraisal profiles that have adaptive functions in dealing with quintessential contingencies in animal and human life.
The mechanism postulated by the basic emotion model is deterministic on a macro level—a given stimulus or event will determine the occurrence of one of the basic emotions (through a process of largely automatic appraisal). By contrast, appraisal theorists are deterministic on a micro level—specific appraisal results or combinations thereof are expected to determine, in a more molecular fashion, specific action tendencies and the corresponding physiological and motor responses. Most importantly, appraisal theorists espouse emergentism, assuming that the combination of appraisal elements in a recursive process is unfolding over time and that the ensuing reactions will form emergent emotions that are more than the sum of their constituents and more than instantiations of rigid categories, namely unique emotional experiences in the form of qualia (Scherer 2004, in press a). In fact, appraisal theorists also engage in mild constructivism in that the process of categorization and labelling of the non-verbal representation of an emotion episode, including somatosensory proprioceptive feedback, allows for an active search for the construction of individual, cultural or situational meaning. In comparison, modern constructivist theorists are generally anti-deterministic and define constructivism in a strong sense, i.e. individuals search to define their worlds and experiences based on contextual cues that may be more or less related to an eliciting event. While they admit that core affect is produced by a large number of different factors including appraisal, they have not elaborated specific hypotheses about the mechanisms that determine the position of core affect in valence/arousal space or about the organization of components over time.
Let us now return to the issue of computation. Which theory or model out of the three shown in figure 1 would seem the most useful for this purpose? Let us proceed by exclusion. If one wants to compute emotion rather than constructively assign labels of emotion likely to vary greatly from one individual to another in a rather unpredictable way, constructivist theories need to be discarded (a decision comforted by the serious underspecification of the determining factors and the absence of precise predictions of mechanisms). Basic emotion theories have been and still are to some extent the models of choice, especially in computer sciences and engineering. However, as shown above, if one accepts the central features of emotion outlined above, they do not fare so well, both from the point of view of mapping theory to underlying processes and with respect to the specification of mechanisms that allow to model the essentially emergent nature of dynamic emotion processes. Ergo, the choice should fall, and not only because it seems the only option remaining, on appraisal theories of emotion.
Scherer (in press b) argues that the CPM provides a suitable blueprint for computational models of emotion. This specific appraisal model will be briefly outlined below.
Figure 1d shows the basic architecture of the model, including the dynamic, recursive emotion processes following an event that is highly pertinent to the needs, goals and values of an individual. As shown in the flow diagram, the CPM suggests that the event and its consequences are appraised with a set of criteria on multiple levels of processing. The result of the appraisal will generally have a motivational effect, often changing or modifying the motivational state before the occurrence of the event. Based on the appraisal results and the concomitant motivational changes, efferent effects will occur in the autonomic nervous system (ANS; in the form of somatovisceral changes) and in the somatic nervous system (in the form of motor expression in face, voice and body). All of these components, appraisal results, action tendencies, somatovisceral changes and motor expressions are centrally represented and constantly fused in a multimodal integration area (with continuous updating as events and appraisals change). Parts of this central integrated representation may then become conscious and subject to assignment to fuzzy emotion categories as well as being labelled with emotion words, expressions or metaphors.
As recent descriptions of the model and reviews of the extensive empirical evidence from research in psychology and the neurosciences can be found elsewhere (Scherer 2001, 2004, 2005a, in press a; Scherer et al. 2001), in the following section only some of the major elements are summarized based on the earlier presentations. Because the major function of a theoretical model is to guide empirical research, the focus is on the predictions made and the concrete hypotheses that follow from them.
The CPM suggests that there are four major appraisal objectives that an organism needs to reach to adaptively react to a salient event: (i) how relevant is this event for me? Does it directly affect me or my social reference group? (relevance); (ii) what are the implications or consequences of this event and how do they affect my well-being and my immediate or long-term goals? (implications); (iii) how well can I cope with or adjust to these consequences? (coping potential); and (iv) what is the significance of this event for my self-concept and for social norms and values? (normative significance). To attain these objectives, the organism evaluates the event and its consequences on a number of criteria or stimulus evaluation checks (SECs), with the results reflecting the organism's subjective assessment of consequences and implications on a background of personal needs, goals and values (which may well be unrealistic or biased).
Appraisal theorists generally agree about the major criteria or dimensions that are required to determine the behavioural meaning of an event to the organism (Scherer 1999; Ellsworth & Scherer 2003). Column 1 in table 1 lists the criteria as they are defined in the CPM (for further details and references, see Scherer 2001, in press a; Sander et al. 2005). The verbal description of these criteria or checks to be processed in the appraisal process seems to require a complex cognitive calculus. However, this is not necessarily the case. Leventhal & Scherer (1987) showed that all of the criteria can be processed at three hierarchically organized levels: (i) the sensory-motor level, in which the checking mechanisms are mostly genetically determined and the criteria consist of appropriate templates for pattern matching and similar mechanisms (cf. the notion of ‘biological preparedness’; Öhman 1987); (ii) the schematic level, based on social learning processes, occurring in a fairly automatic, unconscious fashion; and (iii) the conceptual level, primarily via cortical association areas and requiring consciousness, involving propositional knowledge and underlying cultural meaning systems. The different levels continuously interact, producing top-down and bottom-up effects (see also Van Reekum & Scherer 1997). The CPM is sometimes accused of ‘cognitivistic bias’, accompanied by the claim that it is too slow and cumbersome to account for the rapid onset of emotional reactions. This critique is surprising, given the early and repeated insistence on parallel processing on the three levels of information processing that are vastly different with respect to automaticity, rapidity, computational power and the need for consciousness. Clearly, these levels need further refinement to reflect current knowledge about the underlying perceptual and cognitive processes. Thus, it may be useful to split the schematic level into two layers, consisting of well-formed prepotent schemata (based on repeated earlier experiences) on the lower level and facilitated configurations for the spread of associations on the higher level, with both levels sharing a high degree of automaticity and the potential for unconscious processing.
The appraisal mechanism described earlier requires interaction between many cognitive functions and their underlying neural circuits to compare the features of stimulus events to stored schemata, representations in memory and self-concept, and expectations and motivational urges of high priority. In addition, this process controls attention deployment and relies heavily on implicit or explicit computation of probabilities of consequences, coping potential and action alternatives. Figure 2 illustrates the postulated sequence, the cognitive and motivational inputs and the effects on response systems (illustrated below). The architecture assumes bidirectional influences between appraisal and various cognitive functions. For example, minimal attention needs to be given for appraisal to start, but a relevant outcome will immediately deploy further attention to the stimulus. Stimulus features are compared with schemata in memory but strongly relevant stimulus features will, following appropriate appraisal, be stored as emotional schemata in memory. Event consequences are compared with current motivational states, but particular appraisal outcomes will change the motivation and produce adaptive action tendencies. These bidirectional effects between appraisal and other cognitive functions are illustrated by the arrows in the upper part of figure 2.
The CPM claims that the SECs are processed in sequence, following a fixed order, consisting of four stages in the appraisal process that corresponds to the appraisal objectives described. This sequence assumption is justified in terms of systems economy and logical dependencies—the results of the earlier SECs need to be processed before later SECs can operate successfully, that is, yield a conclusive result. Expensive information processing should occur for only those stimuli that are considered relevant for the organism. Consequently, relevance detection is considered to be a first selective filter through which a stimulus or event needs to pass to merit further processing. Extensive further processing and preparation of behavioural reactions are indicated only if the event concerns a goal or need of major importance, or when a salient discrepancy with an expected state is detected, suggesting that the implications for the organism are assessed next in the sequence. Further, the causes and implications of the event need to be established before the organism's coping potential can be conclusively determined, as the latter is always evaluated for a specific demand.
The CPM assumes that the microgenetic unfolding of the emotion-antecedent appraisal processes parallels both phylogenetic and ontogenetic development in the differentiation of emotional states. The earlier SECs, particularly the novelty and the intrinsic pleasantness checks, are present in most animals, including newborn humans, and one can argue that these low-level processing mechanisms take precedence as part of our hard-wired detection capacities and occur rapidly after a stimulus event occurs. More complex evaluation mechanisms are successively developed at more advanced levels of phylogenetic and ontogenetic development: natural selection operates towards more sophisticated information processing ability in phylogenesis, and maturation and learning increase the individual's cognitive capacity in ontogenesis (see Scherer 1984, pp. 313–314; Scherer et al. 2004).
The sequence assumption is often criticized as being overly restrictive and inconsistent with the idea that massive parallel processing of information occurs in different systems. This criticism overlooks the CPM postulate that external or internal event changes maintain a recursive appraisal process until the monitoring subsystem signals termination of or adjustment to the stimulation that originally elicited the appraisal episode. Thus, the checking process repeats the sequence continuously, constantly updating the appraisal results that change rapidly with changing events and evolving evaluation. The level of processing can be expected to move up in the course of this sequential course, given both the nature of the computation and the likelihood that lower levels have been unable to settle the issue. The normative significance of the event, that is, its consequences for the self and its normative or moral status, is expected to be appraised last, as it requires comprehensive information about the event and comparison with high-level propositional representation.
Therefore, the proposed mechanism is highly compatible with the assumption of parallel processing. All SECs are expected to be processed simultaneously, starting with relevance detection. However, the essential criterion for the sequence assumption is the time at which a particular check achieves preliminary closure, that is, yields a reasonably definitive result, one that warrants efferent commands to response modalities, as shown by the descending arrows in figure 2. The sequence theory postulates that, for the reasons outlined earlier, the result of a prior processing step (or check) must be in before the consecutive step (or check) can produce a conclusive result with efferent consequences. It is indeed feasible to assume that the results of parallel processes for different evaluation criteria will be available at the different times, given differential depth of processing.
As shown in figure 2, the fundamental assumption of the CPM is that the appraisal results drive the response patterning in other components by triggering efferent outputs designed to produce adaptive reactions that are in line with the current appraisal results (often mediated by motivational changes). Thus, emotion differentiation is the result of the net effect of all subsystem changes brought about by the outcome profile of the SEC sequence. These subsystem changes are theoretically predicted on the basis of a componential patterning model, which assumes that the different organismic subsystems are highly interdependent and that changes in one subsystem will tend to elicit related changes in other subsystems. As illustrated in figure 2, this process, similar to appraisal, is highly recursive, which is what one would expect from the neurophysiological evidence for complex feedback and feedforward mechanisms between the subsystems (see neural architecture discussion). As shown in figure 2, the result of each consecutive check is expected to differentially and cumulatively affect the state of all other subsystems.
The CPM makes specific predictions about the effects of the results of certain appraisal checks on the autonomic and somatic nervous systems, indicating which somatovisceral changes and which motor expression features are expected. These predictions are briefly summarized in column 3 in table 1. They are based on both the general functions of the emotion components and the specific functions of each SEC (see column 2 in table 1). In particular, specific motivational and behavioural tendencies are expected to be activated in the motivation component in order to serve the specific requirements for the adaptive response demanded by a particular SEC result. In socially living species, adaptive responses are required not only for the internal regulation of the organism and motor action for instrumental purposes (organismic functions), but also for interaction and communication with conspecifics (social functions).
The CPM assigns a special status to the feeling component in the emotion process, as it monitors and regulates the component process and enables the individual to communicate its emotional experience to others. If subjective experience is to serve a monitoring function, it needs to integrate and centrally represent all information about the continuous patterns of change and their coherence in all other components. Thus, feeling is an extraordinarily complex conglomerate of information from different systems. Figure 3 shows how the different components of the emotion process might be integrated and represented in a unitary fashion in what philosophers have described as qualia (see Scherer in press a). As shown on the left side of the figure, the ANS, the somatic or motor nervous system (SNS) and motivation components are driven by the appraisal component (which are in turn influenced by the changes that occur in these other components and which may be in part the results of component-specific factors). The current state of each of these components is then represented in an integrated fashion in the feeling component. Quality, intensity and duration of the feeling are determined by these integrated inputs. Appraisal results will be represented by the patterning of the appraisal check results and the weights that are assigned to individual appraisal checks. Both SNS and ANS will be represented as a function of the respective response patterns and their amplitudes. Finally, the motivation component will be represented by the nature of the action tendencies that have been elicited as well as by the estimated urgency of action.
Mechanism of component integration.
Scherer (2004) describes in detail which integration tasks need to be achieved in the process. Information needs to be integrated in the cognitive component as different appraisal results may vary greatly with respect to the nature of the outcome. Information integration is also required for the response components as the response modalities, e.g. physiological variables and expressive behaviours, vary greatly with respect to their underlying metric. Finally, multi-component integration is required to bring all the separate information channels together. In addition, temporal integration has to be achieved to create the notion of a coherent episode.
Additional questions concern the nature of the emergence of the qualia into consciousness and the process of categorization and verbalization. The model offers a conceptualization of the problem as shown in figure 4, using a Venn diagram in which a set of overlapping circles represent the different aspects of feeling. The first circle (A) represents the sheer reflection or representation of changes in all synchronized components in some kind of monitoring structure in the central nervous system (CNS). This structure is expected to receive massive projections from both cortical and subcortical CNS structures (including proprioceptive feedback from the periphery). The second circle (B), only partially overlapping with the first, represents that part of the integrated central representation that enters awareness and thereby becomes conscious, thus constituting the feeling qualities, the qualia that philosophers and phenomenologically minded psychologists have been most concerned with. Thus, this circle corresponds most directly to what is generally called ‘feelings’. The conscious part of the feeling component feeds the process of controlled regulation, much of which is determined by self-representation and socio-normative constraints. It is hypothesized that it is the degree of synchronization of the components (which might in turn be determined by the pertinence of the event as appraised by the organism) that will generate conscious experience.
A Venn model of component integration and the role of conscious feeling.
Unfortunately, all we can currently measure is the individual's verbal account of a consciously experienced feeling, represented by the third circle (C) in figure 4. Drawing this circle as only partially overlapping with the circle representing conscious experience (B) is meant to suggest that the verbal account of feelings captures only part of what is consciously experienced. This selectivity can be owing, in part, to control intentions—the individual may not want to report certain aspects of his/her innermost feelings. Most importantly, verbal report relies on language and thereby on the emotion categories and other pragmatic devices available to express the qualia that are consciously experienced. Apart from capacity constraints (the stream of consciousness cannot be completely described by a discrete utterance), it may not be unreasonable to claim that these linguistic devices are incapable of completely capturing the incredibly rich texture of conscious experience. This proposal is described in greater detail in Scherer (in press a).
This model has been developed over a period of about 25 years and there are copious empirical studies that have provided validation through extensive confirmation of a large set of the hypotheses generated by the model. It would largely exceed the available space to review this evidence here. Major summaries of the empirical studies that have empirically tested elements of the CPM can be found in the chapters by Johnstone, Van Reekum & Scherer, Kaiser & Wehrle, Pecchinenda, in Scherer et al. (2001) and in Scherer (in press a).
Several attempts at computational modelling of the model and its predictions have been made. A first approach consisted in formalizing the appraisal predictions in the form of a simple expert system and to subject the system to empirical test (GENESE—Geneva expert system on emotion). The system invites participants to think of a situation in which they experienced a strong emotion and proposes to diagnose the emotion felt on the basis of the responses to questions that represent simplified versions of the SECs. The Euclidean distances between this input vector to the theoretically postulated, prototypical vectors for 14 major emotions are computed and weighted and a diagnosis based on the smallest distance is returned. The participants judge whether the diagnosis is correct or not. If the answer is no, a second diagnosis, corresponding to the second smallest distance, is suggested. In the first study, with automatic administration of the system at a book fair, with over 200 emotion situations reported by different subjects, an accuracy percentage of 78 per cent was obtained for the expert system's diagnoses (Scherer 1993). This publication triggered an interesting scientific exchange on the role of computer modelling as a tool in appraisal research (Chwelos & Oatley 1994; Wehrle & Scherer 1995). An improved version of GENESE is available for testing on the web (www.affective-sciences.org/genese) and has been used by several thousand participants in the last years. A report of the results is currently being undertaken (Grandjean & Scherer in preparation).
Based on this approach, Wehrle & Scherer (2001) developed a prototype of a simulation system for the development and testing of appraisal theories of emotion. The underlying idea of this system is to allow the theorist to specify the hypothesized components of the appraisal model in sufficient detail and degree of formalization to generate concise predictions on the basis of hypothetical or empirical datasets. Furthermore, the theorist is given the possibility to rapidly and interactively change the major parameters of the system for observing the consequences on the outcomes. The simulation environment was conceived in such a way that not only verbal labels, i.e. decisions on categorical classifications, represent possible outcomes but also non-verbal response modalities such as facial expression (with the possibility for extensions into vocal expression and physiological patterning). An important aspect of the system is its close connection to empirical databases—the system has modules to acquire, analyse and evaluate subject-generated data.
Another, rather modest, approach to computational modelling of the CPM was based on neural modelling, using the Interactive Activation and Competition network (IAC) shell provided by McClelland & Rumelhart (1988). The program (K. R. Scherer 1995, unpublished; available by writing to the author) uses the SEC predictions (Scherer 1984, 2001) as the basis for the matrix of activation and inhibition patterns in the IAC shell, allowing one to set inputs that correspond to the result of specific SECs, and generates activation patterns of hidden units and corresponding output patterns with respect to a number of discrete emotion labels. If one sets the input patterns according to the theoretical predictions (i.e. Scherer 1986, table 2, p. 147) the program produces results that correspond to the hypotheses, confirming the internal consistency of the model. In addition, the program allows generating insights that are not predicted, such as testing the effect of setting the inputs sequentially rather than simultaneously, having the system run through several cycles before successive inputs, and comparing the differences in output between sequential and parallel processing (see discussion in Scherer 1993). Furthermore, systematically varying patterns of input strength, or input profiles, that deviate more or less from the predicted, prototypical profiles provide interesting insights and intuitions. Another issue of interest is the role of interactions between appraisal dimensions.
More recently, a more formal model of the process mechanisms that may underlie the sequential check and componential patterning theories in a neural network model was provided (figure 5; adapted from Scherer 2001), suggesting that appraisal processing occurs in an information-processing system similar to that described by Cowan (1988).
Potential architecture of the appraisal process as part of a general information processing system.
On a first pass, contents of the brief sensory storage are processed or ‘coded’ by a range of procedures from simple pattern matching to logical inference. Pertinent schemata are recruited in a largely automatic fashion to determine whether a satisfactory match (and consequently a promising adaptational response) can be selected. In many cases, this is followed by controlled processing based on propositional content activated in long-term memory, giving rise to more elaborate evaluation and inference processes (see the lower part of figure 5). The results of both types of processing, constantly updated as the appraisal process unfolds, activate a network of representational units that correspond to the appraised characteristics and significance of the event and thus represent the overall significance of the event for the individual.
As suggested above, the SECs do the job of providing the four essential types of information required for action preparation. Given the presumed network architecture, the contents of the representational units corresponding to individual SECs are continuously integrated with respect to these classes of information. In figure 5, the connections between the SEC units and the boxes represent the integration of the SEC-based appraisal registers with respect the basic information types of appraisal objectives (relevance, implication, coping and normative significance). The different SECs are likely to be integrated through context-dependent weighting functions, giving them differential importance in the combination.
As shown in the upper part of figure 5, the profile of integrated information in the four appraisal objectives will directly activate, on a molecular level, the neuro-endocrine system (NES), the ANS, and the SNS to provide rudimentary preparation for action. In addition, in a more molar, motivational form, the appraisal profile will activate several potential response mechanisms in the form of action tendencies (Frijda 2007), which in turn will also recruit the support systems.
Admittedly, this is only a very preliminary sketch of a potential model that will need to be developed in much greater detail to allow serious modelling and testing. However, the choice of the type of model is decisive at this stage. It can be argued that neural network models are much better suited to model emergent emotion processes than are hierarchical decision tree or rule systems (e.g. Chwelos & Oatley 1994; Gratch & Marsella 2004; Spackman 2004). Advanced neural network models are preferable as they can represent the architecture of the CPM and the emergent nature of the emotion process, such as the simultaneity of massively parallel processing with a sequential and cumulative decision and efference structure, as described above. As many circuits are continuously active and emotional quality changes continuously because of changes in input, appraisal and proprioceptive feedback, binary decision structures are hardly appropriate to model the process. A multi-layered outcome of the processing, as shown in figure 5, also corresponds to the central tenet of the CPM which states that there is almost an endless variety and subtlety of different emotional qualities (the philosophers' qualia). If one wants to model the emergent process of emotion, it is this level of outcome complexity that needs to be targeted.
As briefly described in §3e above, the categorization and labelling of the centrally represented qualia, for example as one of the ‘big six’ basic emotions, is a secondary process which is largely dependent on consciousness, salience of mental categories and availability of labels (among many other factors). The complexity of these categorization and labelling processes, and the central role of consciousness, is analysed in detail in Scherer (2005a, in press a). One may want to model this process, with a category or a label as an outcome, rather than the emergent multimodal features of the emotion process. However, as shown elsewhere (Scherer in press b), this requires first, that one has a way of representing the emergent process outcomes as an input to the categorization process at a specific moment in time; second, that one is able to model the nature of the cognitive operators in categorization and labelling; and third, that one recognizes the language and culture dependency of categories and labels.
This contribution focuses on modelling the emergence of a rich scope of qualitative multimodal emotion episodes. As pointed out earlier (Scherer 2004), the issue of multimodal and temporal integration is of particular importance and has been largely neglected. This is all the more problematic as it is this integration which is at the basis of all monitoring and regulation (see also Scherer 2005a,b, in press a). The attempt to model emergent information integration in such a fashion as to allow mathematical simulation and empirical investigation will require a high degree of theoretical specification and research sophistication. Apart from precise definition and adequate scaling of the relevant variables, a specification of the transfer functions involved will be needed (Kappas 2001). Most likely, linear functions, as exemplified by the rules in Anderson's (1989) integration model, will not provide an appropriate model of the functions in all cases. Weighted integration in the form of dynamically adaptive differential activation and inhibition thresholds are likely to be more appropriate modelling principles as compared with fixed weights in regression or decision structures.
As suggested earlier (Scherer 2000), we may need to adopt nonlinear dynamic systems approaches for modelling complex, context-bound nonlinear functions, rather than our classic statistics and modelling tool boxes, which are largely based on linear functions. Below, two brief examples of the use of this approach drawn from the earlier chapter is provided (for greater detail and references see Scherer 2000).
As shown above, an emotion episode consists of a certain level of synchronization of the organismic subsystems, driven by the results of the appraisal checks. Given the recursive nature of the system processes, the appraisal process is affected by prior changes in the different system components (figure 2). For example, feedback of increasing arousal from the physiological system or changes in the motivational system can affect attention deployment or change perception and judgment thresholds. This suggests that emotion episodes can be profitably considered as processes of self-organization among neurophysiological systems that are mapped into cultural meaning structures. Specifically, appraisal is expected to drive the synchronization of coupled neurophysiological oscillators in a process of entrainment by networks of CNS activity.
It is important to note that the various subsystems of the organism display highly variable response (attack and decay) and regulation characteristics. Therefore, the components of the emotion process are unlikely to correlate in a direct, linear fashion. Instead, we expect lagged covariation, nonlinearity, differential damping and many other aspects of complex synchronization. In consequence, nonlinear dynamics systems theory (such as self-organization theory and chaos theory) are optimally suited to develop models and measurement procedures for synchronization (e.g. discreteness within continuity, order within chaos, simplicity within complexity, nonlinear dynamics, emergence of structure, self-organization, complex coupling, synchronization, entrainment of subsystems, sensitivity to initial values, sudden change). Two concepts from this literature are particularly interesting for emotion modelling—attractor and hysteresis.
Synchronized subsystems or oscillators can be ‘drawn into’ or entrained to specific synchronized modes that have a tendency to be more stable than other, continuously changing states. The existence of such attractor basins has been demonstrated for many states in biological systems (e.g. sleep, respiration). The notion of forced synchronizations in coupled systems in regulatory physiological systems can be extended to the role of psychological factors serving as drivers of underlying biological oscillators (Redington & Reidbord 1992).
Conceptualizing emotions as attractor states of limited duration in the service of rapid adaptation to changed conditions correspond to the evolutionary origin of the emotions, which is generally seen in the facilitation of adaptation to emergency situations (Nesse 2009). Thus, Haken (1991) suggested that physiological systems, in their normal functioning, are close to instability points, as this allows the system to adapt to new situations rapidly by transiting to more synchronized attractor states (Freeman 1992). A similar architecture might underlie the emotion-driving appraisal in humans. Appraisal results may drive coupled psychophysiological oscillators that, as a consequence, undergo a state transition from previously chaotic behaviour leading to synchronization through increased coupling and mutual entrainment. Different patterns of appraisal results may produce sets of order parameters that ‘push’ the synchronization process into the direction of specific attractor basins. The emotion episode will come to an end when the specific attractor loses its force owing to changing appraisal results, which leads to a steady weakening of the synchronization, a decrease in the degree of coupling of the component systems and a transition back to a more or less chaotic state (or, in some cases, owing to a new set of organized appraisal input, moves, by an abrupt transition, to a new attractor state representing another type of modal emotion).
What is the role of causality in the mechanisms suggested here? Because of the constant recursivity of the process, the widespread notion of linear causality (a single cause for a single effect) cannot be applied to these mechanisms. Appraisal is a process with constantly changing results over very short periods of time and, in turn, constantly changing driving effects on subsystem synchronization (and, consequently, on the type of emotion). Specific appraisal profiles that move subsystem synchronization into an attractor basin that characterizes a modal emotion episode is the end result (in terms of a time slice) of sequential information accumulation and refinement, yielding a relatively stable outcome. Appraisal initiates and drives the synchronization process but is in turn also affected by the changes in the various subsystems and their synchronization. Thus, as is generally the case in self-organizing systems, there is no simple, unidirectional sense of causality (see also Lewis 1996).
Emotion states viewed as attractor basins of coupled oscillators can be mapped into mentally represented meaning systems. However, as mentioned earlier, emotion categorization and labelling are quite separate phenomena, related to emotional communication and the representation of cultural knowledge structures. These processes cannot be understood in the sense of oscillating systems but require categorical analysis. Most likely, the results of categorization and labelling, once having occurred, will influence appraisal and emotional regulation, and thus constitute an important input to the synchronization process. One of the major issues for the future is to understand the relationship between continuous time series or oscillations, on the one hand, and more stable, discrete states amenable or accessible only through categorization, on the other. The notion of attractors—describing relatively stable patterns of repeated coupled oscillations with similar characteristics—may be useful in the context of describing ‘modal emotions’, categories that are generally labelled with specific words or expressions (Scherer 1994).
A second illustration of the use of dynamic nonlinear systems models draws from chaos theory, in particular the branch known as catastrophe theory (Stewart & Peregoy 1983; Abraham & Gilgen 1995; Sprott 2003). The notion of hysteresis, referring to a nonlinear part of a function that is inaccessible and that doubles back in its course allows explaining certain dynamic aspects of emotion processes. Figure 6 plots the relationship between the degree of frustration (e.g. owing to someone or something preventing us from reaching a highly relevant goal) and the anger elicited as a function of its increase over time. A hysteresis function predicts that the intensity of anger will change abruptly for specific degrees of frustration, whereas a linear function assumes continuously rising anger with increasing frustration. This explains why the point of departure is such an essential part of nonlinear dynamic functions. For example, if I start out with a little frustration, and consequently a little anger, there is a point where rising frustration will make my anger jump to a much higher level without going through any other intermediate stages (e.g. the explosive flaring up of anger). Conversely, if one starts from a high level of anger, produced by strong frustration, and slowly calms down because of a reduction in frustration, it will take much less frustration before a drop to the lower level will occur (owing to the fold in the function). Current conceptualizations of emotion do not allow the prediction of such well-known emotional characteristics.
An illustration of hysteresis in the frustration–anger relationship.
Hysteresis is also an essential feature of much more complex chaos models, even though these contain more dimensions that need to be taken into account in predicting the underlying phenomenon. Zeeman (1976) provided a beautiful example of the application of catastrophe theory to a classic behavioural phenomenon in ethology—the response conflict between attack and flight in a dog faced with an adversary with unknown strength. He postulated a control space consisting of the dimensions controlling the behaviour of the animal (the opposing tendencies to fight or flee) and a response surface upon which the respective position of the animal in the control space is projected. As the motivational tendencies of the dog on the two dimensions change, the change in behaviour of the dog can be plotted as a path on the response surface. The presence of a bifurcation (or cusp, owing to hysteresis) produces a fold in the response surface, which helps to account for a number of well-known characteristics of a dog's emotional behaviour, for example, abrupt changes from one moment to another.
One can easily adapt Zeeman's two-dimensional model to show how dynamic emotion responses can be modelled on the basis of appraisal results. Figure 7 shows a control space with two of the major dimensions postulated by all appraisal theorists: the goal conduciveness gradient (Factor A), characterized by the appraisal of the probability of reaching one's goal (to the left) or not reaching one's goal (to the right). Factor B, control or power, represents the appraisal of the degree of coping potential available to the organism to deal with a given situation (ranging from very little power towards the back to high power towards the front). Using the highly convergent predictions of appraisal theorists (Scherer 1999, 2001; Ellsworth & Scherer 2003), one can project various positions of this two-dimensional control space onto the behaviour or response surface and describe specific regions on this surface by emotion labels (used here as shorthand specifications of the corresponding complex multimodal synchronization patterns described above). Thus, as predicted by most appraisal theorists, anger is predicted to occur in conditions where the organism perceives a goal to be obstructed but considers having sufficient coping potential to deal with the block. What nonlinear dynamic modelling can add to the straightforward appraisal theoretical account is again owing to the bifurcation in the behaviour surface. Using the model, one can imagine how someone faced with adversity, i.e. seeing one's goal attainment increasingly threatened but perceiving a fairly high degree of coping potential or power, will move through states of hope and increasing determination to a point where a sudden switch to anger or even rage will occur. The fact that only a very small change in the perceived obstructiveness and power needs to precede the sudden change is explained by the bifurcation in the behaviour surface. Another example might be that of someone who appraises goal conduciveness as fairly low but increasingly evaluates coping potential to increase. This person moves from anxiety over resentment to a sudden change, owing to the bifurcation, to determination. Again, while the increment in the perceived coping potential is relatively small, the change in the resulting emotion quality is quite dramatic.
An illustrative nonlinear dynamic model of appraisal-driven emotion responses.
Appraisal theorists postulate many more than two underlying control factors and thus the locations of the emotion labels on the behaviour surface are quite approximate. Obviously, the introduction of higher order bifurcation sets in several dimensions allows an enormous variety of outcomes based on a fairly simple control structure. Modelling the dynamic processes involved in the sequence of appraisal changes in this fashion may help appraisal theory to move beyond a rather static prediction of semantic meaning of particularly emotional terms to a more dynamic approach, highlighting the changes that will occur upon incremental changes of the appraisal on particular dimensions. Most importantly, the effects of these changes may be nonlinear. In other words, depending on the region involved and on the combination of the underlying appraisal dimensions, relatively small changes may produce dramatic consequences. Obviously, an empirical investigation of theoretical models of this sort requires a much finer measurement of the appraisal dimensions (e.g. the use of interval scales) as well as a process measurement of appraisal in time rather than retrospective one-point measurement, as has been the case in most appraisal research to date. This is an example of how modelling based on catastrophe theory can guide theoretical and empirical development in one of the central areas of current emotion research. In addition, such modelling promises to do a better job in explaining a number of intuitively obvious characteristics of emotional responses (such as abrupt changes that are difficult to explain by linear functions or dependency of the response on the origin or departure point).
The notion of attractor basins nicely complements the control-response modelling. For example, the regions labelled by emotion terms on the response surface in figure 7 can be seen as attractor basins with auto-organizational properties, representing multidimensional vectors that are differentially affected by changes in the control structure. As mentioned above, the coupling of a normally independent oscillating system is explained by the need for adaptation that results from certain appraisals (represented as positions in the underlying control space in the model). Thus, it is the current state of the control structure that couples the independent oscillators. Of course, changes of the position in the control space must also explain the decoupling of oscillators underlying the travelling of the organism along a path on the response surface over time. However, most physical and biological systems have some degree of inertness built in, requiring a relaxation of prior constraints over some period of time before uncoupling can occur. This characteristic explains the existence of attractor basins that maintain the persistence of a particular state for some time, as reflected in the lingering of some emotions.
Clearly, a more formal attempt to computationally model emergent emotion processes, using current sophisticated tools, and including constraints imposed by what is known by the neural architecture and modelling the dynamic flow of recursive effects, would be of major importance for the further development of theorizing and research in emotion. It would also greatly enhance the development of the CPM. It is hoped that the existence of a validated theoretical model such as the CPM, which encourages neural network modelling, will motivate scholars in the area of nonlinear dynamic modelling to invest in this hitherto neglected domain and to attempt the construction of a computational model of emergent emotion processes. Obviously, one would need to begin by modelling the emotions of a very simple creature, an organismus simplicissimus (Scherer 1984), rather than attempting to start with full-blown human emotions. Given the ever-accelerating increase in both the sophistication of research methods and the resulting understanding of the neural architecture and the dynamic processing underlying our cognitive and affective performances and experiences, the time may be ripe to venture into more complex levels of neural network modelling, including dynamical systems approaches. Unfortunately, the funding policies in the affective computing domain seem to privilege more immediately applicable computational models for agent emotion production and recognition over the basic science concern of building process models to understand complex recursive mechanisms. But, paraphrasing the old adage that nothing is more practical than a good theory, one can venture the prediction that a good computational model of fundamental emotion processes, being able to account for their emergent properties, might well move the practical applicability of computational agent models to a higher level.
One contribution of 17 to a Discussion Meeting Issue ‘Computation of emotions in man and machines’.