|Home | About | Journals | Submit | Contact Us | Français|
The study aimed to explore the humor processing elicited through the manipulation of artistic drawings. Using the Comprehension–Elaboration Theory of humor as the main research background, the experiment manipulated the head portraits of celebrities based on the independent variables of facial deformation (large/small) and addition of affective features (positive/negative). A 64-channel electroencephalography was recorded in 30 participants while viewing the incongruous drawings of celebrities. The electroencephalography temporal and spectral responses were measured during the three stages of humor which included incongruity detection, incongruity comprehension and elaboration of humor. Analysis of event-related potentials indicated that for humorous vs non-humorous drawings, facial deformation and the addition of affective features significantly affected the degree of humor elicited, specifically: large>small deformation; negative>positive affective features. The N170, N270, N400, N600-800 and N900-1200 components showed significant differences, particularly in the right prefrontal and frontal regions. Analysis of event-related spectral perturbation showed significant differences in the theta band evoked in the anterior cingulate cortex, parietal region and posterior cingulate cortex; and in the alpha and beta bands in the motor areas. These regions are involved in emotional processing, memory retrieval, and laughter and feelings of amusement induced by elaboration of the situation.
Mark Twain once stated that, ‘Humor is mankind’s greatest blessing’, as it evokes happy and pleasant feelings in humans (Schopenhauer, 1957). However, we have yet to achieve a theory that can explain all humor experiences (Vrticka et al., 2013; Li, 2016). This study is not concerned with verifying the most ideal theory of humor. Instead, it seeks to adopt the perspective of manipulating artistic drawings to explore the cognitive processes underlying the perception of humorous drawings. The 18–19th century philosophers, Immanuel Kant and Arthur Schopenhauer, believed that humor arises from the violation of our existing patterns of thinking, that is, when we suddenly perceive an incongruity between an idea and reality. This subsequently led to preliminary discussions on the Incongruity Theory (Monro, 1988). After several decades of research, Suls proposed a two-stage process of humor: the Incongruity Resolution Theory, which addressed the issue that ‘incongruity’ alone is unable to generate humor, and that comprehension of incongruity is also required for the cognitive processes of humor (Suls, 1972; Katz, 1993; Ritchie, 1999; Galloway, 2009; Samson et al., 2009). Wyer and Collins further extended this theory, and suggested that incongruity detection and comprehension are not sufficient to fully interpret all humor experiences. For instance, the incongruity detection and comprehension of bad jokes cannot elicit humor and still require the humor elaboration stage (Wyer and Collins, 1992). Hence, they proposed the Comprehension–Elaboration Theory (Wyer and Collins, 1992; Chan et al., 2012), which includes three stages, namely, incongruity detection, incongruity resolution and elaboration of humor (Chan et al., 2012; Chan et al., 2013). Based on the Comprehension–Elaboration Theory, this study manipulated the experimental stimuli (humorous drawings) to influence the main processes underlying humor cognition. The process of humor elicitation through comprehension–elaboration involves the incongruous projection of new signs onto the original objects, which then causes an inconsistency between the manipulations and the initial image impression (Chapman and Foot, 2013). Individuals retrieve past experiences or prior knowledge to form their existing patterns of thinking, which may be violated by external information. When such violations transcend their prior expectations of people, objects or affairs and are interpreted as pleasurable experiences, this leads to feelings of humor (Mulder and Nijholt, 2002; Morreall, 2011; MacHovec, 2012; Perks, 2012).
Incongruity alone is insufficient to explain humor, and the perception of conceptual incongruity evokes event-related potentials (ERPs) related to incongruity, specifically the N270 (Mensen et al., 2014) and N400 components (Coulson and Lovett, 2004; Feng et al., 2014). In their observations on humor, Mensen et al. (2014) found that compared to neutral stimuli, humorous stimuli evoked a negative deflection at N270. Feng et al. (2014) used jokes to examine the cognitive responses of humor and evoked the N400 component, which reflects incongruity detection. Comprehension is the active resolution of incongruity, which involves creating clues through logical thinking or rules based on real experiences. This process has been shown to evoke the N600-800 component (Du et al., 2013; Tu et al., 2014). Du et al. (2013) used a three-stage model to examine the humor responses of joke comprehension, and elicited the N600-800 component. Coulson and Kutas (2001) attempted using word play to differentiate the ERPs evoked during the incongruity detection stage from the incongruity resolution stage. Their results showed that incongruity evoked the N400 component, whereas joke comprehension elicited later slow waves at 500–900ms. An overview of past ERP studies (Coulson and Lovett, 2004; Du et al., 2013; Feng et al., 2014; Mensen et al., 2014) on humorous jokes has mainly focused on observations of N270, N400, N600-800, or slow waves up to 1500ms. The elaboration of humor is the re-interpretation of the situation subsequent to incongruity detection and comprehension, which then elicits humor and feelings of amusement. The study infers that this process is related to the slow waves evoked after incongruity resolution (Chan et al., 2013; Feng et al., 2014; Tu et al., 2014), and the present results from ERP and event-related spectral perturbation (ERSP) experiments to verify this claim (please see subsequent sections of Results and Discussion).
This study attempted to investigate the humor processing elicited by the head portrait drawings of celebrities. A review of ERP waveforms related to facial recognition indicates that N170 reflects facial information processing (Bentin et al., 1996; Olivares et al., 1999; Eimer, 2000; Rossion et al., 2003; Miyoshi et al., 2004; Caharel et al., 2005; Rossion and Caharel, 2011) and emotional facial processing (Batty and Taylor, 2003; Eimer and Holmes, 2007). Regarding the literature related to electroencephalography (EEG) bands, it has been shown that the synchronization of delta and theta bands reflects emotional processing (Knyazev et al., 2009) whereas anterior-right theta and right-hemisphere gamma bands reflect emotional facial processing (Balconi and Lucchiari, 2008; Balconi and Pozzoli, 2009). The study previously performed an EEG to uncover the association between humor and exaggerated visual features. Such visual features allow us to form associations with the original subject matter from different angles, which then generates feelings of amusement or frustration toward the character depicted. In a pilot study, the study explored the association between incongruity and drawing features within the framework of the Comprehension–Elaboration Theory (Wang and Kuo, 2016). The results showed that adding facial deformations or affective facial features elicited humor in participants; adding negative affective features, in particular, elicited greater feelings of humor. Du et al. (2013) used funny jokes to investigate the brain regions related to humor and found that the anterior cingulate cortex (ACC) was involved in breaking mental expectations and forming novel associations. Based on the findings of the above literature and the pilot studies, we performed independent component analysis (ICA) to investigate the activation of various brain regions and used ERSP to observe the responses of different spectral bands. The research hypotheses are as follows. H1: The greater the facial deformations in artistic drawings will apparently affect humor processing and involve distinct processing phases of incongruity, comprehension and elaboration that can be observed by EEG phenomena of ERP and ERSP. H2: Addition of negative affective features in artistic drawings will apparently affect humor processing and involve distinct processing phases of incongruity, comprehension and elaboration more than positive affective ones that can be observed by EEG phenomena of ERP and ERSP. H3: The interaction effect between the greater the facial deformations and the addition of negative affective features in artistic drawings will apparently affect humor processing and involve distinct processing phases of incongruity, comprehension and elaboration that can be observed by EEG phenomena of ERP and ERSP.
In this experiment, 30 participants (15 males, 15 females) were recruited in Taiwan. The mean age was 24.3 years (21–27 years), education level was undergraduate and above, and corrected visual acuity was above 0.8. Considering the abnormal interferences of personality factors (Beck and Steer, 1988; Beck et al., 1988; Watson et al., 1988), the study recruited mental health subjects who do not have any histories of mental illness. The experimental site was located in the NTUST Design Perceptual Awareness Lab (D:PAL). With reference to the oddball paradigm in past literature (Gierych et al., 2005), the present experiment required the participants to focus their attention on counting the number of green spots (oddballs) while also using their remaining attention to elicit humorous responses to the stimuli. The aim was to extract a stronger humorous effect toward the experimental stimuli. This rule implies that participants who gave more than 10% incorrect responses to the number of green spots had to be excluded. Hence, one male participant was excluded from the experiment. This study complied with the Declaration of Helsinki (World_Medical_Association, 2000, 2001, 2013) and was approved by the Institutional Review Board of the Cathay General Hospital. All participants signed informed consents prior to the experiment and were reimbursed with appropriate remunerations. All methods were carried out in accordance with the approved guidelines.
A Latin square experimental design (Winer, 1962; Kiefer and Wynn, 1981; Dean, 2015) involves generating the experimental stimuli based on all combinations of the independent variables. This design allows the impact of a single factor to be extracted from the experiment using a small number of trials, thus enabling us to observe the individual impact of the two independent variables (factors A and B) on the dependent variable. The constraints for this experimental design were that the treatment dimension (n) in each row and column factor had to be identical; a n×n Latin square design was employed, which included n types of treatments in two factors (A1 to An, B1 to Bn). A randomized Latin square was used to solve the problem of counterbalancing the stimuli such that the results would not be influenced by the order of treatment presentation (Dean, 2015). Latin square designs are suitable for all types of experiments and have been extensively applied in EEG experiments, including in auditory, tactile and visual discrimination tasks, mental arithmetic, language, sleep, biofeedback, pain and emotion (Kreitman and Shaw, 1965; Dimond and Farrington, 1977; Philip et al., 1994; Nieuwenhuis et al., 2004; Guilleminault et al., 2006; Gordon et al., 2010; Jensen et al., 2013). A 2×2 Latin square design was employed in this experiment, with four different types of experimental stimuli, to explore the differences between the experimental groups. After repeated presentation of the stimuli, the impact of factor A on the participant was analyzed, i.e. the statistical differences between the experimental groups (A1 and A2) were observed. The impact of factor B on the participant was also analyzed, i.e. the statistical differences between the experimental groups (B1 and B2) were observed.
The head portraits of well-known celebrities were selected in the previous study as drawing stimuli to manipulate the cognitive processes of humor. Purposive sampling is widely used in qualitative research particularly when studies have more than one specific objective (Palinkas et al., 2015). The purposive sampling sample included 59 male and female film or political celebrities each who were topics of discussion and controversy in Taiwan in 2014, with a total of 118 celebrities. These celebrities had been featured in Apple Daily, which is the newspaper with the highest readership in Taiwan. From these, 90 participants selected 10 of the most recognizable male and female celebrities each. Then, a Latin square design (2×2) was used to manipulate the independent variable of two factors: facial deformation and addition of affective features. Creating experimental stimuli of celebrities with facial deformations involved exaggerating (enlarging/shrinking) the sizes of their facial features. The concept originated from the Incongruity Resolution Theory and its derivative Comprehension–Elaboration Theory, which both suggest that incongruity is a necessary process for humor perception (Suls, 1972; Wyer and Collins, 1992). In other words, when participants perceive the humorous and novel image stimuli, they would compare the stimuli with their existing pattern of thinking, thus eliciting incongruity detection and incongruity comprehension. Another aspect of manipulating the experimental stimuli is through the addition of affective features, which were selected from among images in the International Affective Picture System (IAPS). This concept arose from the observation that humor is often regarded as a complex affective process, which is related to positive and negative emotions (Yip and Martin, 2006; Vrticka et al., 2013).
The IAPS is a stimulus set that is frequently used in emotion-related research (Lang et al., 1999; Mikels et al., 2005; Garcia-Molina et al., 2013). It was established by the US National Institute of Mental Health and is a cross-cultural and standardized system for emotion-related research. Using the Self-Assessment Manikin scale (Bradley and Lang, 1994; Morris, 1995), participants were asked to subjectively evaluate the pleasure, arousal and dominance associated with the images in IAPS. Hence, the images have affective consistency and retest reliability and can be widely used in emotion-related research (Lang and Bradley, 2007). Each IAPS image has an affective valence and arousal rating. Presenting these images during the experiment would evoke a similar degree of emotional response, which has been applied and verified in a variety of emotion-related research, such as the theory of affective picture processing (Briggs and Martin, 2009), emotion recognition and visualization (Liu et al., 2010), emotion-regulation choice (Sheppes et al., 2011), affective picture modulation (De Cesarei and Codispoti, 2011; Leite et al., 2012), affective image classification (Machajdik and Hanbury, 2010) and affective picture processing in children (Hajcak and Dennis, 2009).
Figure 1A shows the experimental stimuli: humorous drawings with large/small facial deformation and addition of positive/negative affective feature from IAPS. Each of the four conditions included 20 drawings, providing 80 drawings in total. Using a Latin square (2×2) design for the two independent variables, they were able to observe the individual impact of a single independent variable (Winer, 1962; Kiefer and Wynn, 1981; Dean, 2015) through 40 experimental stimuli in the study (as shown in Figure 1A). Samples in the control group were the normal-sized, recognizable, realistic looking head portraits of the celebrities, which included 20 real pictures and 20 pictorial illustrations (Mayer and Geher, 1996; Mayer et al., 2001); hence, 40 pictures were presented without deformation beyond normal proportions of local facial features and without addition of IAPS affective features. Another 20 pictures of a green spot (oddball stimulus) were also interspersed among the experimental stimuli (Gierych et al., 2005).
In the field of art and design, the act of drawing an actual object always implies a certain extent of simplification (Hsu and Wang, 2010). The creation of artistic drawings will deviate from the original appearance of the object. Therefore, small facial deformation was defined as realistic simplified drawing in the study. The conditions for large facial deformation involved the manipulation of local facial features by enlarging or shrinking them beyond normal proportions. In addition, reference was made to the positive/negative ratings of the emotional valence in the IAPS (Lang et al., 1999; Garcia-Molina et al., 2013) in order to add affective features for the design of the celebrity stimuli (Russell, 2003; Fujimura and Okanoya, 2012), thereby adding a sense of incongruous amusement or strangeness. Added positive affective features included birds, flowers, fruits and ice cream, whereas negative affective features included spiders, skulls, cockroaches and bombs.
The experimental equipment included the Neuroscan EEG recording and analysis system (Scan 4.3.3 & STIM2), which included an electrode cap (Quik-Cap) and a SynAmps2 Amplifier. The 64-channel EEG was recorded in accordance with the International 10–10 system of electrode placement. The experimenter observed the monitor placed on a table with a height of 74cm, ensuring that the center of the monitor was placed within 10–20° of the participant’s line of sight and at a distance of 60–70cm away from the participant.
The experimental procedure is shown in Figure 1B. The experimental design involved the randomized presentation of four types of experimental stimuli, one type of control stimulus and one type of oddball stimulus (image of green spot). At the start of the experiment, participants were given instructions to count the total number of times the green spot was presented. The green spot was randomly presented during the experiment in accordance with the oddball paradigm. After the instructions were given to the subject, a fixation cross (+) would appear, followed by the stimulus. Aside from trials with the green spot, a behavioral question on the degree of humor was presented after each stimulus. The fixation cross would appear again, followed by a response of ‘Yes’ or ‘No’ to a question on humor behavior (e.g. ‘Was this picture humorous?’). Each stimulus was presented per trial, and the experiment lasted for about 20min.
Responses to the behavioral question (degree of humor) were computed for each stimulus. Then, two-way analysis of variance (ANOVA) was performed to test the significance of differences between the four groups of humorous drawings and the control stimuli. This was followed by Bonferroni post hoc comparisons of the influence of the two independent variables, facial deformation and additional affective features to the face, on the degree of humor. EEG analysis was then performed (as shown in Figure 2).
Channel-based EEG analysis was performed for ERP analysis and statistical testing. EEG has very high temporal resolution, and ERP technique is able to reflect the cognitive processes in the brain. Reference was made to the ERP analysis guidelines (Picton et al., 2000; Holder et al., 2010; Hirsch et al., 2013) and ERP experiments related to humorous jokes and facial perception (Bentin et al., 1996; Batty and Taylor, 2003; Coulson and Lovett, 2004; Feng et al., 2014) to analyze the 64-channel EEG signals acquired. ERP analysis first involved marking abnormal segments caused by eye blinks and large body movements. After performing the eye-blink/eye-movement corrections and discarding noise interference, zero-phase shift low-pass filter was applied, and the filter frequency was 30Hz. The continuous EEG recordings were then segmented into epochs of −100 to 1500ms, and linear detrend was applied to remove low-frequency signal drift. Then, epochs with potential amplitudes exceeding±75μV were discarded, followed by baseline removal based on the mean amplitude of the baseline period. Finally, averaging and summation were performed to remove noise and to produce the ERP waveforms. The overall trial rejection rate due to movement, blink and muscular artifacts was 4.2% (s.d.=0.048; SE=0.009) across all subjects and conditions. The mean rejection rate for each condition was 4.6% (s.d.=0.042; SE=0.009) for small deformation, 4.8% (s.d.=0.046; SE=0.009) for large deformation, 4.6% (s.d.=0.04; SE=0.009) for addition of positive affective feature, 4.8% (s.d.=0.044; SE=0.009) for addition of negative affective feature and 4.2% (s.d.=0.036; SE=0.008) for the control group. In the observation of EEG data for each independent variable, there were at least 30 trials for each condition from each participant.
Based on literature related to facial signals and humor, we observe the components N170 (130–180ms), N270 (220–300ms), N400 (350–450ms), N600-800 (600–800ms) and slow waves beyond 900ms. We calculated the statistical differences for single channel maxima. The ‘humorous vs non-humorous’ conditions were used to test whether there were statistical differences among the components (P<0.05) (Figures 3 and and4).4). In accordance with the 2×2 Latin square design, the following conditions were tested: ‘large deformation vs control group’, ‘small deformation vs control group’, ‘addition of positive affective feature vs control group’ and ‘addition of negative affective feature vs control group’. Statistical testing was then performed on the single channel maxima between the ERP components of different groups using the Wilcoxon signed-rank test (P<0.05) (Figures 5 and and6).6). To account for the issue of false positives that often occur in ERP studies (Pernet et al., 2015), the study adopted the false discovery rate (FDR) controlling procedures recommended by statisticians Benjamini and Yekutieli (Benjamini and Yekutieli, 2001; Benjamini et al., 2006). This procedure effectively controls the traditional familywise error. Finally, the aoctool function in the MATLAB Statistics toolbox was used to calculate the covariance, to examine the statistical correlation between the earlier and later components. The corr function was used to calculate the Pearson’s correlation coefficient.
After applying 1Hz high-pass filtering and 100Hz low-pass filtering on the original EEG raw data, the sampling rate of the filtered EEG signals was down-sampled to 250Hz to reduce data storage and analysis time. Infomax ICA algorithm (Bell and Sejnowski, 1995; Makeig et al., 1997) from the EEGLab toolbox (Delorme and Makeig, 2004) was employed to effectively remove noises in the EEG recording (e.g. eye movement and blinking, single-electrode noise, muscle activities) (Giannakopoulos et al., 1999; Jung et al., 2000; Jutten and Karhunen, 2004). Accordingly, the filtered EEG signals were separated into independent components corresponding to the different brain regions. The weights from each independent component projecting toward the channels were used to obtain the scalp map of the independent components, in order to investigate the activated brain regions across all participants.
Subsequently, DIPFIT2 routines (Oostenveld and Oostendorp, 2002) were used to compute the equivalent dipole locations of the independent components in the brain for further analysis. Then, K-means clustering (MacQueen, 1967) was performed on the independent components of all participants, simultaneously considering weight on scalp map, power spectrum density and equivalent dipole location for clustering into 13 common brain regions: frontal midline region, left frontal region, right frontal region, central midline region, left temporal region, right temporal region, left motor region, right motor region, parietal region, left occipital region, occipital midline region, right occipital region and limbic system region. Finally, the Talairach xyz coordinates (Lancaster et al., 2000; Gramann et al., 2010) of the region centroids were mapped to the corresponding Brodmann areas (Brodmann, 2007) in order to understand the specific brain functions of those regions.
Because ERP analysis is unable to detect the influence of each frequency rhythm in the EEG data, this study employed the ERSP technique (Delorme and Makeig, 2004) provided in the EEGLab toolbox. ERSP uses time as the horizontal axis and frequency as the vertical axis, in order to observe the spectral perturbations in the brain evoked by specific events (Lin et al., 2015). First, the analytical procedure segmented the continuous EEG recordings into epochs (−1000 to 3000ms). Then, wavelet transformation (Mallat, 1989; Burrus et al., 1997) was applied to convert the signals from time series into temporal and frequency data. The data were then normalized according to the spectral power of the baseline period. Finally, the spectral perturbations from all trials were averaged to obtain the ERSP image. This allowed us to observe the Delta (1–3Hz), Theta (4–7Hz), Alpha (8–13Hz), Beta (14–30Hz) and Gamma band (>30Hz) spectral powers and map them to the corresponding neurophysiological significance of each brain region.
ERSP intensity is directly proportional to color brightness; a darker color implies a stronger degree of spectral perturbation. Following stimuli presentation, ERSP was presented in red if EEG spectral power was increased or blue if the spectral power was decreased, while compared to baseline spectrum. The green portions in the ERSP image indicate that EEG perturbations were not significantly different from baseline, implying the absence of increased or decreased activities within that particular period (Wei et al., 1998; Pfurtscheller and Da Silva, 1999). The parametric testing was performed to determine the statistically significant differences between independent groups (P<0.01).
As shown in Figure 1C, Facial deformation and affective features added to the face had significant effects on humor behavior rate. Two-way ANOVA of behavioral responses indicated a significant main effect of facial deformation [F(2,58)=182.88, P<0.001]. Post hoc comparisons indicated that the humor behavior rate of large facial deformation was greater than that of small facial deformation, and this difference was significant (t=3.73, P<0.001). The humor behavior rate of small deformation was greater than the control group (no deformation), and the difference was significant (t=14.20, P<0.001).
The main effect of adding affective features [F(1,29)=7.76, P<0.01] showed a significant difference. Post hoc comparisons indicated that the humor behavior rate of negative affect was significantly greater than the control group (no addition of IAPS affective features) (t=16.38, P<0.001). However, there were no interactive effects between facial deformation and addition of affective features on the degree of humor [F(1,29)=0.01, P=0.93].
Wilcoxon signed-rank test was performed to analyze the 64-channel EEG data. The experimental results showed that for ‘humorous vs non-humorous’ drawings, there were statistically significant differences for N170 (130–200ms), N270 (220–300ms), N400 (350–450ms), N600-800 (600–800ms) and N900-N1200 (900–1200ms), respectively. Figure 3 shows the statistical analysis of ERPs evoked by humorous vs non-humorous drawings among all participants. The N170 component showed statistically significant differences for channels AF4, F1, FZ, F2, FCZ, FC2, P7, CP6, TP8, P4, P6, P8, PO6 and PO8, respectively. The N270 component showed statistically significant differences for channels FP1, FPZ, FP2, AF3, AF4, F7, F5, F3, F1, FZ, F2, F4, F6, FT7, FC3, FC1, FCZ, FC2, FC4 and C2, respectively. The N400 component showed statistically significant differences for channels FP1, FPZ, FP2, AF3, AF4, F1, FZ, F2, F4, F6, F8, FC1, FCZ, FC2, FC4, C2, P3, P1, PZ and P2, respectively. The N600-800 component showed statistically significant differences for channels FP2, AF4, F7, F2, F4, F6, F8, FC2, FC4, FC6, FT8, C2, C4, C6, T8, CP6, TP8, FT7, FC5, C5, TP7, CP3, CP1, P5, P3, P1 and PZ, respectively. The N900-1200 component showed statistically significant differences for channels FP1, FPZ, FP2, AF3, AF4, F1, FZ, F2, F4, F6, F8, FC1, FCZ, FC2, FC4, FC6, FT8, C2, C4, C6, T8, CP6, TP8, P8, P1 and PZ, respectively. The significant differences among the electrodes above were all subjected to FDR correction (P<0.05).
Figure 4A shows the average 64-channel ERP waveforms for ‘humorous vs non-humorous’ from one subject. Statistical analysis of ‘humorous vs non-humorous’ indicates that the electrodes that were significant for all ERP components were AF4, F2 and FC2, which were all located in the right prefrontal region. The FDR-corrected P values for all components were all smaller than 0.0087 (P<0.05) and the electrode of F2 was adopted as an instance for Figure 4B. It shows that N170, N270, N400, N600-800 and N900-1200 were evoked in all participants at the electrode F2 and presented significant differences between humorous vs non-humorous drawings (P<0.05). Figure 4B shows the mean amplitude and standard deviation at F2 for all participants, which are as follows. N170: humorous (Mean = −3.445; s.d.=1.894; SE=0.352), non-humorous (Mean=−2.775; s.d.=2.176; SE=0.404); N270: humorous (Mean=−4.253; s.d.=2.253; SE=0.418), non-humorous (Mean=−3.067; s.d.=2.489; SE=0.462); N400: humorous (Mean=−4.438; s.d.=2.746; SE=0.509), non-humorous (Mean=−3.085; s.d.=2.455; SE=0.455); N600-800: humorous (Mean=−5.012; s.d.=2.573; SE=0.478), non-humorous (Mean=−3.846; s.d.=2.219; SE=0.412); and N900-1200: humorous (Mean=−3.594; s.d.=2.105; SE=0.390), non-humorous (Mean=−2.269; s.d.=1.471; SE=0.273).
The statistical correlation among N270, N400, N600-800 and N900-1200 evoked by humorous vs non-humorous drawings was calculated. The highest to the lowest correlation values were as follows: the correlation coefficient between N400 and N600-800 was the highest at R=0.838 (P<0.001). The correlation coefficient between N270 and N400 was R=0.741 (P<0.001). The correlation coefficient between N400 and N600-800 was R=0.617 (P<0.001). The correlation coefficient between N270 and N600-800 was R=0.598 (P<0.001). The correlation coefficient between N270 and N900-1200 was R=0.536 (P<0.001). The correlation coefficient between N600-800 and N900-1200 was R=0.502 (P<0.001).
Wilcoxon signed-rank test was performed to analyze the 64-channel EEG data. The results showed that when the experimental stimuli were ‘large deformation vs control group’ and ‘small deformation vs control group’, there were statistically significant differences for N270, N400, N600-800 and N900-1200. The statistically significant electrodes are shown in Figure 5A. All electrodes showed statistically significant differences for the previously mentioned components (FDR-corrected P<0.05). The most significant electrode was F2, and the FDR-corrected P values of the four components were all smaller than 0.0352. Hence, the mean amplitude of the most significant electrode F2 was analyzed. The top panel in Figure 5B shows the average ERP waveform for all participants. The bottom panel in Figure 5B shows the mean amplitude and standard deviations of the F2 electrode for all participants, which are as follows: N270: large deformation (Mean=−4.542; s.d.=2.305; SE=0.428), small deformation (Mean=−3.854; s.d.=2.214; SE=0.411), control group (Mean=−3.115; s.d.=2.714; SE=0.504); N400: large deformation (Mean=−4.761; s.d.=2.727; SE=0.506), small deformation (Mean=−4.098; s.d.=2.692; SE=0.500), control group (Mean=−2.879; s.d.=2.991; SE=0.555); N600-800: large deformation (Mean=−5.221; s.d.=2.745; SE=0.509), small deformation (Mean=−5.377; s.d.=2.448; SE=0.454), control group (Mean=−3.505; s.d.=2.569; SE=0.477); N900-1200: large deformation (Mean=−3.694; s.d.=1.985; SE=0.368), small deformation (Mean=−3.604; s.d.=2.027; SE=0.376), control group (Mean=−2.546; s.d.=1.783; SE=0.331).
The results showed that when the experimental stimuli were ‘positive affect vs control group’ and ‘negative affect vs control group’, there were statistically significant differences for N270, N400, N600-800 and N900-1200. The statistically significant electrodes are shown in Figure 6A. All electrodes showed statistically significant differences for the previously mentioned components (FDR-corrected P<0.05). The most significant electrode was F2, and the FDR-corrected P values of the four components were all smaller than 0.0313. Hence, the mean amplitude of the most significant electrode F2 was analyzed. The top panel in Figure 6B shows the average ERP waveform for all participants. The bottom panel in Figure 6B shows the mean amplitude and standard deviations of the F2 electrode for all participants, which are as follows: N270: positive affect (Mean=−4.003; s.d.=2.240; SE=0.416), negative affect (Mean=−4.327; s.d.=2.279; SE=0.423), control group (Mean=−3.115; s.d.=2.714; SE=0.504); N400: positive affect (Mean=−4.489; s.d.=2.967; SE=0.551), negative affect (Mean=−4.600; s.d.=2.566; SE=0.476), control group (Mean=−2.879; s.d.=2.991; SE=0.555); N600-800: positive affect (Mean=−4.766; s.d.=2.656; SE=0.493), negative affect (Mean=−5.962; s.d.=2.534; SE=0.470), control group (Mean=−3.505; s.d.=2.569; SE=0.477); and N900-1200: positive affect (Mean=−3.512; s.d.=2.068; SE=0.384), negative affect (Mean=−3.950; s.d.=1.814; SE=0.336), control group (Mean=−2.546; s.d.=1.783; SE=0.331).
Observations of all the common brain regions revealed that there were significant differences between humorous and non-humorous group (P<0.01) in Theta-band power in the ACC (BA32), parietal region (BA7) and posterior cingulate cortex (PCC, BA30), as well as in Alpha- and Beta-band spectral power in the left motor region. ERSP results of humorous and non-humorous drawings indicate that there were significant differences between the two in the ACC at 4–6Hz during 200–400 and 600–800ms and at 4–7Hz during 1700–2000ms, in the parietal region at 4–8Hz during 500–1400ms, in the PCC at 3–7Hz during 0–250 and 600–900ms, and in the left motor region at 7–22Hz during 1250–2000ms (Figure 7A–D).
As shown in Figure 6A–F, both facial deformation and addition of affective features to the faces evoked significant Theta-band in the ACC, parietal region and PCC (P<0.01). ERSP results of facial deformation are shown in Figure 8A–C. In the ACC, there were significant differences between the control group and large deformation at 3–6Hz during 250–500ms and at 4–9Hz during 1600–2250ms. There were also significant differences between the control group and small deformation at 3–5Hz during 700–1000ms, at 5–8Hz during 1300–1500ms and at 4–7Hz during 2250–2500ms. In the parietal region, there were significant differences between the control group and large deformation at 4–9Hz during 500–900ms and between the control group and small deformation at 5–8Hz during 500–800ms. In the PCC, there were significant differences between the control group and large deformation at 4–7Hz during 400–1250ms and between the control group and small deformation at 3–6Hz during 600–750 and 1600–2000ms.
ERSP results for the addition of affective features are shown in Figure 8D–F. In the ACC, there were significant differences between the control group and positive affective features at 3–6Hz during 250–1250ms and between the control group and negative affective features at 3–6Hz during 250–500 and 750–1250ms. In the parietal region, there were significant differences between the control group and positive affective features at 4–9Hz during 500–1500ms and between the control group and negative affective features at 4–9Hz during 500–1000ms. In the PCC, there were significant differences between the control group and positive affective features at 3–7Hz during 0–400, 600–1250, 1750–2000 and 2250–2500ms and between the control group and negative affective features at 4–7Hz during 600–1250ms.
This study aimed to examine the cognitive processes that underlie the elicitation of humor by artistic drawings. The experiment manipulated the artistic drawings of head portraits by adding facial deformations (large vs small deformation) and affective features (positive vs negative affect) to explore the cognitive processing of humorous drawings. ERP analysis was performed to verify humor processing of incongruity, comprehension and elaboration of humor. ERSP analysis was performed to verify the frequency responses of brain regions toward humor processing of incongruity, comprehension and elaboration of humor.
The behavioral data indicated that compared to the control group, the humor behavior rates toward stimuli with facial deformation and the addition of affective features were significantly higher. Large deformation was more humorous than small deformation, and small deformation was more humorous than the control group. The addition of positive affective features was more humorous than negative affective features, and the addition of negative affective features was more humorous than the control group. Humor is the transmission of a unique emotion that is closely related to a sense of joy. Different scholars have classified this emotion as feelings of amusement and entertainment (Martin, 2010) and have claimed that humor evolved from mock-aggressive play in apes (Morreall, 1986, 2011). The hints of harmless jesting in humor provide humans with the thrill of attacking others. This type of mockery elicits feelings of humor (Zillmann and Bryant, 1980) or induces laughter, as participants understand the jokes and jests that are implicit in these behaviors. Therefore, the behavioral results indicate that the incongruous components in the humorous drawings were closely related to the three stages of incongruity detection, comprehension and humor elaboration reflected in the EEG signals that link to the different humor theories specified in the previous section of Introduction.
In the comparison of humorous vs non-humorous stimuli, this study found that there were significant differences in N170 at frontocentral and lateral posterior temporal sites. Past studies have found that N170 evoked at the occipitotemporal region reflected holistic facial processing, especially in the right hemisphere (Bentin et al., 1996; Rossion et al., 2003). As the stimuli used in this experiment were all celebrities, the participants were familiar with the facial stimuli. Hence, the N170 component could still be evoked despite the addition of facial deformation or affective features. This was due to holistic facial recognition and is consistent with the results of past literature. Blau et al. have found that N170 is related to emotional facial expression, which supports the experimental results.
This study found that N270 was significantly different for large facial deformation and the addition of negative affective features compared to the control group (non-deformation), and the difference was mainly reflected in the prefrontal regions. As the images of celebrities were the target of experimental manipulation, the resulting incongruity in facial recognition evoked this particular component (Ortega et al., 2008). N270 has been shown to be related to feedback-related negativity (FRN) and can be used to detect visual processing and perceptual conflict (Wang et al., 2002; Jia et al., 2007; Proverbio and Riva, 2009). Visual processing experiments have shown that N270 reflects the processing of the supra-modal conflict system (Wang et al., 2002). Furthermore, visual recognition experiments have shown that images that require greater comprehension would evoke a larger N270 component during early recognition (Proverbio and Riva, 2009). A recent study has also indicated that in addition to being related to FRN, N270 was also related to expectedness (Yeung and Nieuwenhuis, 2009; Alexander and Brown, 2011). These support the finding that the normal-sized celebrity images, the use of celebrity head portraits with exaggerated facial deformation and added affective features in this study led to disparities with the participants’ expectations, thereby evoking this ERP component.
In this experiment, humorous drawings with large facial deformation and the addition of positive and negative affective features had a significant impact on not only N270 but also N400. N400 is related to inconsistencies in expected information processing. Compared to old scientific innovations, novel scientific innovations evoked a larger N400 component. Novel scientific innovations involve retrieving prior experiences to generate novel solutions. In addition to causing inconsistencies in cognition, they lead to new comprehension (Luo et al., 2013). This experiment used the exaggeration of facial features and addition of accessories to change the participants’ existing patterns of thinking about celebrities. The EEG data indicate that the stimuli generated inconsistencies in perception; however, the participants were able to comprehend the new stimuli (new images of celebrities). Past research using functional magnetic resonance imaging (fMRI) to examine the humor responses of participants when viewing nonsense cartoons and funny stories found that the frontal/prefrontal cortex was involved in humor comprehension (Samson et al., 2009; Chan et al., 2012). This finding is related to the significant regions for N400 observed in the experimental result shown in Figures 5A and and6A6A.
N600-800 showed significant differences in the temporo-prefrontal regions for humorous vs non-humorous drawings, groups with large deformation, and the addition of positive and negative affective features for the study. Marinkovic observed using magnetoencephalography (MEG) that the left-dominant temporo-prefrontal areas were involved in integration difficulty (Marinkovic et al., 2011). A previous experiment on humorous jokes showed that there was an inconsistency in comprehension prior to humor elicitation, and this comprehension process evoked the N600-800 component (Du et al., 2013). Humor comprehension in word play or cartoons mostly evoked waveforms at 500–800ms. Past literature has shown that N500-800 is associated with N400 and reflects the comprehension process of humorous surprise in cartoons (Tu et al., 2014). Researchers have suggested that negative component of the longer latency following N400 is similar to the N400 component and could be regarded as a later response (a second N400) (Shibata et al., 2009). The claims support the correlation coefficients measured among the different components in the experiment (please refer to ‘ERP results of humorous vs non-humorous drawings’ subsection). Moreover, the N600-900 component is frequently associated with complex and incongruous drawing stimuli (West and Holcomb, 2002).
In the study of N900-1200 showed significant differences in the frontal lobe for humorous vs non-humorous drawings. It found that N900-1200 was evoked in the right frontal region for stimuli with large deformation and the addition of positive and negative affective features. The neural circuit of laughter is subserved by the frontal lobe, supplementary motor area and subcortical nucleus, which are responsible for fulfilling different functions (Takeda et al., 2010). Marinkovic observed using MEG that the right prefrontal region is sensitive to funniness at 900–1150ms (Marinkovic et al., 2011). Patients with lesions in the right frontal cortex have difficulties in appreciating humor (Shammi and Stuss, 1999; Takeda et al., 2010). Tu et al. also observed that deflections evoked at 1000–1600ms were related to the formation of novel associations. These findings support the deconstruction of humor components using artistic drawings in the experiment and reflect stage of humor elaboration.
The study used ERSP techniques to examine the brain regions activated by humorous drawings. Drawing stimuli with large facial deformation and addition of negative affective features evoked theta band in the ACC (BA32), parietal cortex (BA7) and PCC (BA30). Humorous drawings also showed significant differences in the spectral powers of alpha and beta bands in the left motor region (BA4).
Previous studies have shown that the activation of the ACC played a crucial role in the salience network, reflecting the novelty and frequency of occurrence of stimulus salience (Downar et al., 2002; Ham et al., 2013). Novelty is the quality of being new, unusual and something that has not been experienced before and so is interesting (Sinclair, 1987). fMRI study also found that word play or jokes (Nie Qi-Yang, 2012) and cognitive conflicts (Fritz and Dreisbach, 2013) cause the activation of the ACC. Immanuel Kant suggested that the elicitation of humor must involve a component of absurdity, and that humor is an affection arising from the dissipation of strained expectations. According to Roger Scruton, amusement can be described as an attentive demolition of a person, which uses a variety of methods to devalue them, thereby forming feelings of superiority and happiness (Deigh, 2013). Humor is a complex high-order cognitive and emotional process (Vrticka et al., 2013; Clark et al., 2015), and the reported theta synchronization in the ACC is associated with emotional processing. Aftanas et al. (2003) observed theta synchronization dysregulation in the ACC in alexithymic and non-alexithymic subjects viewing affective pictures. They support that the incongruous components of the artistic drawings in the study elicit emotional responses. Devinsky claimed that the anterior cingulate gyrus provides emotional consciousness to an individual’s experience, which also involves the frontal cortex (Devinsky et al., 1995).
The study found that theta rhythm was activated in the parietal region, which has been observed in past literature to be associated with memory retrieval (Jacobs et al., 2006) and working memory functions (Sauseng et al., 2005). Incongruity detection activates memory retrieval in the brain while working memory is used to perform incongruity resolution and comprehension. The superior parietal cortex plays a critical role in the manipulation of information in working memory (Koenigs et al., 2009). Chan et al. (2013) used jokes to explore the brain regions involved in the incongruity resolution stage and found that the parietal lobe showed greater activation. The emotions of surprise caused by features that are inconsistent with psychological expectations activate the temporoparietal junction and anterior superior temporal cortex (Vrticka et al., 2013; Clark et al., 2015). The study found the activation of theta rhythm in different brain regions. Interregional theta synchronization was regarded as being related to the integration of functions in different brain regions (Sauseng et al., 2010). The parietal region has involved in cognitive integration (Wang et al., 2016).
The PCC is activated when evaluating the quality of practical jokes (Cannon et al., 2008) and is involved in self-reflection and judgments about others (Johnson et al., 2002; Ochsner et al., 2004). The pilot study showed that the addition of a negative affective image enabled the participants to experience the effect of mocking and attacking others, which elicited feelings of humor (Wang and Kuo, 2016). It is consistent with past findings that hostile and non-hostile jokes activated different brain regions, which reflected different cognitive functions (Chan et al., 2016); Talk et al. (2004) observed that the theta rhythm in the PCC might be related to the hippocampus, which is critical for processing of learning. Another study claimed that pleasant and unpleasant words activated the PCC, and the PCC mediates the interactions of emotional and memory-related processes (Maddock et al., 2003).
Aside from activating the ACC and parietal region, the incongruous drawing components led to the detection of alpha and beta power in the left motor region (BA4) when the participants were viewing humorous drawing stimuli of the experiment. Past research on mirthful laughter induced by funny stimuli and visual humor (sight gags) has shown that funny cartoons activate the motor regions and the emotion-related ACC (Mobbs et al., 2003; Caruana et al., 2015). Krolak-Salmon observed that the supplementary motor area is involved in detecting happiness (Krolak‐Salmon et al., 2006). Herbert Spencer claimed that laughter is an ‘economic phenomenon’, wherein surplus energy is released through laughter to regulate the mobilized energy caused by false expectations (Meyer, 2000; Buijzen and Valkenburg, 2004). The expressions of laughter can be explained as a form of pressure-relief for the nervous system (Wilkins and Eisenbraun, 2009; Raskin, 2012). The nervous system connects the sense organs and the muscles; laughter involves the motor areas (Foki et al., 2007). It was observed in this experiment that participants showed motor area alpha desynchronization for incongruous drawings. The alpha desynchronization phenomenon has been observed in the literature (Pfurtscheller and Aranibar, 1977; Southgate et al., 2009). The decrease of beta band is also related to motor behaviors (Pogosyan et al., 2009).
This study found that adding facial deformations and affective features to artistic drawings had an influence on humor processing of incongruity, comprehension and elaboration, which can serve as a reference for future EEG research on humor artistic drawing. The results verify that humorous drawings evoked larger N170, N270, N400, N600-800 and N900-1200 components; theta rhythm in the ACC, parietal region and PCC; and alpha and beta rhythms in the left motor area. As the emotional responses induced by humor is very complex, this study attempted using EEG to decompose the three stages of incongruity detection, comprehension and humor elaboration in the components of humorous drawings. Each type of EEG phenomena of ERP and ERSP could explain each stage in the Comprehension–Elaboration Theory: N170 indicates facial information processing; N270 indicates incongruity detection; N400 indicates incongruity comprehension; N600-800 indicates the late response of incongruity comprehension; and N900-1200 indicates the feelings of humor and amusement produced during the elaboration of humor following incongruity comprehension. Activation of the ACC reflects incongruity comprehension of humor and emotional responses; the parietal region and PCC reflect the incongruity comprehension in humor; alpha and beta rhythms in the motor areas reflect laughter induced by the perception of humor elaboration or pressure relief.
Editage is appreciated for the professional English language editing assistance.
This work was supported by the Aim for the Top University Project, Ministry of Education, Republic of China (Taiwan) [105H451710].
Conflict of interest. None declared.