PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of wtpaEurope PMCEurope PMC Funders GroupSubmit a Manuscript
 
J Neurosci. Author manuscript; available in PMC 2010 September 3.
Published in final edited form as:
PMCID: PMC2839485
EMSID: UKMS28922

INTERNAL AND EXTERNAL FEATURES OF THE FACE ARE REPRESENTED HOLISTICALLY IN FACE -SELECTIVE REGIONS OF VISUAL CORTEX

Abstract

The perception and recognition of familiar faces depends critically on an analysis of the internal features of the face (eyes, nose, mouth). We therefore contrasted how information about the internal and external (hair, chin, face-outline) features of familiar and unfamiliar faces is represented in face-selective regions. There was a significant response to both the internal and external features of the face when presented in isolation. However, the response to the internal features was greater than the response to the external features. There was significant adaptation to repeated images of either the internal or external features of the face in the FFA. However, the magnitude of this adaptation was greater for the internal features of familiar faces. Next, we asked whether the internal features of the face are represented independently from the external features. There was a release from adaptation in the FFA to composite images in which the internal features were varied but the external features were unchanged, or when the internal features were unchanged but the external features varied, demonstrating a holistic response. Finally, we asked whether the holistic response to faces could be influenced by the context in which the face was presented. We found that adaptation was still evident to composite images in which the face was unchanged but body features were varied. Together, these findings show that although internal features are important in the neural representation of familiar faces, the face's internal and external features are represented holistically in face-selective regions of the human brain.

Keywords: fMRI, Object Recognition, Vision, Temporal, Occipital, Adaptation

INTRODUCTION

Although recognizing human faces is a simple process for most people, the differences between faces are small compared with the variation that occurs between many non-face objects. A variety of evidence suggests that the mechanisms involved in face processing are distinct from those used to process other categories of objects (Yin, 1969; Tanaka and Farah, 1993; McNeil and Warrington, 1993; Pitcher et al., 2009). These mechanisms are thought to primarily analyse the internal features of the face (Valentine, 1991; Leopold et al., 2001; Maurer et al., 2002; Yovel and Kanwisher, 2004), with the ability to make use of differences in internal features improving as faces become familiar (Hancock et al., 2000; Burton et al., 2005). For example, behavioural studies have shown that internal features become more salient than external features when recognizing familiar faces, but that perceivers make use of both internal and external features of unfamiliar faces (Ellis et al., 1979; Young et al., 1985, O'Donnell and Bruce, 2001).

Despite the clear importance of the internal features in the perception and recognition of faces, other studies have suggested the importance of holistic processing, in which both the internal and external features of the face are combined to create an overall representation. For example, the same internal features of a face can be perceived differently according to the face setting in which they are viewed (Young et al., 1987; Sinha and Poggio, 1996, 2001) – combining even highly familiar internal features (eyes, nose and mouth) with an inappropriate set of external features (hair, chin and face outline) produces a composite face whose constituent parts can be difficult to recognise.

Our aim was to draw on these behavioural findings to elucidate how information about the internal and external features of the face is represented in face-selective regions of the human brain (Haxby et al., 2000; Fairhall and Ishai, 2007). Several studies suggest that one region – the fusiform face area (FFA) – is important for processing facial identity. For example, the response in the FFA is modulated by successful face recognition (Grill-Spector et al., 2004) and shows a reduced response (adaptation) to repeated images with the same identity (Grill-Spector et al., 1999; Andrews and Ewbank, 2004; Rotshtein et al., 2004; Yovel and Kanwisher, 2005). Despite evidence for the role of the FFA in face recognition, the relative importance of the internal and external features of the face in activating this region has not been tested explicitly, and the majority of studies have used unfamiliar faces as stimuli. First, we measured the responses to the internal and external features of familiar and unfamiliar faces when presented in isolation. Next, we asked to what extent the internal and external features of the face have independent or holistic neural representations, using composite face images in which the internal features of one face were combined with the external features of another face. Finally, we determined whether the holistic response to faces could be influenced by the context (such as a body) in which a face is presented.

METHODS

Subjects

21 participants took part in Experiment 1 (12 females; mean age, 22), 23 subjects took part in Experiment 2 (14 females; mean age, 24) and 21 subjects took part in Experiment 3 (13 females; mean age, 23). All participants were right-handed, and had normal to corrected-to-normal vision. Written consent was obtained for all subjects and the study was approved by the York Neuroimaging Centre Ethics Committee. Visual stimuli (approx. 8° × 8°) were back-projected onto a screen located inside the magnetic bore, approximately 57cm from subjects' eyes.

Imaging parameters

All experiments were carried out using a GE 3 Tesla HD Excite MRI scanner at the York Neuroimaging centre (YNiC) at the University of York. A Magnex head-dedicated gradient insert coil was used in conjunction with a birdcage, radio-frequency coil tuned to 127.4 MHz. A gradient-echo EPI sequence was used to collect data from 38 contiguous axial slices. (Experiment 1 &2: TR 3s, TE=25ms, FOV 28 * 28 cm, Matrix size = 128 * 128, slice thickness 3 mm; Experiment 3: TR 2s, TE=30 ms, FOV 24 * 24 cm, Matrix size = 128 * 128, slice thickness 4 mm).

Stimuli

Face images were taken from the Psychological Image Collection at Stirling (PICS; http://pics.psych.stir.ac.uk/), and a variety of internet sources. Body images were taken from a body image collection at Bangor (Downing et al., 2001). Image of other categories were taken from a variety of web-based sources. Images were presented in grey scale. In the adaptation scans, the mean change in image intensity across images was calculated by taking the average of the absolute differences in grey value at each pixel for successive pairs of images within a block. Images were presented using Neurobehavioural Systems Presentation 13.0.

Familiar faces were chosen on the basis of familiarity ratings obtained from a separate group of subjects. The familiarity of the faces used in the adaptation scan was confirmed by a test that was performed outside the scanner in which subjects were asked whether the faces were familiar and whether they could report the name and occupation. In the familiarity test participants from experiment 1 and 2 reported 95% (± 0.9) to be familiar. They were also able to report the correct occupation (92% ± 1.6) and name (87% ± 1.6) from the majority of the familiar face images. Different images of the familiar identities were used in the familiarity test and in Experiments 1 and 2. The unfamiliar faces were chosen to match familiar faces for variation in age and appearance.

Localizer scan

To identify regions responding selectively to faces in the visual cortex, a localiser scan was carried out for each subject. Subjects viewed 20 blocks of 10 images. Each block contained images from one of the five different categories: faces, bodies, objects, places, or fourier scrambled images of the former categories. Each image was presented for 700ms followed by a 200ms fixation cross. Stimulus blocks were separated by a 9s fixation grey screen. Each condition was repeated four times, and arranged in a counterbalanced block design.

Experiment 1 – Adaptation to internal and external features of the face

To determine how the internal and external features of the face are represented in face-selective regions of the brain, we used an adaptation paradigm that included 6 conditions: (i) same internal features, (ii) different internal features, (iii) same external features, (iv) different external features, (v) same whole face, (vi) different whole faces. A blocked design was used to present the stimuli. Each stimulus block consisted of 9 images from one of the conditions. In each block, images were shown for 800ms followed by a 200ms fixation cross. The mean image change between successive stimuli in each condition is shown in Suppl. Table 1. Stimulus blocks were separated by a 9s fixation grey screen. Each condition was repeated eight times in a counterbalanced order giving a total of 48 blocks per scan and each scan was repeated for each subject with familiar and unfamiliar faces. To monitor attentional load across stimulus conditions, subjects had to press a button to report the occurrence of a red dot, which was superimposed on 15% of the images.

Experiment 2 – Adaptation to composite (internal/external) face images

To determine whether the internal and external features of the face are represented independently or holistically in face-selective regions of the brain, we used an adaptation paradigm with four image conditions: (i) same internal features, same external features; (ii) same internal features, different external features; (iii) different internal features, same external features; (iv) different internal features, different external features. Composite images were generated using image editing software. A blocked design was used to present the stimuli. Each stimulus block consisted of 9 images from one of the conditions. In each block, images were shown for 800ms followed by a 200ms fixation cross. The mean image change between successive stimuli in each condition is shown in Suppl. Table 2. Stimulus blocks were separated by a 9s fixation grey screen. Each condition was repeated six times giving a total of 48 (24 unfamiliar, 24 familiar) blocks. To monitor attentional load across stimulus conditions, a red dot task was used.

Experiment 3 – The role of context in face processing

To determine whether the neural representation in face-selective regions could be influenced by changes in the immediate context in which the face was presented, we used composite images in which the faces could be superimposed on different bodies or busts. There were four image conditions: (i) same face, same bust; (ii) same face, different bust; (iii) different face, same bust; (iv) different face, different bust. The faces were unfamiliar to the subjects. A blocked design was used to present the stimuli. Each stimulus block consisted of 10 images from one of the conditions. In each block, images were shown for 800ms followed by a 200ms fixation cross. The mean image change between successive stimuli in each condition is shown in Suppl. Table 3. Stimulus blocks were separated by a 10s fixation grey screen. Each condition was repeated six times giving a total of 24 blocks. To monitor attentional load across stimulus conditions, a red dot was superimposed on one or two images in each block. Subjects were required to respond, with a button press, as soon as they saw the image containing the target.

fMRI Analysis

Statistical analysis of the fMRI data was carried out using FEAT (http://www.fmrib.ox.ac.uk/fsl). The initial 9s of data from each scan were removed to minimize the effects of magnetic saturation. Motion correction was followed by spatial smoothing (Gaussian, FWHM 6mm) and temporal high-pass filtering (cut off, 0.01 Hz). For the localizer scan, face selective regions of interest (ROI) were determined by the contrast face > place thresholded at P < 0.001 (uncorrected). The time series of the resulting filtered MR data at each voxel was converted from units of image intensity to percentage signal change by subtracting and then normalizing the mean response of each scan ([x − mean]/mean*100). All voxels in a given ROI were then averaged to give a single time series in each ROI for each subject. Individual stimulus blocks were normalized by subtracting every time point by the zero point for that stimulus block. The normalized data were then averaged to obtain the mean time course for each stimulus condition. The peak response calculated as an average of the response at 9 and 12 seconds in Experiment 1 and 2 and the average of the response at 8, 10 and 12 seconds in Experiment 3 after the onset of a block. Repeated-measures ANOVA were used to determine significant differences in the peak response to each stimulus condition in each experiment.

RESULTS

Localizer scan

Figure 1 shows the location of three different regions in the occipital and temporal lobe that showed face-selective activity (face>place): fusiform face area (FFA); occipital face area (OFA); and superior temporal sulcus (STS) (Haxby et al, 2000). The coordinates of each region are shown in Table 1. Each region was defined separately for each individual and all further analyses were performed on the mean time courses of voxels in these ROI. The average time-courses of activation in the face-selective regions are also shown in Figure 1. The curves confirm that the model used to analyze the data (face>place) provides a good description of the raw MR-activity and illustrate the differences in responses between faces and other categories of objects. There was no difference in the pattern of response between the right and left hemispheres. Accordingly, all subsequent analyses were based on a pooled analysis in which the right and left hemisphere voxels were combined.

Figure 1
Localiser scan. (A) Location of face-selective regions (FFA: fusiform face area, OFA: occipital face area, STS: superior temporal sulcus) across all subjects in a whole brain group analysis. These scan images follow radiological convention, with the left ...
Table 1
Mean MNI coordinates of face-selective regions-of-interest.

Experiment 1 – Adaptation to internal and external features of the face

Figs. Figs.22--33 show the responses in the different regions to internal and external features of familiar and unfamiliar faces. The FFA was localized in all (21) subjects, whereas the OFA and STS were localized in 17 subjects. The data were first analyzed using a 3 × 2 × 2 × 3 repeated measures ANOVA with ROI (FFA, OFA, STS), Familiarity (familiar, unfamiliar), Identity (same, different) and Feature (internal, external, whole) as the main factors. There were significant effects of ROI (F(2,28) = 75.0, P < 0.0001), Familiarity (F(1,14) = 4.5, P < 0.05), Feature (F(2,28) = 25.0, P < 0.0001) and Identity (F(1,14) = 85.2, P < 0.0001). There was a significant interaction between ROI and Identity (F(2,28) = 52.1, P < 0.0001) and between Familiarity and Identity (F(2,28) = 10.1, P < 0.01). The data were then analyzed in each region using a 2 × 3 repeated-measures ANOVA with Identity (same, different) and Feature (internal, external, whole) as the main factors.

Figure 2
Experiment 1 - Adaptation to internal and external features of familiar faces. (Left) Examples of familiar faces. (Right) The average time-course of response across subjects in the fusiform face area (FFA), the occipital face area (OFA), and the superior ...
Figure 3
Experiment 1 - Adaptation to internal and external features of unfamiliar faces. (Left) Examples of unfamiliar faces. (Right) The average time-course of response across subjects in the fusiform face area (FFA), the occipital face area (OFA), and the superior ...

First, we determined whether there was a significant effect of Feature (whole, internal, external) in the face-selective regions. In the FFA, there was an effect of Feature for familiar (F(2,40) = 33.4, P < 0.001) and unfamiliar (F(2,40) = 27.9, P < 0.001) faces. The effect of Feature in the FFA was due to a larger response to the internal features compared to the external features (familiar: t(20) = 6.3, P < 0.001; unfamiliar: t(20) = 5.1, P < 0.001). Similarly, the response to the whole face was greater than the response to external features for both the familiar (t(20) = 7.59, P < 0.001) and unfamiliar (t(20) = 7.2, P < 0.001) faces. However, there was no difference in the response of the FFA to the internal features compared to the whole face for either the familiar (t(20) = 1.2, P = 0.51) and unfamiliar (t(20) = 1.5, P = 0.28) faces.

There was a significant effect of Feature for familiar (F(2,32) = 3.6, P < 0.05) and unfamiliar (F(2,32) = 6.9, P < 0.005) faces in the OFA. This was due to a progressive change in the response to the external features (familiar: 1.05 ± 0.10 %, unfamiliar: 0.95 ± 0.12 %), internal features (familiar: 1.42 ± 0.10 %, unfamiliar: 1.25 ± 0.10 %) and whole faces (familiar: 1.43 ± 0.12 %, unfamiliar: 1.33 ± 0.12 %).

There was also an effect of Feature in the STS for familiar (F(2,32) = 20.8, P < 0.001) and unfamiliar faces (F(2,32) = 28.7, P < 0.001). This was due to a lower response to the external features compared to the internal features (familiar: t(16) = −3.8, P < 0.005; unfamiliar: t(16) = −8.15, P < 0.001) and the whole face (familiar: t(16) = −5.3, P < 0.001; unfamiliar: t(16) = −5.1, P < 0.001).

Next we determined the effect of Identity (same, different) in the face-selective regions. There was a significant effect of Identity in the FFA for familiar (F(1,20) = 82.7, P < 0.001) and unfamiliar faces (F(1,20) = 69.6, P < 0.001). The effect of identity was due to a smaller response (adaptation) to the same face conditions compared to the corresponding different faces conditions for familiar (diff. whole > same whole: t(20) = 7.2, P < 0.001; diff. internal > same internal: t(20) = 8.8, P < 0.001; diff. external > same external: t(20) = 7.04, P < 0.001) and unfamiliar (diff. whole > same whole: t(20) = 6.6, P < 0.001; diff. internal > same internal: t(20) = 5.94, P < 0.001; diff. external > same external: t(20) = 5.3, P < 0.001) faces.

There was also a significant effect of Identity in the OFA for familiar (F(1,16) = 116.7, P < 0.001) and unfamiliar faces (F(1,16) = 18.6, P < 0.001). This was due to a smaller response (adaptation) to the same face conditions compared to the corresponding different faces conditions for familiar faces (diff. whole > same whole: t(16) = 6.61, P < 0.001; diff. internal > same internal: t(16) = 9.6, P < 0.001; diff. external > same external: t(16) = 6.09, P < 0.001), and unfamiliar faces (diff. whole > same whole: t(16) = 4.3, P < 0.005; diff. external > same external: t(16) = 2.5, P < 0.05). There was a trend, but no significant differences, between the internal features (diff. internal > same internal: (t(16) = 2.3, P = 0.07) of unfamiliar faces.

There was a significant effect of Identity in the STS for familiar (F(1,16) = 15.67, P < 0.001) but not unfamiliar faces (F(1,16) = 0.02, P = 0.90). The effect of Identity for familiar faces was due to the difference between the same and different whole faces and internal features (diff. whole > same whole: t(16) = 4.33, P < 0.001; diff. internal > same internal: t(16) = 2.63, P < 0.05). No adaptation was evident for external features (diff. external > same external: t(16) = 0.88, P = 0.78).

Finally, we determined whether there was a significant interaction between Feature and Identity. In the FFA, although there was no significant interaction (F(2,40) = 0.97, P = 0.39) for unfamiliar faces, there was a trend for an interaction between Identity and Feature (F(2,40) = 2.67, P = 0.08) for familiar faces (Fig. 4). A post-hoc 2 × 2 ANOVA specifically aimed to examine differences between internal and external representations revealed a significant interaction in the FFA between Feature (internal, external) and Identity (same, different) for familiar faces (F(1,20) = 5.52, P < 0.05) but not unfamiliar faces (F(1,20) = 0.75, P = 0.40). This was caused by a larger difference in the adaptation effect (different-same) for the internal (0.63 ± 0.07 %) relative to the external (0.48 ± 0.07 %) features of familiar faces, but no difference between adaptation to internal (0.49 ± 0.08 %) and external (0.40 ± 0.07 %) features of unfamiliar faces.

Figure 4
Experiment 1 - Adaptation to internal and external features of faces. The average adaptation response in the FFA to internal and external features of familiar and unfamiliar faces in Experiment 1. There was a significant interaction between Feature and ...

Unlike the FFA, there was no interaction in the OFA between Identity and Feature for familiar (F(2,32) = 0.82, P = 0.45) or unfamiliar (F(2,32) = 1.67, P = 0.21) faces. Similarly, there was no interaction between Identity and Feature for either familiar (F(2,32) = 1.12, P = 0.34) or unfamiliar faces (F(2,32) = 0.03, P = 0.97).

Experiment 2 – Adaptation to composite (internal/external) face images

Figs. Figs.55--66 show the responses in the FFA, OFA and STS to different composites of the internal and external features of either familiar or unfamiliar faces. The FFA was localized in all (23) subjects, the OFA was localized in 21 subjects and the STS was localized in 19 subjects. The data were analyzed using a 2 × 2 repeated-measures ANOVA with Internal (same, different) and External (same, different) as the main factors.

Figure 5
Experiment 2 – Adaptation to composite (internal/external) images of familiar faces. (Top) Example of images from different conditions (row 1: same internal, same external; row 2: same internal, different external; row 3: different internal, same ...
Figure 6
Experiment 2 – Adaptation to composite (internal/external) images of unfamiliar faces. (Top) Example of images from different conditions (row 1: same internal, same external; row 2: same internal, different external; row 3: different internal, ...

In the FFA, there was a significant effect of External for both familiar (F(1,22) = 11.2, P < 0.005) and unfamiliar (F(1,22) = 8.2, P <0.01) faces. Although there was no effect of Internal for either familiar (F(1,22) = 2.65, P = 0.12) or unfamiliar (F(1,22) = 0.36, P = 0.56) faces, there was a significant interaction (familiar: F(1,22) = 18.6, P <0.001; unfamiliar: F(1,22) = 8.6, P <0.01). A post-hoc analysis revealed that there was a smaller response to the same internal, same external condition compared to all other conditions for familiar (same internal, diff. external: t(22) = 6.23, P < 0.001; diff. internal, same external: t(22) = 3.46, P < 0.005; diff. internal, diff. external: t(22) = 4.65, P < 0.001) and unfamiliar (same internal, diff. external: t(22) = 4.22, P < 0.001; diff. internal, same external: t(22) = 2.24, P = 0.07; diff. internal, diff. external: t(22) = 2.43, P < 0.05) faces. There was no significant difference between any of the other conditions.

In the OFA, there was a significant effect of External for familiar (F(1,20) = 8.4, P < 0.01), but not unfamiliar (F(1,20) = 2.2, P = 0.15) faces. Although there was no effect of Internal for either familiar (F(1,20) = 0.09, P = 0.77) or unfamiliar (F(1,20) = 0.06, P = 0.81) faces, there was a significant interaction (familiar: F(1,20) = 21.9, P <0.001; unfamiliar: F(1,20) = 6.9, P <0.05). A post-hoc analysis revealed there was a smaller response (adaptation) to same internal features, same external features compared to same internal, diff. external (t(20) = 5.66, P < 0.001) and diff. internal, same external (t(20) = 2.68, P < 0.05) but not to diff. internal, diff. external (t(20) = 2.10, P = 0.10) of familiar faces. For unfamiliar faces, the only significant difference was between the same internal, same external condition compared to same internal, diff. external condition (t(20) = 2.92, P < 0.05).

There was no significant effect of External (familiar: F(1,18) = 0.51, P = 0.49; unfamiliar: F(1,18) = 0.59, P = 0.45) or Internal (familiar: F(1,18) = 0.02, P = 0.90; unfamiliar: F(1,18) = 0.01, P = 0.92) in the STS. There was also no interaction between Internal and External features (familiar: F(1,18) = 0. 92, P = 0.35; unfamiliar: F(1,18) = 2.4, P = 0.14).

Experiment 3 – The role of context in face processing

Fig. 7 shows the effect of changing the context in which a face is presented on the responses in the FFA, OFA and STS. The FFA was localized in 21 subjects, the OFA was localized in 17 subjects, and the STS was localized in 14 subjects. The data were analyzed using a 2 × 2 repeated-measures ANOVA with Face (same, different) and Bust (same, different) as the main factors.

Figure 7
Experiment 3 – The role of context in face processing. (Top) Example of composite images (row 1: same face, same bust; row 2: same face, diff bust; row 3: diff face, same bust; row 4: diff face, diff bust). (Bottom) The average time-course of ...

There was a significant effect of Face (F(1,20) = 25.9, P < 0.0001) in the FFA. Post-hoc analysis revealed that this pattern of results was due to a smaller response (adaptation) to the same face, same bust condition compared to the different face conditions (diff. face, same bust: t(20) = 4.1, P < 0.001; diff. face, diff. bust: t(20) = 5.5, P < 0.0001). Similarly, there was a smaller response to the same face, diff. bust condition compared to the different face conditions (diff. face, same bust: t(20) = 2.1, P < 0.05; diff. face, diff. bust: t(20) = 3.9, P < 0.001). There was also a significant, but smaller, effect of Bust (F(1,20) = 6.4, P < 0.05) in the FFA. This effect was due to a small release from adaptation in the same face, diff. bust condition compared to the same face, same bust condition (t(20) = 3.2, P < 0.005). However, there was no significant difference between the diff. face, same bust and diff. face, diff. bust conditions (t(20) = 0.8, P = 0.4).

There was a significant effect of Face (F(1,16) = 18.1, P < 0.001), but no effect of Bust (F(1,16) = 0.45, P = 0.51) in the OFA. This pattern of results was due to due to a smaller response (adaptation) to a difference between conditions in which the same face was repeated (same face/same bust: 1.33 ± 0.16 %, same face, diff. bust: 1.40 ± 0.17 %) compared to when different faces were presented (same face,same bust: 1.62 ± 0.20 %, same face,diff. bust: 1.65 ± 0.19 %).

There was no significant effect of Face (F(1,13) = 0.33, P = 0.57) or Bust (F(1,13) = 0.14, P = 0.71) in the STS face region.

DISCUSSION

The aim of this study was to probe how the internal (eyes, nose and mouth) and external (hair, face shape) features of a face are represented in face-selective regions of the human brain. The main results from this study are: (1) There was a larger response to the internal features of the face compared to the external features; (2) There was more adaptation to the internal features compared to the external features of familiar faces in the FFA, but there was no difference in adaptation between the internal and external features of unfamiliar faces; (3) Changes to either the internal or external features of a face caused a complete release from adaptation, demonstrating that these features are represented holistically; (4) The holistic response to faces does not extend to the context in which the face is presented.

The importance of the internal features of the face in the recognition of familiar faces has been shown in a number of behavioural studies (Ellis et al., 1979; Young et al., 1985, O'Donnell and Bruce, 2001). For example, Ellis et al. (1979) used a memory task in which participants had to remember a set of famous or unfamiliar faces. Participants were then presented with either the internal or external features of faces, and asked to recall whether they had seen the face during the learning phase. The results showed that participants were better able to recognize the identities of familiar faces from the internal features compared to the external features. In contrast, there was no difference in recognition rate between the internal and external features of unfamiliar faces. Young et al. (1985) found similar results using a matching task in which participants had to judge whether two images were the same person. Participants were faster at making this judgement using the internal features compared to the external features with familiar faces. However, there was no difference between the internal and external features when matching unfamiliar faces. Interestingly, Moscovitch and Moscovitch (2000) reported that the patient CK, who has a severe object agnosia with preserved face recognition, showed a more pronounced difficulty in recognizing external features compared to control subjects. Our imaging results fit well with these behavioural studies by showing greater response to the internal features compared to the external features of the face. Moreover, there was significantly more adaptation to the internal features of the face compared to the external features in the FFA for familiar faces; but no difference in adaptation for unfamiliar faces (see Fig. 4). These results are also consistent with recent results showing a difference in the way that familiar and unfamiliar faces are represented in the FFA (Ewbank and Andrews, 2008).

Although face-selective regions responded more to the internal features of the face, significant responses to the external features were evident in these regions. This fits with behavioural results, which show that, while recognition is better when the internal features of the face are shown, it is still possible to identity faces based solely on the presence of the external features (Ellis et al., 1979). Indeed, it is possible for the same internal features to be recognized as different individuals just based on differences in the appearance of the external features of the face (Sinha and Poggio, 1996, 2002). More recently, Freiwald et al (2009) reported that neurons in the macaque temporal lobe are particularly sensitive to the aspect ratio or shape of the face. So, although our results demonstrate that the external features of faces are represented in face-selective regions, they are not specific about which aspects of the external features are pre-eminent.

To determine whether the internal and external features of faces are represented independently, we generated composite face images in which the internal features of one face were combined with the external features of another face. If face regions represent internal and external features of faces independently, a reduced response (adaptation) would be expected when either the internal or external features of the face are repeated, regardless of any change to the image. However, if the FFA relies on some form of holistic representation, then changes to either the internal or external features of the face should result in a release from adaptation.

Most models of face processing emphasize the importance of the configuration of features of the face, but there are different senses in which the idea of ‘configuration’ has been invoked. Particularly important in this respect is the difference between a holistic configurational representation, in which information from all parts of the face is indissolubly combined, and a ‘second-order’ configurational representation that specifies how the various parts of the face are positioned with respect to each other (Maurer et al., 2002; Yovel and Kanwisher, 2005; Maurer et al., 2007). In terms of this distinction, recombining the internal and external features of different faces (as in our Experiment 2) disrupts any holistic representation, but should have little effect on a second-order configurational representation (since the spacing of the critical internal features themselves remains unchanged). Our findings that there was a release from adaptation in the FFA to composite images in which the internal features were unchanged but the external features varied on successive images, or when the external features were unchanged and the internal features were varied, strongly suggest that the FFA processes information using a holistic code in which the internal and external features of the face are both important. These findings fit with other evidence that face parts are processed interactively rather than independently (Yovel and Axelrod, 2009, although see Betts and Wilson, 2009). For example, the combination of highly familiar internal features with an inappropriate set of external features produces a composite face that can be difficult to recognise accurately (Young et al., 1987). The importance of a holistic representation of faces is also apparent when the lower half of one face is aligned with the upper half of another face to produce a composite image (Young et al., 1987); the resulting novel configuration impairs recognition of the two halves of the face, suggesting that face perception is based on a holistic process. Consistent with these behavioural findings and the results in our study, a larger adaptation effect in the FFA is apparent for misaligned compared to aligned composite images in which one half of the face is unchanged (Schiltz and Rossion, 2006).

Finally, we determined whether holistic response to faces could be influenced by the non-face context in which the face was presented. The importance of context has been shown in the response of the FFA to images (such as a body) that imply the presence of a face (Cox et al., 2004). However, it has been argued that these findings could be explained by activity of body-selective neurons in the Fusiform Gyrus (Peelen and Downing, 2004; Schwarzlose et al., 2005). To determine the role of ‘context’, we generated composite face images in which the face was unchanged, but the body features on which it was superimposed varied. In the FFA, there was a slight increase in response (i.e. release from adaptation) for this composite image compared to repetitions of the same face and body. However, the response was still significantly lower than stimulus blocks in which the face varied across successive presentations. We then determined the response to composite images in which the face varied, but the body was unchanged. This stimulus generated a response that was not significantly different from stimulus blocks in which different faces and bodies were shown. In other words, the FFA shows a holistic response to faces, but does not show a holistic response to different combinations of faces and bodies. Together, these results suggest that the neural representation in the face-selective regions such as the FFA is predominantly selective for faces.

Adaptation to repeated images of internal or external features of the face was evident in the FFA and OFA, but not in the STS. This fits with models of face processing that emphasize the difference between inferior temporal processes involved in facial recognition and superior temporal processes involved in understanding dynamic aspects of faces (Bruce and Young, 1986; Haxby et al., 2000; Hoffman and Haxby, 2000; Andrews and Ewbank, 2004; Winston et al., 2004; Ewbank and Andrews, 2008). The STS did however show a larger response to internal compared to external features of the face. This is consistent with the importance of the internal features in providing cues about expression and gaze.

In conclusion, the response to isolated features of the face provide clear neurological support for the idea that the dominant role of internal features in the neural representation of familiar faces. However, the release from adaptation when viewing composite faces suggests that both the internal and external features are important in face perception. Finally, the absence of a holistic response between faces and non-face objects suggests that the FFA is primarily involved in processing information about faces.

Supplementary Material

Supplemantary Tables

Acknowledgements

We would like to thank Chris Alderson, Nick Arthur, Johan Carlin, Rick Chen, Gemma Cheney, Dettie Freedman, Andre Gouws, Anna Gebhardt, Rachel Harlow, Ama Hirsch, Anneka Holden, Ed Ingamells and Helen Williams for their help during the initial stages of this project. JD-T is supported by an ESRC studentship. This work was supported by a grant from the Wellcome Trust (WT087720MA).

REFERENCES

  • Andrews TJ, Ewbank MP. Distinct representations for facial identity and changeable aspects of faces in the human temporal lobe. NeuroImage. 2004;23:905–913. [PubMed]
  • Betts LR, Wilson HR. Heterogeneous Structure in Face-selective Human Occipito-temporal Cortex. Journal of Cognitive Neuroscience. 2010 doi: 10.1162/jocn.2009.21346. [PubMed]
  • Bruce V, Young AW. Understanding face recognition. British Journal of Psychology. 1986;77:305–327. [PubMed]
  • Burton AM, Jenkins R, Hancock PJB, White D. Robust representations for face recognition: The power of averages. Cognitive Psychology. 2005;51:256–284. [PubMed]
  • Cox D, Meyers E, Sinha P. Contextually evoked object-specific responses in human visual cortex. Science. 2004;304:115–117. [PubMed]
  • Ellis HD, Shepherd JW, Davies GM. Identification of familiar and unfamiliar faces from internal and external features. Perception. 1979;8:431–439. [PubMed]
  • Ewbank MP, Andrews TJ. Differential sensitivity for niewpoint between familiar and unfamiliar faces in human visual cortex. NeuroImage. 2008;40:1857–1870. [PubMed]
  • Fairhall SI, Ishai A. Effective connectivity within the distributed cortical network for face perception. Cerebral Cortex. 2007;17:2400–2406. [PubMed]
  • Freiwald WA, Tsao DY, Livingstone MS. A face feature space in the macaque temporal lobe. Nat. Neurosci. 2009;12:1187–1196. [PMC free article] [PubMed]
  • Grill-Spector K, Kushnir T, Hendler T, Edelman S, Itzchak Y, Malach R. Differential processing of objects under various viewing conditions in human lateral occipital complex. Neuron. 1999;24:187–203. [PubMed]
  • Grill-Spector K, Knouf N, Kanwisher N. The fusiform face area subserves face perception, not generic within-category identification. Nat Neurosci. 2004;7:555–562. [PubMed]
  • Hancock PJB, Bruce V, Burton AM. Recognition of unfamiliar faces. Trends in Cognitive Sciences. 2000;4:330–337. [PubMed]
  • Haxby JV, Hoffman EA, Gobbini MI. The distributed human neural system for face perception. Trends in Cognitive Sciences. 2000;4:223–233. [PubMed]
  • Hoffman EA, Haxby JV. Distinct representations of eye gaze and identity in the distributed human neural system for face perception. Nat Neurosci. 2000;3:80–84. [PubMed]
  • Leopold DA, O'Toole AJ, Vetter T, Blanz V. Prototype-referenced shape encoding revealed by high-level aftereffects. Nat Neurosci. 2001;4:89–94. [PubMed]
  • Maurer D, Le Grand R, Mondloch CJ. The many faces of configural processing. Trends in Cognitive Sciences. 2002;6:255–260. [PubMed]
  • Maurer D, O'Craven KM, Le Grand R, Mondloch CJ, Springer MV, Lewis TL, Grady CL. Neural correlates of processing facial identity based on features versus their scrambling. Neuropsychologia. 2007;45:1438–1451. [PubMed]
  • Moscovitch M, Moscovitch DA. Super face-inversion effects for isolated internal or external features, and for fractured faces. Cognitive Neuropsychology. 2000;17:201–219. [PubMed]
  • McNeil JE, Warrington EK. Prosopagnosia: a face-specific disorder. Quarterly Journal of Experimental Psychology. 1993;46A:1–10. [PubMed]
  • O'Donnell C, Bruce V. Familiarisation with faces selectively enhances sensitivity to changes made to the eyes. Perception. 2001;30:755–764. [PubMed]
  • Peelen MV, Downing PE. Selectivity for the human body in the fusiform gyrus. Journal of Neurophysiology. 2005;93:603–608. [PubMed]
  • Pitcher D, Charles L, Devlin JT, Walsh V, Duchaine B. Triple Dissociation of Faces, Bodies, and Objects in Extrastriate Cortex. Current Biology. 2009;19:319–324. [PubMed]
  • Rotshtein P, Henson R, Treves A, Driver J, Dolan RJ. Morphing Marilyn into Maggie dissociates physical and identity face-representations in the brain. Nat. Neurosci. 2004;8:107–113. [PubMed]
  • Schwarzlose R, Baker C, Kanwisher N. Separate face and body selectivity on the fusiform gyrus. J. Neurosci. 2005;25:11055–11059. [PubMed]
  • Schiltz C, Rossion B. Faces are represented holistically in the human occipito-temporal cortex. Neuroimage. 2006;32:1385–1394. [PubMed]
  • Sinha P, Poggio T. I think I know that face. Nature. 1996;384:404. [PubMed]
  • Sinha P, Poggio T. ‘United’ we stand. Perception. 2002;31:133. [PubMed]
  • Tanaka JW, Farah M. Parts and wholes in face recognition. Quarterly Journal of Experimental Psychology. 1993;46A:225–245. [PubMed]
  • Valentine T. A unified account of the effects of distinctiveness, inversion and race in face recognition. Quarterly Journal of Experimental Psychology. 1991;43A:161–204. [PubMed]
  • Winston JS, Henson RNA, Fine-Goulden MR, Dolan RJ. fMRI-Adaptation reveals dissociable neural representations of identity and expression in face perception. Journal of Neurophysiology. 2004;92:1830–1839. [PubMed]
  • Yin RK. Looking at upside-down faces. Journal of Experimental Psychology. 1969;81:141–145.
  • Young AW, Hay DC, McWeeny KH, Flude BM, Ellis AW. Matching familiar and unfamiliar faces on identity and expression. Perception. 1985;14:737–746. [PubMed]
  • Young AW, Hellawell D, Hay DC. Configurational information in face perception. Perception. 1987;16:747–759. [PubMed]
  • Yovel G, Kanwisher N. Face perception: domain specific, not process specific. Neuron. 2004;44:889–898. [PubMed]
  • Yovel G, Kanwisher N. The neural basis of the behavioral face-inversion effect. Current Biology. 2005;15:2256–2262. [PubMed]
  • Yovel G, Axelrod V. Non-preferred stimuli modify the representation of preferred stimuli in the fusiform face area. 603.4 2009 Neuroscience Meeting Planner; Society for Neuroscience; Chicago, IL. 2009. 2009. Online. [PubMed]