Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Int J Psychophysiol. Author manuscript; available in PMC 2018 January 1.
Published in final edited form as:
PMCID: PMC5222687

Methodological Issues in the Use of Individual Brain Measures to Index Trait Liabilities: The Example of Noise-Probe P3


Recent research initiatives have called for an increased use of biological concepts and measures in defining and studying mental health problems, but important measurement-related challenges confront efforts in this direction. This article highlights some of these challenges with reference to an intriguing measure of neural reactivity: the probe P3 response, a mid-latency brain potential evoked by an intense, unexpected acoustic-probe stimulus. Using data for a large adult sample (N = 418), we report evidence that amplitude of probe P3 response to unwarned noise bursts occurring in a picture-viewing task exhibits robust, independent associations with two distinct trait constructs: weak inhibitory control (or disinhibition; DIS) and threat sensitivity (THT). Additionally, we report a selective association for THT with attentional suppression of probe P3 response during viewing of aversive pictures compared to neutral. These results point to separable elements of variance underlying the probe P3 response, including one element reflecting DIS-related variations in cognitive-elaborative processing, and others reflecting THT-related variations in aversive foreground engagement and abrupt defensive reorientation. Key measurement issues are considered in relation to these specific findings, and methodological and statistical approaches for addressing these issues are discussed in relation to advancement of a quantitatively sound, biologically informed science of psychopathology.

Keywords: psychopathology, P3, inhibitory control, disinhibition, threat sensitivity, fear

Methodological Issues in the Use of Individual Brain Measures to Index Trait Liabilities: The Example of Noise-Probe P3

Empirical research over the past decade has called into question the categorical classification of psychopathology as reflected in the Diagnostic and Statistical Manual of Mental Disorders (DSM; American Psychiatric Association, 2013), pointing instead to a dimensional organization of clinical problems cutting across traditional mental disorder categories. In line with this shift, the National Institute of Mental Health (NIMH) has proposed a biologically oriented framework for pursuing research on transdiagnostic problem dimensions, the Research Domain Criteria (RDoC) matrix (Insel et al., 2010). The NIMH RDoC framework calls for investigation of basic biobehavioral processes relevant to differing forms of mental illness using multiple units of analysis, from molecular (genomic) variables to overt behavioral measures. Two RDoC matrix constructs have particular relevance for understanding psychopathology: inhibitory control and threat sensitivity (Blair, White, Meffert, & Hwang, 2014; Patrick, Durbin, & Moser, 2012; Patrick & Drislane, 2015; Yancey, Venables, Hicks, & Patrick, 2013; Yancey, Venables, & Patrick, 2016). Weak inhibitory control plays a role in various externalizing conditions, including conduct disorder, adult antisocial behavior, and substance use disorders (Krueger, Hicks, Patrick, Carlson, Iacono, & McGue, 2002), whereas threat sensitivity has been implicated in fear-related internalizing conditions such as social phobia, specific phobia, and panic disorder (B. D. Nelson et al., 2013). Thus, these two RDoC constructs are relevant to a number of the most commonly occurring mental disorders. A deeper understanding of these constructs in neurobiological terms will thus contribute to new perspectives on a wide range of mental illnesses and help to inform prevention and treatment efforts.

A major topic of interest in biologically oriented clinical research has been the concept of “biomarkers,” referring to biological variables that index liabilities for or expressions of psychopathology. In recent years, this concept has come under criticism for its oversimplification of the relationship between biological phenomena and outcomes (Lenzenweger, 2013; Miller & Rockstroh, 2013). Across scientific disciplines, the identification of biomarkers for psychopathology has been a major funding priority, but investigative efforts have often failed to consider measurement issues of importance to this endeavor. For example, biomarkers generally account for only a small portion of the variance in clinical outcomes they are intended to index (Patrick & Bernat, 2010; Patrick, Venables, Yancey, Hicks, Nelson, & Kramer, 2013). Additionally, individual biomarkers can contain variance related to different clinical outcomes, raising questions about their etiological coherence and diagnostic specificity (Cicchetti, 1984). The current study highlights overlooked issues in the use of neurobiological variables to index psychopathology-related characteristics, through reference to a distinct variant of the P3 brain event-related potential (ERP), the noise-probe P3 response.

Probe P3 as a Neural Indicator of Psychopathology

The noise-probe P3 (or probe P3) response is a member of the larger P3 family of brain-ERP responses. (The term “P300” refers to the response to infrequent target stimuli in the well-known oddball task, whereas the more generic label “P3” is used for variants of this response occurring in different tasks [Rugg & Coles, 1995].) Broadly, P3 responses are positive ERP deflections, typically maximal in amplitude at parietal scalp sites, that tend to peak between 300 and 600 ms following the presentation of task stimuli, depending on the context of processing. Prior work using P3 response to visual stimuli suggests that this response reflects cognitive postprocessing, with greater amplitude reflecting greater cortical resources devoted to the stimulus’s meaning (Lang, Bradley, & Cuthbert, 1992; Rugg & Coles, 1995).

The probe P3 is a variant of this type of ERP response that occurs in response to a sudden, intense acoustic stimulus, such as a burst of white noise. Like other variants of P3, it is a mid-latency response to a salient perceptual stimulus. However, it differs from other P3s in that it is elicited by a stimulus that is inherently noxious, due to its intensity and unpredictability. As such, the probe P3 has been characterized as indexing a “cortical call-to-arms” — a rapid marshalling of cognitive-attentional capacities to interpret and contend with a strong, unanticipated event (Drislane, Vaidyanathan, & Patrick, 2013; see also Graham, 1979). Additionally, prior research has demonstrated that, in contrast with the faster-latency (< 100 ms) blink-reflex response, which increases during viewing of aversive foregrounds (relative to neutral) and decreases during viewing of pleasurable ones, probe P3 amplitude is reduced for probes presented during both aversive and pleasurable foregrounds (Cuthbert, Schupp, Bradley, McManis, & Lang, 1998). This effect has been interpreted as reflecting reduced availability of cognitive resources for post-processing of noxious probe stimuli as a function of enhanced foreground-attentional engagement (Cuthbert et al., 1998; Drislane et al., 2013; Graham, 1979).

Thus, while P3 responses are generally viewed as indexing cognitive processing, the probe P3 appears to operate as a biological index of psychological processes related to both cognition and emotion: It indexes a rapid defensive-interrupt process (Graham, 1979; see also Miller, Patrick, & Levenston, 2002) to discern the action-relevance of abrupt, intense events; in addition, it taps into affective-foreground engagement at the time of noise-probe occurrence, as reflected in P3-amplitude suppression during affective relative to neutral foreground contexts. As discussed below, distinct portions of variance in probe P3 reflect 1) cognitive processes in common with other P3 variants and 2) affective processes related to the aversive nature of the probe; these portions are differentially associated with psychopathology-related dispositions.

Disinhibition (DIS)

One relevant psychological variable in the study of cognitive processing is inhibitory control, represented by the construct of response inhibition in the Cognitive Systems domain of the RDoC matrix. Weak inhibitory control, characterized in individual-difference terms as trait disinhibition (DIS; L. D. Nelson, Strickland, Krueger, Arbisi, & Patrick, 2016; Patrick et al., 2013), has been identified as a liability factor for externalizing problems of various types, including child and adult antisocial behavior and maladaptive substance use (Krueger, Hicks, Patrick, Carlson, Iacono, & McGue, 2002; Yancey et al., 2013). Patrick and colleagues (2013) constructed a “psychoneurometric” index of DIS, incorporating self-report-scale measures along with brain-response variables, that effectively predicted both clinical-symptom and physiological criterion measures. Within this model of disinhibition, amplitude of P3 response to noise probes presented during viewing of neutral pictures was shown to covary with scale measures of DIS as well as with P3 responses to target and novel stimuli in a separate visual oddball task. Results from this study dovetail with other work showing that trait disinhibition is associated with reductions in different variants of P3 response (Bernat, Nelson, Steele, Gehring, & Patrick, 2011; L. D. Nelson, Patrick, & Bernat, 2011; Yancey et al., 2013), indicating that high DIS involves a general, cross-task deficit in cognitive post-processing of task-related stimuli (Patrick, Bernat, Malone, Iacono, Krueger, & McGue, 2006; Hall, Bernat, & Patrick, 2007; Venables, Hall, Yancey, & Patrick, 2015). The finding by Patrick et al. (2013) that probe P3 covaried with DIS in a manner similar to target P3 responses suggests that the noise-elicited P3 includes an element of variance reflecting this DIS-related impairment in cognitive post-processing. As highlighted above and discussed next, the probe P3 response also appears to contain separate elements of variance related to affective processing that may operate as indices of threat sensitivity.

Threat Sensitivity (THT)

Another psychological characteristic with broad relevance to clinical problems is threat sensitivity (THT), represented by the construct of acute threat in the Negative Valence Systems domain of the RDoC matrix. Conceptualized in individual-difference terms, this construct connects to the general trait dimension shown to underlie psychological-scale measures of fear versus fearlessness in relation to stimuli and situations of various types (Kramer, Patrick, Krueger, & Gasperi, 2012). When operationalized in terms of scores on this fear/fearlessness dimension, variations in threat sensitivity are independent of (i.e., uncorrelated with) variations in inhibitory control, quantified as trait disinhibition (L. D. Nelson et al., 2016; Venables, Sellbom, et al., 2015). High scores on this fear/fearlessness dimension have been found to be related to anxiety disorders of various types, particularly those involving context-bound fear (L. D. Nelson et al., 2016; Yancey et al., 2016); low scores on this dimension, by contrast, are associated with affective-interpersonal (“Factor 1”) symptoms of psychopathy (Patrick & Bernat, 2009; Patrick & Drislane, 2015).

Related to the latter point, a study of incarcerated male offenders by Drislane et al. (2013) reported a negative association between Factor 1 symptoms of psychopathy (as assessed by the Psychopathy Checklist-Revised [PCL-R]; Hare, 2003) and amplitude of probe P3 response during a picture-viewing task (both for noises presented during pictures of differing types and during intervals between pictures). Given prior evidence linking Factor 1 psychopathy features to deficient fear response (e.g., Benning, Patrick, & Iacono, 2005; Flor, Birbaumer, Hermann, Ziegler, & Patrick, 2002; Patrick, 1994), the authors interpreted this result as evidence for a reduced “cortical call-to-arms” in relation to sudden noxious events among offenders exhibiting the core affective-interpersonal symptoms of psychopathy. Based on the notion of a continuum of threat sensitivity ranging from psychopathic fearlessness at one end to fear-disorder susceptibility at the other, individuals high in THT would be expected to show enhanced probe P3 amplitude; however, this possibility needs to be tested directly. Notably, and in contrast with above-noted findings from Patrick et al. (2013), Factor 2 features of psychopathy (which relate more to trait disinhibition; Venables & Patrick, 2012) showed only a weak, nonsignificant association with reduced probe P3 response in Drislane et al.’s (2013) study. The null relationship for disinhibitory symptoms of psychopathy in this study may reflect the high prevalence of such symptoms in incarcerated offenders and resultant problems of range restriction. Thus, research with other participant samples is needed to further test for possible associations of both THT and DIS with probe P3 amplitude.

In addition to a positive association with general amplitude of probe P3 response (i.e., across stimulus conditions), we also expected that THT would show a relationship with the degree of amplitude modulation (i.e., extent of reduction in P3 response) for noise probes occurring during aversive as compared to neutral picture foregrounds. This hypothesis is based on the conceptualization of threat sensitivity as involving heightened attention to cues for potential danger in the environment (Dennis & Chen, 2007), along with evidence pointing to attentional biases in threat detection as a factor contributing to the development of anxiety disorders (Stein & Nesse, 2011; Mathews & MacLeod, 1985). From this standpoint, individuals high in THT are expected to devote disproportionate attentional resources to processing of aversive foreground images in a picture-viewing task — and thus lesser resources to processing the noise probe — than those with lower THT scores. If so, high-THT individuals should show greater probe P3 amplitude suppression for aversive compared to neutral images than low-THT individuals, whereas THT level should not affect amplitude suppression for pleasurable versus neutral images. Drislane et al. (2013) did not specifically test for a relationship between Factor 1 features of psychopathy and aversive-neutral modulation of the probe P3 response, so the possibility of an association with THT remains to be evaluated.

The Present Study

The present study addressed these gaps in the literature by examining whether differing elements of variance in the P3 response to abrupt noise probes within a picture-viewing task might operate as physiological indicators of both INH and THT. Our specific hypotheses were as follows:

  1. As a member of the P3 family of brain-ERP responses, known to index cognitive postprocessing of salient stimuli (L. D. Nelson et al., 2011; Patrick et al., 2013), the probe P3 was predicted to show a negative relationship with variations in DIS, such that higher disinhibitory tendencies would be associated with smaller amplitude of probe P3 response.
  2. Based on recent research documenting a negative association between probe P3 amplitude and affective-interpersonal features of psychopathy (Drislane et al., 2013), and other work tying these features of psychopathy to low fearfulness (Patrick & Bernat, 2009; Patrick & Drislane, 2015), we hypothesized that THT would be related to general amplitude of probe P3 response (i.e., across foreground stimulus conditions) in a manner opposite to, and independent of, the predicted relationship for DIS (i.e., greater P3 amplitude with increasing levels of THT).
    1. Given evidence that threat sensitivity involves heightened attention to cues for potential danger, we hypothesized that high-THT participants would show enhanced modulation of P3 response to noise probes occurring in the context of aversive picture foregrounds (i.e., greater amplitude suppression relative to responses evoked during neutral foregrounds) as a function of increased allocation of attention to aversive foregrounds. That is, greater fearfulness was predicted to confer particular engagement in aversive images, resulting in a larger probe P3 amplitude difference between aversive and neutral conditions.
  3. Based on evidence that the association between probe P3 and DIS reflects a process in common with other variants of P3 (Patrick et al., 2013), we predicted that the association between the two in the current study would be accounted for by overlap between the probe P3 and another well-established ERP indicator of DIS, the P3 response to rare target stimuli in a visual oddball task (cf. Patrick et al., 2006; Yancey et al., 2013).
    1. Given the conceptualization of THT as separate from DIS (L. D. Nelson et al., 2016) and involving different neural circuitry (Patrick & Bernat, 2009; Patrick et al., 2012), we hypothesized that the variance in probe P3 related to THT reflects a different process than that related to DIS (i.e., affective, as opposed to cognitive), and thus would be unrelated to oddball-target P3 response.

Evidence in support of these hypotheses would serve to highlight differing elements of trait-relevant variance in the probe P3 response and establish this ERP measure as a neurophysiological indicator of two key dispositional constructs known to relate to multiple forms of psychopathology.



The base sample for the current study consisted of 508 adult twins (257 female), recruited from the greater Twin Cities metropolitan area, who participated for a payment of $100. Most participants were tested concurrently with their same-gender co-twin on the same day, by different experimenters in separate laboratory testing rooms. Participants were selected for lab testing based on levels of threat sensitivity (THT), as indexed by scores on the Trait Fear Inventory (measure described below), such that half of the test sample (one member of each twin pair) was pre-selected based on THT scores, to ensure comprehensive representation of individuals at high and low levels of threat sensitivity. Specifically, approximately one third were chosen to be high in THT (i.e., highest 18% of screening sample), one third low (lowest 18%), and the remaining third intermediate (19th to 82nd percentile of scorers; see L. D. Nelson et al., 2016, for further details of subject recruitment.) Participants were also screened and determined to be free of visual or hearing impairments as assessed by a screening questionnaire.

From among the base sample, 32 participants were excluded from analyses due to missing self-report data, 35 were excluded due to missing or artifact-ridden probe P3 data, and 23 others were excluded due to missing or artifact-ridden target P3 data. These exclusions resulted in a final N of 418. The mean age of study participants was 29.49 (SD = 4.84), and 213 (51.0%) were female. Consistent with the demographic makeup of the Twin Cities metropolitan area, 96.7% of participants included in analyses were Caucasian. All participants provided informed written consent, and study procedures were approved by the University of Minnesota’s Institutional Review Board.

Dispositional Measures

Disinhibition (DIS): Trait Disinhibition Scale

Participants were administered a 100-item version of the Externalizing Spectrum Inventory (ESI; Krueger, Markon, Patrick, Benning, & Kramer, 2007), a 415-item questionnaire designed to index differing expressions of disinhibitory tendencies, including irresponsibility, various forms of impulsivity, blame externalization, variants of aggression (physical, relational, and destructive), deficient empathy, rebelliousness, excitement seeking, and alcohol, drug, and marijuana use/problems. Items were answered on a four-point scale (true, somewhat true, somewhat false, false). Higher scores on the ESI can be viewed as reflecting a general lack of inhibitory control (DIS) that is associated with problematic behaviors (e.g., impulsive-aggressive behavior and substance use). Scores on the 100-item version (ESI-100) have been shown to correlate very highly (> .95) with scores on the full-form ESI (Hall et al., 2007).

The scale measure of DIS used in the current analyses was a 30-item index of trait disinhibition (DIS-30; Yancey et al., 2013) derived from the ESI-100. The 30 items of this scale are drawn from the seven ESI subscales that load most strongly and selectively on the ESI’s general disinhibitory factor (Krueger et al., 2007): Irresponsibility, Dependability, Problematic Impulsivity, Impatient Urgency, Planful Control, Alienation, and Theft (see Yancey et al., 2013). Scores on the scale were computed as mean endorsement in the keyed direction across items, on a 0 to 3 scale, with higher scores reflecting greater disinhibitory tendencies. Descriptive statistics for the DIS-30 in the analysis sample were: M = .40, SD = .34, range = .00 to 1.83. Internal consistency reliability for this scale in the current sample was high (Cronbach’s α = .87).1

Threat Sensitivity (THT): Trait Fear Inventory

Participants were assessed for THT in the self-report domain using the 55-item Trait Fear Inventory (TF-55), a scale designed to index a broad fear/fearlessness dimension identified through structural modeling analyses (Kramer et al., 2012; see also Vaidyanathan, Patrick, & Bernat, 2009; Vizueta, Patrick, Jiang, Thomas, & He, 2012). The TF-55 consists of items extracted from various established self-report questionnaires of fear and fearlessness, including the Fear Survey Schedule-III (Wolpe & Lang, 1977), the Fearfulness subscale of the EASI Temperament Survey (Buss and Plomin, 1984), the Harm Avoidance subscale of the Temperament and Personality Questionnaire (Cloninger, 1987), subscales composing Factor 1 of the Psychopathic Personality Inventory (Lilienfeld & Andrews, 1996), and the Thrill/Adventure Seeking subscale of the Sensation Seeking Scale (Zuckerman, 1979). Scores on this scale correlate very highly (r > .9) with scores on the general fear/fearlessness factor from the structural model of these differing inventories (Kramer et al., 2012; see also Patrick et al., 2012; Vaidyanathan et al., 2009). A total score was computed for each participant as the average score across individual items, each coded 0 to 3, such that higher scores reflected greater dispositional fearfulness (THT+). Descriptive statistics for this TF-55 score variable in the current analysis sample were: M = 1.12, SD = .46, range = .04 to 2.38. Internal consistency reliability (Cronbach’s α) in the analysis sample was .96.2 Within the current study sample, scores on this scale measure of THT correlated minimally with disinhibitory tendencies as indexed by the DIS-30 scale, r = .05, p = .30.

Procedure and Experimental Paradigms

Data for the current study were collected as part of a larger physiological assessment protocol. While seated in a padded recliner, participants completed a set of questionnaires including the ESI-100 and TF-55. During questionnaire administration, an electroencephalographic (EEG) cap was attached to record brain-response data. During testing, participants viewed the task stimuli on a 53.3 cm computer monitor, situated 1 m away at eye level. Stimuli were presented using a PC running E-Prime software (Psychology Software Tools, Inc.), and physiological data were collected using a second PC running Scan 4 software (Neuroscan, Inc.).

Affective Picture-Viewing Task

Probe P3 responses were elicited by presentation of noise probes during an affective picture-viewing task, which included 90 images (30 neutral, 30 aversive, 30 pleasurable) selected from the International Affective Picture System (IAPS; Lang, Bradley, & Cuthbert, 1999).3 Each picture was presented for 6 s, followed by an intertrial interval of 12 s preceding the next picture presentation, during which time a fixation cross was displayed. Neutral pictures depicted 10 household objects, 10 buildings, and 10 neutral faces. Aversive scenes comprised 20 threat pictures (aimed guns, attacking animals) and 10 mutilation pictures (injured bodies, limbs, faces). Pleasurable pictures depicted 10 erotic, 10 nurturant (babies and small animals), and 10 adventure scenes. During 81 of the 90 picture stimuli (27 per condition), noise probes (50 ms, 105 dB, 10 µs rise time) were presented at 3, 4, or 5 s into the 6 s presentation interval to elicit startle-blink and brain-ERP responses (see Yancey, Vaidyanathan, & Patrick, 2015, for a report of the startle-blink findings). Within and across orders, picture stimuli and noise probes were counterbalanced such that all picture valence categories (pleasurable, neutral, and aversive) were represented equally across orders at each serial position, with the constraint that no more than two pictures of the same valence occurred consecutively within any stimulus order, and pictures of the same content category never appeared consecutively or across orders; in addition, positions of picture stimuli were rotated across orders so that particular pictures occurred in both probed and unprobed conditions.

Visual Oddball Task

The target P3 ERP was assessed in a visual oddball task that included stimuli of three types, presented for 100 ms each and separated by 4 to 5 s intervals: frequent standards (simple ovals; 70% of 240 total trials), infrequent targets (schematic heads; 15%), and infrequent novels (affective and neutral pictures; 15%); a detailed description of the task is reported in Yancey et al. (2013). Participants responded with a left or right button-press on target “head” trials to indicate whether the “ear” appeared on the left or right side of the head.

Data Acquisition, Processing, and Reduction

EEG activity was recorded from 54 scalp sites positioned according to the 10–20 system using Neuroscan Quik-Caps inlaid with sintered Ag-AgCl electrodes. Electrodes were positioned above and below the left eye to monitor vertical electrooculogram (VEOG) activity, and adjacent to the outer canthi of the left and right eyes to monitor horizontal electrooculogram (HEOG) activity. All electrode impedances were kept below 10 KOhms. EEG signals were digitized on-line at 1000 Hz during data collection with an analog band pass filter of .05 to 200 Hz. Data were referenced to electrode site Cz during on-line data collection and arithmetically re-referenced offline to the average of left and right mastoid electrodes for subsequent processing and analysis. Data epochs from −1000 to 2000 ms were extracted from the continuous EEG recordings using EDIT version 4.3 software (Neuroscan Inc.), and corrected for eye movements using the algorithm developed by Semlitsch, Anderer, Schuster, and Presslich (1986), as implemented within the EDIT software. The segmented and eyeblink-corrected EEG data were then imported into Matlab (Mathworks, Inc.) for subsequent processing, including downsampling to 128 Hz using the Matlab resample command, which applies a low pass anti-aliasing filter before downsampling.

Trials for which activity exceeded ±75 µV in either the pre- (−1000 to 0 ms) or poststimulus (0 to 2000 ms) period were excluded from further processing. EEG activity in response to probe stimuli during the affective picture-viewing task was averaged across picture trials within each valence condition (neutral, aversive, and pleasurable) to yield three ERP waveforms. Visual inspection of these waveforms for each participant was undertaken to evaluate the effectiveness of the aforementioned criteria. Electrodes deemed to contain excessive artifact were replaced by their nearest neighboring sites. In the event that neighboring sites also contained excessive artifact, data for those subjects were excluded entirely. The aforementioned criteria resulted in the exclusion of 35 participants (7.35% of the sample having complete self-report data).

Probe P3 Quantification (Affective Picture-Viewing Task)

Probe P3 was quantified from the aggregate waveform for each picture condition as the peak amplitude occurring during a window of 250 to 351.56 ms following the onset of noise probes, relative to a 300 ms pre-probe baseline (see Figure 1; cf. Patrick et al., 2013). All analyses were conducted using peak scores from electrode site Pz, where probe P3 response in this sample was found to be maximal, consistent with prior research (e.g., Drislane et al., 2013). Split-half reliability coefficients for P3 peak scores were calculated based on average waveforms for odd- vs. even-numbered trials and corrected using the Spearman-Brown prophecy formula (Brown, 1910; Spearman, 1910) to account for the smaller number of trials included in the split halves. Reliability was very high for each picture-valence category (neutral: r = .90, p < .001; aversive: r = .92, p < .001; pleasurable: r = .90, p < .001), and even higher as expected for overall (i.e., averaged) amplitude across picture categories (r = .96, p < .001). Split-half reliabilities for condition-difference scores were expectably weaker (r for aversive-neutral difference = .29, p < .001; r for pleasurable-neutral difference = .23, p < .01), owing to subtraction of substantial systematic variance.

Figure 1
(a) Average event-related potential (ERP) waveforms for participants as a whole (N = 418) at electrode site Pz for noise probes presented while viewing pictures of differing valences (pleasurable, neutral, aversive). (b) Color topographic plot (“head ...

Target P3 Quantification (Visual Oddball Task)

EEG activity from the visual oddball task was aggregated by stimulus type to yield ERP waveforms for target, novel, and standard stimuli. Following prior research (Yancey et al., 2013), target P3 amplitude was calculated as the maximum voltage peak occurring at electrode site Pz (where the response was found to be maximal) between 297 and 602 ms relative to a preceding baseline (−136 to −8 ms). Split-half reliability, calculated in the corrected manner described above, was very high (r = .92, p < .001). Although measured in a separate task procedure, target P3 amplitude correlated to a moderate positive degree (r = .32, p < .001) with noise-probe P3 amplitude across conditions in the current participant sample.

Data Analyses

A one-way repeated-measures analysis of variance (ANOVA), with picture type (pleasurable, neutral, aversive) as the within-subjects factor, was used to test for the expected modulatory effect of foreground valence on probe P3 amplitude (Cuthbert et al., 1998; Drislane et al., 2013). Follow-up contrasts comparing the neutral condition to the aversive and pleasurable conditions were used to clarify the basis of the omnibus valence effect.

Two-way mixed-model ANOVAs, including trait score (either DIS or THT) as a continuous between-subjects factor along with picture type as a discrete within-subjects factor, were performed to examine associations of each trait with probe P3 amplitude. In addition to main effect of each trait on P3 amplitude as a whole (i.e., across picture types; Hypotheses 1 and 2), a trait by picture type interaction was also predicted for the analysis involving the THT variable. To test our specific hypothesis that THT would show a negative association with the aversive-neutral, but not the pleasurable-neutral, difference in probe P3 amplitude (Hypothesis 2a), follow-up analyses were run including THT as a factor along with either (a) aversive and neutral picture conditions, or (b) pleasurable and neutral picture conditions.

In addition to the mixed-model ANOVAs, correlational analyses were performed to quantify associations of each trait variable (DIS, THT) with (a) general probe P3 reactivity, and (b) aversive-foreground modulation of probe P3; to separate these two response parameters, analyses of general reactivity associations focused on probe P3 amplitude for neutral-picture trials, and analyses of aversive-modulation effects focused on P3 difference scores for aversive minus neutral trials. Simple (zero-order) correlations were computed for each trait as a predictor of each response parameter, and regression analyses utilizing the two traits as concurrent predictors were used to test for unique relations of each with general reactivity (Hypotheses 1 and 2), and of THT with aversive modulation (Hypothesis 2a). Along with these analyses, hierarchical regression analyses were performed to test for overlap between probe P3 and target P3 in predicting DIS (Hypothesis 3), but not THT (Hypothesis 3a). In these analyses, target P3 was entered as a predictor of one or the other trait in step 1, followed by probe P3 in step 2, to assess whether probe P3 contributed to prediction as a function of, or separately from, its association with target P3.


The initial one-way ANOVA revealed a significant main effect of picture type on probe P3 amplitude for participants as a whole, F2,416 = 149.87, p < .001, with follow-up contrasts revealing reduced peak amplitude for each of the affective conditions relative to the neutral condition: aversive versus neutral F1,417 = 181.26, p < .001; pleasurable versus neutral F1,417 = 278.01, p < .001.

Consistent with prediction, the two-way mixed-model ANOVAs including DIS or THT along with picture type as independent variables (IVs) demonstrated significant main effects for each trait on probe P3 amplitude (i.e., across picture types): for DIS, F1,416 = 7.56, p < .01; for THT, F1,416 = 8.15, p = .005. In addition, as expected, a significant trait-by-picture-type interaction was evident in the analysis for THT, F2,415 = 3.22, p < .05 (but not DIS, F2,415 = .42, p = .66), with follow-up tests revealing a significant effect of THT on the degree of probe P3 modulation for aversive relative to neutral pictures (i.e., greater amplitude suppression for aversive with higher THT), F1,416 = 6.45, p = .01, but not for pleasurable pictures relative to neutral, F1,416 = 2.16, p = .14.

Regression Analyses

DIS and THT each showed significant zero-order correlations with general probe P3 reactivity (quantified as amplitude of P3 response to noise probes presented during viewing of neutral pictures), rs = −.13 and .16, respectively, ps < .005 (see Figure 2). However, THT alone showed a significant zero-order correlation with aversive-neutral modulation, r = .12, p = .01 (r for DIS = −.02, p = .64). When a regression analysis was run with DIS and THT included together as predictors of neutral-condition probe P3 amplitude, the model as a whole was significant, R = .21, p < .001, with both traits contributing uniquely to prediction in opposing directions: Higher DIS predicted decreased probe P3 amplitude (β = −.14, p < .005), whereas higher THT predicted increased amplitude (β = .17, p < .005). When the two traits were entered together as predictors of the aversive-neutral difference score reflecting modulation of probe P3 amplitude under aversive viewing conditions, the model as a whole was again significant, R = .13, p < .05, but with a unique predictive association evident for THT only, β = −.13, p = .01 (β for DIS = .02, p = .64).

Figure 2
Scatter plots for the study sample as a whole (N = 418) depicting relationships between participants’ trait scores and general probe P3 reactivity (i.e., amplitude of P3 response to noise probes presented during viewing of neutral pictures). The ...

Results in line with hypotheses also emerged from the hierarchical regression analyses with target P3 amplitude entered in step 1 and general probe P3 reactivity (i.e., amplitude during neutral trials) entered in step 2 as predictors of one or the other trait. In the regression analysis for DIS, significant prediction was evident for target P3 at step 1 (R2 = .03, F1,416 = 12.24, p = .001), but probe P3 reactivity did not account for additional variance at step 2 (F1,415 = 3.29, p > .07). By contrast, in the analysis for THT, target P3 did not predict significantly at step 1 (R2 < .001, F1,416 = .003, p = .96), whereas probe P3 reactivity did show significant prediction at step 2 (R2 change = .03, F1,415 = 11.90, p = .001).


The present study identified separate components of variance within the noise-probe P3 brain response related to two biobehavioral traits — disinhibition (DIS) and threat sensitivity (THT) — that are known to account for appreciable variance in psychological problems of various types (L. D. Nelson et al., 2016; Patrick et al., 2012, 2013). Consistent with hypotheses based on prior work (e.g., Drislane et al., 2013; Patrick et al., 2013), DIS was associated with reduced amplitude and THT with increased amplitude of general probe P3 reactivity.4 Notably, the correlation of DIS with general probe P3 reactivity overlapped with that for target P3 response from an oddball task, whereas the association for THT did not, consistent with the idea that separate processes underlie relationships for the two traits. Additionally, high-THT participants showed increased affect-driven attention (Lang, Bradley, & Cuthbert, 1997) to aversive foregrounds (Matthews & MacLeod, 1985), as evidenced by greater attentional suppression of probe P3 during viewing of aversive versus neutral pictures, relative to low-THT participants.

Unique Predictive Associations of DIS and THT with Probe P3 Response

The observed negative association for DIS with general probe P3 reactivity (i.e., decreasing amplitude of probe P3 with increasing levels of DIS) mirrors relationships with other variants of P3 reported consistently in prior published work, including responses to oddball task stimuli (both target and novel stimuli), target stimuli in a flanker discrimination task, and feedback cues in a choice-feedback paradigm (L. D. Nelson et al., 2011; Patrick et al., 2006, 2013; Venables & Patrick, 2014; see also Iacono, Carlson, Malone, & McGue, 2002). Taken together, these findings indicate that high trait disinhibition involves a neural processing deviation that affects P3 responding across different tasks and processing contexts. Research with twin samples has shown this DIS-P3 relationship to be mediated largely by common genetic influences between the two (Yancey et al., 2013; see also Hicks et al., 2007). Together with longitudinal work demonstrating that reduced P3 responding in young individuals at risk for externalizing problems predicts the later emergence of such problems (e.g., Berman, Whipple, Fitch, & Noble, 1993; Iacono et al., 2002), this literature suggests that reduced P3 amplitude reflects a neural process associated with underlying genetic liability for externalizing problems (cf. Krueger et al., 2002).

What DIS-related processing deviation does reduced P3 amplitude reflect? Notably, while impaired in terms of P3 responding, highly disinhibited individuals display intact processing of perceptual elements of stimuli, such that they are effectively able to discriminate briefly presented visual stimuli (Hall et al., 2007; Patrick et al., 2006) and show expected brain-response differentiation between emotional and neutral stimulus events (Patrick & Bernat, 2009). As a salient example of this, Bernat et al. (2011) reported that participants high in trait disinhibition processed the affective significance of feedback stimuli normally, as evidenced by expected enhancement of the brain feedback-related negativity response following loss versus gain feedback, while at the same time showing reduced amplitude of subsequent P3 response to feedback stimuli of both types. Consistent with this, DIS scores in the current study were negatively related to general probe P3 reactivity, but unrelated to affective-modulation of the response, as evidenced by the lack of any DIS-by-picture-type interaction. Based on these and other data, Patrick and Bernat (2009) postulated that high disinhibition involves a selective impairment in deeper elaborative (connotative) processing of task stimuli that normally occurs along with more basic processing related to performance of the task. They suggested that elaborative processing of this type involves comparing and integrating ongoing perceptual events with neuro-cognitive representations stored in long-term memory (Miller & Cohen, 2001; see also Ericsson & Kintsch, 1995) and is essential to adaptive anticipation, reflection, and self-regulation. However, this theoretical account remains speculative, and further research is needed to clarify the precise nature of neural processing deficits underlying reduced P3 amplitude in relation to trait disinhibition.

The contrasting positive relationship for THT with general noise-probe P3 response fits with prior research demonstrating an opposing relationship for offenders scoring high on affective-interpersonal features of psychopathy, which are theorized to reflect a deficiency in fear and correlate with psychological-scale and physiological-response measures in ways consistent with this view (e.g., Benning, Patrick, & Iacono, 2005; Benning, Patrick, Salekin, & Leistico, 2005; Brislin, Drislane, Smith, Edens, & Patrick, 2015; Vaidyanathan et al., 2009; Vaidyanathan, Hall, Patrick, & Bernat, 2011). Within the current study sample (as in other participant samples, e.g., Brislin et al., 2015; Venables et al., 2015), scores on THT were uncorrelated with scores on DIS (r = .05, p = .30), and inclusion of these two trait variables together in a regression model confirmed separate associations for each with general (i.e., neutral-condition) probe P3 response. In addition, hierarchical regression analyses incorporating target-P3 amplitude as a predictor revealed that the variance in probe P3 related to THT, in contrast with that related to DIS, was independent of this cognitive-task P3 variant. The implication is that some other process underlies the general enhancement of probe P3 for participants high as compared to low in THT. We postulate that this enhancement reflects a distinct, defensive-alerting process triggered by the unexpected, noxious nature of the noise-probe stimulus. Drislane et al. (2013), citing Graham (1979), characterized this process as a “cortical call-to-arms,” that is, a specific allocation of cognitive-attentional resources for purposes of ascertaining the need to mobilize for evasive action. This interpretation of enhanced P3 response to noxious noise probes in relation to THT dovetails with other work demonstrating enhanced somatic and visceral responding (e.g., startle-blink, corrugator muscle, heart rate) to aversive visual stimuli in high-THT participants (Vaidyanathan et al., 2009; Yancey et al., 2016).

In addition to a positive association with general probe P3 amplitude, THT also showed a relationship with attentional modulation of probe P3 during aversive-picture foregrounds (i.e., higher THT predicted greater dampening of probe P3 response during viewing of aversive relative to neutral pictures). No such effect was evident for pleasurable-picture modulation. As discussed earlier, threat sensitivity (e.g., among individuals with anxiety disorders) has been theorized to involve heightened attentiveness to cues for potential danger in the environment. Our finding for P3 amplitude modulation is consistent with this literature, indicating that high-THT participants in the current study attended more strongly to aversive visual foregrounds than low-THT participants, resulting in decreased resources to process auditory probes at the time of their occurrence. The lack of a corresponding relationship with pleasurable-picture modulation highlights the specificity of this hyperattentiveness effect to aversive visual foregrounds. As discussed in the next subsection, this selective dampening effect of THT on probe P3 during aversive picture viewing is especially intriguing because it occurred in connection with an amplifying effect of THT on general probe P3 reactivity (e.g., under neutral viewing conditions). This finding points to distinct THT-related processes affecting probe P3: one having to do with enhanced defensive alerting to noxious noise stimuli in general, and the other with increased allocation of attention to aversive visual foregrounds specifically.

Measurement Issues Highlighted by Current Study Findings

Our findings as described above point to potential uses of noise-probe P3 for indexing core trait constructs that play a role in clinical problems of various types (L. D. Nelson et al., 2016; Patrick et al., 2013; Yancey et al., 2016). However, in moving toward the use of brain and other physiological variables as indicators of psychological traits, some important methodological issues need to be addressed. Findings from the current work serve to underscore some of these.

Reliability of physiological indicators

One key methodological issue is that the validity of a physiological variable as an indicator of a psychological characteristic of interest is constrained first by its reliability (Anastasi & Urbina, 1997). Physiological variables with weak reliability are capable of showing only modest relations (at best) with other variables. Difference-score variables, though widely used in psychophysiological research on individual differences, are problematic from a reliability standpoint due to exclusion of systematic (true-score) variance through subtraction, resulting in scores with a lower true-variance/error-variance ratio. The attenuating impact of subtraction becomes larger as the correlation between variables used to form difference scores increases, as covariance reflects true-score variance in each. This consideration is illustrated in the current work by the weak reliability of difference scores used to index modulatory effects of foreground-attention on probe P3 response for aversive and pleasurable pictures relative to neutral. Split-half reliability coefficients for these difference-score variables were modest (~.25) compared to the very high coefficients for P3 response scores in individual picture conditions (~.90), which correlated strongly with one another (rs = .88 to 89). Given their weak reliability, modulation scores can be expected to correlate only weakly to moderately with other variables, depending upon their domain of measurement and score reliability, and are less likely to show replicable relations across smaller-N samples. However, as discussed under the “Difference-score variables” subsection below (with reference to current study data), difference scores can have unique advantages for certain purposes, and there are statistical methods that can be used to augment the measurement signal they contain.

It should also be noted that alternatives exist to basic classical test theory approaches to reliability estimation (e.g., split-half or test-retest methods). One of these is generalizability (g) theory, which accounts for variability in assessment conditions (e.g., setting, time, items, raters) that can affect measurements (Clayson & Miller, in press). Another is item-response theory, in which reliability is estimated for varying levels of a score continuum, rather than for a ‘test’ (measured attribute) as a whole. Alternative approaches such as these may be useful for overcoming some of the limitations of classical reliability methods and moving the field forward in interesting ways.

Trait-related variance in physiological indicators

Another key issue spotlighted by the current work is that only a portion of the systematic, reliable variance in a candidate physiological indicator can be expected to relate to a target psychological characteristic that is quantified through self- or other-report (e.g., as scores from a questionnaire or interview). The reason is that variables from a particular domain of measurement contain method variance unique to that domain. Consequently, measures of highly similar constructs from separate domains of measurement (e.g., self-report, physiology) are likely to correlate only moderately at best (i.e., .4 to .6), and measures of only somewhat related constructs from separate domains are likely to correlate only modestly at best (i.e., .1 to .3; Campbell & Fiske, 1959). For example, in the current study, the correlations of DIS and THT with general probe P3 reactivity were robust (see Figure 2) but small in magnitude (rs = −.13 and .16, respectively). Given that score reliability for the index of general reactivity (i.e., probe P3 amplitude during neutral picture trials) was .90, this means that only about 2% of the systematic variance in general probe P3 response was related to DIS (i.e., .132/.90 * 100) and only about 3% was related to THT.

The fact that only a small portion of the reliable variance in probe P3 response relates to these trait variables reflects the fact that probe P3 is not a direct measure of either trait; rather, it is an indicator assessed in a separate domain (i.e., neural-response) from the traits (self-report). More broadly, relationships of any physiological response variable — whether somatic, visceral, electrocortical, or neuroimaging — with individual-difference constructs assessed using report-based measures (questionnaires or informant-/clinician-ratings) are likely to be only modest in magnitude (i.e., .1 to .3); correlations larger than this may emerge in certain small, select samples, but generally not in large, representative samples. For this reason, individual physiological indicators alone cannot be used as substitutes for report-based scale measures of psychological attributes: Too little of the variance in any particular physiological variable will reflect the trait construct of interest. However, as discussed in the closing section below, this “low-signal” problem can be addressed by (a) aggregating across different physiological indicators that contain trait-related variance, and (b) formulating cross-domain measurement models that include trait indicators from both physiological-response and psychological-report domains.

Difference-score variables

The portions of variance in general probe P3 reactivity related to the two traits are interesting to compare with the portion of variance in aversive-neutral modulation related to THT specifically. As a condition-difference variable, the aversive-neutral modulation score was markedly less reliable (split-half reliability = .29) than the raw P3 amplitude scores for individual conditions. As a function of this, the correlation of the P3 modulation score with THT (r = −.12), while lower in absolute terms than the r for general reactivity (r = .16), translates into a larger proportion of systematic variance in the modulation score that is associated with the trait variable (i.e., .122/.29 * 100 ≈ 5%). That is, more of the true-score variance in the modulation score (reflected in its reliability) is related to THT. This analytic picture aligns with the conceptual view that a condition-difference score can serve to index a specific process of interest (e.g., aversive-foreground engagement) by removing the variance it contains in common with a control condition (e.g., neutral-foreground engagement). In the current study, subtracting the neutral-condition variance from the aversive condition in the present case was especially critical for indexing attentional suppression because the relationship of THT with the variance unique to the aversive condition, reflected in the difference score, was in the opposite direction (i.e., negative) from its association with neutral-condition reactivity (i.e., positive). This particular finding of the current study illustrates how difference-score indicators can provide unique information for certain measurement purposes; however, as discussed in the closing section below, statistical methods must be used to distinguish systematic variance of interest in such indicators from the larger proportion of error variance contained within them.

Single brain measures can index different traits

A final issue highlighted by the current findings is that physiological indicators can contain systematic variance related to more than one psychological trait construct. The probe P3 response is a particularly interesting example, given that other variants of P3 (e.g., oddball-target P3) relate exclusively to DIS, whereas probe P3 relates to THT as well as DIS. It seems likely that distinctive features of the current task — i.e., the abrupt, unexpected nature of the noise-probe stimulus used to evoke the P3 response during picture-viewing — resulted in a distinct type of P3 containing variance related to THT. This finding illustrates the broader point that the parameters of a task context can affect sources of trait-related variance in a particular physiological response. In cases where a physiological indicator contains variance related to more than one trait, the challenge becomes one of isolating portions of variance that are relevant to one trait versus another. Efforts to do so can contribute to our understanding of different neuropsychological processes underlying a particular physiological response variable and help to clarify aspects of psychological traits that intersect with the physiological domain.

The next, closing section discusses approaches that can be taken to cope with the foregoing methodological challenges, in the interest of advancing our knowledge of intersections between biobehavioral processes and clinical problems.

Strategies for Addressing Measurement Challenges in Psychophysiological Research on Individual Differences

The work presented here highlights measurement issues in attempting to integrate variables across different domains of measurement (units of analysis). Consistent with the aims of the RDoC initiative, the current work focused on biobehavioral traits — weak inhibitory control (DIS) and threat sensitivity (DIS) — that reflect psychological processes relevant to multiple forms of psychopathology. We used data for the probe P3 to illustrate how a physiological-response measure can contain multiple elements of variance reflecting separate neural processes, some of which may relate to different psychological attributes. As an illustration of this point and its implications for physiologically oriented clinical assessment, the current study featured some notable strengths, including the use of a moderately large participant sample recruited to represent a broad range of trait scores, reliance on well-validated measures of DIS and THT with known relations to multiple forms of psychopathology (e.g., L. D. Nelson et al., 2016; Patrick et al., 2012, 2013), and the use of multiple regression to parse apart trait-related variance in physiological measures.

However, a major limitation of the current study, characteristic of much existing psychophysiological work on individual differences and psychopathology, is that it focused primarily on a single measure of physiological response — namely, amplitude of the noise-elicited P3 response. While this narrow focus was useful for illustrating how differing portions of variance in a single physiological indicator can relate to different psychological attributes, we advocate for increasing use of a multivariate approach in psychophysiological research on individual differences (Gilmore, Malone, & Iacono, 2010; L. D. Nelson et al., 2011). Results from the current work illustrate that only a small fraction of the systematic variance in a physiological indicator can be expected to relate to a psychological attribute assessed using a report-based approach. To increase the psychological-trait “signal” in physiological assessments, it will be necessary to aggregate across different physiological indicators that contain common trait-related variance, treating each as an “item” of a composite trait measure, rather than as a measure of the trait in itself (cf. L. D. Nelson et al., 2011). Taking this approach, individual indicators with modest reliability that include meaningful amounts of trait-related variance (e.g., condition-difference variables) can serve a useful measurement role — contributing both to measurement precision and aggregate reliability — when combined with other indicators into a composite index of the trait.

A critical step in moving toward a multivariate approach to physiologically oriented clinical assessment is to identify different physiological indicators of a clinical target construct that covary with one another and thus can be profitably combined. Two major practices in the field have impeded progress in this direction. One of these, emphasized heavily in RDoC publications (Insel et al., 2010; Kozak & Cuthbert, 2016), has been the focus for many years on arbitrary diagnostic categories as targets for research on physiological mechanisms and measures. The other is the “after-the-fact” approach that has been taken in efforts to identify physiological-response correlates of clinical conditions or clinically relevant attributes: Conditions or attributes are first characterized in clinical-psychological terms, without reference to biobehavioral systems/processes, and then work is undertaken to identify their physiological correlates.

In recent writings (Patrick & Bernat, 2010; Patrick et al., 2012; Yancey et al., 2016), we have proposed a psychoneurometric research paradigm for addressing these problems. This paradigm calls for use of biobehavioral trait constructs — dispositional counterparts to process constructs from the RDoC matrix (e.g., DIS and THT, corresponding to RDoC constructs of response inhibition and acute threat, respectively) — as referents for research on physiological indicators of psychopathology, rather than using diagnostic conditions or symptoms. Focusing on traits of this type is advantageous because they relate directly to biobehavioral systems/processes as well as to clinical problems of various types. Psychological scales provide useful starting points for operationalizing target traits and mapping psychological correlates because they have strong psychometric properties and are efficient to administer. Once multiple physiological indicators of a clinically relevant trait such as DIS or THT have been identified, they can be aggregated together — either unto themselves (L. D. Nelson et al., 2011) or in conjunction with scale indicators (e.g., Patrick et al., 2013; Yancey et al., 2016) — to form reliable trait composites that correlate robustly with physiological as well as clinical criterion measures. However, the aim of this research strategy is not simply to identify correlates of scale-assessed psychological attributes in an “after-the-fact” manner. The psychoneurometric approach also provides a means whereby knowledge of physiological correlates and their interrelations can feed back into conceptualizations of the traits themselves, leading to new biopsychological conceptions of traits that directly incorporate data from the domain of physiological response (Patrick et al., 2012; Yancey et al., 2016).

Findings from the current study serve to illustrate how knowledge of physiological correlates can feed into conceptualizations of traits themselves. Our finding that scale measures of DIS and THT showed separate, opposing relationships with probe P3 indicates that these two reported dispositions affect post-perceptual processing of an abrupt, noxious stimulus in different ways. Complementing this, we found that (a) DIS alone was related to oddball-target P3 response, as a function of target P3’s overlap with probe P3, and (b) THT alone was related to unique variance in probe P3 response within the aversive-picture condition (i.e., variance reflecting foreground-attentional engagement, as distinct from general reactivity). Taken together, the findings for DIS point to a general impairment in elaborative-associative processing of stimuli, separate from basic affect-driven processing, that is reflected in reduced P3 responding across a range of lab tasks (and, potentially, real-world contexts). As indicators of a common DIS-related neural process, P3 variants from different tasks can be aggregated into a composite neurometric index of DIS that relates strongly to brain-response criteria (L. D. Nelson et al., 2011), or combined together with scale measures of DIS into a psychoneurometric composite that relates robustly to clinical as well as brain-response criteria (Patrick et al., 2013). On the other hand, current findings for THT point to heightened attention to ongoing aversive stimuli, and amplified reactivity to phasic aversive events, as neural processes relevant to variations in threat sensitivity that contribute to fear pathology (or conversely, to psychopathic tendencies involving low fear). As with DIS, physiological indicators of THT can be combined with one another and with scale measure of THT to form composites for use in research on neural mechanisms of fear-related problems (Yancey et al., 2016). In line with classic writings on construct validity (Cronbach & Meehl, 1955; Loevinger, 1957), composite measures that incorporate physiological indicators can serve as points of reference for reconceptualizing psychological traits in ways that connect more clearly to biological systems/processes (Patrick et al., 2013; Yancey et al., 2016).

As a final point, powerful alternative methods exist for aggregating different indicators (“items”) into composite measures (“scales”) beyond the simple summation or averaging approaches used in standard report-based assessment. Statistical modeling methods provide a means for quantifying systematic overlap among sets of indicators, as well as for addressing instances in which individual indicators contain separable elements of variance reflecting distinguishable attributes or processes. Two such approaches are structural equation modeling (SEM) and item-response theory (IRT) modeling. SEM can be used to specify latent variables (factors) reflecting interrelated portions of variance among different measured variables. Because latent factors are defined only by systematic overlapping variance (i.e., covariance) among different indicators, they are free of measurement error associated with individual indicators and thus exhibit higher reliability, allowing for stronger correlations with criterion measures. In principle, SEM could be used to define latent factors reflecting the covariance among indicators from different measurement domains (Balsis, Choudhury, Geraci, & Patrick, 2016) — for example, to combine scale and neurophysiological indicators of DIS or EXT into hybrid trait factors. SEM can also be used to partition variance in a particular indicator that relates to separate attributes or processes, by specifying loadings for the indicator on different latent variables in the structural model. For example, in a model that included multiple indicators of both DIS and THT, a variable such as general probe P3 reactivity could be specified as cross-loading on separate factors corresponding to the two traits. IRT is another statistical modeling approach that can be used in similar ways, but with greater emphasis on the comparative measurement properties of individual variates as indicators of latent attributes or processes. (For a detailed discussion of applications of IRT modeling to multi-domain assessment, see Balsis et al., 2016.)

In sum, while modest in scope and in need of replication and extension, the current work highlights a number of important measurement issues confronting efforts to quantify clinically relevant person characteristics using physiological-response measures. Together with other articles in this special issue, it challenges the prevailing view that individual physiological indicators (so-called biomarkers) can serve as effective measures of psychological attributes — a perspective that has been rightfully criticized for its oversimplification of relationships that account for only a tiny portion of observed variance. Additionally, in line with the RDoC initiative, our findings underscore the importance of combining data for multiple indicators, across different domains of measurement, to operationalize core biobehavioral constructs in physiologically oriented studies of clinical problems. It is only through rigorous attention to basic measurement principles and the application of suitable statistical methods to data from multiple response domains that systematic progress will be made toward an effective, biologically informed science of psychopathology.


  • Probe P3 ERP response relates to multiple clinically relevant psychological traits
  • Findings discourage use of biomarkers as proxies for self-report variables
  • Measurement issues must be considered in biological research on mental health

Supplementary Material


This work was supported by grants P50 MH072850 and RC1 MH089727 from the National Institute of Mental Health, and grant W911NF-14-1-0027 from the US Army. The content of this paper is solely the responsibility of the authors and does not necessarily reflect the official views of the U.S. Government, Department of Defense, Department of the Army, Department of Veterans Affairs, or U.S. Recruiting Command. Funding sources had no role in the study design in the collection, analysis and interpretation of data, in the writing of the report, or in the decision to submit the article for publication.


Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

1Due to missing scores for certain items in a small portion of cases, Cronbach’s α for the DIS-30 was computed for a somewhat reduced sample (n = 409).

2Due to missing scores for certain items in a portion of cases, Cronbach’s α for the TF-55 as a whole was computed for a reduced participant sample (n = 380).

3IAPS numbers for the picture stimuli used in the current study were as follows. Neutral: 2038, 2102, 2190, 2215, 2280, 2393, 2397, 2480, 2840, 2890, 5510, 5740, 7000, 7004, 7010, 7020, 7035, 7041, 7050, 7059, 7100, 7150, 7175, 7179, 7185, 7187, 7491, 7510, 7700, 7705. Aversive: 1050, 1205, 1220, 1300, 1525, 2692, 2811, 3000, 3010, 3053, 3060, 3064, 3071, 3080, 3102, 3120, 3130, 3280, 6200, 6210, 6213, 6230, 6243, 6244, 6250, 6260, 6300, 6370, 6570, 6830. Pleasurable: 1440, 1710, 1750, 2040, 2058, 2071, 2080, 2150, 2154, 2340, 2530, 4180 (male only), 4210 (male only), 4232 (male only), 4538 (female only), 4542 (female only), 4572 (female only), 4659, 4660, 4670, 4681, 4687, 4695, 5621, 8030, 8050, 8080, 8180, 8185, 8186, 8200, 8370, 8490.

4The correlation for DIS with general probe P3 reactivity in the current sample (−.13) was weaker than that for THT (.16). A factor contributing to this may be that high-DIS scorers were not strongly represented in the current study (Figure 1, upper plot), whereas due to pre-selection as described in the Method section, high-THT scorers were (Figure 1, lower plot). In future studies of this type, it will be valuable to pre-select for both trait dimensions to ensure strong represention of extreme scorers on each.

The authors have no financial disclosures or competing interests.


  • American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 5th. Washington, DC: Author; 2013.
  • Anastasi A, Urbina S. Psychological testing. 7th. New York: Pearson; 1997.
  • Balsis S, Choudhury T, Geraci L, Patrick CJ. Multi-domain assessment of psychological attributes: A review and illustration focusing on Alzheimer’s disease. Manuscript submitted for publication. 2016
  • Benning SD, Patrick CJ, Iacono WG. Psychopathy, startle blink modulation, and electrodermal reactivity in twin men. Psychophysiology. 2005;42(6):753–762. [PMC free article] [PubMed]
  • Benning SD, Patrick CJ, Salekin RT, Leistico AR. Convergent and discriminant validity of psychopathy factors assessed via self-report: A comparison of three instruments. Assessment. 2005;12(3):270–289. [PMC free article] [PubMed]
  • Berman SM, Whipple SC, Fitch RJ, Noble EP. P3 in young boys as a predictor of adolescent substance use. Alcohol. 1993;10(1):69–76. [PubMed]
  • Bernat EM, Nelson LD, Steele VR, Gehring WJ, Patrick CJ. Externalizing psychopathology and gain-loss feedback in a simulated gambling task: Dissociable components of brain response revealed by time-frequency analysis. Journal of Abnormal Psychology. 2011;120(2):352–364. [PMC free article] [PubMed]
  • Blair RJR, White SF, Meffert H, Hwang S. Disruptive behavior disorders: Taking an RDoC(ish) approach. Current Topics in Behavioral Neuroscience. 2014;16:319–336. [PubMed]
  • Brislin SJ, Drislane LE, Smith ST, Edens JF, Patrick CJ. Development and validation of triarchic psychopathy scales from the Multidimensional Personality Questionnaire. Psychological Assessment. 2015;27(3):838–851. [PMC free article] [PubMed]
  • Brown W. Some experimental results in the correlation of mental abilities. British Journal of Psychology. 1910;3:296–322.
  • Buss AH, Plomin R. Temperament: early developing personality traits. Hillsdale, NJ: Erlbaum; 1984.
  • Campbell DT, Fiske DW. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin. 1959;56(2):81–105. [PubMed]
  • Cicchetti D. The emergence of developmental psychopathology. Child Development. 1984;55:1–7. [PubMed]
  • Clayson PE, Miller GA. Psychometric considerations in the measurement of event-related brain potentials: Guidelines for measurement and reporting. International Journal of Psychophysiology. in press. [PubMed]
  • Cloninger CR. A systematic method for clinical description and classification of personality variants: A proposal. Archives of General Psychiatry. 1987;44(6):573–588. [PubMed]
  • Cronbach LJ, Meehl PE. Construct validity in psychological tests. Psychological Bulletin. 1955;52(4):281–302. [PubMed]
  • Cuthbert BN, Schupp HT, Bradley M, McManis M, Lang PJ. Probing affective pictures: Attended startle and tone probes. Psychophysiology. 1998;35:344–347. [PubMed]
  • Dennis TA, Chen CC. Neurophysiological mechanisms in the emotional modulation of attention: The interplay between threat sensitivity and attentional control. Biological Psychology. 2007;76(1–2):1–10. [PMC free article] [PubMed]
  • Drislane LE, Vaidyanathan U, Patrick CJ. Reduced cortical call to arms differentiates psychopathy from antisocial personality disorder. Psychological Medicine. 2013;43:825–835. [PubMed]
  • Ericsson KA, Kintsch W. Long-term working memory. Psychological Review. 1995;102(2):211–245. [PubMed]
  • Flor H, Birbaumer N, Hermann C, Ziegler S, Patrick CJ. Aversive Pavlovian conditioning in psychopaths: Peripheral and central correlates. Psychophysiology. 2002;39:505–518. [PubMed]
  • Gilmore CS, Malone SM, Iacono WG. Brain electrophysiological endophenotypes for externalizing psychopathology: A multivariate approach. Behavior Genetics. 2010;40(2):186–200. [PMC free article] [PubMed]
  • Graham FK. Distinguishing among orienting, defense, and startle reflexes. In: Kimmel HD, van Olst EH, Orlebeke JF, editors. The orienting reflex in humans. Hillsdale, NJ: Erlbaum; 1979. pp. 137–167.
  • Hall JR, Bernat EM, Patrick CJ. Externalizing psychopathology and the error-related negativity. Psychological Science. 2007;18:326–333. [PMC free article] [PubMed]
  • Hare RD. The Psychopathy Checklist – Revised. 2nd. Toronto: Multi-Health Systems; 2003.
  • Hicks BM, Bernat E, Malone SM, Iacono WG, Patrick CJ, Krueger RF, McGue M. Genes mediate the association between P3 amplitude and externalizing disorders. Psychophysiology. 2007;44:98–105. [PMC free article] [PubMed]
  • Iacono WG, Carlson SR, Malone SM, McGue M. P3 event-related potential amplitude and the risk for disinhibitory disorders in adolescent boys. Archives of General Psychiatry. 2002;59(8):750–757. [PubMed]
  • Insel T, Cuthbert B, Garvey M, Heinssen R, Pine DS, Quinn K, Wang P. Research Domain Criteria (RDoC): Toward a new classification framework for research on mental disorders. American Journal of Psychiatry. 2010;167(7):748–751. [PubMed]
  • Kozac MJ, Cuthbert BN. The NIMH Research Domain Criteria initiative: Background, issues, and pragmatics. Psychophysiology. 2016;53(3):286–297. [PubMed]
  • Kramer MD, Patrick CJ, Krueger RF, Gasperi M. Delineating physiologic defensive reactivity in the domain of self-report: Phenotypic and etiologic structure of dispositional fear. Psychological Medicine. 2012;42(6):1305–1320. [PubMed]
  • Krueger RF, Hicks BM, Patrick CJ, Carlson SR, Iacono WG, McGue M. Etiologic connections among substance dependence, antisocial behavior, and personality: Modeling the externalizing spectrum. Journal of Abnormal Psychology. 2002;111:411–424. [PubMed]
  • Krueger RF, Markon KE, Patrick CJ, Benning SD, Kramer MD. Linking antisocial behavior, substance use, and personality: An integrative quantitative model of the adult externalizing spectrum. Journal of Abnormal Psychology. 2007;116:645–666. [PMC free article] [PubMed]
  • Lang PJ, Bradley MM, Cuthbert BN. A motivational analysis of emotion: Reflex-cortex connections. Psychological Science. 1992;3(1):44–49.
  • Lang PJ, Bradley MM, Cuthbert BN. Motivated attention: Affect, activation, and action. In: Lang PJ, Simons RF, Balaban MT, editors. Attention and Orienting: Sensory and Motivational Processes. Mahwah, NJ: Erlbaum; 1997. pp. 97–135.
  • Lang P, Bradley M, Cuthbert B. International Affective Picture System (IAPS): Technical Manual and Affective Ratings. Gainesville, FL: University of Florida Center for Research in Psychophysiology; 1999.
  • Lenzenweger MF. Thinking clearly about the endophenotype-intermediate phenotype-biomarker distinctions in developmental psychopathology research. Developmental Psychopathology. 2013;25(4 pt. 2):1347–1357. [PubMed]
  • Lilienfeld SO, Andrews BP. Development and preliminary validation of a self-report measure of psychopathic personality traits in noncriminal populations. Journal of Personality Assessment. 1996;66(3):488–524. [PubMed]
  • Loevinger J. Objective tests as instruments of psychological theory. Psychological Reports Monograph Supplement. 1957;3:635–694.
  • Mathews A, MacLeod C. Selective processing of threat cues in anxiety states. Behaviour Research & Therapy. 1985;23(5):563–569. [PubMed]
  • Miller EK, Cohen JD. An integrative theory of prefrontal cortex function. Annual Review of Neuroscience. 2001;24:167–202. [PubMed]
  • Miller GA, Rockstroh B. Endophenotypes in psychopathology research: Where do we stand? Annual Review of Clinical Psychology. 2013;9:177–213. [PubMed]
  • Miller MW, Patrick CJ, Levenston GK. Startle responding during text-prompted imagery: Probing specific emotions, personal experiences, and mechanisms of modulation. Psychophysiology. 2002;39:519–529. [PubMed]
  • Nelson BD, McGowan SK, Sarapas C, Robison-Andrew EJ, Altman SE, Campbell ML, Shankman SA. Biomarkers of threat and reward sensitivity demonstrate unique associations with risk for psychopathology. Journal of Abnormal Psychology. 2013;122(3):662–671. [PMC free article] [PubMed]
  • Nelson LD, Patrick CJ, Bernat EM. Operationalizing proneness to externalizing psychopathology as a multivariate psychophysiological phenotype. Psychophysiology. 2011;48:64–72. [PMC free article] [PubMed]
  • Nelson LD, Strickland CM, Krueger RF, Arbisi PA, Patrick CJ. Neurobehavioral traits as transdiagnostic predictors of clinical problems. Assessment. 2016;23(1):75–85. [PMC free article] [PubMed]
  • Patrick CJ. Emotion and psychopathy: Startling new insights. Psychophysiology. 1994;31:319–330. [PubMed]
  • Patrick CJ, Bernat EM. Neurobiology of psychopathy: A two process theory. In: Berntson GG, Cacioppo JT, editors. Handbook of Neuroscience for the Behavioral Sciences. Vol. 2. Hoboken: John Wiley & Sons Inc.; 2009. pp. 1110–1131.
  • Patrick CJ, Bernat EM. Neuroscientific foundations of psychopatholoy. In: Millon T, Krueger RF, Simonsen E, editors. Contemporary directions in psychopathology: Scientific foundations of the DSM-V and ICD-II. New York: Guilford; 2010. pp. 519–452.
  • Patrick CJ, Bernat EM, Malone SM, Iacono WG, Krueger RF, McGue M. P300 amplitude as an indicator of externalizing in adolescent males. Psychophysiology. 2006;43:84–92. [PMC free article] [PubMed]
  • Patrick CJ, Drislane LE. Triarchic model of psychopathy: Origins, operationalizations, and observed linkages with personality and general psychopathology. Journal of Personality. 2015;83(6):627–643. [PubMed]
  • Patrick CJ, Durbin CE, Moser JS. Reconceptualizing antisocial deviance in neurobehavioral terms. Development and Psychopathology. 2012;24(3):1047–1071. [PubMed]
  • Patrick CJ, Venables NC, Yancey JR, Hicks BM, Nelson LD, Kramer MD. A construct-network approach to bridging diagnostic and physiological domains: application to assessment of externalizing psychopathology. Journal of Abnormal Psychology. 2013;122(3):902–916. [PubMed]
  • Rugg MD, Coles MGH. Electrophysiology of mind. New York: Oxford University Press; 1995.
  • Semlitsch HV, Anderer P, Schuster P, Presslich O. A solution for reliable and valid reduction of ocular artifacts, applied to the P300 ERP. Psychophysiology. 1986;23:695–703. [PubMed]
  • Spearman C. Correlation calculated from faulty data. British Journal of Psychology. 1910;3:271–295.
  • Stein DJ, Nesse RM. Threat detection, precautionary responses, and anxiety disorders. Neuroscience & Biobehavioral Reviews. 2011;35(4):1075–1079. [PubMed]
  • Vaidyanathan U, Hall JR, Patrick CJ, Bernat EM. Clarifying the role of defensive reactivity deficits in psychopathy and antisocial personality using startle reflex methodology. Journal of Abnormal Psychology. 2011;120:253–258. [PMC free article] [PubMed]
  • Vaidyanathan U, Patrick CJ, Bernat EM. Startle reflex potentiation during aversive picture viewing as an index of trait fear. Psychophysiology. 2009;46:75–85. [PMC free article] [PubMed]
  • Venables NC, Hall JR, Yancey JR, Patrick CJ. Factors of psychopathy and electrocortical response to emotional pictures: Further evidence for a two-process theory. Journal of Abnormal Psychology. 2015;124(2):319–328. [PMC free article] [PubMed]
  • Venables NC, Patrick CJ. Validity of the externalizing spectrum inventory in a criminal offender sample: Relations with disinhibitory psychopathology, personality, and psychopathic features. Psychological Assessment. 2012;24:88–100. [PMC free article] [PubMed]
  • Venables NC, Patrick CJ. Reconciling discrepant findings for P3 brain response in criminal psychopathy through reference to the concept of externalizing proneness. Psychophysiology. 2014;51(5):427–436. [PMC free article] [PubMed]
  • Venables NC, Sellbom M, Sourander A, Kendler KS, Joiner TE, Drislane LE, Sillanmaki L, Elonheimo H, Parkkola K, Multimaki P, Patrick CJ. Separate and interactive contributions of weak inhibitory control and threat sensitivity to prediction of suicide risk. Psychiatry Research. 2015;226(2–3):461–466. [PMC free article] [PubMed]
  • Vizueta N, Patrick CJ, Jiang Y, Thomas KM, He S. Dispositional fear, negative affectivity, and neuroimaging response to visually suppressed emotional faces. Neuroimage. 2012;59:761–771. [PMC free article] [PubMed]
  • Wolpe J, Lang PJ. Manual for the Fear Survey Schedule (revised) San Diego, CA: Educational and Industrial Testing Service; 1977.
  • Yancey JR, Vaidyanathan U, Patrick CJ. Aversive startle potentiation and fear pathology: Mediating role of threat sensitivity and moderating impact of depression. International Journal of Psychophysiology. 2015;98(2 pt. 2):262–269. [PMC free article] [PubMed]
  • Yancey JR, Venables NC, Hicks BM, Patrick CJ. Evidence for a heritable brain basis to deviance-promoting deficits in self-control. Journal of Criminal Justice. 2013;41(5):309–317. [PMC free article] [PubMed]
  • Yancey JR, Venables NC, Patrick CJ. Psychoneurometric operationalization of threat sensitivity: Relations with clinical symptom and physiological response criteria. Psychophysiology. 2016;53(3):393–405. [PMC free article] [PubMed]
  • Zuckerman M. Sensation seeking: Beyond the optimal level of arousal. Hillsdale, NJ: Erlbaum; 1979.