|Home | About | Journals | Submit | Contact Us | Français|
ObjectiveTo describe the qualitative development of the Patient-Reported Outcome Measurement Information System (PROMIS®) Pediatric Stress Response item banks.MethodsStress response concepts were specified through a literature review and interviews with content experts, children, and parents. A library comprising 2,677 items derived from 71 instruments was developed. Items were classified into conceptual categories; new items were written and redundant items were removed. Items were then revised based on cognitive interviews (n = 39 children), readability analyses, and translatability reviews.Results2 pediatric Stress Response sub-domains were identified: somatic experiences (43 items) and psychological experiences (64 items). Final item pools cover the full range of children’s stress experiences. Items are comprehensible among children aged ≥8 years and ready for translation.ConclusionsChild- and parent-report versions of the item banks assess children’s somatic and psychological states when demands tax their adaptive capabilities.
Patient-reported outcomes (PROs) include individuals’ perceptions of their own health, illness, and well-being. These outcomes are now considered important in clinical, pharmaceutical, and translational research, in addition to classic endpoints such as survival time, laboratory values, and function tests (Cella et al., 2007; Reeve et al., 2007). For children, PROs are assessed through child or proxy (e.g., parent, teacher) report. Multiple initiatives are underway to ensure that PRO assessments are rigorously defined and tested for use in clinical research and care. For example, the Food and Drug Administration recently issued a set of methodological standards that PRO assessment instruments must meet if they are used to formulate claims for medication and medical devices (Food & Drug Administration, 2006). In addition, the National Institutes of Health (NIH) support several trans-NIH initiatives to develop and standardize health and well-being PRO measurement tools for research (Cella et al., 2007; Gershon, Cella, & Fox, 2010; Reeve et al., 2007).
In 2004, the NIH established the Patient-Reported Outcome Measurement Information System (PROMIS®) to develop a standardized set of self-reported outcome tools for research and clinical practice (Cella et al., 2007; Reeve et al., 2007). The cooperative group of PROMIS research sites and centers has developed and applied a unique mixed-methods approach to generating dozens of PRO measures and an informatics platform that enables web-based static and computerized adaptive administration of the instruments. PROMIS measurement tools are designed for use across disease groups and the life course. Many previous PROs have been designed for specific diseases, but the goal here was to develop measures that assess outcomes that are common across diseases.
PROMIS organizes self-reported health into physical, mental, and social dimensions (Riley et al., 2000). Within the mental health dimension, PROMIS has developed pediatric measures for depression (Irwin et al., 2010), anxiety (Irwin et al., 2010), and anger (Irwin et al., 2012), and is developing measures of subjective well-being (Forrest et al., 2012). In this article, we describe the qualitative steps in the creation of the child- and parent-report PROMIS item banks to measure somatic, cognitive, and psychological stress responses among children aged 5–17 years. Our goal was to create instruments that are appropriate for use with children of diverse cultures and developmental levels.
Stress is increasingly recognized as a predisposing factor in the development of physical and mental disease or disorder (Barr, Boyce, & Zeltzer, 1996). Epidemiologic studies of children have documented significant associations between stress and infectious illness (Graham, Douglas, & Ryan, 1986), injury (Boyce, Sobolewski, & Schaefer, 1989; Horwitz, Morgenstern, DiPietro, & Morrison, 1988), and psychiatric disorder (Rutter, 1989). In addition to the effects of over-active or poorly regulated stress response on children’s concurrent health, stress in childhood is a precursor to later-life disease (Barr et al., 1996). Increasingly, researchers are identifying the complex mechanisms that underlie associations between stress and the manifestation of disease and disorder (Long et al. 2011; Marin, Chen, Munch, & Miller, 2009; Wright, 2009). In some cases, failure to effectively regulate responses to traumatic stressors (e.g., abuse, exposure to violence), life stressors (e.g., family or peer conflict, school or academic challenges), or regular developmental demands (e.g., puberty, changing social roles) alters immune and other system functioning, leading to poorer health over time (Cohen & Manuck, 1995). For example, exposure to an earthquake produced changes in helper–suppressor cell ratios and pokeweed mitogen response that were associated with increased incidence of respiratory illness among kindergarten students (Boyce et al., 1993). The effects of stress exposure on clinical asthma outcomes are mediated by elevated eosinophil counts, greater lymphocyte proliferation and airway inflammation in response to allergic triggers, and heightened in vitro production of inflammatory cytokines implicated in asthma (Marin et al., 2009). Once a disease or disorder is manifested, symptoms, treatment regimens, and other consequences of the health condition may act as stressors. For example, stressors experienced by children with inflammatory bowel disease include the occurrence of pain, embarrassing bowel-related symptoms (e.g., fecal incontinence), intrusive diagnostic and treatment procedures (e.g., endoscopies, suppositories, or enemas), and hospitalizations that undermine opportunities to engage in age-appropriate activities, peer socialization, and schooling (King, 2003; Reigada et al., 2011). As a primary psychological correlate of disease and disorder, an improved understanding of children’s stress response is of significant relevance to the discipline of pediatric psychology.
Despite significant associations between stress and illness, the magnitude of the associations in many studies is modest. This is because not all individuals confronted with stress experience negative sequelae. Investigators have tried to elucidate individual differences that help explain vulnerability (or resistance) to stress. The quality and magnitude of an individual’s stress response is thought to partially explain individual differences in the association between exposure to stressors and the manifestation of disease (Boyce et al., 1995; Cohen & Manuck, 1995; Lovallo & Gerin, 2003).
Stress has long been conceptualized as a dynamic transactional process involving stimuli exposure (i.e., a stressor), detection and cognitive appraisal of the stimuli, and the response to stimuli that are perceived as potentially threatening (Gould, Hussong, & Keeley, 2008). Stress responses operate at the levels of biology (e.g., neuroendocrine stress response system), self (e.g., somatic, cognition, emotion experiences), and through self–environment interactions (e.g., coping). A substantial body of research focuses on biologic reactions to stress including mobilization of the autonomic nervous system and the hypothalamic–pituitary–adrenal axis. Exposure to traumatic or chronic stressors may result in biologic dysfunction such that the stress response system becomes dysregulated—unresponsive or hyper-reactive to new stressors (McEwen, 1998; McEwen & Stellar, 1993; Sapolsky, Krey, & McEwen, 1984). Stress experienced at the level of self includes somatic (e.g., rapid heartbeat, dizziness, pains, agitation), cognitive (e.g., concentration problems, intrusive thinking), and psychological (e.g., worry, moodiness, irritability) reactions. Experiences of stress are thoughts, feelings, and bodily sensations that occur in response to threatening events. They are differentiated from coping, which encompasses voluntary responses aimed at avoiding or mitigating the stressful experience by altering the self or self–environment relationships (e.g., taking action to minimize the stressor, seeking social support) (Compas, Connor-Smith, Saltzman, Thomsen, & Wadsworth, 2001; Folkman & Lazarus, 1991; Gould et al., 2008). Whereas experiences of stress occur involuntarily, coping includes purposeful attempts to manage the stressor through behavior, cognitive restructuring, and emotional self-regulation.
Differentiation among stress responses that operate at the biologic, self, and self–environment levels is important because activation of a response at one level does not necessitate activation of a response at another level. Modest and divergent associations between the biologic and perceived (self) stress responses have been observed (Hjortskov, Garde, Ørbæk, & Hansen, 2004; Lackschewitz, Huther, & Kroner-Herwig, 2008; Schlotz et al., 2008). For example, a child who has a biologic response to a stressor may fail to experience a somatic, cognitive, or psychological stress response. Moreover, strategies used to assess biologic stress response (e.g., cortisol, heart rate, respiratory sinus arrhythmia) differ from those used to measure responses that occur at the self or self–environment levels. Unlike biologic outcomes, the measurement of perceived stress responses is best achieved through direct questioning of the child or an individual who has adequate knowledge of the child’s experiences and behavior (Bevans, Moon, Riley, & Forrest, 2010; Bevans & Forrest, 2010).
Many measures of stress assess the frequency of exposure to stressful events. However, people differ greatly in their interpretation of events and thus experience stress related to those events to varying degrees. Because perceptions of threat differ among people, stress experiences are individually determined and must be self-reported. Although there are a few validated self-report tools that assess adults’ self-reported stress experiences (e.g., Cohen’s concept of perceived stress; Cohen & Williamson, 1988), these instruments lack developmental sensitivity, both in terms of content and the cognitive demands they place on respondents. We sought to develop a standard measure of perceived or experienced stress responses that would transcend cultural, linguistic, and developmental differences. To do so requires sensitivity to cultural and developmental differences and the avoidance of colloquial language, which is a challenge because many terms that are commonly used to describe stress experiences (especially among youth) are familiar and time/location specific (e.g., in the United States at the time of this writing, “freaking out,” or “slammed”). Thus, the instruments were developed successively with guidance from developmental, cultural, and linguistic experts and based on input from an international group of youth.
Qualitative development of the PROMIS pediatric Stress Response item banks followed a well-established set of methods (DeWalt, Rothrock, Yount, & Stone, 2007; Forrest et al., 2012). The process includes the following steps: domain/item bank concept specification; content expert interviews; child/parent interviews; structured literature review and instrument identification; item classification, selection, and new item development; item technical review and revision; cognitive interviews; translatability review; and reading level analysis. Once item banks are developed using rigorous qualitative methods, they are subjected to quantitative psychometric evaluation (Reeve et al., 2007). Thus, evidence of the instruments’ reliability, validity, and sensitivity to changes in children’s health is generated through a combination of qualitative and quantitative procedures. This article describes the qualitative development of the PROMIS Pediatric Stress Response item banks. Quantitative testing of the item banks is currently underway and will be described in a subsequent publication.
The conceptual framework underlying the Stress Response item banks was initially developed through a review of theoretical literature and thereafter refined based on findings from semi-structured interviews conducted with pediatric stress experts, children, and their parents.
Our initial conceptualization of children’s stress response was informed by three theories. The first theory supports the inclusion of stress response in a health outcome framework by highlighting associations between stress and biologically manifested health dimensions (Worthman & Kuzara, 2005). The second theory recognizes that stress responses are individually determined and experienced because one’s cognitive and emotional appraisals of an event, rather than the event itself, produce stress (Dienstbier, 1989; Lazarus & Folkman, 1984). When the appraisals lead to interpretation of an event as a challenge rather than a threat, different physiological and psychological stress experiences result (Dienstbier, 1989). The third theory emphasizes the role of developmental change in stress response and posits that stress effects are both moderated by an individual’s developmental stage and shape the way that the next stage of development unfolds (Boyce et al., 1995; Lupien, McEwen, Gunnar, & Heim, 2009).
Ten theorists, researchers, and clinicians with a minimum of three peer-reviewed publications on children’s stress response were interviewed to inform the conceptual framework. Before the interview, a preliminary version of the Stress Response framework was presented to interviewees. Initially, the framework was composed of three sub-domains: body sensations (the sensations of physical arousal associated with responses to internal or external challenges, e.g., energy, nausea, restlessness, tension), cognitive experiences (thoughts about one’s expectations of his or her ability to meet the demands of one’s inner and outer worlds, e.g., perceptions of threat, controllability, competence), and emotional experiences (moods or feelings in response to environmental or internal challenges, e.g., fear, vigilance, alertness, excitement). Interviewees were asked to respond to the framework’s structure, organization, and component definitions. The framework was iteratively revised such that new versions were produced after each interview. Study investigators trained in the semi-structured procedure conducted interviews, which were audio recorded, transcribed, and subjected to thematic analysis.
Child and parent interviews were conducted to ensure that the conceptual framework comprehensively and meaningfully covered children’s stress response experiences. Semi-structured interviews were conducted with 17 children aged 8–17 years (29% aged 8–10 years, 29% aged 11–13 years, 42% aged 14–17 years, 29% male, 22% White, 50% African American, 28% of other race, 28% Hispanic, 39% with a chronic health condition) and 6 parents of 8 children aged 5–12 years (children were 63% male, 50% White, 13% African American, 38% of other race, 0% Hispanic, 63% with a chronic health condition). Interviewees were asked to identify recent events in their lives (or their children’s lives) that they perceived as stressful and describe the thoughts, feelings, and bodily sensations they experienced in response to that event. Study investigators trained in the semi-structured procedure conducted interviews, which were audio recorded, transcribed, and subjected to thematic analysis. Interviews were conducted until saturation was achieved. Saturation was indicated when participants of a broad age range failed to provide novel information (Lasch et al., 2010). We verified concept saturation by documenting each interviewee’s identification of stress experience concepts (Table I). A concept was considered for saturation if it was elicited in at least one, but not the last, interview and if enough information had been elicited to fully understand the meaning and importance of the concept to children (Lasch et al., 2010).
Once the Stress Response framework was established, item pools were generated through a systematic review of the measurement literature. Items were classified into stress response sub-categories and selected for inclusion in the item banks to ensure both comprehensive coverage of the concepts and minimal item redundancy. All items were rewritten to comply with PROMIS standards (PROMIS®, 2012), and new items were generated as needed. Items were reviewed for clarity through the application of cognitive interviews and readability analyses. Finally, items were subjected to translatability review to ensure that they were linguistically and culturally appropriate for translation into Spanish and German.
The purpose of the literature search was to identify child- or proxy-report instruments that have been used to measure children’s stress response. The instruments were compiled into a library of stress response items. We adopted the search methodology of Klem and colleagues (2009), who previously applied their procedure to the development of PROMIS item banks. This method is comprehensive in scope, systematic, and reproducible. Working with a team of biomedical librarians at the University of Pennsylvania, we searched MEDLINE, CINHAL, PsychINFO, and HaPI (Health and Psychosocial Instruments), the databases that contain the most information about health outcome measurement instruments. Vocabulary terms that could be used for searching the databases were developed in tandem by the investigators and librarians. Vocabulary lists were constructed to capture three distinct aspects of the desired literature: constructs of stress response (e.g., psychological adaptation, perceived stress), self-report instruments (e.g., personal monitoring, self analysis), and measurement (e.g. behavior rating scales, checklists). The search vocabulary was refined using database thesauri and thereafter used in combination with additional search conventions including sample (e.g., involving youth <18 years of age) and article (e.g., English only) restrictions to identify relevant citations. Two study investigators reviewed all article titles and abstracts to identify articles that described the development or application of a youth-report stress experience measurement tool. These tools were collected from the instrument authors or from published works. Item stems and response categories and their linkages to source instruments were catalogued.
A team of three investigators (concealed for peer review: Bevans, Gardner, Pajer) reviewed all items and through a consensus process removed items that failed to represent children’s subjective stress response experiences. Investigators then sorted items into sub-domains and smaller conceptually distinct categories of somatic or psychological symptoms called facets (Table II). The process of systematically grouping items into facets allowed investigators to observe conceptual redundancy among items. At this stage, the goal of item reduction was to decrease the larger item pool to a representative set of item concepts that met the following criteria: (1) item concept was consistent with the sub-domain and facet definitions; (2) the item concept represented a unique manifestation of stress response; and (3) the item concept was universally applicable, meaning that its expression did not depend on presence of a specific disease or context. Formation of the item library helped us to refine a concept map that covered each facet. Where facets were not adequately covered, we generated new item concepts and wrote new items.
The item concepts were transformed into expressions through rewording of the item context, stem, and response options consistent with PROMIS standards. Item wording complied with the following criteria: (1) items could stand alone without reference to any other item (to facilitate presentation of items using a computerized adaptive test format where the order in which items are presented is determined by the computerized algorithm); (2) the context of all items was “In the past 7 days …”; (3) items were worded to inquire about the frequency of a stress response; (4) response options were those previously used to measure the frequency of a symptom or event in PROMIS instruments (Never—Rarely—Sometimes—Often—Always); and (5) items were as concise and simply worded as possible.
Cognitive interviews were conducted with 39 children aged 8–17 years to identify problems with item comprehension, recall, and other cognitive processes that could be remediated through question rewording, reordering, or more extensive instrument revisions (Fortune-Greeley et al., 2009). Participants were recruited from primary care practices and school settings. Cognitive interviewees were 46% male, 15% Asian, 67% African American, 10% White, 8% of another race, 2% Hispanic, and 33% with a chronic health condition. An item sampling method was applied to ensure that up to 35 items were tested with each child. By this method, each stress response item was reviewed by five participants, including at least two who were 8–11 years of age. Before the cognitive interview, participants completed a paper-and-pencil questionnaire with the items assigned to them. Once the child completed the questionnaire, a research assistant trained in cognitive interview procedures interviewed the child using standardized probes. For each item, interviewees read the question aloud and stated the meaning of the question using their own words. The interviewer then asked, “How did you answer that question?” and “Why did you choose that answer?” Responses to cognitive interview probes were audio recorded, transcribed, and coded for the degree to which children’s understanding of the question was consistent with the item definition (1 = poor/different than intended; 2 = partial; 3 = fully consistent). Items with average ratings of less than 2 were removed or revised (examples of problematic items are presented in Table III). Revised items were re-evaluated using the same cognitive interview procedure.
To ensure that PROMIS item banks are translatable, experts in translation reviewed each item to identify idiomatic expressions, complex sentences, and concepts that are not easily translated into Spanish and German. The items that were retained after cognitive interviewing were translated following a universal language approach (one translation per language) following recommendations of the International Society for Pharmacoeconomics and Outcomes Patient-Reported Outcomes Translation and Linguistic Validation Task Force (Wild et al., 2005, 2009), international guidelines published by the International Quality of Life Assessment Project (Aaronson et al., 1992) and the MAPI and Medical Outcomes Study Institutes (MAPI, 2011; Medical Outcomes Study, 2011). In addition, the PROMIS pediatric translation process was based on extensive experience gleaned through decades of standard international translations of pediatric health related quality of life instruments (e.g. EQ-5D, KIDSCREEN; Ravens-Sieberer et al., 2006). A forward–backward–forward translation approach was applied coupled with child stress expertise to achieve optimal conceptual and cultural equivalence between the English and other-language versions rather than on achieving a linguistic or literal equivalence only (Wild et al., 2009).
Items were reviewed for readability by calculating the Flesch-Kincaid Grade Level equivalent using Microsoft Word software. The readability score analyzed and rated text on a U.S. grade-school level based on the average number of syllables per word and words per sentence (Klare, 1976). For example, a score of 8.0 means that an eighth grader would be expected to understand the text.
Interviewees generally supported the proposed conceptual framework, but identified several components of the model that needed further development. Most notably, interviewees highlighted the need to further consider developmental differences in the manifestation of children’s stress response and their capacity to report on those experiences. Several experts felt that younger children may have difficulty expressing their thoughts and feelings about stress, but could report stress-related bodily sensations. The capacity to identify cognitive or emotional stress responses, which may require meta-cognition, was expected to develop in early adolescence (Bevans & Forrest, 2010; Bevans et al., 2010).
The experts identified two broad stress response categories that were not initially included in the framework. The first was sleep problems such as trouble falling asleep, frequent waking, nightmares, and difficulty waking. Sleep problems, which become increasingly common as children enter adolescence, may operate as a cause or consequence of stress and are associated with a multitude of other problematic outcomes (e.g., depression, poor school performance) (Brand & Kirov, 2011). We added a sleep problem facet to the conceptual framework based on this expert feedback.
Second, experts suggested that the disruption of social relationships should be included in a pediatric stress response framework. Undoubtedly, the rapid expansion and increased emphasis on social networks is of great importance among older children and adolescents (DeGoede, Branje, & Meeus, 2009; Molix & Bettencourt, 2010). However, we considered the disruption of social interactions to be both a stressor and a potential consequence of stress that impacts youth at the self–environment level. Given our focus on stress response at the level of self, we excluded social relationships from the conceptual framework, but attempted to ensure that those experiences that may undermine children’s social relationships were included (e.g., negative mood, anger, fearfulness, agitation).
Experts also highlighted the need to further consider associations between stress response and related concepts, most notably coping and anxiety. Because the Stress Response item banks were designed to assess the emotional, cognitive, and somatic reactions to stress that are experienced at the level of self, coping strategies aimed at reducing stress by changing the self–environment relationship were excluded from the conceptual framework. In contrast, symptoms of anxiety are subsumed within both somatic (e.g., heart pounding, nausea, dizziness, sleep disturbance) and cognitive/psychological (e.g., worry, fear, irritability) domains. As with all PROMIS instruments, the Stress Response item banks were designed to be responsive to clinical changes, but they are not intended for use as diagnostic tools. Accordingly, anxiety concepts are integrated throughout the instrument and do not constitute a single item bank.
Among children, the most commonly identified stressors included interpersonal challenges (e.g., conflict with friends, classmates, parents, and teachers), changes in family structure (e.g., death or birth of a family member, parents divorcing), and academic demands (e.g., tests, applying to college). In addition to these challenges, parents noted that their younger children experienced stress related to being in novel situations such as attending a new school or trying a new activity (e.g., joining a new sports team).
Both youth and parents defined stress in general terms (e.g., when you are going through a hard time, have something stuck in your mind, struggling to do something) and with reference to specific stress responses (e.g., fatigue, sadness, anger, headache, stomachache, heart pounding, troubling focusing, breaking or throwing objects). The concept saturation matrix presented in Table I demonstrates that there is significant consensus among children on the most common and important stress experiences. Concept saturation was achieved after only 10 child interviews, but we conducted a total of 17 interviews to ensure full representation of youth of various ages. Parents identified most, but not all, of the stress experiences elicited from children, thereby lending support to the importance of obtaining health information directly from youth.
In general, youth of all ages failed to distinguish between cognitive and affective stress response. For example, when asked about the thoughts they had in response to a stressor, youth commonly described emotional reactions such as “feeling heartbroken” or “worried.” They clearly distinguished emotional/cognitive experiences from bodily (somatic) sensations such as headache, stomachache, and feeling tense. As a result, we combined the cognitive and emotional stress response outcomes to form a single sub-domain for psychological experiences of stress, which is distinct from somatic stress response outcomes.
Youth identified several novel stress responses that were not emphasized in the theoretical literature or expert interviews. These included agitation and trouble concentrating. As the experts predicted, youths frequently referenced the negative impact that stress has on interpersonal relationships, which may occur because of feeling irritable/short-tempered or because youth socially isolate themselves (e.g., “stay quiet” and “don’t participate”).
Finally, although most measures of youths’ stress response assess psychiatric symptoms (e.g., symptoms of generalized anxiety, posttraumatic stress disorder, and depression), most youth described stress responses that are not indicative of psychiatric disorder. Often, youth respond positively to stressors with enhanced energy or feelings of satisfaction or pride on successful resolution of a challenge. In other circumstances, youth experience negative reactions to stress; because these experiences are not consistent with diagnostic criteria for psychiatric conditions, they may go unnoticed when children’s stress responses are evaluated using many traditional health status assessment tools.
This sequence of activities culminated in a final conceptual framework for stress response that was theoretically grounded, conceptually meaningful, and consistent with children’s experiences (Table II). The framework specifies psychological experiences of stress (cognitive–perceptual disruption, controllability/manageability, anger, fear) and somatic experiences of stress (arousal, agitation, sleep problems, gastrointestinal distress) that occur at the level of self when environmental or internal demands tax children’s adaptive capacities.
Searches conducted with four databases yielded 1,191 citations (478 from MEDLINE, 430 from CINHAL, 100 from PsychINFO, and 183 from HaPI). Of these, 1,140 (95.7%) citations were unique. A total of 89 articles were retained, from which 85 unique instruments were identified. Of these, 71 (83.5%) were obtained (authors refused: 3, unattainable: 11). A total of 2,677 items were derived from the 71 instruments. These items were entered into a database for subsequent review and classification (see Table A1).
A team of three content experts (concealed for peer review: XXX) sorted the initial pool of 2,677 items into the psychological and somatic experiences of stress sub-domains. This process resulted in deletion of 1,958 items that were deemed out-of-scope of the experiences of stress definitions, leaving 719 items for classification of which 290 were identified as duplicates and deleted. The resulting item pools contained 335 psychological items and 94 somatic items. The concepts measured by the items were recorded and sorted into facet categories, thereby creating a concept map for the facets. As expected, there was significant redundancy in item content within facets. For example, 4 items that assessed feeling threatened were categorized within the Fear facet. After removal of 304 redundant items, pools were made up of 79 distinct psychological item concepts and 46 somatic item concepts. Finally, investigators evaluated the resulting domain–subdomain–facet–item concept map to ascertain whether conceptual saturation had been achieved. Investigators generated 11 new psychological and 20 new somatic item concepts to ensure comprehensive conceptual coverage of the stress responses identified in expert, child, and parent interviews. For example, children’s sense that they can manage daily life demands was added to the item concept map. Investigators generated several items to assess this concept (e.g., “I could manage things in my life; I felt like I had too much going on”). The resulting item pools were composed of 90 psychological items and 66 somatic items.
The retained items had diverse styles of language, recall periods, and response options. Items were rewritten in accordance with the technical criteria described above, resulting in an item pool that contained 156 items, which were further reviewed for item clarity and translatability.
Of the 156 items subjected to cognitive interviews, 105 (67.3%) were retained without revision, 41 (26.3%) were removed because of poor comprehension, and 10 (6.4%) were revised and re-evaluated. Examples of poorly understood items are presented in Table III. On completion of the cognitive interview process, the average item rating was 2.80 (SD = 0.32).
Translators identified 10 problematic items (7.6% of those subjected to translation review). These items contained idiomatic expressions (e.g., “Lots of things got on my nerves”) or referenced concepts for which there is no exact Spanish or German translation (e.g., “I felt bothered by things”). Of the 10 problematic items, 8 were removed from the bank. Two items were revised and will be further evaluated for measurement equivalence using a modern measurement approach (differential item functioning by language) in the future. See Table A2 for the final item pools.
On average, items were written at a second-grade level (M = 2.4, SD = 0.2) and ranged in estimated grade equivalent from 0.0 to 8.75. Somatic items were written at a slightly lower grade level than Psychological items (Somatic: M = 2.2, SD = 2.2, range = 0.0–8.1; Psychological: M = 2.5, SD = 1.9, range = 0.0–8.5).
The PROMIS Pediatric Stress Response item pools assess children’s somatic and cognitive/psychological states when situations tax their adaptive capabilities. Given bidirectional associations between stress and illness across the life course, children’s subjective stress experiences are important potential disease- or disorder-related outcomes that are of particular relevance to pediatric psychology. Recognizing the importance of children’s perceived stress responses follows two more general trends: (1) increased recognition that PROs are essential endpoints in clinical, pharmaceutical, and translational research (Cella et al., 2007; Food & Drug Administration, 2006; Revicki & Cella, 1997); and (2) advances in measurement methods that account for the unique ways that children manifest and experience health/illness outcomes as well as the cognitive limitations that hinder children’s self-reporting capabilities (Bevans & Forrest, 2010; Bevans et al., 2010). The Stress Response item pools were developed and refined using advanced and varied qualitative methods to ensure their appropriateness for use across developmental levels, diverse cultures, and time. Items were not designed to be disease specific.
By supplementing a review of existing theoretical literature with expert opinion and child and parent perspectives on stress, the conceptual framework underlying the Stress Response item pools advances knowledge of how children perceive stress and manifest stress response at the level of self. Expert, child, and parent informants added to the conceptual framework by identifying stress responses that were not previously emphasized in the theoretical and measurement literatures. For example, although researchers have observed increased frequency and severity of sleep problems and fatigue during adolescence, these symptoms are seldom identified as stress responses. Based on expert input, trouble with sleep and lack of energy are prominently featured in the Stress Response item pools. Both children and parents reported that a common stress response is to feel overwhelmed, the perception of being unable to effectively predict, control, manage, or resolve daily life challenges (e.g., school assignments, conflict with peers or family member). Although this notion is assessed by the most commonly used measure of perceived stress (Perceived Stress Scale), this instrument was developed and validated for adults and has poor psychometric properties when applied to children (Cohen & Williamson, 1988; Cohen, Kamarck, & Mermelstein, 1983). Because the literature review of extant instruments yielded few items that assess children’s sense that their lives are unpredictable, uncontrollable, and overloaded, we developed and qualitatively evaluated new items to assess these concepts. Finally, in describing children’s stress response, children and parents rarely referred to the manifestation of symptoms at levels indicative of specific clinically significant or diagnosable disorders (e.g., panic disorder, gastrointestinal disorder). Instead, children’s stress response often constitutes reactions that are important to the child, but occur at levels such that they would not be detected by traditional diagnostic testing. Because the item pools cover a broad range and level of stress responses, they are expected to be more sensitive to subtle changes in children’s perceived stress response than would traditional disorder-based assessment measures. Thus, the Stress Response measurement tools may be useful in evaluations of interventions for which the reduction of stress is a hypothesized primary or secondary endpoint.
Development of the Stress Response item pools confirmed that children are valuable reporters of their own health outcomes and that their perspectives are unique and complementary to those of their parents. Children identified a number of stress responses that were not commonly identified by parents (e.g., rumination, fear, shakiness, tension, lack of perceived controllability). Further, cognitive interviews suggested that with the exception of some phrases that were not well understood by younger children (e.g., jumpy, at ease), children as young as 8 years old are generally familiar with stress response concepts and terminology. Furthermore, stress response is individually determined and cannot be imposed by others’ expectations, nor assumed based on the frequency of exposure to specific stressors. All of these factors make children’s self-reports vital to the measurement of their stress response experiences.
We expect that the PROMIS Pediatric Stress Response measurement tools will provide reliable, accurate, and efficient measurement of children’s psychological and somatic stress response. The strength of the PROMIS approach to developing and validating PRO measurement tools lies in the multiple, varied, and rigorous methods applied throughout the instrument development process. Each method provided unique and complementary information about the stress response conceptual model and items. Still, the instrument has several limitations. Although the item banks assess novel aspect of children’s stress response, there are limits to the information the items provide. As with many instruments used in pediatric psychology, children’s responses to somatic items may reflect either stress response or disease-related symptoms (Holmbeck et al. 2008). We recommend that end users remain cognizant of children’s disease state when interpreting somatic stress response scores. Another limitation is that a relatively small number of parents provided input into the stress response conceptualization. Parents identified many of the same stress experiences as did children, but additional interviews may have revealed a broader range of child stress responses that are observed by parents, especially for younger children. The content of the item pools and the underlying conceptual framework should be expanded to include stress responses that occur through self–environment interactions (e.g., problem-focused coping, disruption of social relationships). Future expansions of the measurement system should also focus on positive or growth-promoting outcomes of stress such as feelings of satisfaction or learning a new skill through successful resolution of a challenge. Positive outcomes of stress were identified by youth but are currently underemphasized in the theoretical and measurement literatures. Finally, the translatability review was essential to ensuring that the Stress Response concepts and items are approximately equivalent across three languages. Translatability into German and Spanish were selected because several European nations are currently funding patient-reported outcome measure development and these languages ensure the tools’ applicability across large regions of Europe. However, we recognize that other translations conducted outside of the United States and Western Europe may reveal more significant cultural and linguistic differences in perceived stress response outcomes.
The present work laid a strong foundation for the Stress Response instruments through qualitative development and refinement of two item pools that are appropriate for use with children and adolescents. Currently, the item pools are being administered in large field trials to youth aged 8–20 years and parents of 5–17-year-olds. These data will be subjected to a series of psychometric tests to evaluate the instruments’ dimensionality; ensure that items measure the full range of the dimensions with minimal floor/ceiling effects, gaps in coverage, or item redundancy; identify items that are biased against subgroups (e.g., age, gender, language, reading level, health status); and generate short-form and computerized adaptive test versions of the instruments for future use. In accordance with the PROMIS approach, the instruments were designed to assess stress response among all children, regardless of the degree to which they manifest specific diseases or disorders. Clinical validation studies are needed to ensure that the instruments are appropriate, useful, and responsive to change in stress response when applied to children with specific diseases or disorders. Psychometric analyses are expected to demonstrate that the instruments measure critical endpoints in a standardized manner, thereby supporting comparisons across multiple clinical studies to assess the relative effectiveness of interventions aimed at reducing children’s negative stress response experiences.
The project described was supported by Award Number U01AR057956 from the National Institute of Arthritis and Musculoskeletal and Skin Diseases. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Arthritis and Musculoskeletal and Skin Diseases or the National Institutes of Health.
Conflicts of interest: None declared.
The Patient-Reported Outcomes Measurement Information System (PROMIS) is an NIH Roadmap initiative to develop a computerized system measuring PROs in respondents with a wide range of chronic diseases and demographic characteristics. PROMIS II was funded by cooperative agreements with a Statistical Center (Northwestern University, PI: David Cella, PhD, 1U54AR057951), a Technology Center (Northwestern University, PI: Richard C. Gershon, PhD, 1U54AR057943), a Network Center (American Institutes for Research, PI: Susan (San) D. Keller, PhD, 1U54AR057926), and thirteen Primary Research Sites which may include more than one institution (State University of New York, Stony Brook, PIs: Joan E. Broderick, PhD, and Arthur A. Stone, PhD, 1U01AR057948; University of Washington, Seattle, PIs: Heidi M. Crane, MD, MPH, Paul K. Crane, MD, MPH, and Donald L. Patrick, PhD, 1U01AR057954; University of Washington, Seattle, PIs: Dagmar Amtmann, PhD, and Karon Cook, PhD, 1U01AR052171; University of North Carolina, Chapel Hill, PI: Darren A. DeWalt, MD, MPH, 2U01AR052181; Children’s Hospital of Philadelphia, PI: Christopher B. Forrest, MD, PhD, 1U01AR057956; Stanford University, PI: James F. Fries, MD, 2U01AR052158; Boston University, PIs: Stephen M. Haley, PhD, and David Scott Tulsky, PhD [University of Michigan, Ann Arbor], 1U01AR057929; University of California, Los Angeles, PIs: Dinesh Khanna, MD, and Brennan Spiegel, MD, MSHS, 1U01AR057936; University of Pittsburgh, PI: Paul A. Pilkonis, PhD, 2U01AR052155; Georgetown University, PIs: Carol. M. Moinpour, PhD [Fred Hutchinson Cancer Research Center, Seattle], and Arnold L. Potosky, PhD, U01AR057971; Children’s Hospital Medical Center, Cincinnati, PI: Esi M. Morgan DeWitt, MD, MSCE, 1U01AR057940; University of Maryland, Baltimore, PI: Lisa M. Shulman, MD, 1U01AR057967; and Duke University, PI: Kevin P. Weinfurt, PhD, 2U01AR052186). NIH Science Officers on this project have included Deborah Ader, PhD, Vanessa Ameen, MD, Susan Czajkowski, PhD, Basil Eldadah, MD, PhD, Lawrence Fine, MD, DrPH, Lawrence Fox, MD, PhD, Lynne Haverkos, MD, MPH, Thomas Hilton, PhD, Laura Lee Johnson, PhD, Michael Kozak, PhD, Peter Lyster, PhD, Donald Mattison, MD, Claudia Moy, PhD, Louis Quatrano, PhD, Bryce Reeve, PhD, William Riley, PhD, Ashley Wilder Smith, PhD, MPH, Susana Serrate-Sztein, MD, Ellen Werner, PhD, and James Witter, MD, PhD. This manuscript was reviewed by PROMIS reviewers before submission for external peer review. See the Web site at www.nihpromis.org for additional information on the PROMIS initiative.
|Instrument name||Author||Year of publication||Number of items|
|Academic Expectations Stress Inventory||Ang||2006||9|
|Adolescent Coping Orientation for Problem Experiences (A-COPE)||Patterson||1987||54|
|Adolescent Minor Stress Inventory (MAC-AMSI)||Ames||2005||102|
|Beck Youth Inventories of Emotional and Social Impairment||Beck||2001||100|
|Child Acute Stress Questionnaire (CASQ)||Winston||2004||48|
|Child Behavioral Style Scale (CBSS Scale-Revised)||Miller||1995||33|
|Child Post Traumatic Cognitions Inventory—Fragile Person in a Scary World (CPTC_FPSW)||Meiser-Stedman||2009||25|
|Child rating scale||Hightower||1998||24|
|Children’s Attributional Style Interview||Conley||2001||16|
|Children’s Attributional Style Questionnaire (CASQ) Revised||Kaslow||1991||24|
|Children’s Emotion Management Scales (CEMS)—A,S,W||Zeman||2001||33|
|Children’s life events scale (CLES)||Coddington||1972||147|
|Children’s Integrated Stress and Coping Scale CISCS||Jose||2005||21|
|Children’s Negative Cognitive Error Questionnaire||Leitenberg||1986||24|
|Children’s Response Styles Questionnaire (Rumination)||Abela||2007||21|
|Children Revised Impact of Event Scale (CRIES)||Dyregrov||2005||13|
|Choi Coping Scale||Choi||1997||6|
|Coping Efficacy Scale||Sandler||2000||7|
|Coping Resources Inventory Scales for Educational Enhancement||Curlette||2000||69|
|Coping Scale for Children and Youth||Brodzinsky||1992||29|
|Coping with Disease Questionnaire (CODI)—American Version||Petersen-Ewert||2007||28|
|Cognitive Emotion Regulation Questionnaire||Garnefski||2005||72|
|Coping health inventory for children||Austin||1994||52|
|Coping Strategies Inventory (CSI) Manual||Tobin||1984||72|
|Daily diary-frequency of stressful events||Schanberg||2000||28|
|Daily diary of Pain||Gil||2000||31|
|Daily Hassles and others||Seidman||1995||16|
|Daily Hassles Questionnaire||Little||2004||4|
|The Daily Life Stressors Scale||Kearney||1998||30|
|Daily Stress Inventory||Brantley||1989||58|
|Diepenmaat Perceived stress questions||Diepenmaat||2006||1|
|Electronic Momentary Log||Bjorling||2009||3|
|Emotion Expression Scale for Children (EESC)||Penza-Clyve||2002||16|
|Emotional Distress scale||Reynolds||1994||21|
|Experiences of worry questionnaire||Brown||2006||9|
|How I Cope Under Pressure Scale||Ayers||1991||54|
|Impact of Event Scale||Horowitz||2006||15|
|Impact of Event Scale—Revised||Weiss||1997||22|
|Inventory of high school students recent life experiences||Kohn||1993||81|
|Inventory of Psychological Attitudes (IPPA-32R)||Kass||2000||32|
|Life Events and Coping Inventory||Dise-Lewis||1988||175|
|Life Experiences Questionnaire||Payne||2000||85|
|Meta-Cognitions Questionnaire—Positive beliefs about worry||Cartwright-Hatton||1998||69|
|Multidimensional Measure of Children’s Perception of Control||Connell||1985||48|
|Perceived Stress Questionnaire (PSQ)||Levenstein||1993||60|
|Perceived Stress Scale (PSS) 14-item||Cohen||2010||14|
|Profile of Mood States (POMS)||McNair||1971||65|
|Responses to Stress Questionnaire (RSQ): Coping and Stress Responses||Connor-Smith (Compas)||2000||70|
|The Revised Manifest Anxiety Scale||Richmond||2008||3|
|The Revised Worry-Emotionally Scale||Meijer||2001||34|
|Schoolagers’ Coping Strategies Inventory-II (SCSI-II-COLOR)||Ryan-Wenger||2004||42|
|Self-Report Coping Scale||Causey||1992||35|
|Shamai Lebanon Stress||Shamai||2007||21|
|Social, Attitudinal, Familial, and Environmental (SAFE-C)—English||Chavez||1997||40|
|State-Trait Anxiety Inventory (STAI)||Spielberger||1970||20|
|Strengths and Difficulties Questionnaire (11- to 17-year old)||Goodman||2005||67|
|Stress in Children Questionnaire||Osika||2007||21|
|Stress Overload Inventory||Girdano||2003||10|
|Stress Resiliency Profile||Thomas||1992||8|
|The Stress Response Scale||Chandler||1982||40|
|The Stress Response Scale For Adolescents||Adams||1991||10|
|The Student-Life Stress Inventory||Gadzella||1991||10|
|Threat Appraisal Scale Manual||Sandler||1999||24|
|Ways of Coping Questionnaire||Folkman||1988||66|
|5 point self rating stress score||Lim||2005||16|
|The 14-Item Resilience Scale (RS-14)||Wagnild||1993||15|
|Psychological Experiences of Stress|
|I felt distracted.|
|I had trouble paying attention.|
|Thoughts about upsetting things popped into my head.|
|I had trouble controlling my thoughts.|
|I felt unable to remember answers, even for questions I knew the answer to.|
|I had trouble concentrating.|
|I lost my train of thought.|
|I felt unable to react to something that bothered me.|
|I forgot things.|
|My thoughts went very fast.|
|I felt so upset that I could not remember what happened or what I did.|
|I had trouble making decisions.|
|I had trouble thinking clearly.|
|I felt like my thinking was slower than usual.|
|I was slow to react to things.|
|Pictures of upsetting things popped into my head.|
|I forgot things I needed to remember.|
|I felt I had enough time for everything I needed to do.|
|I felt in control of my time.|
|I felt I had too many things to do.|
|I felt concerned about what was going on in my life.|
|I felt organized.|
|I felt that my problems kept piling up.|
|I felt in control of my life.|
|I felt I had too much going on.|
|I felt overwhelmed.|
|I felt I had enough time to get things done.|
|I felt unable to manage things in my life.|
|I felt able to change my life.|
|I felt really mad.|
|I felt relaxed.|
|I wanted to scream.|
|I felt like throwing things.|
|I felt angry.|
|I felt like hitting something.|
|I felt mad.|
|Small things upset me.|
|I felt alarmed.|
|I felt like fighting.|
|I felt frustrated.|
|I felt like breaking things.|
|I felt grouchy.|
|I felt annoyed.|
|Everything bothered me.|
|I wanted to hurt someone.|
|I lost my temper easily.|
|I felt terrified.|
|I felt threatened.|
|I felt nervous about what was going on in my life.|
|I felt afraid.|
|I felt fearful.|
|I felt as if something bad could happen.|
|I felt worried.|
|I felt secure.|
|I felt stressed.|
|I felt safe.|
|I felt distressed.|
|I felt scared.|
|I felt that I could not stop bad things from happening to me.|
|I felt panicky.|
|I felt worried that something bad would happen to me.|
|I had to watch out for danger all the time.|
|I felt frightened.|
|I felt under pressure.|
|Somatic Experiences of Stress|
|My palms were sweaty.|
|My heart pounded, even when I was not exercising or playing hard.|
|I had to go to the bathroom more than usual.|
|My body shook.|
|I was sweaty, even when I was not exercising or playing hard.|
|My breathing was fast, even when I was not exercising or playing hard.|
|My mouth was dry.|
|My heart beat faster than usual, even when I was not exercising or playing hard.|
|My hands shook.|
|A sudden noise made me jump.|
|I felt dizzy.|
|I had trouble breathing, even when I was not exercising or playing hard.|
|My breathing was fast.|
|My legs shook.|
|I could not stay still for long.|
|It was hard for me to stay in my seat at school.|
|I felt as if I needed to move my legs a lot.|
|It was hard for me to sit still.|
|My muscles felt tight.|
|My back hurt.|
|My neck felt tight.|
|I had a headache.|
|I had pain that really bothered me.|
|My neck hurt.|
|I woke up feeling tired.|
|It was hard for me to wake up in the morning.|
|I had trouble falling asleep.|
|I grinded my teeth at night.|
|I had trouble going to bed.|
|When I woke up, I felt tired.|
|I had trouble staying asleep.|
|I woke up and had thoughts that made me afraid.|
|I had bad dreams or nightmares.|
|I was afraid in the middle of the night.|
|It was hard for me to get up in the morning.|
|I was afraid to go to bed.|
|I threw up.|
|My appetite changed.|
|I had hard poop.|
|I had loose, watery poop.|
|I had a bad stomach ache.|
|It hurt me to poop.|
|I felt some food coming up into my throat.|
Note. All items are prefaced with the context, “In the past 7 days …”; response categories for all items are as follows: Never—Rarely—Sometimes—Often—Always (coded on a 1–5 scale such that higher values indicate more problematic stress response experiences).