|Home | About | Journals | Submit | Contact Us | Français|
This paper describes a large-scale administration of the Patient-Reported Outcomes Measurement Information System (PROMIS) pediatric items to evaluate measurement characteristics.
Each child completed one of seven test forms containing items from a pool of 293 PROMIS items and four legacy scales. PROMIS items covered six domains (physical function, emotional distress, social role relationship, fatigue, pain, and asthma).
From January 2007 to May 2008, 4,129 children aged 8–17 were enrolled. The sample was 51% female, 55% aged 8–12, 42% minority race and 17% were Hispanic ethnicity. Approximately, 35% of the children participating in the survey consulted a clinician for a chronic illness diagnosis or treatment within 6 months prior to study enrollment.
The final PROMIS pediatric item banks include physical function (n = 52 items), emotional distress (n = 35 items), social role relationships (n = 15 items), fatigue (n = 34 items), pain (n = 13 items), and asthma (n = 17 items). The initial calibration data were provided by a diverse set of children with varying health states (e.g., children with a variety of common chronic illnesses) and racial/ethnic backgrounds.
The Patient-Reported Outcomes Measurement Information System (PROMIS) project, a National Institutes of Health Roadmap for Medical Research initiative, was developed to advance the science and application of patient-reported outcomes (PRO) associated with chronic diseases . One main goal of the PROMIS initiative is to develop a set of item banks and computerized adaptive tests for the clinical research community. The PROMIS pediatric project focused on the development of self-report PRO item banks across several health domains for youth aged 8–17. The primary focus was on the measurement of generic health domains that are important across a variety of illnesses, including physical function, pain, fatigue, emotional distress, and social function . Additionally, one disease-specific item bank was developed for children with asthma to explore the relationships between general and disease-specific measures. Asthma was chosen because it is the most common chronic childhood disease, and PRO measurement is an essential component in evaluating outcomes for children with asthma [3–5].
This paper describes the large-scale administration and evaluation of the PROMIS pediatric item banks in a diverse population. The methods utilized for the large-scale field testing as well as the final sample characteristics are reported. Other manuscripts will describe in detail the psychometric properties of the items banks administered during large-scale field testing.
We developed the PROMIS pediatric item banks using a strategic item generation methodology adopted by the PROMIS Network . Six phases of item development were implemented: identification of existing items, item classification and selection, item review and revision, focus group input on domain coverage, cognitive interviews with individual items, and final revision before field testing. Identification of items refers to the systematic search for existing items in currently available pediatric scales. This was utilized to identify an initial item pool of over 3,345 items. Expert item review and revision were conducted by trained professionals who reviewed the wording of each item and revised as appropriate for conventions adopted by the PROMIS network [2, 6]. Focus groups were used to confirm domain definitions and to identify new areas of item development for future PROMIS item banks . Cognitive interviews were used to examine and refine the wording of individual items . The pediatric items were written in the past tense with a 7-day recall period and utilized a standard set of response options . Items successfully screened through the cognitive interview process were sent to field testing. The final item set contained 293 items across six domains (Physical Function = 70 items; Emotional Distress = 49 items; Social Role Relationships = 74 items; Fatigue = 39 items; Pain = 27 items; Asthma = 34 items) .
The purpose of the population sampled was to obtain a diverse set of children with a variety of experiences including health states (e.g., children with a variety of common chronic illnesses), age, and race/ethnicity groups. This sampling strategy was designed to derive a range of representation across the latent traits.
To be eligible to participate in the large-scale testing survey, subjects were required to meet the following inclusion criteria: aged between 8 and 17; able to speak and read English; and able to see and interact with a computer screen, keyboard, and mouse. They provided informed assent prior to study entry, and a parent or guardian provided informed consent. Both the informed assent and the informed consent were administered in English, so parents were also required to read and speak English. Parent reports were used to determine whether or not the child had any limitations (e.g., physical or cognitive impairment) that would make it too difficult to complete a computer-administered survey.
Participants were recruited in hospital-based outpatient general pediatrics and subspecialty clinics and in public school settings. Potential clinic participants were identified through a variety of methods such as a review of pediatric clinic appointment rosters or while in the clinic waiting rooms according to protocols approved by the institutional review boards (IRBs) of The Children’s Hospital at Scott and White (S&W) in Texas, the University of North Carolina (UNC), and Duke University pediatrics clinics. Pediatric patients within the appropriate age range who had clinic appointments were recruited while waiting for their clinic appointments. The UNC, Duke, and S&W general pediatric clinics were representative of health issues for which children have physician office visits (e.g., well-child visits, acute illnesses, and some chronic illnesses). The specialty clinics, which included Pulmonology, Allergy, Gastroenterology, Rheumatology, Nephrology, Obesity, and Endocrinology, primarily saw children with more serious chronic illnesses. Children with asthma were over sampled during recruitment because asthma-specific items were tested.
School-based participants were recruited through the Chapel Hill-Carrboro (NC) Public School System including elementary after-school programs as well as required middle and high school health classes. Rosters for both the after-school programs and health classes were obtained through the school or after-school administrative offices. An informational packet was given directly to, or mailed to, all parents with children enrolled in any of the after-school programs or health classes to inform them about the study. This packet contained general information about the study, the informed consent documents, and parental forms (i.e., the sociodemographic forms) to complete and return to the school.
Parents signed an informed consent document and children signed an informed assent document that outlined the following: purpose of the study, participation requirements, potential benefits and risks of participation, and the measures implemented to protect participant privacy. Child participants received a $10 gift card in return for their time and effort. The study protocols were approved by the institutional review boards at each institution. Data were collected during the time period from January 2007 to May 2008.
The survey was administered on laptop computers in a private location. Children completed the survey at the time of recruitment without parent or peer assistance. Each of the children was assigned to complete one of seven test administration forms (see PROMIS test forms section below). These forms contained either PROMIS items or items from widely used fixed-length measures (legacy items) such as the Pediatric Quality of Life Inventory™ (PedsQL™) Generic Core Scale (PedsQL Generic), and Asthma Module (PedsQL Asthma) [3, 9], KIDSCREEN-52 , and the DISABKIDS Asthma Module .
The PedsQL™ Generic Core Scale is a 23-item multidimensional instrument that measures health-related quality of life (HRQOL) in children and adolescents aged 5–18. It measures core domains of health including physical, emotional, and social functioning, as well as school functioning. The PedsQL Asthma Module was designed to provide greater measurement sensitivity for children with asthma. Both PedsQL instruments (Generic and Asthma) are scored on a 0–100 scale, with higher scores indicating better HRQOL .
KIDSCREEN—52 is a 52-item pediatric instrument for quality of life measurement that was originally developed in Europe. This instrument measures ten HRQOL domains (physical well-being, psychological well-being, moods and emotions, self-perception, autonomy, parent relation and home life, financial resources, peer and social support, school environment, and bullying). The DISABKIDS Asthma Module is an 11-item asthma-specific instrument measuring domains of worry and impact related to asthma symptoms. This European instrument was developed as a sister project to KIDSCREEN and measures the HRQOL and level of distress caused by pediatric chronic disease. Both of these instruments are scored on a 0–100 scale, with higher scores indicating better HRQOL .
PROMIS items covered six domains (physical function, emotional distress, social role relationship, fatigue, pain, and asthma) with the domain definitions listed in the “Appendix”.
Parents, guardians, or caregivers of participants completed a paper–pencil questionnaire to assess sociodemographic variables (including the child’s age, sex, race, ethnicity, and education as well as the caregiver’s education level) and medical history of the child. The medical history included diagnoses of any new chronic health conditions within 6 months prior to study enrollment, treatment for existing chronic health conditions within 6 months prior to study enrollment, lifetime diagnosis of asthma and current asthma medication and treatments.
To limit respondent burden, the number of items administered to any respondent was limited to no more than 76 items out of the entire pool of 293 PROMIS items and the legacy questionnaires. The items were written to accommodate low literacy levels . Based on the experience of the research team, it was estimated that the younger children would be able to complete the survey in about 25 min and the adolescents in about 15 min. The 293 PROMIS items were divided among six testing forms and one additional form containing only general ‘legacy’ scales (see Table 1). Some items were administered on more than one form. The inclusion of overlapping items on different forms permits an evaluation of the associations between domains. Each PROMIS item from non-disease-specific banks was administered to at least 754 respondents and each PROMIS asthma item was administered to at least 622 children.
Children were sequentially assigned to complete one of the seven testing forms. Children with asthma were assigned to one of two forms with asthma items including one form with PROMIS asthma items and one form with legacy asthma items. Children without asthma were assigned to one of five forms (four forms with PROMIS items and a few legacy general items, and one form containing only legacy scales). The legacy form was administered about half as often as the other forms. This sampling plan was developed for collecting responses to the candidate items from the targeted PROMIS domains and was designed to accommodate multiple objectives: (1) confirm the factor structure of the domains; (2) evaluate items for local dependence (LD) and differential item functioning (DIF); (3) calibrate the items for each domain using item response theory (IRT); and (4) estimate profile scores for children with asthma.
Descriptive analyses were conducted to describe the demographic and clinical characteristics of the study population. Mean scores for legacy scales were calculated per instrument instructions [3, 10, 14]. For all legacy instruments, scores ranged from 0 to 100 with higher scores indicating better HRQOL. Detailed psychometric properties of the item banks administered during large-scale field testing are described in detail in the domain-specific papers.
From January 2007 to May 2008, we enrolled 4,129 respondents, with 3,890 recruited from pediatric clinics and 239 from school settings. The sample was 51% female and 49% male. Approximately, 55% were aged 8–12 and 45% were aged 13–17. Fifty-eight percent were white, 23% black, 6% multi-racial, and 10% other (Asian/Pacific Islanders, Native Americans, and other races). Seventeen percent of the sample was of Hispanic ethnicity. The vast majority of the adults providing informed consent for the children were parents of the child (93%) or grandparents (5%). The educational attainment of these parents or guardians ranged from less than high school (8%) to advanced degree (12%), with 24% reporting a college degree, 34% some college, and 22% a high school diploma (Table 2). Most importantly, the sample characteristics were almost identical across the test forms, indicating that the form allocation strategy created similar samples.
Approximately, 35% of the children participating in the survey had consulted a clinician for a chronic illness diagnosis or treatment within 6 months prior to study enrollment (Table 3) and about 9% had two or more chronic illnesses (Table 3). Of these children with a chronic illness diagnosis or treatment, the most common condition was asthma. Attention-deficit disorder/attention-deficit hyperactivity disorder (ADD/ADHD), arthritis, gastrointestinal (GI) disorders, and mental disorders comprised the other most common conditions represented in this sample of children (Table 3).
The mean scale scores for the legacy instruments (PedsQL Generic, PedsQL Asthma Module, KIDSCREEN-52 and DISABKIDS Asthma Module) are shown in Table 4. Figure 1 compares the PedsQL Generic mean scores in the current PROMIS survey to the published normative values for various patient populations. PROMIS clinic and school sample population scores had lower mean total scale scores (74.5) than the published data for healthy children (83.8) . The PROMIS sample had mean scores closer to a group of children with chronic illnesses such as asthma, kidney disease, and psychiatric disease (Fig. 1).
Compared to the sample of children used to establish KIDSCREEN , the PROMIS sample reported worse scores on moods, autonomy, and bullying, and no areas were substantially better (Fig. 2). This confirms the notion that this population was more reflective of children with chronic illness than healthy children.
Children with asthma in the PROMIS sample scored similarly to previously published data for children with asthma on both the DISABKIDS and PedsQL Asthma Modules. (DISABKIDS Asthma published normdata : Impact 65.4 and Worry 79.4; PROMIS sample: Impact 66.0 and Worry 76.2; PedsQL Asthma normdata : Asthma 64.2, Treatment 80.6, Worry 76.3, and Communication 73.7; PROMIS sample: Asthma 68.2, Treatment 79.5, Worry 70.7, and Communication79.2).
The study design described previously allowed for the evaluation of the measurement characteristics of the PROMIS pediatric item banks. Subsequent articles will be published describing the psychometric characteristics of the PROMIS pediatric items evaluated in this testing. The final PROMIS pediatric item banks were developed to provide accurate and efficient assessment of important domains of HRQOL for children including physical function (n = 52 items), emotional distress (n = 35 items), social role relationships (n = 15 items), fatigue (n = 34 items), pain (n = 13 items), and asthma (n = 17 items).
The population used for this testing was derived primarily from clinic populations which may not be representative of those without access to health care. The mean PedsQL Generic Scale and KIDSCREEN-52 scores are lower than the mean score from a population of healthy children and similar to that of a population with chronic illness [10, 15]. This is not surprising considering that children with asthma and other chronic illnesses were oversampled with 9% of the survey population reporting two or more chronic illness diagnoses.
Our study population included children with chronic illnesses and healthy children. This sampling strategy was designed to derive a range of representation across the latent traits. Because we envisioned item banks that measure across the continuum of the traits of interest (e.g., fatigue, physical function), it was important to include children from a variety of experiences. In addition to allowing for broadly measured constructs, by oversampling children with asthma we were able to perform individual analyses within this disease population. We are currently performing cross-sectional testing in other selected chronic illnesses as part of a supplemental project to PROMIS.
The purpose of the population sampled was to obtain a diverse set of children with a variety of experiences including health states (e.g., children with a variety of common chronic illnesses), age, and race/ethnicity groups. The goal of the study was to calibrate the scale’s items (i.e., obtain item parameters) utilizing item response theory which is independent of the sampled population. Hence, population diversity was more important than representativeness. Thus, the goal of this study was not to assess validity but instead to study item parameters. Validity testing will be the focus of future studies that are currently underway. Subsequent papers that describe individual analyses of item banks will further illustrate these challenges and opportunities.
A subset of children in the survey completed the legacy instruments and reported lower domain scores in general than either PedsQL Generic or KIDSCREEN-52 published data for healthy children. Children in the PROMIS sample scored the highest in the Social Functioning domain for PedsQL Generic and in the Social Support and Peers domain for KIDSCREEN-52 scales. The items represented in these domains for these two instruments are quite similar and essentially measure pediatric sociability [10, 14]. Similarly, the children in this study scored lowest on the Emotional Functioning domain in PedsQL Generic and the Moods and Emotions domain in KIDSCREEN; both provide measures of emotional distress [9, 10]. The School Functioning domain in PedsQL Generic primarily focuses on cognitive attention and absence from school, while the School Environment domain in KIDSCREEN-52 primarily focuses on social and emotional aspects of school, which may explain the differences noted [10, 14].
The initial data collection to determine dimensionality and item calibrations for the PROMIS pediatric items banks was performed on an ethnically, racially and experientially diverse group of children aged 8–17. Other manuscripts will describe in detail the psychometric properties of these items banks and the creation of the PROMIS pediatric instruments, version 1.0.
We would like to acknowledge the contribution of Harry A. Guess, MD, PhD, to the conceptualization and operationalization of this research prior to his death. This work was funded by the National Institutes of Health through the NIH Roadmap for Medical Research, Grant 1U01AR052181-01. Information on the Patient-Reported Outcomes Measurement Information System (PROMIS) can be found at http://nihroadmap.nih.gov/ and http://www.nihpromis.org.
Emotional distress commonly refers to unpleasant feelings or emotions that are experienced privately and, therefore, are good candidates for assessment as patient-reported outcomes. Emotional distress among children comprises feelings of anxiety, depression, and anger.
Depressive symptoms among children often include feelings of hopelessness, helplessness, and worthlessness. The PROMIS pediatrics item bank for depression focuses on negative mood (e.g., sadness), decrease in positive affect (e.g., loss of interest), negative views of the self (e.g., worthlessness, low self-esteem), and negative social cognition (e.g., loneliness, interpersonal alienation). This item bank is best characterized as depressive symptoms rather than a complete diagnostic test for depression which may include other physical manifestations.
Symptoms that best differentiate anxiety are those that reflect autonomic arousal and the experience of threat. Children often experience these feelings in a variety of contexts specific to their environment of home, school, and social activities. The PROMIS pediatric item bank for anxiety focuses on fear (e.g., fearfulness), anxious misery (e.g., worry), and hyperarousal (e.g., nervousness).
Anger is distinguished by attitudes of hostility and cynicism and is often associated with experiences of frustration impeding goal-directed behavior. Specific components relate to verbal and nonverbal evidence of interpersonal antagonism. The PROMIS pediatric item bank for anger focuses on angry mood (e.g., irritability, reactivity), and aggression (verbal and physical).
Social health is defined as perceived well-being regarding social activities and relationships, including the ability to relate to individuals, groups, communities, and society as a whole. The term “social health” is used here synonymously with “social function” and refers to a higher-order domain, with measurable subdomains. Components of social functioning include understanding and communication, getting along with people, participation in society, and performance of social roles.
One common goal of childhood is success in socializing with others. This enables one to create positive relationships with family, friends, teachers, and colleagues. Social interaction with peers is the initial focus of PROMIS pediatric investigation in social health.
Physical function is defined as one’s ability to carry out various activities, ranging from self-care (activities of daily living) to more challenging and vigorous activities that require increasing degrees of mobility, strength or endurance. Although several important aspect of physical function can be measured, the initial pediatric item banks will focus on (1) mobility and (2) upper extremity.
This item banks focuses on activities of physical mobility such as getting out of bed or a chair to activities such as running.
This item bank focuses on activities that require use of the upper extremity including shoulder, arm, and hand activities. Examples include writing, using buttons, or opening containers.
Pain is an unpleasant sensory and emotional experience associated with actual or potential tissue damage, or described in terms of such damage. Pain is what the patient says it is—that is, the “gold standard” of pain assessment is self-report. Pain is divided conceptually into components of quality (referring to the nature, characteristics, intensity frequency, and duration of pain), interference with activities (impact upon physical, mental, and social activities), and avoidance behaviors (behaviors one engages into avoid, minimize, or reduce pain). The initial PROMIS pediatric item bank focuses on the interference impact of pain.
Fatigue is defined as an overwhelming, debilitating, and sustained sense of exhaustion that decreases one’s ability to carry out daily activities, including the ability to work effectively (for pediatric populations this primarily translates to school work) and to function at one’s usual level in family or social roles.
Asthma causes several symptoms for children that are not addressed in the generic item banks which include cough, wheeze, shortness of breath, and avoidance of triggers. Asthma is also associated with impacts such as missing school or activities with other children. The PROMIS pediatric asthma item bank focuses on symptoms specific to asthma.
Debra E. Irwin, Department of Epidemiology, University of North Carolina at Chapel Hill, CB #7294, Chapel Hill, NC 27599, USA.
Brian D. Stucky, Department of Psychology, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA.
David Thissen, Department of Psychology, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA.
Esi Morgan DeWitt, Department of Pediatrics, Division of Rheumatology, Cincinnati Children’s Hospital and Medical Center, Cincinnati, OH, USA.
Jin Shei Lai, Department of Medical Social Sciences, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
Karin Yeatts, Department of Epidemiology, University of North Carolina at Chapel Hill, CB #7294, Chapel Hill, NC 27599, USA.
James W. Varni, Department of Pediatrics, College of Medicine, Department of Landscape Architecture and Urban Planning, College of Architecture, Texas A&M University, College Station, TX, USA.
Darren A. DeWalt, Division of General Medicine and Clinical Epidemiology, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA. Cecil G. Sheps Center for Health Services Research, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA.