|Home | About | Journals | Submit | Contact Us | Français|
The Pediatric Cardiac Quality of Life Inventory (PCQLI) is a disease-specific, health-related quality of life (HRQOL) measure for pediatric heart disease (HD). The purpose of this study was to demonstrate the external validity of PCQLI scores.
The PCQLI development site (Development sample) and six geographically diverse centers in the United States (Composite sample) recruited pediatric patients with acquired or congenital HD. Item response option variability, scores [Total (TS); Disease Impact (DI) and Psychosocial Impact (PI) subscales], patterns of correlation, and internal consistency were compared between samples.
A total of 3,128 patients and parent participants (1,113 Development; 2,015 Composite) were analyzed. Response option variability patterns of all items in both samples were acceptable. Inter-sample score comparisons revealed no differences. Median item–total (Development, 0.57; Composite, 0.59) and item–subscale (Development, DI 0.58, PI 0.59; Composite, DI 0.58, PI 0.56) correlations were moderate. Subscale–subscale (0.79 for both samples) and subscale–total (Development, DI 0.95, PI 0.95; Composite, DI 0.95, PI 0.94) correlations and internal consistency (Development, TS 0.93, DI 0.90, PI 0.84; Composite, TS 0.93, DI 0.89, PI 0.85) were high in both samples.
PCQLI scores are externally valid across the US pediatric HD population and may be used for multi-center HRQOL studies.
Children and adolescents with congenital or acquired heart disease (HD) have benefited from recent advancements in pediatric cardiac surgery and heart transplantation, interventional procedures, imaging, and intensive care [1–3]. In the current era, neonatal mortality for children with the most complex congenital HD (e.g. hypoplastic left heart disease) is less than 10% , and cardiac-related mortality during the first two decades remains low . However, the medical, surgical, and catheter-based therapies required to treat the underlying cardiac abnormality result in morbidities that affect the patient's psychosocial [5–7], neurodevelopmental [8–10], and physical functioning [11–13] and have a cumulative impact on their health-related quality of life (HRQOL).
Assessment of HRQOL is increasingly important in the pediatric cardiac population, because it can provide information that may improve clinical decision-making. HRQOL refers to the impact of a specific illness, medical therapy, and/or health services policy on the ability of the patient to function in situational contexts (e.g. family, school, peer) and to draw personal satisfaction from a physical, psychological, and social functioning perspective . Pediatric HRQOL can be measured using both generic and disease-specific instruments [15–23]. However, use of a disease-specific instrument may provide a more comprehensive view of a particular condition or disease and may be more sensitive to change over time. Additionally, a disease-specific instrument may better discriminate differences among disease sub-groups. To evaluate differences in sub-populations and therapeutic regimens, a reliable and valid disease-specific HRQOL measure is needed for future cross-sectional and prospective studies that assess outcomes in the pediatric cardiac population.
Validation of a psychometric scale is an ongoing, evidence-based process that assesses the degree of confidence one should have in inferences made about a test-taker based on their score. For purposes of substantiation of a quality of life instrument, validation is often divided into the domains of “internal” and “external” construct validity [24–26]. “Internal validity” may be thought of as an assessment of content validity and structural validity [24, 25, 27, 28] which includes the following: assessment of the theoretical conceptualization of the respective instrument; the clarity, relevance, and representativeness of the item content; and tool construction. In contrast, establishing “external validity” involves the assessment of convergent and discriminant construct validity and “generalizability” [24, 25, 29]. “Generalizability” may be defined as the ability of a tool to provide valid and reliable information when utilized in different geographic regions and patient populations . Generalizable tools allow researchers to have confidence that data collected from multiple sites and regions are comparable.
Many HRQOL instruments have established the “generalizability” characteristic of external validity by re-evaluating reliability and validity in a new geographic location (region or country), racial group, ethnic group or language [15, 30–34]. The “generalizability” aspect of external validity may be confirmed through comparing components of Classical True Score Theory method including response option variability (i.e. whether questions are discriminating), scores (i.e. whether similar test-takers have equivalent scores), patterns of correlation (i.e. whether patterns of responses correlate across items, subscales, and scale), and internal consistency (i.e. whether reliability is comparable) measurements between the Development site sample and a distinct multi-site sample [15, 30–35]. Substantiation of the “generalizability” part of external validity enables a HRQOL instrument to be used for clinical applications and multi-center research that may serve as a platform for future multi-site cross-sectional and prospective studies using HRQOL as an outcome.
The Pediatric Cardiac Quality of Life Inventory (PCQLI), developed at The Children's Hospital of Philadelphia , is a disease-specific HRQOL tool for children and adolescents with HD. It is self-administered, brief (completion time less than ten minutes), and inclusive of a wide age range (8–18 years) with parent-proxy reporting. Previously, Marino et al.  described pilot reliability and validity characteristics of the PCQLI at a single site and response option variability testing and correlation coefficient data between item response, subscale score, and total score were used to create the final forms of the PCQLI. A subsequent study in a large multi-center, cross-sectional cohort in the United States indicated that the PCQLI forms are reliable and internally valid as measured by test–retest reliability and a construct validity model incorporating cross-informant variance and correlations of PCQLI scores with disease severity, medical care utilization, and established generic HRQOL, self-perception, competency, and behavioral measures .
The purpose of the present study was to demonstrate the “generalizability” aspect of the external validity of the PCQLI scores by comparing item response option variability, scores, patterns of correlation, and internal consistency between the development site sample and a national composite sample.
This study was a prospective, multi-center, cross-sectional study of pediatric patients with congenital or acquired HD in the United States and their parent/guardian. The institutional review boards of all participating institutions approved the study. Cincinnati Children's Hospital Medical Center was the principal data-coordinating site (IRB Protocol No. 07-03-09).
Participants were recruited in the United States at seven large, pediatric cardiology outpatient centers between November 2004 and December 2008. Recruitment sites were chosen for their geographic, racial, and ethnic diversity. The PCQLI was developed at The Children's Hospital of Philadelphia (Development sample). The Composite sample was composed of participants from the other six recruitment sites: Cincinnati Children's Hospital Medical Center, Children's Hospital Boston, Children's Hospital of Wisconsin, Phoenix Children's Hospital, the University of California San Francisco Children's Hospital, and the University of Texas Southwestern Medical Center at Dallas.
All patients that had congenital or acquired HD and were 8–18 years of age, fluent in English, and attending a “routine” outpatient cardiology visit at the participating sites were eligible for the study. Examples of acquired HD include patients with electrophysiologic disease, cardiomyopathy, acquired valvular disease, and Kawasaki disease. “Routine” was defined as a regularly scheduled visit when the patient was in his or her usual state of health. Children (8–12 years old) or adolescents (13–18 years old) who came to clinic because of an acute change in their clinical status or had a significant co-morbid medical condition (e.g. cystic fibrosis, sickle cell disease, human immunodeficiency virus infection) or major developmental delay were excluded. Parents (an inclusive term for parents or guardians of eligible patients) were excluded if they were non-English speaking or had a major developmental delay. All eligible, consenting/assenting patient–parent pairs were included in the study sample.
A research nurse or trained research assistant administered the PCQLI to all enrolled patients and parents prior to the cardiology visit. Participants were supervised while completing the patient and parent-proxy inventories to minimize data contamination resulting from patient–parent discussion. If the patient or parent were unable to read the inventory, the investigator read it aloud.
Patient–parent demographic information and patient medical history data were also acquired. Demographic data included gender, race, ethnicity, age, annual household income, and Hollingshead socioeconomic status score . Medical and surgical data included type of HD (congenital vs. acquired), type of congenital HD (two-ventricle vs. single-ventricle), original diagnostic category, current cardiac status, number of prior catheterizations/interventions, number of prior cardiac surgeries, time since last hospitalization, and number of doctor visits within the past year. For original diagnostic category, patients were classified into a scheme of congenital HD previously shown to be highly associated with hospital mortality based on the number of functioning ventricles and the presence or absence of aortic arch obstruction  (two-ventricle without aortic arch obstruction, two-ventricle with aortic arch obstruction, single-ventricle without aortic arch obstruction, and single-ventricle with aortic arch obstruction) or classified as acquired HD. For current cardiac status, patients were classified into the following types: mild congenital HD (no prior surgical or catheter-based intervention), post-heart surgery or catheter-based intervention, post-heart transplantation, and structurally normal heart. Patient clinical history was obtained from medical records and parent interview.
The PCQLI score results that serve as the basis for this study's external validity analyses are available within the test–retest reliability and construct validity analyses published elsewhere . External validity confirmation proceeded in four discrete stages. First, descriptive statistics were computed for all relevant demographic and clinical variables of the Development and Composite samples. Six different clinical variables were chosen for further analysis for their relevance to families and clinicians caring for pediatric patients with congenital or acquired HD. Depending upon the characteristics of the data, Chi-squared and Wilcoxon rank-sum tests were applied. The type I error (alpha) was adjusted to account for the correlated nature of the six clinical variables (αADJ = 0.008). Second, response option variability was individually assessed using simple and relative frequency measurements for every PCQLI item in each respondent group (Child, Parent of Child, Adolescent, Parent of Adolescent) of the Development and Composite samples. PCQLI item responses were based on a five-point Likert scale (strongly agree to strongly disagree). Child and Parent of Child forms have 14 items in the Disease Impact subscale and 9 items in the Psychosocial Impact subscale. Adolescent and Parent of Adolescent forms have 17 items in the Disease Impact subscale and 12 items in the Psychosocial Impact subscale. If any item on any form in either sample had a single response option composing 90% or more of the item's selected responses, it was identified as potentially problematic and worthy of further review. Such items are not variable enough to add significant meaning to the sum scores generated . For each item for all four forms for both the Development and Composite samples, the highest response option percentage was calculated. Third, the psychometric properties of the PCQLI scores were evaluated by comparing the Development and Composite samples by respondent group. Overall PCQLI scores were compared between the Development and Composite samples. Median Spearman correlations representing four different components of Classic True Score Theory were computed: item–subscale, item–total, subscale–subscale, and subscale–total. In each case, derived scale scores were computed with the item response under analysis excluded. Fourth, internal consistency for each PCQLI respondent group was qualitatively assessed using Cronbach's alpha coefficient analysis of the PCQLI Total score (TS) and principal subscales [Disease Impact (DI) and Psychosocial Impact (PI)]. Cronbach's alpha values greater than 0.70 were considered internally consistent .
All data were analyzed using SAS v9.1. For all statistical comparisons, an a priori significance level of P<0.008 was used. Correlations were interpreted as ≤0.20 poor; 0.21–0.40 fair; 0.41–0.60 moderate; 0.61–0.80 good; and ≥0.81 excellent agreement .
A total of 3,128 patients and parents were evaluated in the PCQLI Testing Trial (total enrollment 3,210; consent rate 85%): the Development sample was composed of 1,113 patient and parent participants and the Composite sample of 2,015 patient and parent participants (Table 1). The age category (Child, Adolescent) distribution of patients was comparable between samples. The majority of patients were males. In contrast, male parent respondents comprised a low percentage of the total. Patients and parents in the Development and Composite samples were predominantly Caucasian and non-Hispanic with African Americans being the largest minority group. African Americans constituted 10.7 and 7.2% of patients in the Development and Composite samples respectively, and 10.1 and 7.0% of parents in the Development and Composite samples respectively. Hispanic/Latino participants constituted 3.7 and 5.9% of patients in the Development and Composite samples respectively, and 2.4 and 5.7% of parents in the Development and Composite samples respectively. The mean ages of PCQLI respondent groups did not differ significantly between samples. For the total sample, the mean ages were 9.9 years for Child, 40.6 years for Parent of Child, 15.2 years for Adolescent, and 45.2 years for Parent of Adolescent. About one-quarter of households had an average household income of less than or equal to $50,000. The average Hollingshead SES scores ranged from 45.5 to 46.5. These Hollingshead scores indicate a predominantly upper-middle class SES for both sample populations.
For each item, no response option was chosen more than 90% of the time. The highest response option percentage for any individual item on any of the four respondent forms was 72% from the Development sample and 71% from the Composite samples. Ranges for the highest response option percentage for the Development sample by PCQLI form were 27–63% for Child, 29–71% for Parent of Child, 31–72% for Adolescent, and 28–71% for Parent of Adolescent. Ranges for the highest response option percentage for the Composite sample were 24–58% for Child, 26–71% for Parent of Child, 27–69% for Adolescent, and 27–70% for Parent of Adolescent.
PCQLI score and scale correlation comparisons between the Development and Composite samples revealed no significant differences. Median item–subscale score and item–TS correlations were moderate for both the Development and Composite samples. There were negligible differences in the median and range of the item–DI subscale score, item–PI subscale score, and item–TS correlations within respondent groups. Subscale–subscale score (DI-PI) and subscale–TS (DI-TS, PI-TS) correlations were high and similar between samples. Patterns of correlations were very similar between the Development and Composite samples across all item–TS, item–subscale, subscale–subscale, and subscale–TS correlation comparisons (Table 2).
Measures of internal consistency were high across all forms and qualitatively indistinguishable between Development and Composite samples (>0.70) (Table 3).
The Development and Composite samples differed significantly with respect to the type of HD (congenital vs. acquired), original diagnostic category, and current cardiac status. The Development sample had a lower percentage of congenital HD patients compared with the Composite sample (P<0.001). However, despite the difference in the overall percentage of patients with congenital HD between the samples, there was no difference in the relative percentage of single-ventricle vs. two-ventricle congenital HD groups as both samples had predominantly two-ventricle physiology (Table 4). The percent distribution of original diagnostic category and current cardiac status differed between samples for children (P<0.001) (Fig. 1). Similarly, the percent distribution of original diagnostic category and current cardiac status differed between samples for adolescents (P<0.001) (Fig. 2). The most common original diagnostic category was two-ventricle congenital HD without aortic arch obstruction for children in both samples and for adolescents in the Composite sample (Figs. 1, ,2).2). In contrast, the most prevalent original diagnostic category in adolescents from the Development sample was acquired HD with a structurally normal heart (Fig. 2). Of note, 32.3% (n = 177) of patients in the Development sample were diagnosed with EP disease in contrast to only 11.7% (n = 117) in the Composite sample. The majority of patients in both samples had a current cardiac status of post-surgery or catheter-based intervention; however, the Development sample had significantly more patients with a structurally normal heart than the Composite sample (P < 0.001) (Figs. 1, ,22).
Although the median number of prior catheterizations/interventions for children and adolescents in both samples was 1, there was significant variation in the adolescent population between the Development and Composite samples (P < 0.001). Additionally, the median number of prior cardiac surgeries for children in both samples was 1, with significant variation between the samples (P < 0.001). The median number of prior cardiac surgeries for adolescents from the Development and Composite samples differed, 0 and 1 respectively, and significant variation existed between the samples (P < 0.001). There were no significant differences in the median number of doctor visits in the past year or the time since last hospitalization between the Development and Composite samples for either the child or adolescent populations (Table 4).
This study demonstrates that the PCQLI is a disease-specific, pediatric, cardiac HRQOL tool that is externally valid across English-speaking patient populations throughout the United States. Subject demographics were similar between the Development and Composite samples. Item-level analysis demonstrated that the response patterns of all items in both samples have variation that allows for discriminatory scoring. PCQLI scores between samples had no significant differences. Median item–subscale and item–total correlation estimates were moderate in each sample. Subscale-level analyses demonstrated that subscale–sub-scale and subscale–total correlations were high in both samples. Additionally, internal consistency measurements were high for subscale and total scores across all respondent groups for both the Development and Composite samples. These results support the external validity and broad applicability of the PCQLI in the United States.
Comparisons between the Development and Composite samples showed statistically significant differences in clinical variables including original diagnostic category, current cardiac status, and number of prior cardiac surgeries and catheterization/interventions. Differences in the original diagnostic category and current cardiac status can be attributed to a greater percentage of patients with electrophysiologic disease and acquired HD in the Development sample in both the child and adolescent populations. Patients with acquired HD may not present until later in childhood or adolescence. Furthermore, patients with primary electrophysiologic disease typically have fewer cardiac surgeries, catheterizations/interventions, and hospitalizations relative to children with complex congenital HD. The marked similarities in response option variability, score comparisons, patterns of correlations, and internal consistency, given the differences in clinical variables between the two samples, further support the broad applicability of the PCQLI among multiple pediatric, cardiac populations in the United States.
The published disease-specific, pediatric, cardiac HRQOL instruments have minimal external validity substantiation for use in the United States. The PedsQL Cardiac Module  and the Congenital Heart Adolescent and Teenager Questionnaire  were developed at single centers in the United States and Canada respectively. The ConQOL was developed through a multi-center study in England , but has yet to demonstrate externally valid outside of England. Based on the lack of external validation data, information gathered through use of these instruments outside of the development site or region may not be reliable and valid.
The PCQLI, as a reliable  and valid (internal  and external) disease-specific HRQOL measure, is a needed and valuable instrument for the evaluation of HRQOL in the pediatric HD population given the increasing number of survivors. In addition to its use as a clinical decision-making tool, the PCQLI can be used for multi-site cross-sectional and prospective studies. Medical treatments will continue to advance and further decrease pediatric HD mortality. Application of the PCQLI may facilitate this process through assessment of treatment protocol efficacy, responsiveness, and sub-population variations to therapeutic regimens. In conjunction, PCQLI research may benefit the increasing number of pediatric HD survivors through identification of factors whose modification would lessen the psychosocial and physical impact of HD morbidities on the patient's HRQOL.
Despite many racial populations being represented in the subject population, the study participants were predominantly Caucasian and non-Hispanic with few Asian and American Indians. During the PCQLI Testing Trial, no goals for enrollment of specific ethnic and racial groups were placed on the sample populations. In addition, it is important to recognize that many Hispanic/Latino potential participants were ineligible because they were non-English speakers. This meant that certain racial and ethnic groups, despite our efforts to increase diversity in the Composite sample, might be underrepresented relative to the larger United States population when the study was completed. Although consistent with other HRQOL studies, the majority of parent respondents were female. Furthermore, the socioeconomic status of the parent respondent population was above average. Any of these population characteristics could limit the broad applicability of the PCQLI. In addition, a comparison of the factor structure between the Development and Composite samples was not performed. Future comparison of the factor structure may be useful to further refine the PCQLI.
PCQLI scores are externally valid across the United States' English-speaking, pediatric cardiac population, and PCQLI testing outside the United States is currently underway. As a well-substantiated disease-specific HRQOL instrument, the PCQLI is a needed and valuable tool for clinical and research settings and may be used for multi-site cross-sectional and prospective HRQOL research studies designed to improve patient outcomes . Future research and translation of the PCQLI will further expand the applications of the PCQLI to include additional ethnic, racial, socioeconomic, lingual, and geographic populations.
The following persons participated in the enrollment of patients, data collection, study coordination or manuscript preparation: Cincinnati Children's Hospital Medical Center—Charlotte Andersen, RN, MSN, PMP, Shawna Hottinger, MS, Aimee Baker, MA, Emily Claybon, MA, Kaleigh Coughlin, BA, Brett Morgan, BS, Baiyang Wang, MS, Loran Carroll, BS, Melanie Riedel, BS, Michael Whalen; The Children's Hospital of Philadelphia—Linda Hurd, MSN, CRNP, Janice Prodell, RN, Lynda Ahearn, RN, Lydia Kruge, BA, Anita Pudusseri, BS, Darryl Powell, Andrew Schissler, BSE, Josie Welkom, BA, Stanley O. Dunn, BA; Children's Hospital Boston—Annette Baker, MSN, CRNP, Jill Cotter, BA, Danielle Martin, BA Erica Denhoff, BA, Ellen McGrath, BSN; University of California San Francisco Children's Hospital—David Teitel, MD, Laura Robertson, MD; Phoenix Children's Hospital—Melissa Hill, PA-C; Children's Hospital of Wisconsin—Stuart Berger, MD, Nancy Ghanayem, MD, Lisa Young-Borkowski, MSN, Mary Krolikowski, MSN, Angie Klemm, Mara Koffarnus; University of Texas Southwestern Medical Center Dallas—Gloria Williams, BA. Funding Sources: National Institute of Child Health and Human Development K23 Grant (5-K23-HD048637), Rockville, MD. American Heart Association – Pennsylvania/Delaware Affiliate (now Great Rivers Affiliate) Beginning Grant-in-Aid (0465467U), Columbus, OH. Cincinnati Children's Hospital Research Foundation, Cincinnati, OH. The Children's Hospital of Philadelphia Institutional Development Fund, Philadelphia, PA.
This study is conducted for the PCQLI Testing Study Consortium Investigators. Other investigators for the PCQLI Testing Study Consortium are listed in the acknowledgments.
Conflict of interest There are no financial relationships or conflicts of interest to disclose relevant to this manuscript.