|Home | About | Journals | Submit | Contact Us | Français|
Fatigue is one of the most debilitating conditions associated with cancer and anticancer therapy. The lack of reliable and valid self-report instruments has prevented accurate assessment of fatigue in pediatric oncology patients. The purpose of this study was to identify the most sensitive and specific score, i.e., the “cut score,” on the Fatigue Scale–Child (FS-C) to identify those children with high cancer-related fatigue in need of clinical intervention. We first used Rasch methods to identify the items on the FS-C that distinguished children with high cancer-related fatigue from other children; our findings indicated that the FS-C needed to be reduced from 14 items to 10 items. We then assessed the 10-item FS-C for its psychometric properties and applied the receiver operating characteristics (ROC) curve analysis to the FS-C responses from 221 children (aged 7–12 years) receiving anticancer treatment. The cut score identified with 75% sensitivity and 73.5% specificity was 12; 73 (33%) patients scored 12 or higher. Findings from this validated instrument provide a needed guide for clinicians to interpret fatigue scores and provide clinical interventions for this debilitating condition to their pediatric patients with cancer.
Pediatric patients with cancer report that fatigue is one of the most prevalent, intense treatment- and illness-related effects that they experience during therapy.1,2 Parents of dying children report cancer-related fatigue to be the most prevalent symptom experienced by their child at the end of life (defined as the final 30 days of life).3 Despite its reported prevalence and intensity, cancer-related fatigue is not routinely assessed during treatment, as recommended in the fatigue evidence-based guidelines from the National Comprehensive Cancer Network.4 This insufficient attention to a distressing and common clinical symptom is related, in part, to the only recent availability of reliable and valid instruments to measure cancer-related fatigue in children.5–8
The advances in instrument development need to be accompanied by accurate interpretation of fatigue scores, so that children with high fatigue can be identified and provided the necessary clinical interventions. The primary purposes of our study were as follows: 1) to identify the items of the Fatigue Scale – Child (FS-C) that most distinguish children with or without high fatigue, and 2) to determine the most sensitive and specific score, i.e., the “cut score,” on the FS-C that identifies children with cancer who have high fatigue meriting clinical intervention. The secondary purpose was to identify the proportion of the studied children with cancer who scored at or above the cut score.
Here we describe the steps taken to identify the FS-C items that discriminate magnitudes of fatigue and to determine a sensitive and specific cut score for the FS-C that indicates high cancer-related fatigue in children 7- to 12-years of age. This process included applying Rasch methods to the original 14-item FS-C and applying the receiver operating characteristics (ROC) curve techniques to the amended FS-C to identify the score that most accurately identifies children with high cancer-related fatigue.
Our study sample consisted of 221 pediatric patients receiving treatment for cancer who participated in one of three fatigue instrument-testing studies (Table 1). The first study included 53 pediatric patients with low- or standard-risk acute lymphocytic leukemia (ALL) who were receiving outpatient continuation treatment (between Weeks 50 and 76) at one of three children’s cancer centers (two in the United States and one in Canada). These patients completed the FS-C, and their parents (n=51) completed the Fatigue Scale – Parent (FS-P) and the Daily Sleep Diary – Parent. The second study included 150 patients from seven childhood cancer centers in the United States. The patients completed the FS-C, and their parents (n=148) completed the FS-P.5 The majority of patients (n=95, 63.3%) were diagnosed with leukemia, and the second largest patient group (n=28, 18.7%) was diagnosed with a solid tumor. All were receiving frontline therapy (n=132, 88%) or treatment for recurrence of disease while on therapy (n=18, 12%). 5 The third study included 18 pediatric oncology patients and parents from two pediatric cancer centers in the United States. Patients had either a solid tumor (n=16, 88.9%) or acute myelocytic leukemia (n=2; 11.1%). These 18 patients comprised a convenience sample of children admitted for a two- to four-day inpatient stay for scheduled chemotherapy. They completed the FS-C, and their parents completed the FS-P each day of the hospitalization.9 Of the 221 patients whose data are included in the analysis reported here, the majority were male (n=119, 53.9%), of white race (159, 71.9%), and had a primary diagnosis of leukemia (n=150, 67.9%).
Three instruments, one patient report (the FS-C) and two parental reports (the FS-P and the Daily Sleep Diary-Parent), were used for the purposes of this study.
This 14-item, self-report instrument measures cancer-related fatigue and provides a fatigue intensity score. Each item has a 5-point Likert-type format; total scale intensity ratings range from 0 (no fatigue symptoms) to 70 (highest possible fatigue score). The FS-C has been evaluated for face, content, and construct validity and for internal consistency (Cronbach’s alpha for the 7-day version was 0.84; for the 24-hour version, 0.81).10 The FS-C requires 4 to 5 minutes to complete.5,9 Most recently, 6-year olds completed the FS-C without difficulties.10
This 14-item instrument documents parents’ perceptions of their child’s fatigue during cancer care and yields a fatigue intensity score. Items are formatted exactly the same as those in the FS-C. The FS-P has been evaluated for face, content, and construct validity and for internal consistency.5 The FS-P requires 3 to 5 minutes to complete.
This 16-item parent report measures the parent’s perceptions of the child’s sleep and nap patterns during the previous 24-hour period. The items were derived from a questionnaire developed and tested by Sadeh to identify sleep characteristics (i.e., latency, efficiency, duration, and quality).11 Additional items relate to naps, tiredness, and energy level. Sadeh reports the items to be strongly correlated with actigraphic findings (r=0.89; P=0.001). The diary requires 4 to 5 minutes to complete.9,10
Responses to nine questions posed during a series of separate focus groups for children 7 to 12 years of age, their parents, and their health care providers were the basis of the conceptual definition of cancer-related fatigue in children and for the actual items that comprised the FS-C instrument.1,8 The resulting 14-item FS-C was tested in a series of studies involving more than 200 children (7–12 years old) and assessed for its internal consistency, ability to distinguish between known groups, and construct validity.5,9,10 We did not examine the ability of FS-C items to discriminate magnitudes of fatigue, nor did we identify the FS-C score that indicates high cancer-related fatigue until we accrued a sufficient sample to complete such analyses.
All analyses were completed on the combined patient responses and parent responses from the three studies. The patient demographic data were analyzed using descriptive statistics. We applied Rasch analytic techniques in the WINSTEPS computer program to the FS-C data to identify items with a poor statistical fit (i.e., those FS-C items that either over- or under assessed the level of cancer-related fatigue in pediatric oncology patients). Fit statistics, as reported via the MnSq (acceptable levels, 0.6–1.4), were calculated for each item to assess fit with the Rasch model. We also examined the separation index among the items to learn the levels of fatigue captured by the items. We combined the Rasch findings about items with a poor statistical fit with the investigator-maintained field notes on patient responses and queries about the FS-C items to identify items that contributed to a poor fit because of item construction or lack of clarity of the item’s meaning. We also examined the distribution of responses across the response options for each item. We used statistical and graphical evidence from the Rasch analysis and conceptual understandings to determine whether or not to retain an item on the FS-C.12,13 Although Rasch analysis has been used in the development of adult quality of life and fatigue scales, only recently has the Rasch model been used to measure fatigue in children with cancer.7
Responses to the FS-C and FS-P were assessed for their internal consistency using Cronbach alpha coefficients. Construct validity was first estimated by using an exploratory factor analysis and then a confirmatory three-factor solution and a principal components procedure with and without varimax orthogonal factor rotations. Concurrent validity was estimated using a Pearson’s correlational analysis between the FS-C and FS-P responses.
We then applied a ROC curve analysis, a statistical procedure that includes the use of graphs of classified scores to gauge the performance of an instrument over an entire range of scores to assist with medical decision making.14,15 This method also offers a means for determining the optimal cut score on an instrument to indicate at-risk individuals or groups on the measured variable. This is accomplished by assessing the sensitivity and specificity of an instrument at any given cut-point in the range of scores, when compared to an independent criterion often referred to as the “gold standard.”16 Sensitivity is an operating characteristic of a test that assesses the test’s capacity to accurately identify the presence of a symptom when it is truly present, as determined by the gold standard. In contrast, specificity is the capacity of a test to accurately identify the absence of a symptom when it is actually absent, as determined by the gold standard.17,18 The end result of this analysis is that one number is extracted to represent the full curve; the area under the curve (AUC) is considered a good measure of accuracy. The AUC is a nonparametric statistic that ranges from 0.50 to 1.0 with higher AUC representing better accuracy. AUCs less than 0.70 represent poor accuracy, and those ranging from 0.70 to 1.0 represent relatively good accuracy.18
We assessed the accuracy of the FS-C in terms of sensitivity and specificity when compared to the parents’ report of their child’s cancer-related fatigue as the gold standard. The statistically significant correlation between child and parent reports of the child’s fatigue has been previously documented.5 For the initial sample, a ROC curve analysis was used to determine the cut-score on the FS-C that achieved maximal sensitivity and specificity. We used the Youden’s Index as the criterion to identify the cut point, because it maximizes the sum of specificity and sensitivity.19 The identified cut-score was then applied to the study sample to determine what proportion of patients would be classified as having high fatigue according to this cut score. Two items, one from the Daily Sleep Diary-Parent form (item 10: Today, did your child seem tired during the day?) and one from the FS-P (item 7: My child has not had energy (today) to participate in daily activities.) were selected to serve as the independent criterion in the ROC procedure. Only the parent response option equivalent to “very tired” from each of the two items was used in the analysis to indicate high fatigue in the child. The two items were selected for their very similar conceptual meaning to the FS-C items and for their significant positive correlation with the FS-C total scale scores (r=0.285, P=0.033 for item 10 and the FS-C, and r=0.327, P<0.0001 for item 7 and the FS-C). ROC analysis requires matched patient and parent reports; thus, our sample numbers varied slightly compared with those of other analyses, because four parents did not complete the questionnaire items used as the gold standard comparison.
One FS-C item (“I have slept more at night.”) had an unacceptable MNSq coefficient, and two items had borderline acceptable MNSq coefficients. A fourth item, as documented in the study field notes maintained by the study team members, elicited multiple clarification-seeking queries from respondents. We deleted these four items from the FS-C, completed a psychometric assessment of the 10-item FS-C, and compared those results with results from the 14-item version.
The 10-item version of the FS-C achieved a Cronbach alpha of 0.76, whereas the standard FS-C achieved a Cronbach alpha of 0.81. The three-factor solution yielded acceptable goodness-of-fit indicators for the 10-item version. The eigenvalues for each of the three factors were 4.67 (Not able to function), 2.14 (Lack of energy), and 1.02 (Altered mood). The final communality estimate was 4.95, and the overall Kaiser’s Measure of Sampling Adequacy (MSA) was 0.732. The Goodness-of-Fit index (GFI) was 0.94, which was just above the criterion (0.90); and the Root Mean Square (RMS) Residual was 0.028, which was close to the zero criterion; and the absolute value of the unstandardized residuals was very small (0.051). The internal consistency estimates for the three factors ranged from 0.673 (Not able to function) to 0.841 (Lack of energy). The correlation between the FS-C and the FS-P in the sleep study was 0.64 (P<0.0001) in 51 patient/parent dyads, and for the total study sample across all three studies, it was 0.441 (P<0.0001).
On the 10-item FS-C, the cut score indicating patients with high fatigue was 12. This was identified with 75% sensitivity, 73.5% specificity, and an AUC of 0.768 (standard error [SE], 0.044; 95% confidence interval [CI], 0.68, 0.85) (Figure 1). The number of cases potentially misclassified was 57 (49 false positives and 8 false negatives).
Seventy-three of 221 (33%) patients had a score of 12 or more on the FS-C. In the first study, 8 of 51 (14.5%) patients scored 12 or more on the first day of measurement; this proportion increased to 18 of 53 (34%) patients on the final day of measurement. In the second study, 48 of 150 (32%) patients had a score of 12 or more. In the third study, cancer-related fatigue appeared to increase during the hospitalization. On the first night, 2 of 16 (12.5%) patients scored 12 or higher; on the second night, 7 of 18 (38.9%); and on the third night, 6 of 15 (40.0%) scored 12 or more.
Results from our Rasch analysis led us to reduce the FS-C from 14 items to 10 items, thereby reducing the burden of time on children to complete the instrument and assessing only those items that best discriminated between patients who experienced high cancer-related fatigue and those who did not. Our confirmatory factor analysis revealed that the 10-item FS-C achieved acceptable internal consistency estimates, concurrent validity, and construct validity; thus, in its reduced form, the instrument was still a reliable and valid indicator of cancer-related fatigue in children.
By applying the ROC techniques to the 10-item FS-C, we identified the cut score of 12 with acceptable sensitivity (75%) and specificity (73.5%). This measure allowed us to distinguish children who had high cancer-related fatigue from those who did not. Our sensitivity and specificity values compared favorably with other ROC analyses in medical decision making.19–21 Furthermore, our AUC analysis yielded a strong finding of 0.768. Together, these results support the cut score of 12 on the FS-C and its ability to distinguish pediatric oncology patients with high cancer-related fatigue. The primary finding of this analysis is a clinically interpretable score of a child’s cancer-related fatigue. The cut-score can now be used by clinicians to guide their clinical assessments of this prevalent adverse effect of cancer and anticancer treatment. This score can also be used to signal the need to implement fatigue-directed treatment and to evaluate the effectiveness of that treatment.
The proportion of children who had cancer-related fatigue in our study sample suggests that fatigue in children with cancer differs in intensity by clinical context, i.e., approximately 33% of outpatients with diverse cancer diagnoses and those with ALL had high fatigue, and 40% of patients receiving chemotherapy as inpatients experienced fatigue at some point during their hospitalization. Furthermore, the proportion of hospitalized children with high fatigue tripled (from 12.5% to 38.9%) by the second night of hospitalization, and that proportion was maintained during the third night. The proportion of hospitalized children with high fatigue implies the need to routinely measure and treat, if necessary, fatigue in children with cancer during hospitalization for chemotherapy.
Certain limitations are apparent in this study. The independent criterion used to complete the ROC analysis was not a child-reported item or score but a parental report. The parent diary item is a well-tested item from an instrument that has been validated using objective child actigraphy scores.11 Proxy ratings are the standard in other medical decision-making studies that use the ROC analysis; thus, our use is similar to that of one or more items from family members or health care professionals as the independent criterion in these other studies. An additional limitation of this study is the lack of a validation sample to further assess the ROC technique with the FS-C and the Daily Sleep Diary–Parent. We plan to include this comparison in our future studies.
Assessing the cut score validation samples that reflect diverse clinical contexts will help confirm the cut score and further study the influence of context on the proportions of patients at risk for high fatigue. As interventions for cancer-related fatigue evolve, consideration of a scoring system to identify a range of fatigue scores (i.e., low, moderate, and high) will also need to be developed. After those ranges have been identified, the feasibility, fit, and effectiveness of interventions for each intensity range will need to be tested. As strong clinical correlates of fatigue are confirmed over time, their use in the ROC as the gold standard of comparison, against the patient-reported fatigue, will also be important to assess and compare with that of the parental perspective.
As a result of the combined analysis of the Rasch findings and the field notes from multiple studies involving patients, the FS-C instrument was reduced to 10 items that proved to have strong psychometric properties. The 10-item FS-C consists of items that discriminate between children who have high cancer-related fatigue and those who do not. The potential clinical utility of using a standard score to identify patients in need of a fatigue intervention will be further supported in planned future prospective studies.
This study was supported in part by Cancer Center Support CA 21765 from the National Cancer Institute; grant R01NR007610 from the National Institute of Nursing Research; a FIRE grant from the Oncology Nurses Foundation; and the American Lebanese Syrian Associated Charities (ALSAC).
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.