|Home | About | Journals | Submit | Contact Us | Français|
We examined the effects of recall on symptom severity ratings by comparing ratings made using 24-hour and 7-day recall periods of the M. D. Anderson Symptom Inventory (MDASI). Forty-two patients in their third to eighth week of chemoradiation rated their symptoms using the MDASI on two separate occasions (T1 and T2), one week apart. At T1, patients were randomly assigned to rate symptoms using either a 24-hour or a 7-day recall. At T2, patients rated symptoms using the recall period not used at their first visit. Comparing the 24-hour and 7-day recall periods, the correlation coefficient for total symptom severity was 0.888. All correlation coefficients for symptom severity items were > 0.7 except for distress (r = 0.67). The percentages of moderate to severe symptoms (rated ≥ 5) were consistent for both recall periods, with no significant difference between recall periods in the prevalence of moderate to severe symptoms. Cronbach α coefficients for both 24-hour and 7-day recalls were > 0.8. Symptoms from both recall periods were more severe for patients with poorer performance status. Twenty patients were cognitively debriefed; 70% thought the 7-day recall was “more appropriate” for the MDASI, but 85% did not think that recall period would influence their answers. This study demonstrated that the MDASI in a 7-day recall format has psychometric properties consistent with the 24-hour recall version, which may allow its use in future cancer clinical trials and may inform the choice of recall period when symptoms are outcome measures.
People with cancer report a variety of symptoms that are as likely to be caused by treatment as they are by the cancer itself.1 Commonly prescribed treatments, such as chemotherapy, radiotherapy, or a combination of the two, are known to produce severe symptoms, including fatigue, pain, disturbed sleep, distress, and lack of appetite, that may vary over the course of treatment.2 Such symptoms can be severe and extremely debilitating.1
Comprehensive assessment of treatment-related symptoms is thus an integral part of patient care. To provide a more representative picture of the patient’s symptom status, a questionnaire that assesses the severity of pain and other symptoms may be administered. Such questionnaires frequently use the patient’s own report to describe symptom severity. Patient-reported outcomes (PROs) have gained acceptance by researchers and regulatory agencies because 1) some treatment effects are known only to the patient; 2) the patient has an important perspective on the effectiveness of a treatment; and 3) valuable information can be lost when a patient’s perspective is filtered through a clinician’s evaluation of the patient’s response to clinical interview questions.3 One of these PRO-based multisymptom assessment tools is the M. D. Anderson Symptom Inventory (MDASI), a well-validated measure designed for use in the general cancer population.4 Patients responding to the MDASI rate the severity of multiple cancer-related symptoms in the past 24 hours and the degree of interference with functioning caused by those symptoms, using a 0–10 numeric scale. In a systematic review of 21 validated symptom assessment instruments,5 the MDASI was judged the best in terms of flexibility, reliability and validity, ease of completion, and utility in symptom management.
The past 24 hours is not always the most appropriate recall period for a clinical trial, however, and it is not unusual for a clinical trial to be designed to assess symptoms weekly or even less often.6,7 On the other hand, a 24-hour recall period allows more frequent assessments for capturing changes over time than does a longer recall period. In 2006, the US Food and Drug Administration (FDA) issued a draft Guidance for Industry – Patient-Reported Outcomes Measures: Use in Medical Product Development to Support Labeling Claims.3 In this guidance, the FDA suggested that the choice of a suitable recall period in a clinical trial should depend on the specific purpose of the trial, the characteristics of the disease, and the treatment to be tested. Thus, careful definition of the recall period chosen is required in clinical trials with PROs as endpoints.
The MDASI has been used with a 24-hour recall period only. We therefore designed a study to assess the psychometric properties of a 7-day recall version of the MDASI in a group of patients undergoing chemoradiation. We selected the patient group and assessment time on the basis of results from a previous treatment study,2 which showed that 1) approximately two thirds of patients undergoing chemoradiation suffer from moderate to severe levels of multiple symptoms, suggesting that this symptomatic population is an ideal target for evaluating an instrument that assesses multiple symptoms; and 2) the lowest weekly rate of change in symptoms occurred midway through chemoradiation therapy (i.e., weeks 3–9 in a 12-week treatment period), suggesting that we could target a time point for the assessments during which symptom severity was relatively stable. The study used a crossover-type design in which each patient responded to both 24-hour and 7-day recall period versions of the MDASI, allowing us to compare between-patient ratings of symptoms from both versions and to examine the effects of recall on symptom severity ratings. We hypothesized that the patient symptom ratings based on 7-day recall would be sufficiently correlated with patient ratings based on a 24-hour recall to support the appropriate use of either measure.
This study was designed in light of the FDA’s PRO labeling guidance to provide data for evaluating the rationale and appropriateness of a 7-day recall period. Our ultimate aim was to obtain a validated 7-day recall version of the MDASI that would have psychometric properties consistent with the 24-hour recall version and that would therefore be useful for assessing symptoms in future clinical trials. The comparisons reported herein should help clinicians and researchers to choose a MDASI version with the appropriate recall period for a specific clinical trial.
Patients with either breast, head and neck, lung, gynecological, gastrointestinal, prostate, or brain cancer were enrolled in the study during the third to eighth week of chemoradiation in the Radiation Treatment Center at The University of Texas M. D. Anderson Cancer Center in Houston. Patients were eligible for the study if they 1) were at least 18 years old, 2) had a pathologically proven diagnosis of cancer, 3) spoke English, and 4) had signed a study-specific informed consent prior to study entry. Patients were not eligible to participate if they were 1) unable to understand the symptom assessment questionnaire in English, 2) unwilling to participate, or 3) unable to comply with protocol requirements. All patients provided informed consent to participate in the study. The M. D. Anderson Cancer Center Institutional Review Board approved the study.
Patients were seen on two occasions, one week apart, during regularly scheduled clinic visits. On each occasion, patients were asked to rate their symptoms using the MDASI. At the initial clinic visit (T1), patients were randomly assigned to rate their symptoms using either a 24-hour recall (Group 1) or a 7-day recall (Group 2). On their next clinic visit (T2), Group 1 patients, who had been assessed with the 24-hour recall version at the first visit, rated their symptoms with the 7-day recall version, whereas Group 2 patients, who had been assessed with the 7-day recall version at the first visit, rated their symptoms with the 24-hour recall version (Fig. 1).
The M. D. Anderson Symptom Inventory4 assesses the severity of 13 symptoms (pain, fatigue, nausea, disturbed sleep, distress, shortness of breath, difficulty remembering, lack of appetite, drowsiness, dry mouth, sadness, vomiting, and numbness or tingling) in the previous 24 hours. Each symptom is rated on a 0–10 numeric scale, with 0 being “not present” and 10 being “as bad as you can imagine.” The MDASI also contains six items that describe how much symptoms have interfered with various aspects of the patient’s life (general activity, mood, normal work (including work outside the home and housework), relations with other people, walking ability, and enjoyment of life) during the past 24 hours. Each interference item is rated on a 0–10 scale, with 0 being “does not interfere” and 10 being “completely interferes.” For this study, a new version of the MDASI with the same symptom and interference items but a 7-day recall period was created.
A symptom severity component score can be derived by calculating the arithmetic mean of the MDASI symptom items. A symptom interference component score can be derived by calculating the arithmetic mean of the MDASI symptom interference items.
Patients have reported that the MDASI severity and interference items are relevant and easy to understand, and that the 0–10 scale is intuitive and easy to use.8 In paper-and-pencil format, the MDASI takes less than five minutes to complete.
Cognitive debriefing was conducted in a subset of patients to gain their perspective on the appropriateness and feasibility of the two recall periods. In structured interviews, patients indicated how well they understood items of the MDASI, how comfortable they were with answering the items, and how well the items reflected their concerns about their disease or treatment.
Research staff conducted chart reviews to obtain patient demographic and disease information, including age, race, marital status, education level, employment status, cancer type and stage, disease status, and the date that chemoradiation was started.
Eastern Cooperative Oncology Group performance status (ECOG PS)9 was recorded at both T1 and T2. ECOG PS is a five-point, physician-rated measure of functional ability, ranging from 0 (fully active; able to carry on all predisease performance without restriction) to 4 (completely disabled; cannot carry on any self care; totally confined to bed or chair). ECOG PS has demonstrated excellent reliability and validity in numerous clinical trials and descriptive studies.10,11
Sample size was based on the need to detect a correlation of at least 0.70 between the two recall periods at the T1 clinic visit. Assuming a two-tailed test at 0.05 level of significance, 42 patients were required to detect this correlation with 90% power.
To test our hypothesis that patient symptom ratings based on 7-day recall would be highly correlated (Pearson’s r > 0.70) with their ratings based on a 24-hour recall, Pearson product moment correlation was used to compare the 24-hour and 7-day recall periods for each symptom in each patient group. Component scores of the five most severe symptoms for each group were generated at each time point. Order effect was tested by comparing symptom severity and interference between T1 and T2. We also compared correlation coefficients between Group 1 and Group 2.
We then combined data from the two groups to calculate the Pearson correlation coefficients between 7-day and 24-hour recall periods, adjusting for group differences. Paired t-tests were used to compare the means of symptom severity and interference between the two recall periods. The percentages of moderate and severe symptoms between the two recall periods were compared using a chi-square test.
Concurrent validity was tested by Pearson correlation between symptom scores for the 7-day and 24-hour recall periods. Known-group validity was evaluated by comparing patients’ 7-day recall MDASI scores stratified by ECOG PS to test whether patients with poor performance status (ECOG PS greater than 1) would report expected increases in levels of symptom severity and interference in both recall periods. The internal consistency of responses was evaluated by Cronbach α.
Forty-two patients with cancer were approached to participate in the study. None of the patients declined to participate and no data was missing from either of the two assessments. Patients were evenly and randomly assigned to either Group 1 or Group 2. The demographic and disease characteristics for respondents in each of the two groups are provided in Table 1. The average age for Group 1 was 60.1 years (standard deviation [SD] = 8.9) and for Group 2 was 54.3 years (SD = 13.8). Most of the patients were women (86% in Group 1 and 67% in Group 2) and most had breast cancer (62% in Group 1 and 48% in Group 2). A greater proportion of patients had metastatic cancer in Group 2 (57%) than in Group 1 (43%), and Group 1 had better ECOG PS overall. No significant differences between the two groups were found in any of the demographic and disease characteristics.
ECOG PS was assessed for each patient at both T1 and T2. At T1, 76% of the patients in Group 1 and 62% in Group 2 had good ECOG PS (0 or 1). At T2, 86% of Group 1 patients and 67% of Group 2 patients had good ECOG PS. No significant difference was found in ECOG PS between Group 1 and Group 2 at either time point and no significant change was found over time for either group (Table 2).
No significant difference was found in the means of all symptom items between T1 and T2 (data not shown), indicating no notable order effect in this study. Pearson correlation coefficients for each symptom and interference item were compared between Group 1 and Group 2, with significant differences found for vomiting, lack of appetite, shortness of breath, distress, activity, and walking. Therefore, partial correlation coefficients were calculated to control the group difference when Group 1 and Group 2 were combined. Pearson coefficients representing the correlation between the 24-hour and 7-day recall assessments were greater than 0.70 for all of the symptom severity and interference items except for distress (r = 0.67) and relations with other people (r = 0.58) (Table 3).
Fig. 2 shows the mean scores for symptom severity and inference for the 24-hour and 7-day recall periods. All mean scores were lower than 4. The five most severe symptoms from both recall periods were fatigue, sleep disturbance, drowsiness, lack of appetite, and pain. Paired t-tests did not find any significant difference between the 24-hour and 7-day recall periods. Percentages of patients reporting moderate to severe symptoms (rated 4 or higher on the MDASI’s 0–10 scale) are demonstrated in Fig. 3. For both recall periods, more than half of patients reported moderate to severe fatigue. Percentages did not differ significantly between the 24-hour and 7-day recall periods.
The concurrent validity of the 7-day recall version was evidenced by significantly high correlation coefficients between 24-hour and 7-day recalls (Table 3). Known-group validity of the 7-day recall version of the MDASI was examined by comparing ECOG PS across time points. At T1, interference ratings and the severity of the five most severe symptoms were significantly higher in patients with poor ECOG PS than in patients with good ECOG PS, regardless of recall period. Seven days later, these ratings remained higher in the patients with poor ECOG PS regardless of recall period, and all differences were significant (Table 4). Cronbach α coefficients of both 24-hour and 7-day recall periods were greater than 0.80, indicating good internal consistency of the responses (Table 5).
Cognitive debriefing was conducted in 20 patients (10 patients from each of the two patient groups). Table 6 shows patients’ opinions on how the recall period affected their report on the MDASI. Most patients thought that the 7-day recall period was easier to remember and was more appropriate for answering the MDASI, although their answers were not significantly influenced by the recall period.
This study found that MDASI ratings of symptom severity and interference made using a 24-hour or a 7-day recall period were highly correlated, and that the psychometric properties of the 7-day recall period were consistent with those of the 24-hour recall period. Our results suggest that, as with the well-established 24-hour recall version of the MDASI, the 7-day recall version is a reliable measure of symptom burden in symptomatic cancer patients. These results and the availability of the MDASI with two recall periods may aid clinical researchers in choosing the most appropriate recall period for rating the severity of multiple symptoms in a specific clinical or research setting.
The patients in each of the two groups were similar in terms of demographic and disease characteristics, and were not significantly different as to ECOG PS status at either T1 or T2 or between T1 and T2. They reported the same five symptoms—fatigue, sleep disturbance, drowsiness, lack of—appetite, and pain as the most severe for both recall periods. Correlation coefficients greater than 0.70 for all but two of the 19 symptom and interference items indicate a high level of concurrency for these items. Known-group validity indicates that the 7-day recall version is as sensitive as the 24-hour version in detecting significant or meaningful group differences between patients by ECOG PS (0 or 1 vs. 2). Although this study needs to be replicated in broader population, the relatively robust correlations suggest similar information about symptom severity and impact could be derived from ratings made using either recall period.
Crossover-type designs have a number of inherent issues that must be addressed. One is the issue of order effects, where the order in which an assessment is administrated—in this study, which recall period was administered first—may affect the outcome. Our comparisons of the means of symptom items did not identify any notable order effect. Moreover, when we combined the two groups, we used partial correlation coefficients to control the difference between groups. A second issue is that of carryover between administrations of an assessment. In practice, carryover can be dealt with by use of a washout period between treatments, or by making observations late enough after the start of a treatment period that any carryover effect is minimized. In our study, the 7-day period between T1 and T2 should be enough to eliminate the effect of the first assessment.
It is also important in a crossover study that the underlying conditions (symptoms, in this study) not change over time. Previous studies reported that accuracy of recall of past symptoms is affected by current symptoms.12–16 All patients in this study were in their third to eighth week of chemoradiation therapy, because our previous study found that the lowest weekly rate of change in symptoms occurred during the middle of a course of chemoradiation therapy (weeks 3–9 in a 12-week treatment period).2 The stability of symptoms during the study lessened the impact of current symptoms on recall accuracy, making the symptom severities from the two recall periods comparable.
A potential source of bias for this study is response shift. The term “response shift” refers to the process that patients’ internal standards, values, and the conceptualization of symptoms or quality of life may change as they adapt to the treatment.17 The then-test is the most used approach for evaluating the effect of response shift.18 Andrykowski19 reported response shift in fatigue severity ratings (one of the MDASI symptoms) in breast cancer patients treated with chemotherapy and/or radiotherapy; these patients retrospectively lowered their perception of the severity of the fatigue they had experienced before they started therapy. In our study, we were not able to evaluate the effect of response shift because we lacked a then-test. Moreover, the two measurements in our study were only one week apart and in the middle course of therapy characterized by stable treatment and stable symptom burden, whereas response shift is usually found after the entire treatment period.19 In addition, if we consider the T2 measurement of Group 1 (7-day) as a then-test, no significant differences in symptom means were found between T1 and T2 (data not shown), suggesting that the probability of response shift may not significant in this study. Future studies of this instrument using a then-test design are needed for weighing the effect of response shift for different recall periods.
The cognitive debriefing examined how patients responded to the 7-day recall version of the MDASI compared with the 24-hour version. The qualitative evidence showing similar reactions to both 7-day and 24-hour recall versions further confirmed the representativeness of the 7-day recall version.
The study was limited in that our sample included a high proportion of patients with breast cancer, which led to more women than men being included in the study. This may affect our ability to extend the findings to men. Further study with a balanced gender ratio may enhance the generalization of our results to men with cancer. Additionally, all patients were undergoing chemoradiation therapy, which has been related to high symptom burden in patients with cancer.2 Lack of comparisons among different aggressive cancer treatments suggests caution when using these results in patients receiving other treatments. However, as was found in this study, other studies of cancer patients undergoing a variety of treatments have consistently identified fatigue, sleep disturbance, drowsiness, lack of appetite, pain, and distress as the most severe cluster of symptoms.8,20,21 This indicates that similar results may be obtained from the 7-day recall version of the MDASI in cancer populations undergoing treatments other than chemoradiation therapy.
The results of this study demonstrate that symptom severity and interference as measured with a 7-day recall version of the MDASI are highly correlated with ratings from the 24-hour recall version, and that these two versions have consistent psychometric properties. This study supports the consideration of a 7-day recall period version of the MDASI as a candidate instrument for assessing symptoms in clinical trials of patients with cancer.
Funding: This project was funded by Pfizer, Inc.