PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Pain Symptom Manage. Author manuscript; available in PMC 2013 February 28.
Published in final edited form as:
PMCID: PMC3584563
NIHMSID: NIHMS441646

Does Recall Period Have an Effect on Cancer Patients’ Ratings of the Severity of Multiple Symptoms?

Qiuling Shi, MD, PhD, MSc,1 Peter C. Trask, PhD, MPH,2 Xin Shelley Wang, MD, MPH,1 Tito R. Mendoza, PhD,1 Winifred A. Apraku, MPH,1 Maggie Malekifar, PTCB,1 and Charles S. Cleeland, PhD1

Abstract

We examined the effects of recall on symptom severity ratings by comparing ratings made using 24-hour and 7-day recall periods of the M. D. Anderson Symptom Inventory (MDASI). Forty-two patients in their third to eighth week of chemoradiation rated their symptoms using the MDASI on two separate occasions (T1 and T2), one week apart. At T1, patients were randomly assigned to rate symptoms using either a 24-hour or a 7-day recall. At T2, patients rated symptoms using the recall period not used at their first visit. Comparing the 24-hour and 7-day recall periods, the correlation coefficient for total symptom severity was 0.888. All correlation coefficients for symptom severity items were > 0.7 except for distress (r = 0.67). The percentages of moderate to severe symptoms (rated ≥ 5) were consistent for both recall periods, with no significant difference between recall periods in the prevalence of moderate to severe symptoms. Cronbach α coefficients for both 24-hour and 7-day recalls were > 0.8. Symptoms from both recall periods were more severe for patients with poorer performance status. Twenty patients were cognitively debriefed; 70% thought the 7-day recall was “more appropriate” for the MDASI, but 85% did not think that recall period would influence their answers. This study demonstrated that the MDASI in a 7-day recall format has psychometric properties consistent with the 24-hour recall version, which may allow its use in future cancer clinical trials and may inform the choice of recall period when symptoms are outcome measures.

Keywords: MDASI, recall, validation, patient-reported outcomes, symptom, assessment

Introduction

People with cancer report a variety of symptoms that are as likely to be caused by treatment as they are by the cancer itself.1 Commonly prescribed treatments, such as chemotherapy, radiotherapy, or a combination of the two, are known to produce severe symptoms, including fatigue, pain, disturbed sleep, distress, and lack of appetite, that may vary over the course of treatment.2 Such symptoms can be severe and extremely debilitating.1

Comprehensive assessment of treatment-related symptoms is thus an integral part of patient care. To provide a more representative picture of the patient’s symptom status, a questionnaire that assesses the severity of pain and other symptoms may be administered. Such questionnaires frequently use the patient’s own report to describe symptom severity. Patient-reported outcomes (PROs) have gained acceptance by researchers and regulatory agencies because 1) some treatment effects are known only to the patient; 2) the patient has an important perspective on the effectiveness of a treatment; and 3) valuable information can be lost when a patient’s perspective is filtered through a clinician’s evaluation of the patient’s response to clinical interview questions.3 One of these PRO-based multisymptom assessment tools is the M. D. Anderson Symptom Inventory (MDASI), a well-validated measure designed for use in the general cancer population.4 Patients responding to the MDASI rate the severity of multiple cancer-related symptoms in the past 24 hours and the degree of interference with functioning caused by those symptoms, using a 0–10 numeric scale. In a systematic review of 21 validated symptom assessment instruments,5 the MDASI was judged the best in terms of flexibility, reliability and validity, ease of completion, and utility in symptom management.

The past 24 hours is not always the most appropriate recall period for a clinical trial, however, and it is not unusual for a clinical trial to be designed to assess symptoms weekly or even less often.6,7 On the other hand, a 24-hour recall period allows more frequent assessments for capturing changes over time than does a longer recall period. In 2006, the US Food and Drug Administration (FDA) issued a draft Guidance for Industry – Patient-Reported Outcomes Measures: Use in Medical Product Development to Support Labeling Claims.3 In this guidance, the FDA suggested that the choice of a suitable recall period in a clinical trial should depend on the specific purpose of the trial, the characteristics of the disease, and the treatment to be tested. Thus, careful definition of the recall period chosen is required in clinical trials with PROs as endpoints.

The MDASI has been used with a 24-hour recall period only. We therefore designed a study to assess the psychometric properties of a 7-day recall version of the MDASI in a group of patients undergoing chemoradiation. We selected the patient group and assessment time on the basis of results from a previous treatment study,2 which showed that 1) approximately two thirds of patients undergoing chemoradiation suffer from moderate to severe levels of multiple symptoms, suggesting that this symptomatic population is an ideal target for evaluating an instrument that assesses multiple symptoms; and 2) the lowest weekly rate of change in symptoms occurred midway through chemoradiation therapy (i.e., weeks 3–9 in a 12-week treatment period), suggesting that we could target a time point for the assessments during which symptom severity was relatively stable. The study used a crossover-type design in which each patient responded to both 24-hour and 7-day recall period versions of the MDASI, allowing us to compare between-patient ratings of symptoms from both versions and to examine the effects of recall on symptom severity ratings. We hypothesized that the patient symptom ratings based on 7-day recall would be sufficiently correlated with patient ratings based on a 24-hour recall to support the appropriate use of either measure.

This study was designed in light of the FDA’s PRO labeling guidance to provide data for evaluating the rationale and appropriateness of a 7-day recall period. Our ultimate aim was to obtain a validated 7-day recall version of the MDASI that would have psychometric properties consistent with the 24-hour recall version and that would therefore be useful for assessing symptoms in future clinical trials. The comparisons reported herein should help clinicians and researchers to choose a MDASI version with the appropriate recall period for a specific clinical trial.

Methods

Patients and Procedures

Patients with either breast, head and neck, lung, gynecological, gastrointestinal, prostate, or brain cancer were enrolled in the study during the third to eighth week of chemoradiation in the Radiation Treatment Center at The University of Texas M. D. Anderson Cancer Center in Houston. Patients were eligible for the study if they 1) were at least 18 years old, 2) had a pathologically proven diagnosis of cancer, 3) spoke English, and 4) had signed a study-specific informed consent prior to study entry. Patients were not eligible to participate if they were 1) unable to understand the symptom assessment questionnaire in English, 2) unwilling to participate, or 3) unable to comply with protocol requirements. All patients provided informed consent to participate in the study. The M. D. Anderson Cancer Center Institutional Review Board approved the study.

Patients were seen on two occasions, one week apart, during regularly scheduled clinic visits. On each occasion, patients were asked to rate their symptoms using the MDASI. At the initial clinic visit (T1), patients were randomly assigned to rate their symptoms using either a 24-hour recall (Group 1) or a 7-day recall (Group 2). On their next clinic visit (T2), Group 1 patients, who had been assessed with the 24-hour recall version at the first visit, rated their symptoms with the 7-day recall version, whereas Group 2 patients, who had been assessed with the 7-day recall version at the first visit, rated their symptoms with the 24-hour recall version (Fig. 1).

Fig 1
Study design.

Assessment Measures

Symptom Assessment

The M. D. Anderson Symptom Inventory4 assesses the severity of 13 symptoms (pain, fatigue, nausea, disturbed sleep, distress, shortness of breath, difficulty remembering, lack of appetite, drowsiness, dry mouth, sadness, vomiting, and numbness or tingling) in the previous 24 hours. Each symptom is rated on a 0–10 numeric scale, with 0 being “not present” and 10 being “as bad as you can imagine.” The MDASI also contains six items that describe how much symptoms have interfered with various aspects of the patient’s life (general activity, mood, normal work (including work outside the home and housework), relations with other people, walking ability, and enjoyment of life) during the past 24 hours. Each interference item is rated on a 0–10 scale, with 0 being “does not interfere” and 10 being “completely interferes.” For this study, a new version of the MDASI with the same symptom and interference items but a 7-day recall period was created.

A symptom severity component score can be derived by calculating the arithmetic mean of the MDASI symptom items. A symptom interference component score can be derived by calculating the arithmetic mean of the MDASI symptom interference items.

Patients have reported that the MDASI severity and interference items are relevant and easy to understand, and that the 0–10 scale is intuitive and easy to use.8 In paper-and-pencil format, the MDASI takes less than five minutes to complete.

Cognitive Debriefing

Cognitive debriefing was conducted in a subset of patients to gain their perspective on the appropriateness and feasibility of the two recall periods. In structured interviews, patients indicated how well they understood items of the MDASI, how comfortable they were with answering the items, and how well the items reflected their concerns about their disease or treatment.

Other Measures

Research staff conducted chart reviews to obtain patient demographic and disease information, including age, race, marital status, education level, employment status, cancer type and stage, disease status, and the date that chemoradiation was started.

Eastern Cooperative Oncology Group performance status (ECOG PS)9 was recorded at both T1 and T2. ECOG PS is a five-point, physician-rated measure of functional ability, ranging from 0 (fully active; able to carry on all predisease performance without restriction) to 4 (completely disabled; cannot carry on any self care; totally confined to bed or chair). ECOG PS has demonstrated excellent reliability and validity in numerous clinical trials and descriptive studies.10,11

Statistical Analysis

Sample Size

Sample size was based on the need to detect a correlation of at least 0.70 between the two recall periods at the T1 clinic visit. Assuming a two-tailed test at 0.05 level of significance, 42 patients were required to detect this correlation with 90% power.

Statistical Methods

To test our hypothesis that patient symptom ratings based on 7-day recall would be highly correlated (Pearson’s r > 0.70) with their ratings based on a 24-hour recall, Pearson product moment correlation was used to compare the 24-hour and 7-day recall periods for each symptom in each patient group. Component scores of the five most severe symptoms for each group were generated at each time point. Order effect was tested by comparing symptom severity and interference between T1 and T2. We also compared correlation coefficients between Group 1 and Group 2.

We then combined data from the two groups to calculate the Pearson correlation coefficients between 7-day and 24-hour recall periods, adjusting for group differences. Paired t-tests were used to compare the means of symptom severity and interference between the two recall periods. The percentages of moderate and severe symptoms between the two recall periods were compared using a chi-square test.

Psychometric Validation of the 7-Day Recall MDASI

Concurrent validity was tested by Pearson correlation between symptom scores for the 7-day and 24-hour recall periods. Known-group validity was evaluated by comparing patients’ 7-day recall MDASI scores stratified by ECOG PS to test whether patients with poor performance status (ECOG PS greater than 1) would report expected increases in levels of symptom severity and interference in both recall periods. The internal consistency of responses was evaluated by Cronbach α.

Results

Demographic and Disease Characteristics

Forty-two patients with cancer were approached to participate in the study. None of the patients declined to participate and no data was missing from either of the two assessments. Patients were evenly and randomly assigned to either Group 1 or Group 2. The demographic and disease characteristics for respondents in each of the two groups are provided in Table 1. The average age for Group 1 was 60.1 years (standard deviation [SD] = 8.9) and for Group 2 was 54.3 years (SD = 13.8). Most of the patients were women (86% in Group 1 and 67% in Group 2) and most had breast cancer (62% in Group 1 and 48% in Group 2). A greater proportion of patients had metastatic cancer in Group 2 (57%) than in Group 1 (43%), and Group 1 had better ECOG PS overall. No significant differences between the two groups were found in any of the demographic and disease characteristics.

Table 1
Demographic and Disease Characteristics (n = 42)

ECOG PS was assessed for each patient at both T1 and T2. At T1, 76% of the patients in Group 1 and 62% in Group 2 had good ECOG PS (0 or 1). At T2, 86% of Group 1 patients and 67% of Group 2 patients had good ECOG PS. No significant difference was found in ECOG PS between Group 1 and Group 2 at either time point and no significant change was found over time for either group (Table 2).

Table 2
ECOG PS by Group and Time Point

Comparison of the 24-Hour and 7-Day MDASI Versions

Correlation Coefficients and Order Effects

No significant difference was found in the means of all symptom items between T1 and T2 (data not shown), indicating no notable order effect in this study. Pearson correlation coefficients for each symptom and interference item were compared between Group 1 and Group 2, with significant differences found for vomiting, lack of appetite, shortness of breath, distress, activity, and walking. Therefore, partial correlation coefficients were calculated to control the group difference when Group 1 and Group 2 were combined. Pearson coefficients representing the correlation between the 24-hour and 7-day recall assessments were greater than 0.70 for all of the symptom severity and interference items except for distress (r = 0.67) and relations with other people (r = 0.58) (Table 3).

Table 3
Pearson Product Moment Correlation Coefficients for Scores from 7-day and 24-Hour Recall Periods

Symptom Severity

Fig. 2 shows the mean scores for symptom severity and inference for the 24-hour and 7-day recall periods. All mean scores were lower than 4. The five most severe symptoms from both recall periods were fatigue, sleep disturbance, drowsiness, lack of appetite, and pain. Paired t-tests did not find any significant difference between the 24-hour and 7-day recall periods. Percentages of patients reporting moderate to severe symptoms (rated 4 or higher on the MDASI’s 0–10 scale) are demonstrated in Fig. 3. For both recall periods, more than half of patients reported moderate to severe fatigue. Percentages did not differ significantly between the 24-hour and 7-day recall periods.

Fig 2
Average MDASI scores, 7-day and 24-hour recall periods.
Fig 3
Percentage of patients with moderate to severe symptoms.

Psychometric Validation of the 7-Day Recall MDASI

Psychometric Properties

The concurrent validity of the 7-day recall version was evidenced by significantly high correlation coefficients between 24-hour and 7-day recalls (Table 3). Known-group validity of the 7-day recall version of the MDASI was examined by comparing ECOG PS across time points. At T1, interference ratings and the severity of the five most severe symptoms were significantly higher in patients with poor ECOG PS than in patients with good ECOG PS, regardless of recall period. Seven days later, these ratings remained higher in the patients with poor ECOG PS regardless of recall period, and all differences were significant (Table 4). Cronbach α coefficients of both 24-hour and 7-day recall periods were greater than 0.80, indicating good internal consistency of the responses (Table 5).

Table 4
Known-Group Validity, Mean (SD)
Table 5
Internal Consistency – Cronbach α

Cognitive Debriefing of the 7-Day Recall MDASI

Cognitive debriefing was conducted in 20 patients (10 patients from each of the two patient groups). Table 6 shows patients’ opinions on how the recall period affected their report on the MDASI. Most patients thought that the 7-day recall period was easier to remember and was more appropriate for answering the MDASI, although their answers were not significantly influenced by the recall period.

Table 6
Cognitive Debriefing of Recall Periods, by Group

Discussion

This study found that MDASI ratings of symptom severity and interference made using a 24-hour or a 7-day recall period were highly correlated, and that the psychometric properties of the 7-day recall period were consistent with those of the 24-hour recall period. Our results suggest that, as with the well-established 24-hour recall version of the MDASI, the 7-day recall version is a reliable measure of symptom burden in symptomatic cancer patients. These results and the availability of the MDASI with two recall periods may aid clinical researchers in choosing the most appropriate recall period for rating the severity of multiple symptoms in a specific clinical or research setting.

The patients in each of the two groups were similar in terms of demographic and disease characteristics, and were not significantly different as to ECOG PS status at either T1 or T2 or between T1 and T2. They reported the same five symptoms—fatigue, sleep disturbance, drowsiness, lack of—appetite, and pain as the most severe for both recall periods. Correlation coefficients greater than 0.70 for all but two of the 19 symptom and interference items indicate a high level of concurrency for these items. Known-group validity indicates that the 7-day recall version is as sensitive as the 24-hour version in detecting significant or meaningful group differences between patients by ECOG PS (0 or 1 vs. 2). Although this study needs to be replicated in broader population, the relatively robust correlations suggest similar information about symptom severity and impact could be derived from ratings made using either recall period.

Crossover-type designs have a number of inherent issues that must be addressed. One is the issue of order effects, where the order in which an assessment is administrated—in this study, which recall period was administered first—may affect the outcome. Our comparisons of the means of symptom items did not identify any notable order effect. Moreover, when we combined the two groups, we used partial correlation coefficients to control the difference between groups. A second issue is that of carryover between administrations of an assessment. In practice, carryover can be dealt with by use of a washout period between treatments, or by making observations late enough after the start of a treatment period that any carryover effect is minimized. In our study, the 7-day period between T1 and T2 should be enough to eliminate the effect of the first assessment.

It is also important in a crossover study that the underlying conditions (symptoms, in this study) not change over time. Previous studies reported that accuracy of recall of past symptoms is affected by current symptoms.1216 All patients in this study were in their third to eighth week of chemoradiation therapy, because our previous study found that the lowest weekly rate of change in symptoms occurred during the middle of a course of chemoradiation therapy (weeks 3–9 in a 12-week treatment period).2 The stability of symptoms during the study lessened the impact of current symptoms on recall accuracy, making the symptom severities from the two recall periods comparable.

A potential source of bias for this study is response shift. The term “response shift” refers to the process that patients’ internal standards, values, and the conceptualization of symptoms or quality of life may change as they adapt to the treatment.17 The then-test is the most used approach for evaluating the effect of response shift.18 Andrykowski19 reported response shift in fatigue severity ratings (one of the MDASI symptoms) in breast cancer patients treated with chemotherapy and/or radiotherapy; these patients retrospectively lowered their perception of the severity of the fatigue they had experienced before they started therapy. In our study, we were not able to evaluate the effect of response shift because we lacked a then-test. Moreover, the two measurements in our study were only one week apart and in the middle course of therapy characterized by stable treatment and stable symptom burden, whereas response shift is usually found after the entire treatment period.19 In addition, if we consider the T2 measurement of Group 1 (7-day) as a then-test, no significant differences in symptom means were found between T1 and T2 (data not shown), suggesting that the probability of response shift may not significant in this study. Future studies of this instrument using a then-test design are needed for weighing the effect of response shift for different recall periods.

The cognitive debriefing examined how patients responded to the 7-day recall version of the MDASI compared with the 24-hour version. The qualitative evidence showing similar reactions to both 7-day and 24-hour recall versions further confirmed the representativeness of the 7-day recall version.

The study was limited in that our sample included a high proportion of patients with breast cancer, which led to more women than men being included in the study. This may affect our ability to extend the findings to men. Further study with a balanced gender ratio may enhance the generalization of our results to men with cancer. Additionally, all patients were undergoing chemoradiation therapy, which has been related to high symptom burden in patients with cancer.2 Lack of comparisons among different aggressive cancer treatments suggests caution when using these results in patients receiving other treatments. However, as was found in this study, other studies of cancer patients undergoing a variety of treatments have consistently identified fatigue, sleep disturbance, drowsiness, lack of appetite, pain, and distress as the most severe cluster of symptoms.8,20,21 This indicates that similar results may be obtained from the 7-day recall version of the MDASI in cancer populations undergoing treatments other than chemoradiation therapy.

The results of this study demonstrate that symptom severity and interference as measured with a 7-day recall version of the MDASI are highly correlated with ratings from the 24-hour recall version, and that these two versions have consistent psychometric properties. This study supports the consideration of a 7-day recall period version of the MDASI as a candidate instrument for assessing symptoms in clinical trials of patients with cancer.

Acknowledgments

Funding: This project was funded by Pfizer, Inc.

References

1. Cleeland CS. Symptom burden: multiple symptoms and their impact as patient-reported outcomes. J Natl Cancer Inst Monogr. 2007;(37):16–21. [PubMed]
2. Wang XS, Fairclough DL, Liao Z, et al. Longitudinal study of the relationship between chemoradiation therapy for non-small-cell lung cancer and patient symptoms. J Clin Oncol. 2006;24(27):4485–4491. [PubMed]
3. U.S. Department of Health and Human Services, Food and Drug Administration. Guidance for Industry. [Accessed Jun 11, 2009];Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Available from: URL: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM071975.pdf. [PMC free article] [PubMed]
4. Cleeland CS, Mendoza TR, Wang XS, et al. Assessing symptom distress in cancer patients: the M.D. Anderson Symptom Inventory. Cancer. 2000;89(7):1634–1646. [PubMed]
5. Kirkova J, Davis MP, Walsh D, et al. Cancer symptom assessment instruments: a systematic review. J Clin Oncol. 2006;24(9):1459–1473. [PubMed]
6. Gabrilove JL, Perez EA, Tomita DK, Rossi G, Cleeland CS. Assessing symptom burden using the M. D. Anderson symptom inventory in patients with chemotherapy-induced anemia: results of a multicenter, open-label study (SURPASS) of patients treated with darbepoetin-alpha at a dose of 200 microg every 2 weeks. Cancer. 2007;110(7):1629–1640. [PubMed]
7. Northouse LL, Mood DW, Schafenacker A, et al. Randomized clinical trial of a family intervention for prostate cancer patients and their spouses. Cancer. 2007;110(12):2809–2818. [PubMed]
8. Gning I, Trask PC, Mendoza TR, et al. Development and initial validation of the thyroid cancer module of the M. D. Anderson Symptom Inventory. Oncology. 2009;76(1):59–68. [PubMed]
9. Oken MM, Creech RH, Tormey DC, et al. Toxicity and response criteria of the Eastern Cooperative Oncology Group. Am J Clin Oncol. 1982;5(6):649–655. [PubMed]
10. Cleeland CS, Ryan KM. Pain assessment: global use of the Brief Pain Inventory. Ann Acad Med Singapore. 1994;23(2):129–138. [PubMed]
11. Cleeland CS, Gonin R, Baez L, Loehrer P, Pandya KJ. Pain and treatment of pain in minority patients with cancer. The Eastern Cooperative Oncology Group Minority Outpatient Pain Study. Ann Intern Med. 1997;127(9):813–816. [PubMed]
12. Eich E, Reeves JL, Jaeger B, Graff-Radford SB. Memory for pain: relation between past and present pain intensity. Pain. 1985;23(4):375–380. [PubMed]
13. Harvey AG, Bryant RA. Memory for acute stress disorder symptoms: a two-year prospective study. J Nerv Ment Dis. 2000;188(9):602–607. [PubMed]
14. Meek PM, Lareau SC, Anderson D. Memory for symptoms in COPD patients: how accurate are their reports? Eur Respir J. 2001;18(3):474–481. [PubMed]
15. Miranda H, Gold JE, Gore R, Punnett L. Recall of prior musculoskeletal pain. Scand J Work Environ Health. 2006;32(4):294–299. [PubMed]
16. Wells JE, Horwood LJ. How accurate is recall of key symptoms of depression? A comparison of recall and longitudinal reports. Psychol Med. 2004;34(6):1001–1011. [PubMed]
17. Sprangers MA, Schwartz CE. The challenge of response shift for quality-of-life-based clinical oncology research. Ann Oncol. 1999;10(7):747–749. [PubMed]
18. Schwartz CE, Sprangers MA. Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Soc Sci Med. 1999;48(11):1531–1548. [PubMed]
19. Andrykowski MA, Donovan KA, Jacobsen PB. Magnitude and correlates of response shift in fatigue ratings in women undergoing adjuvant therapy for breast cancer. J Pain Symptom Manage. 2009;37(3):341–351. [PMC free article] [PubMed]
20. Ivanova MO, Ionova TI, Kalyadina SA, et al. Cancer-related symptom assessment in Russia: validation and utility of the Russian M. D. Anderson Symptom Inventory. J Pain Symptom Manage. 2005;30(5):443–453. [PubMed]
21. Yamagishi A, Morita T, Miyashita M, Kimura F. Symptom prevalence and longitudinal follow-up in cancer outpatients receiving chemotherapy. J Pain Symptom Manage. 2009;37(5):823–830. [PubMed]