Impact of Cultural and Linguistic Factors on Symptom Reporting by Patients With Cancer

doi:10.1093/jnci/djq097

J Natl Cancer Inst. 2010 May 19; 102(10): 732–738.

PMCID: PMC2873184

Impact of Cultural and Linguistic Factors on Symptom Reporting by Patients With Cancer

Xin Shelley Wang, Charles S. Cleeland, Tito R. Mendoza, Young Ho Yun, Ying Wang, Toru Okuyama, and Valen E. Johnson

Affiliations of authors: Department of Symptom Research (XSW, CSC, TRM) and Department of Biostatistics (VEJ), The University of Texas M. D. Anderson Cancer Center, Houston, Texas; Department of Family Medicine, National Cancer Center, Goyang, Gyeonggi, Korea (YHY); Tianjin Cancer Hospital, Research and Educational Department, Tianjin Medical University Cancer Institute and Hospital, Tianjin, China (YW); Department of Psychiatry, Nagoya City University Graduate School of Medical Sciences, Nagoya, Japan (TO)

Corresponding author.

Correspondence to: Xin Shelley Wang, MD, MPH, Department of Symptom Research, The University of Texas M. D. Anderson Cancer Center, 1515 Holcombe Blvd, Unit 1450, Houston, TX 77030 (e-mail: xswang/at/mdanderson.org).

Received August 5, 2009; Revised March 1, 2010; Accepted March 2, 2010.

This article has been cited by other articles in PMC.

Abstract

Background

Patient reporting of the severity and impact of symptoms is an essential component of cancer symptom management and cancer treatment clinical trials. In multinational clinical trials, cultural and linguistic variations in patient-reported outcomes instruments could confound the interpretation of study results.

Methods

The severity and interference of multiple symptoms in 1433 cancer patients with mixed diagnoses and treatment status from the United States, China, Japan, Russia, and Korea were measured with psychometrically validated language versions of the M. D. Anderson Symptom Inventory (MDASI). Mixed-effect ordinal probit regression models were fitted to the pooled data to compare the magnitude of the effect of “country” (nation and linguistic factors) with between-subjects effects on symptom reporting, adjusted for patient and clinical factors (age, sex, performance status, and chemotherapy status).

Results

For the pooled sample, fatigue, disturbed sleep, distress, pain, and lack of appetite were the most severe patient-reported MDASI symptoms. The magnitude of the variance of the country random effects was only one-fourth to one-half of the interpatient variation (σ² = 0.23–0.46) for all symptoms, except nausea and vomiting.

Conclusions

Cultural and linguistic variations in symptom reporting among the five language versions of the validated MDASI were limited. Ordinal probit modeling provided a simple mechanism for accounting for cultural and linguistic differences in patient populations. The equivalence among MDASI translations in this study suggests that symptom ratings collected from various cultural and language groups using the MDASI can be interpreted in a similar way in oncology practice, clinical trials, and clinical research.

CONTEXTS AND CAVEATS

Prior knowledge

The results of clinical trials or treatment guidelines for patients of different nationalities and languages may be difficult to interpret because of linguistic and cultural differences in patient-reported outcomes. The M. D. Anderson Symptom Inventory (MDASI) is used to assess 13 symptoms commonly associated with cancer or its treatment.

Study design

Pooled MDASI data were analyzed from validation studies conducted with 1433 patients from the United States, China, Japan, Russia, and Korea. Variations in MDASI scores attributable to linguistic and cultural variations were compared with intersubject variations in patient responses.

Contribution

National and linguistic variations in patient responses to the MDASI were small relative to individual patient-related factors. Adjusting for patient and clinical factors, the country effect accounted for only one-fourth to one-half of the patient-to-patient variation in MDASI symptom severity ratings, especially for the most severe symptoms.

Implications

Symptom data obtained using various language versions of well-validated patient-report measures such as the MDASI can be pooled to analyze multinational clinical research and can provide reliable symptom assessment for oncology practice in other parts of the world.

Limitations

Only one to three treatment centers were sampled in each country, so the effect of country may be confounded with that of individual sample site. Cancer stage was not used as a covariate because data for this variable from one country were missing.

From the Editors

Cancer-related symptoms produced either by the disease itself or by the toxicities of treatment are what patients report to clinicians as subjective negative feelings that may be physical (such as pain, fatigue, and shortness of breath), cognitive (such as memory problems), or affective (such as sadness and emotional distress). These multiple symptoms collectively impose a symptom burden that greatly affects a patient’s quality of life and daily activities (1).

Characterizing symptom burden requires accurate measurement of symptom severity and the degree to which symptoms interfere with daily life. Psychometrically validated tools for assessing patient self-report of symptoms and interference (patient-reported outcomes [PROs]) are widely accepted in oncology practice and are increasingly used as primary or secondary outcomes for clinical trials (2).

Psychometrically sound, culturally valid, standardized PRO assessment tools are available for administration to patients from diverse racial, ethnic, and cultural and language groups. However, many PRO measures (including symptom scales) and most guidelines for cancer symptom management (3–6) are initially developed and validated in English and later translated into other languages. When clinical trials that include patient self-report or the application of treatment guidelines are conducted in patients with diverse linguistic backgrounds, cultural differences can confound the accuracy and cross-similarity of the PROs, thus complicating the interpretation of the trial results and the application of the clinical guidelines. Knowing the degree to which symptom ratings might vary as a function of language or nationality is therefore important for both the clinical trials and the treatment of symptoms and requires empirical evidence of the effects of language on the performance of a symptom measure (7).

Several well-validated PRO assessment tools are used transnationally in oncology, including the Functional Assessment of Cancer Therapy (FACT), the European Organisation for Research and Treatment of Cancer (EORTC) QLQ-C30, and the M. D. Anderson Symptom Inventory (MDASI) (8). Whereas the FACT and QLQ-C30 are measures of health-related quality of life, the MDASI focuses specifically on measuring the severity and interference of cancer-related symptoms caused by disease and the treatment process. A systematic review of cancer symptom assessment tools by Kirkova et al. (9) rated the MDASI highly in terms of flexibility, reliability and validity, ease of completion, and utility in symptom management. The MDASI has been both linguistically and psychometrically validated in multiple languages.

In this study, we examined the effects of language on symptom report relative to patient and clinical factors in an analysis of pooled data from the English (8), Chinese (10), Japanese (11), Russian (12), and Korean (13) MDASI validation studies. We hypothesized that the differences in MDASI scores attributable to the language and culture in which the instrument was administered would be small compared with the between-subject variation in patients exhibiting similar demographic and clinical characteristics, such as age, sex, and performance status.

Methods

Patient Samples

This study was an analysis of data gathered in five MDASI language validation studies. Patients in each validation study were recruited from clinics or inpatient units at The University of Texas M. D. Anderson Cancer Center in Houston, TX; Tianjin Medical University Cancer Institute and Hospital, Tianjin, China; the National Cancer Center Hospital East, Kashiwa, Chiba, Japan; four city hospitals in St Petersburg, Russia; and five cancer centers and university hospitals in Korea. International collaboration between the study investigators and the Department of Symptom Research at M. D. Anderson Cancer Center ensured that all study protocol procedures were similar for each validation protocol. All patients were at least 18 years old, had a pathological diagnosis of cancer, were able to read, understand, and complete the questionnaires in their native language, and did not have a diagnosis of severe mental or cognitive disorder. There were no limitations as to type of cancer diagnosis, staging, or type or timing of cancer treatment. The studies were approved by the institutional review boards of M. D. Anderson Cancer Center and of the participating cancer hospitals in each country. All patients provided consent to participate.

Symptom Assessment

The MDASI (8) is a brief, psychometrically validated, multisymptom assessment tool that assesses 13 symptoms commonly associated with cancer or its treatment, including pain, fatigue, nausea, disturbed sleep, distress, shortness of breath, difficulty remembering, lack of appetite, drowsiness, dry mouth, sadness, vomiting, and numbness or tingling. Patients rate the severity of each of these symptoms in the past 24 hours on an 11-point scale ranging from 0 (not present) to 10 (as bad as you can imagine). Six additional items measure the degree to which symptoms have interfered with various facets of the patient's daily life during the past 24 hours, including general activity, mood, normal work (both work outside the home and housework), relations with other people, walking, and enjoyment of life. Interference items are also rated on an 11-point scale ranging from 0 (does not interfere) to 10 (completely interferes).

The four non-English language versions of the MDASI used in this analysis were developed and tested using a consistent translation/back-translation procedure (14). The psychometric properties for the foreign-language translations of the MDASI included in this analysis have been shown to be satisfactory and comparable to the English version (8,10–13). The internal consistency (reliability) of the symptom and interference items in each of the five language versions of MDASI has been demonstrated by Cronbach alpha coefficients (15), which are calculated by subtracting from one the ratio of the sum of the component score variances to the true score variance. A coefficient value lower than 0.70 suggests either that one or more items have high variability or that the items are not all measuring the same underlying construct. A high degree of internal consistency exists in all five language versions, with Cronbach alpha coefficients of 0.85–0.93 for the symptom severity subscales and greater than or equal to 0.90 for the symptom interference subscales.

Statistical Analysis

The five countries represented the nationality and/or language variable, defined as “country,” in our analyses. Descriptive statistics, including proportions and SDs, were used to describe the characteristics of the sample from each country. All statistical tests for symptom severity by sex and chemotherapy status were two-tailed, and a P value less than .05 was considered to be statistically significant. P values reported herein represent two times the smaller of the posterior probabilities that the parameter of interest was either less than or greater than zero.

Ordinal probit regression modeling (16) was used to estimate the effect of country on symptom report, with each MDASI symptom score treated as an ordinal response. Country was modeled as a random effect. The SD of the random effects attributable to country was implicitly scaled relative to the intersubject variance of 1.0. Ordinal probit regression models allowed us to account for subject-specific explanatory variables as fixed effects. The covariate vectors were age, sex, Eastern Cooperative Oncology Group performance status (ECOG PS) score (17), and whether or not the patient was receiving chemotherapy at the time of assessment.

To define an ordinal probit regression model, let equation n1

, r = 0, 1, …, 9 denote the cumulative categorical probabilities that patient i responds in category r or less (note that P_i_,10 = 1), let β denote a regression parameter, let x_i denote a vector of patient covariates for patient i, and let z_i denote a five-level factor vector indicating the ith patient's culture/language. Then, the regression model for each MDASI item can be described by the equation equation n2

, where the five components of γ, equation n3

, are assumed to be random effects associated with jth culture/language, and equation n4

denotes the standard normal distribution function. The parameters u_r denote category thresholds. We further assume that the random-effects γ_j are independently distributed according to a normal distribution with mean 0 and variance σ². To complete the model specification, we assume that the prior distribution on σ² is proportional to Cauchy density truncated to the positive real axis, that the prior densities on the components of β are uniform on the real line, and that the prior densities on the components of u_r are uniform subject to the constraint that equation n5

. We used a Gibbs sampling scheme to obtain a posterior sample of parameters (16). After 2000 burn-in iterations, 100

000 updates of each parameter were performed to obtain a joint posterior sample on all unknown parameter values.

This mixed-effects ordinal probit regression modeling method facilitated the interpretation of the country random-effects variance, σ². According to the mixed-effect probit model, the between-patient variation for patients who have the same covariate values is defined to be 1.0 on the latent probit scale. Thus, the magnitude of the random-effects variance has a simple interpretation in terms of its magnitude relative to the unit interpatient variability.

There are several reasons for modeling country effects as random effects. First, our intent in performing these analyses was to demonstrate that the MDASI can be used to assess symptoms from many international patient populations, not just the five countries for which data were available. Therefore, we regarded the countries available for analysis as a sample (even if not random) of the countries that we are potentially interested in studying. Second, it is not uncommon to fit random-effects models with four or five random effects. We were then able to compare the “average country effect” with the effects of other variables (eg, ECOG PS). Regarding country effects as fixed effects would have required us to make country-specific comparisons to the effects of other variables, thus proliferating the number of comparisons. Finally, because sufficient sample sizes from each country were included in the analysis, the estimated country effects (the posterior mean estimates) obtained from the random-effects model were almost identical to those that would be obtained from the countries in the random-effects model or from a corresponding fixed-effects model. The shrinkage effect on the parameters associated with the country effects that results from modeling these effects as random was negligible in our analysis.

The focus of our study (rather than to test whether the effects of country are zero) was to estimate the relative importance of country on symptom reports in relation to other known factors, such as ECOG PS. Therefore, we did not perform power calculations against prespecified alternative values of the country effect sizes. Also, because the missing data rates were small from each country (0.04%–2.40% on MDASI symptom items), we simply excluded missing data from our analysis.

Results

Sample Characteristics

Data from 1433 patients were included in our analyses: 524 patients from the United States, 249 from China, 256 from Japan, 226 from Russia, and 178 from Korea. Patient demographic and disease information is summarized in Table 1. Compared with the other samples, the Japanese sample included fewer patients undergoing active treatment because much of the Japanese data were collected at clinic follow-up visits (Table 1).

Table 1

Demographic and disease characteristics^*

Symptom Severity and Prevalence

Analysis of symptom data revealed that cancer patients from the five national and linguistic groups were similar in their symptom experiences across the various stages of disease and treatment. Cross-nationally, fatigue was consistently the most prevalent moderate to severe symptom [rated ≥5 on the MDASI 0–10 scale (18)]. The prevalence of moderate to severe MDASI symptoms for the pooled sample and for each nation is presented in Table 2. In the analysis of symptom severity overall, fatigue was followed by disturbed sleep, distress, pain, lack of appetite, and drowsiness (Table 2). Treatment-induced symptoms such as nausea, vomiting, and numbness were consistently rated as the least severe of the 13 MDASI symptoms (Table 3).

Table 2

Most prevalent moderate to severe M. D. Anderson Symptom Inventory (MDASI) symptoms by country^*

Table 3

Comparison of mean and variance of symptom items and random effects variance of country^*

Effect of Country on Symptom Reporting

Table 3 presents the posterior mean estimates for the mixed-effects ordinal probit regression models that include random-effects variables to reflect the influence of cultural and linguistic factors on symptom scores. The random-effects variance (σ²) of the country effects for each symptom and the coefficients for age, sex, ECOG PS, and chemotherapy status can be compared with the between-patient variance of 1.0. Positive coefficients are associated with increased symptom severity.

The variances of random effects due to country for all symptom ratings were less than 1.0 (Table 3). The magnitude of the variance of the country random effects was only one-fourth to one-half, approximately, of the interpatient variation (σ² = 0.23–0.46) for all symptoms, except nausea and vomiting. The range of the posterior means of the variances of the country effect was the lowest (σ² = 0.23–0.28) for fatigue, disturbed sleep, and sadness (Table 3). Fatigue and disturbed sleep were also the most severe symptoms.

The posterior mean of the variances of the random effects due to country for each of the MDASI interference items was also smaller than individual differences, demonstrated by random-effects estimates less than 1.0 for each of the interference items. The smallest random-effects variance was observed for the interference with work item (σ² = 0.14), whereas the relations with others item (σ² = 0.85) was the most affected. The variances for the items activity, walking, mood, and enjoyment of life ranged from 0.30 to 0.40.

Effect of Performance Status and Demographics on Symptom Severity

To facilitate the comparison of random effects due to country, we next compared the SDs of the random effects for country (σ, which is on the same scale as the regression coefficients) with the ECOG PS variable for each symptom. We found that patients with poor performance status (ECOG PS = 2–4) consistently reported more severe symptoms, on average, than patients with good performance status (ECOG PS = 0–1) and that the difference in the magnitude of mean effect between good and poor ECOG PS was much larger than the SD of the random effect for country. Table 4 shows that, for most symptom items, the estimates of the random-effect SD associated with country were roughly comparable to the magnitude of the difference in symptom responses for patients with ECOG PS 1 vs 0.

Table 4

Comparison of Eastern Cooperative Oncology Group performance status (ECOG PS) effect on symptom item reports and SD of country effects*

After adjusting for country effects, we found that women reported statistically significantly more severe fatigue, sadness, and distress (all Ps < .001), and disturbed sleep (P = .003). As expected, patients receiving chemotherapy reported more severe nausea and vomiting (all Ps < .001).

Discussion

The results of this study indicate that national and linguistic (country) variations in patient responses to the MDASI are small relative to individual patient-related factors. Analysis of MDASI symptom and interference ratings from cancer patients in five countries—the United States, China, Japan, Russia, and Korea—revealed that the variance of the random effects for country was between 20% and 50% of the intersubject variance. These results give some reassurance that symptom data obtained using various language versions of the MDASI, and possibly patient-reported symptom data from other measures, can be pooled to analyze multinational clinical research.

Previous studies have begun to address cross-cultural equivalence in patient-reported health-related quality-of-life measures. Although progress has been made toward identifying the arenas in which equivalency should be established, few studies have been designed to test assertions about cultural applicability. Such studies have noted that adapting a PRO measure for cross-cultural use requires a careful accounting for the differential impact of culture on results (19) and the establishment of conceptual equivalency (7). Confirmation of an adapted measure's psychometric validity and reliability is insufficient evidence of its suitability for use across cultures (20). A few studies of cross-cultural comparison have examined the dimensional structure of certain PRO instruments (21,22) but not within the cancer population. Other studies conducted in patients with cancer examined multidimensional scaling for cancer pain (23) and the use of differential item functioning for the European Organisation for Research and Treatment of Cancer 30-item quality-of-life questionnaire (24) and FACT–Breast (25) but did not examine the magnitude of effects of language translation and culture/nationality on how people respond to these measures. Thus, additional investigation in internationally coordinated projects is needed to provide sufficient evidence of measurement equivalence for various language versions of major health-related quality-of-life measures (26). Findings of cross-cultural equivalency would support the international application of clinical guidelines for symptom management because such guidelines are often based on symptom ratings.

In this study, we compared variations in MDASI scores attributable to linguistic and cultural variations with inherent intersubject variations in patient responses. In the interpretation of retrospective data collected from a single nationality, the results from our analyses suggest that the magnitude of various cultural or linguistic backgrounds is likely to be only one-fourth to one-half of the interpatient variation of MDASI symptom reports obtained from an otherwise homogeneous population.

In future studies that use PROs, this type of probit analysis method would allow investigators to estimate the impact on symptom reports of patient or clinical variables, such as sex or ECOG PS, thus making it possible to determine whether the effects of such variables are sufficiently important to be included in subsequent data analyses. For example, in this study, the average difference between symptom reports of patients with poor performance status (ECOG PS = 2–4) and patients with good performance status (ECOG PS = 0–1) was larger than the country effect for most symptoms. We note that it is common practice to collapse ECOG status into poor and good categories when analyzing PRO data; our analyses thus suggest that ignoring the language effects associated with the administration of the MDASI instrument is likely to have a smaller effect on study conclusions than collapsing ECOG categories in this way. We also found small but statistically significant effects for sex across samples, with women reporting more severe fatigue, sadness, sleep disturbance, and distress. Again, these differences were small compared with overall individual patient variability.

A finding of a statistically significant country effect in cross-national and cross-cultural studies affecting symptom report data might derive from several sources. First, a poor translation from the original linguistic version could compromise the instruments’ comparability (27). Although there is no empirical evidence in favor of one specific method of translation of a PRO tool, we used an internally consistent procedure for translating all versions of the MDASI (28). In addition, in contrast to the more abstract concepts assessed in most quality-of-life measures, the MDASI assesses only symptoms; it uses single words or simple phrases for its items and a straightforward 0 to 10 numeric rating scale, making the translation of MDASI items relatively simple and the establishment of equivalency less challenging. In this study, we demonstrated that the more severe symptoms—fatigue, disturbed sleep, distress, and pain—are less subject to nation and linguistic effects, as evidenced by the small effect of country on these symptom items.

Symptom management practice is known to vary from country to country (29), as well as from one treatment site to another within a country (30). It is thus not unreasonable to expect that symptoms might be more severe in countries and sites with less aggressive symptom management. With the current models, we did not take into account differences in symptom management practice between countries as a fixed-effect factor. However, the consistency in this study in the most severe symptoms reported by patients, regardless of the characteristics from each sample, as well as the consistency in patients with poor ECOG PS reporting more severe symptoms, indicates that the MDASI functioned similarly across language versions in characterizing symptom burden. In addition, the very small effect of country on pain ratings from this study also supports our procedures, given that one would expect much greater cross-national variation in pain control (ie, variations in practice of prescribing opioids, which are the World Health Organization's recommended standard for management of severe cancer pain) (31) than in fatigue management (because of lack of widely used therapeutic methods to control fatigue).

The study had several limitations. First, the effect of country was confounded with the effect of the individual sample site because only one to three treatment centers were sampled in each country. This could prevent a full examination of patient cultural differences within the same language from country to country. Second, although we expect that language differences in symptom reporting will be even smaller with more homogeneous samples, this expectation needs to be tested empirically. The current analysis used pooled data from heterogeneous samples and did identify one MDASI interference item—relations with others—that was affected by country. Additional research is needed, and caution is warranted when interpreting the meaning of “relations with others” in international MDASI data. Third, we were not able to use cancer stage as a covariate in the modeling because data from one country for this variable were missing. However, the similarity in the percentage of patients with metastasis indicates a comparable disease status across all samples.

In conclusion, this analysis suggests that once psychometrically sound translations of the MDASI have been established, various language versions can be used to gather symptom severity and interference ratings that can be interpreted in a similar way across patient nationalities. The generalizability is meaningful for interpreting the results across various cultural and language groups and provides greater utility in symptom assessment for oncology practice, clinical trials, and clinical research—not only among the diversity of patients in the United States but also for patients with cancer in other parts of the world.

Funding

National Cancer Institute of the National Institutes of Health (R01 CA026582 to C.S.C.); United States Cancer Pain Relief Committee; Hawn Foundation, Dallas, TX.

Footnotes

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Cancer Institute or the National Institutes of Health. The study sponsor had no role in the design of the study; the collection, analysis, and interpretation of the data; the writing of the manuscript; or the decision to submit the manuscript for publication.

The authors acknowledge the editorial assistance of Jeanie F. Woodruff, ELS.

References

1. Cleeland CS. Symptom burden: multiple symptoms and their impact as patient-reported outcomes. J Natl Cancer Inst Monogr. 2007;37:16–21. [PubMed]

2. US Food and Drug Administration. Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research, Center for Devices and Radiological Health. Guidance for industry (draft). Patient-reported outcome measures: use in medical product development to support labeling claims. 2009 http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM071975.pdf. Accessed March 15, 2010. [PMC free article] [PubMed]

3. Benedetti C, Brock C, Cleeland C, et al. NCCN practice guidelines for cancer pain. Oncology (Huntington) 2000;14(11A):135–150. [PubMed]

4. Berger AM, Abernethy AP, Atkinson A, et al. NCCN Clinical Practice Guidelines in Oncology: cancer-related fatigue. 2008 http://www.nccn.org/professionals/physician_gls/PDF/fatigue.pdf. Accessed December 10, 2008.

5. Swarm R, Anghelescu DL, Benedetti C, et al. Adult cancer pain. J Natl Compr Canc Netw. 2007;5(8):726–751. [PubMed]

6. Eaton LH, Tipton JM. Putting Evidence Into Practice: Improving Oncology Patient Outcomes. 1st ed. Pittsburgh, PA: Oncology Nursing Society; 2009.

7. Bowden A, Fox-Rushby JA. A systematic and critical review of the process of translation and adaptation of generic health-related quality of life measures in Africa, Asia, Eastern Europe, the Middle East, South America. Soc Sci Med. 2003;57(7):1289–1306. [PubMed]

8. Cleeland CS, Mendoza TR, Wang XS, et al. Assessing symptom distress in cancer patients: the M.D. Anderson Symptom Inventory. Cancer. 2000;89(7):1634–1646. [PubMed]

9. Kirkova J, Davis MP, Walsh D, et al. Cancer symptom assessment instruments: a systematic review. J Clin Oncol. 2006;24(9):1459–1473. [PubMed]

10. Wang XS, Wang Y, Guo H, Mendoza TR, Hao XS, Cleeland CS. Chinese version of the M.D. Anderson Symptom Inventory: validation and application of symptom measurement in cancer patients. Cancer. 2004;101(8):1890–1901. [PubMed]

11. Okuyama T, Wang XS, Akechi T, et al. Japanese version of the M.D. Anderson Symptom Inventory: a validation study. J Pain Symptom Manage. 2003;26(6):1093–1104. [PubMed]

12. Ivanova MO, Ionova TI, Kalyadina SA, et al. Cancer-related symptom assessment in Russia: validation and utility of the Russian M.D. Anderson Symptom Inventory. J Pain Symptom Manage. 2005;30(5):443–453. [PubMed]

13. Yun YH, Mendoza TR, Kang IO, et al. Validation study of the Korean version of the M.D. Anderson Symptom Inventory. J Pain Symptom Manage. 2006;31(4):345–352. [PubMed]

14. Wang XS, Mendoza TR, Gao SZ, Cleeland CS. The Chinese version of the Brief Pain Inventory (BPI-C): its development and use in a study of cancer pain. Pain. 1996;67(2–3):407–416. [PubMed]

15. Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16(3):297–334.

16. Johnson VE, Albert J. Ordinal Data Modeling. New York, NY: Springer-Verlag; 1999.

17. Oken MM, Creech RH, Tormey DC, et al. Toxicity and response criteria of the Eastern Cooperative Oncology Group. Am J Clin Oncol. 1982;5(6):649–655. [PubMed]

18. Serlin RC, Mendoza TR, Nakamura Y, Edwards KR, Cleeland CS. When is cancer pain mild, moderate or severe? Grading pain severity by its interference with function. Pain. 1995;61(2):277–284. [PubMed]

19. de Haes JC, Olschewski M. Quality of life assessment in a cross-cultural context: use of the Rotterdam Symptom Checklist in a multinational randomised trial comparing CMF and Zoladex (Goserlin) treatment in early breast cancer. Ann Oncol. 1998;9(7):745–750. [PubMed]

20. Wagner AK, Gandek B, Aaronson NK, et al. Cross-cultural comparisons of the content of SF-36 translations across 10 countries: results from the IQOLA Project. International Quality of Life Assessment. J Clin Epidemiol. 1998;51(11):925–932. [PubMed]

21. Allik J. Personality dimensions across cultures. J Pers Disord. 2005;19(3):212–232. [PubMed]

22. Kim M, Han HR, Phillips L. Metric equivalence assessment in cross-cultural research: using an example of the Center for Epidemiological Studies–Depression Scale. J Nurs Meas. 2003;11(1):5–18. [PubMed]

23. Cleeland CS, Nakamura Y, Mendoza TR, Edwards KR, Douglas J, Serlin RC. Dimensions of the impact of cancer pain in a four country sample: new information from multidimensional scaling. Pain. 1996;67(2–3):267–273. [PubMed]

24. Scott NW, Fayers PM, Bottomley A, et al. Comparing translations of the EORTC QLQ-C30 using differential item functioning analyses. Qual Life Res. 2006;15(6):1103–1115. [PubMed]

25. Hahn EA, Holzner B, Kemmler G, Sperner-Unterweger B, Hudgens SA, Cella D. Cross-cultural evaluation of health status using item response theory: FACT-B comparisons between Austrian and U.S. patients with breast cancer. Eval Health Prof. 2005;28(2):233–259. [PubMed]

26. Anderson RT, Aaronson NK, Bullinger M, McBee WL. A review of the progress towards developing health-related quality-of-life instruments for international clinical studies and outcomes research. Pharmacoeconomics. 1996;10(4):336–355. [PubMed]

27. Acquadro C, Conway K, Hareendran A, Aaronson N. Literature review of methods to translate health-related quality of life questionnaires for use in multinational clinical trials. Value Health. 2008;11(3):509–521. [PubMed]

28. Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46(12):1417–1432. [PubMed]

29. World Health Organization. Cancer pain relief: with a guide to opioid availability. 1994. http://whqlibdoc.who.int/publications/9241544821.pdf. Accessed July 14, 2009.

30. Cleeland CS, Gonin R, Hatfield AK, et al. Pain and its treatment in outpatients with metastatic cancer. N Engl J Med. 1994;330(9):592–596. [PubMed]

31. Laugsand EA, Kaasa S, de CF, Hanks G, Klepstad P. Intensity and treatment of symptoms in 3,030 palliative care patients: a cross-sectional survey of the EAPC Research Network. J Opioid Manag. 2009;5(1):11–21. [PubMed]

Articles from JNCI Journal of the National Cancer Institute are provided here courtesy of
Oxford University Press