PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of hlthqualBioMed CentralBiomed Central Web Sitesearchsubmit a manuscriptregisterthis articleHealth and Quality of Life OutcomesJournal Front Page
 
Health Qual Life Outcomes. 2008; 6: 12.
Published online Jan 31, 2008. doi:  10.1186/1477-7525-6-12
PMCID: PMC2276197
Reliability and validity of the Gastrointestinal Symptom Rating Scale (GSRS) and Quality of Life in Reflux and Dyspepsia (QOLRAD) questionnaire in dyspepsia: A six-country study
Károly R Kulich,corresponding author1 Ahmed Madisch,2 Franco Pacini,3 Jose M Piqué,4 Jaroslaw Regula,5 Christo J Van Rensburg,6 László Újszászy,7 Jonas Carlsson,1 Katarina Halling,1 and Ingela K Wiklund1
1AstraZeneca R&D, Medical Science, Mölndal, S-431 86, Sweden
2Medical Department I, Technical University Hospital, Dresden, 01307, Germany
3Azienda Ospedaliera Careggi, U. O. di Gastroenterologia ed Endoscopia digestiva, Villa Medicea, Viale Pieraccini, 17, 50139, Firenze, Italy
4Servicio de Gastroenterología, Hospital Clinic de Barcelona, Villarroel 170, 08036, Spain
5Klinika Gastroenterologii CMKP, Centrum Onkologii, Roentgen Street 5, 02-781, Warszawa, Poland
6Gastroenterology Unit, Tygerberg Hospital, Cape Town, 7505, South Africa
7Semmelweis Hospital, Internal Medicine, Csabai Kapu 9-11, 3501, Miskolc, Hungary
corresponding authorCorresponding author.
Károly R Kulich: Karoly.Kulich/at/astrazeneca.com; Ahmed Madisch: ahmed.madisch/at/uniklinikum-dresden.de; Franco Pacini: pacinif/at/ao-careggi.toscana.it; Jose M Piqué: jmpique/at/clinic.ub.es; Jaroslaw Regula: jregula/at/coi.waw.pl; Christo J Van Rensburg: c.j.vr/at/telkomsa.net; László Újszászy: lujszasz.1bel/at/semmelweis-miskolc.hu; Jonas Carlsson: jonas.carlsson/at/astrazeneca.com; Katarina Halling: katarina.halling/at/astrazeneca.com; Ingela K Wiklund: ingela.k.wiklund/at/gsk.com
Received August 21, 2007; Accepted January 31, 2008.
Background
Symptoms of dyspepsia significantly disrupt patients' lives and reliable methods of assessing symptom status are important for patient management. The aim of the current study was to document the psychometric characteristics of the Gastrointestinal Symptom Rating Scale (GSRS) and the Quality of Life in Reflux and Dyspepsia questionnaire (QOLRAD) in Afrikaans, German, Hungarian, Italian, Polish and Spanish patients with dyspepsia.
Methods
853 patients with symptoms of dyspepsia completed the GSRS, the QOLRAD, the 36-item Short-Form Health Survey (SF-36) and the Hospital Anxiety and Depression scale.
Results
The internal consistency reliability of the GSRS was 0.43–0.87 and of the QOLRAD 0.79–0.95. Test-retest reliability of the GSRS was 0.36–0.75 and of the QOLRAD 0.41–0.82. GSRS Abdominal pain domain correlated significantly with all QOLRAD domains in most language versions, and with SF-36 Bodily pain in all versions. QOLRAD domains correlated significantly with the majority of SF-36 domains in most versions. Both questionnaires were able to differentiate between patients whose health status differed according to symptom frequency and severity.
Conclusion
The psychometric characteristics of the different language versions of the GSRS and QOLRAD were found to be good, with acceptable reliability and validity. The GSRS and QOLRAD were found to be useful for evaluating dyspeptic symptoms and their impact on patients' daily lives in multinational clinical trials.
The definition and characterization of dyspepsia continue to challenge clinical investigators. At the Rome II consensus conference held in 1999, the recommended definition of dyspepsia was 'pain or discomfort centred in the upper abdomen' [1,2]. Yet, symptoms such as heartburn, nausea, post-prandial fullness, early satiety or bloating may also be present [3-7]. Recent guidelines suggest that gastroesophageal reflux disease (GER) is the likely diagnosis if heartburn is the only presenting symptom or the predominant symptom [8-10].
The label 'functional dyspepsia' is often used interchangeably with that of 'dyspepsia', but actually refers to patients whose dyspepsia has been investigated and has no known structural or biochemical cause [8,11]. Studies on the prevalence of dyspepsia in primary care originate mainly from a few developed countries, where the disorder accounts for 1–4% of all primary care consultations [6,12,13]. Estimates of the prevalence of dyspepsia in the adult general population are noticeably higher, ranging from 20–40% [7,11]. The frequency of medical visits for dyspepsia has been found to increase with increasing age [14], and with increasing severity of dyspeptic symptoms [15].
Dyspepsia symptoms impair patients' daily functioning [16-21]. The symptom-driven nature of patient management places particular importance on a reliable estimation of symptom status. Hence, the assessment of how symptoms of dyspepsia affect patients' lives provides important information about the patients' health status and their perception of the treatment regime [22]. Moreover, this information helps enable clinicians to tailor treatment to the individual patient's needs.
Despite increasing interest in evaluating the impact of dyspepsia symptoms on patients' daily lives, measuring this impact in clinical trials can be problematic due to a paucity of validated instruments [23]. Furthermore, clinical trials are increasingly being conducted as multinational studies. This requires the availability of validated non-English language versions of the instrument to be used. Three patient-reported outcomes (PRO) instruments have been validated in patients with dyspepsia in clinical trials: the Severity of Dyspepsia Assessment (SODA) [24], the Nepean Dyspepsia Index (NDI) [25] and the Dyspepsia Symptom Severity Index (DSSI) [26]. However, all three instruments have only been validated as English-language versions. A further concern is that the NDI has not been validated for treatment effects, and the SODA was validated in a non-representative patient population comprising 96% men [24].
To be a viable measure of treatment outcome in clinical trials, PRO instruments need to be extensively documented to meet scientific standards [27,28] and to satisfy regulatory criteria, particularly with regard to claims for labelling and promotion [28,29]. The regulatory criterion is twofold: linguistic cross-cultural adaptation, and psychometric documentation. The US Food and Drug Administration (FDA) has recently issued a draft guidance on PRO measures [28]. In line with these guidelines, instruments need to show reliability, validity and an ability to detect change.
This report aims to document the psychometric characteristics of two PRO instruments, the Gastrointestinal Symptom Rating Scale (GSRS) and the Quality of Life in Reflux and Dyspepsia questionnaire (QOLRAD), in patients with dyspepsia. The GSRS and the QOLRAD were developed in US English. Several translations of both questionnaires have been psychometrically validated previously in patients with GERD [30-35]. In addition, English versions of the questionnaires have been used in dyspeptic populations (AstraZeneca, data on file). For the present report, a series of studies was undertaken in patients with dyspepsia in South Africa, Germany, Hungary, Italy, Poland and Spain. To enable comparisons between the various countries studied, results from all countries are presented here together in the current paper.
Patients
Consecutive patients with predominant symptoms of dyspepsia attending gastroenterology clinics between December 2000 and August 2003 were included in the study. In Germany and Spain, patients were also included from primary care. Dyspepsia was defined as persistent and/or recurrent pain or discomfort centred in the upper abdomen. Inclusion criteria required a history of episodes of dyspepsia for 6 months or longer, and episodes of dyspepsia on at least 1 day during the previous 7 days. The diagnosis based on the patient reports, and was verified at the discretion of the investigator. Where the diagnosis was uncertain, patients were not included. No physical examinations were performed. The following were exclusion criteria: GERD, irritable bowel syndrome, peptic ulcer disease, dementia or any other significant medical, psychiatric or surgical disease. Patients receiving daily treatment with acetyl salicylic acid or other nonsteroidal anti-inflammatory drugs were also excluded. Patients had to be able to complete the PRO instruments unaided, as no proxy assessment or interpreter was allowed. The administration of the questionnaires took place at visit 1 before any medical procedures were performed. Patients who were judged to be in stable phase were scheduled for a second visit. The second visit occurred 7 days after visit 1. The study was approved by local ethics committees in all countries requiring such approval for a non-drug related study. Good Clinical Practice was followed and the patients were free to discontinue participation in the study at any time.
Demographic and clinical variables
Clinicians recorded patient demographics (including age, sex, ethnicity, and marital and employment status), medical history (including history of gastrointestinal diseases) and frequency of dyspepsia symptoms (number of days with episodes during the last 7 days). Investigators also assessed the severity of patients' dyspepsia symptoms by asking the patients and then recording the answer using a four-point graded scale (0 = none: no symptoms; 1 = mild: awareness of sign or symptom, but easily tolerated; 2 = moderate: discomfort sufficient to cause interference with normal activities; 3 = severe: incapacitating with inability to perform normal activities). All data were recorded in a case-report form.
PRO instruments
Patients completed two disease-specific and two generic PRO instruments: the GSRS, the dyspepsia version of the QOLRAD, the 36-Item Short-Form Health Survey (SF-36), and the Hospital Anxiety and Depression scale (HAD) (except in South Africa, where patients were not given the HAD). Questionnaires were completed in an electronic data capture (EDC) device (Apple Newton Pad, Apple Computers Inc, Cupertino, CA, USA), except in South Africa, where paper and pencil were used. The method of using electronic patient-reported outcomes (ePRO) for recording of symptoms and health-related quality of life has previously been shown to improve the quality of the data and to be well received by patients [36]. Study personnel were trained in how to use the electronic device and how to instruct the patients in a standardized way to minimize bias and enhance compliance.
Gastrointestinal Symptom Rating Scale (GSRS)
The GSRS is a disease-specific instrument of 15 items combined into five symptom clusters depicting Reflux, Abdominal pain, Indigestion, Diarrhoea and Constipation. The GSRS has a seven-point graded Likert-type scale where 1 represents absence of troublesome symptoms and 7 represents very troublesome symptoms. The reliability and validity of the GSRS are well-documented [37], and norm values for a general population are available [38].
Quality of Life in Reflux and Dyspepsia questionnaire (QOLRAD)
The dyspepsia version of the QOLRAD is a disease-specific instrument, including 25 items combined into five dimensions: Emotional distress, Sleep disturbance, Vitality, Food/drink problems and Physical/social functioning. The questions are rated on a seven-point graded Likert scale; lower values indicate a more severe impact on daily functioning. The reliability and validity of the QOLRAD have been documented in patients with dyspepsia [39]. Previous studies have shown that a difference of approximately 0.5 points represents a clinically relevant change [40]. The factor structure of the QOLRAD has been replicated for several language versions [41].
36-item Short-Form Health Survey (SF-36)
The SF-36 is an extensively used generic questionnaire containing 36 items clustered into eight dimensions (Bodily pain, General health, Mental health, Physical functioning, Role – emotional, Role – physical, Social functioning, Vitality). Item scores for each dimension are coded, summed and transformed to a scale from 0 (worst possible health status) to 100 (best possible health status). The SF-36 is well-documented in terms of reliability and validity in all available language versions [42,43]. This study used the acute version of the SF-36 covering a 1-week recall period [44].
Hospital Anxiety and Depression scale (HAD)
The HAD is a screening tool developed for use in medical settings. It consists of 14 items divided into two subscales for anxiety (seven items) and depression (seven items), in which the patient rates each item on a four-point scale. The higher scores indicate the presence of problems. A cut-off of > 11 implies definite cases of anxiety or depression, a cut-off of 8–10 a probable case, and < 7 not a case. The validity and reliability of the HAD have been reported in several studies [45,46]. The HAD was not used in South Africa because the Afrikaans translation was not available at the start of the study.
All instruments have been translated and linguistically validated according to international guidelines [47]. The linguistic validation of a questionnaire is not a literal translation of the original questionnaire, but the production of a translation, which is conceptually equivalent to the original and culturally acceptable in the country in which the translation will be used. This translation process includes forward- and back-translations by different, independent translators. Several translations of the GSRS and the QOLRAD are available, including Afrikaans, German, Hungarian, Italian, Polish and Spanish versions, and all have been psychometrically validated previously in patients with GERD [30-35].
Psychometric evaluation of the GSRS and the QOLRAD
Reliability
Internal consistency refers to the extent to which the items within each dimension are interrelated. Cronbach's alpha is the most widely used measurement for assessing internal consistency reliability [48]. A high alpha coefficient (≥ 0.70) suggests that the items within a dimension measure the same construct and supports the construct validity.
Test-retest reliability measures the stability of a score derived from serial administrations of a measure by the same rater. Repeated measurements are made in the same individuals, at a time interval long enough to ensure independence. Here, patients whose symptoms were judged to be stable, and in whom the treatment – not study mandated – remained unchanged, were assessed between visits 1 and 2. An intraclass correlation coefficient (ICC) above 0.70 was considered to be acceptable [49].
Construct validity
Construct validity is concerned with whether the indicator actually measures the underlying attribute. The construct validity was examined by convergent, discriminant and known-groups validity.
Convergent validity consists of showing that a postulated dimension of the instrument correlates appreciably with all other dimensions that theory suggests should be related to it. Here, it was examined by: (a) correlating the GSRS with the QOLRAD; (b) correlating the GSRS and the QOLRAD with the SF-36 dimensions; and (c) correlating the GSRS and the QOLRAD with the HAD (except in South Africa) and with clinician-rated severity of dyspepsia symptoms. Similar dimensions in these instruments were expected to have high correlations with each other as shown by Pearson's product moment correlation. A strong correlation was considered to be over 0.60, a moderate between 0.30 and 0.60 and a low (very low) correlation below 0.30 [50]. Low correlations were expected between those dimensions that are theoretically unrelated constructs, thereby testing the discriminant validity of the instruments.
Known-groups validity consists of showing that an instrument can differentiate between groups of patients whose health status differs according to the characteristics of patients' disease, in this case clinician-rated frequency and severity of dyspepsia symptoms [51,52].
Statistical methods
Statistical analyses were performed using Statistical Analysis System (SAS, version 8.02) [53]. Bonferroni correction was used to adjust for the multiplicity (significance level, P < 0.0003), corresponding to a correlation of 0.32 or above [54]. If data were missing from one or more item in a PRO instrument, the mean of the completed items in the same dimension was used provided that more than half of the items in that dimension had been completed [55]. The percentage of missing data in the GSRS and QOLRAD, respectively, were as follows: South Africa, 2.7% and 2.7%; Germany, 6.4% and 12.7%; Hungary, 8.8% and 8.8%; Italy, 6.3% and 1.1%; Poland, 0.7% and 0.7%; Spain, 0.6% and 0.6%. In cases where more than 6% of the data were missing (Germany, Hungary and Italy), characteristics (age, gender, duration and frequency of dyspepsia symptoms) of patients with missing data were compared with those who completed the questionnaires fully. There were no significant differences between the two groups.
Demographic and clinical characteristics
A total of 853 patients with dyspepsia were included in the study. Patient demographics and clinical characteristics are shown in Table Table1.1. Patients' ages ranged from 18 to 79 years (range, mean age: 42.2–48.3 years; standard deviation [SD]: 11.8–15.9 years), and the majority (61.1–72.1%) of patients were female. Virtually all participants were Caucasian, except in South Africa, where 89% of patients were of mixed ethnic origins. In all countries, more than half of patients were married (range: 59.5–69.7%). Full-time employment was highest in Hungary and lowest in South Africa and Spain (overall range: 41.4–64.0%).
Table 1
Table 1
Patient demographics and clinical data for all countries (total N = 853). Values are in percentages unless otherwise specified.
The majority (59.3–70.9%) of patients had moderate dyspepsia symptoms over the past week, except in South Africa, where the majority (55%) had severe symptoms. In terms of frequency, the majority (61.0–64.9%) of patients in South Africa, Germany and Poland had symptoms on at least five days in the previous week, as did approximately half of patients in Hungary (47.1%) and Spain (48.4%). In Italy, the majority (53.7%) of patients had symptoms on 3 to 4 days in the previous week.
Psychometric evaluation
Reliability
Cronbach's alpha scores ranged from 0.43 (Abdominal pain, German version) to 0.87 (Constipation, Polish version) for the GSRS, and from 0.79 (Vitality, Afrikaans version) to 0.95 (Emotional distress, Spanish version) for the QOLRAD (Table (Table2).2). Scores were high (range: 0.72–0.87) for all language versions of the GSRS in the Indigestion, Diarrhoea and Constipation domains, thus demonstrating high internal consistency. Scores in the Reflux domain were also high for the German (0.72), Hungarian (0.73) and Italian (0.84) versions of the GSRS, but were below 0.7 for the Afrikaans, Polish and Spanish versions. Scores in the Abdominal pain domain were below 0.7 for all six countries. For the QOLRAD, Cronbach's alpha scores were high for all language versions in all domains.
Table 2
Table 2
Cronbach's alpha at visit 1.
Intraclass correlation coefficients for the two visits spaced about 1 week apart ranged from 0.36 (Abdominal pain, German version) to 0.75 (Indigestion, Hungarian and Spanish versions) for the GSRS, and from 0.41 (Vitality, Polish version) to 0.82 (Sleep disturbance, German version) for the QOLRAD (Table (Table3).3). Intraclass correlation coefficients were high (> 0.7) for the Hungarian, Polish and Spanish versions of the GSRS in the Indigestion domain, for the Hungarian version in the Diarrhoea domain, and for the Afrikaans, Hungarian, Polish and Spanish versions in the Constipation domain. Intraclass correlation coefficients for the GSRS Reflux and Abdominal pain domains were below 0.7 for all countries. For the QOLRAD, intraclass correlation coefficients were high for all domains in the Hungarian version, and for all domains except Food/drink problems in the German version. Intraclass correlation coefficients were below 0.7 for all QOLRAD domains in the Afrikaans and Polish versions, and for all but one (Food/drink problems) domain in the Spanish version. Test-retest reliability was not assessed in Italy, as a follow-up with the physicians revealed that many patients were not in a stable phase between visits 1 and 2: some patients changed their self-medication between the two visits, and some were endoscoped.
Table 3
Table 3
Test-retest reliability (intraclass correlation coefficient [ICC]) for Gastrointestinal Symptom Rating Scale (GSRS) and Quality of Life in Reflux and Dyspepsia questionnaire (QOLRAD) domains between visits 1 and 2.
Construct validity
Correlation of the GSRS with the QOLRAD
In the Afrikaans versions, correlations (as defined by a Pearson's product moment correlation = 0.3) were only observed between the GSRS Abdominal pain domain and the QOLRAD Emotional distress and Physical/social functioning domains. In the German versions, the GSRS Abdominal pain and Indigestion domains correlated with all QOLRAD domains, but the GSRS Reflux domain correlated only with the QOLRAD Food/drink problems domain. The relevant GSRS domains Reflux, Abdominal pain and Indigestion correlated with all QOLRAD domains in the Hungarian version of the questionnaires. This was also the case for the Italian versions of the questionnaires, with the exception of the GSRS Reflux and Indigestion domains, which did not correlate with the QOLRAD Food/drink problems domain. In the Polish versions of the questionnaires, the GSRS Reflux and Abdominal pain domains correlated with the QOLRAD Food/drink problems, Physical/social functioning, and Sleep disturbance domains. As was the case with the Italian versions, the GSRS Reflux, Abdominal pain and Indigestion domains correlated with all QOLRAD domains in the Spanish version of the questionnaires.
Correlation of the GSRS with the SF-36
Each domain of the GSRS correlated negatively with each domain of the SF-36 in all language versions of the questionnaires. Significant correlations between GSRS domains and SF-36 domains (Pearson's product moment correlation = 0.3) are shown in Table Table44.
Table 4
Table 4
Significant correlations between Gastrointestinal Symptom Rating Scale (GSRS), Quality of Life in Reflux and Dyspepsia Questionnaire (QOLRAD) and 36-item Short-Form Health Survey (SF-36) domains (Pearson's product moment correlation ≥ 3).
Correlation of the QOLRAD with the SF-36
QOLRAD domains correlated significantly with the majority of SF-36 domains in most language versions. Significant correlations between QOLRAD domains and SF-36 domains (Pearson's product moment correlation = 0.3) are shown in Table Table44.
Correlation of the GSRS with the HAD
Correlations between GSRS domains and the HAD anxiety and depression scores are shown in Table Table5.5. The GSRS Abdominal pain and Indigestion domains correlated significantly with the HAD anxiety score in the German, Spanish and Hungarian populations. The GSRS Reflux domain correlated with the HAD depression score only in the Hungarian population. (Note: HAD was not assessed in South Africa.)
Table 5
Table 5
Pearson's product moment correlations between Gastrointestinal Symptom Rating Scale (GSRS) and Quality of Life in Reflux and Dyspepsia Questionnaire (QOLRAD) domains and Hospital Anxiety and Depression Scale (HAD).
Correlation of the QOLRAD with the HAD
Correlations between QOLRAD domains and the HAD anxiety and depression scores are shown in Table Table5.5. All QOLRAD domains correlated with the HAD anxiety scores in the Hungarian, Italian and Spanish versions of the questionnaire, and all except Sleep disturbance correlated with the HAD anxiety scores in the German and Polish versions. Furthermore, all QOLRAD domains correlated with the HAD depression scores in the Italian and Hungarian versions of the questionnaire, and all except Food/drink problems correlated with the HAD depression score in the Spanish version.
Correlation of the GSRS with physician-assessed severity of symptoms
The relevant GSRS Abdominal pain domain correlated significantly with physician-assessed severity of symptoms in the Afrikaans version of the GSRS. The GSRS Abdominal pain and Indigestion domains correlated significantly with physician-assessed severity of symptoms in the Spanish version of the questionnaire, and with physician-assessed frequency and severity in the Hungarian version. Taken together, these correlations show convergent validity.
Correlation of the QOLRAD with physician-assessed severity of symptoms
All QOLRAD domains correlated with physician-assessed severity and frequency of symptoms in the Hungarian version of the questionnaire, and all domains except Sleep disturbance correlated with physician-assessed severity and frequency in the Spanish version. In the German versions, the QOLRAD emotional distress domain correlated with physician-assessed frequency of symptoms, and the QOLRAD Sleep disturbance domain correlated with physician-assessed severity. The QOLRAD Vitality domain correlated with physician-assessed severity in the Italian version of the questionnaire. In the Polish version, the QOLRAD Physical/social functioning and Vitality domains correlated with physician-assessed frequency and severity of symptoms, and the QOLRAD Emotional distress domain correlated with physician-assessed frequency of symptoms.
Known-groups validity of the GSRS and the QOLRAD
All domains of the GSRS and the QOLRAD questionnaires were able to differentiate between groups of patients whose health status differed according to the physician-assessed overall frequency and severity of dyspepsia symptoms, thereby confirming the known-groups validity of the instruments. GSRS scores increased with increasing frequency and severity of dyspepsia symptoms, while QOLRAD scores decreased with increasing frequency and severity of symptoms (representing a decrease in daily functioning) (results not shown).
The GSRS and the QOLRAD are two of the most established, validated, reliable and responsive disease-specific instruments available for assessing gastrointestinal symptoms and their impact on patients' daily functioning [37,39]. Both questionnaires have been proven to have very good psychometric characteristics when tested in clinical trials in patients with GERD and dyspepsia [20,56,57]. This paper documents the psychometric validation of the Afrikaans, German, Hungarian, Italian, Polish and Spanish translations of the GSRS and the QOLRAD in patients with dyspepsia. The study was conducted mainly in gastroenterology centres and, therefore, results are particular to patients referred for gastroenterological investigation. Because of the standardized methodology used, results from all countries are directly comparable.
Questionnaires were completed in an electronic data capture device (ECD), which ensures that all questions are answered in full. Although ECD technology is becoming an increasing part of clinical trials, it is not yet widely available, and certainly not that widespread in clinical practice. Thus, in South Africa, paper versions of the questionnaires were used for the present study, and this may have affected the results obtained in this country, especially as the use of ECD technology has been shown to increase patient compliance and improve the quality of the data [36].
The demographic and clinical characteristics of the patient populations were comparable in the different countries studied in terms of the number of patients recruited, their age, gender, symptom status and medication use. The Afrikaans patient population had the highest incidence of abdominal pain as the predominant dyspeptic symptom as well as the highest proportion of patients with a history of previous peptic ulcer and/or ulcerative reflux esophagitis. This was probably due to the fact that all South African subjects were recruited from a single gastroenterology clinic in a healthcare system that may refer only the most severely affected patients.
In terms of psychometric characteristics, internal consistency was high in all domains of all language versions of the QOLRAD, thus supporting construct validity. Internal consistency was also high in the Indigestion, Diarrhoea and Constipation domains of the GSRS in all language versions. However, although abdominal pain is the typical dyspepsia symptom, internal consistency was only moderate in the Abdominal pain domain of the GSRS. This generally low internal consistency in the Abdominal pain domain probably reflects the difficulty of measuring this often fluctuating symptom, as well as the fact that the GSRS Abdominal pain domain contains only two items. In previous international studies conducted in patients with GERD, internal consistency also tended to be lower in the Abdominal pain domain of the GSRS than in its other domains [30-35].
Test-retest reliability was good for most or all domains in the Hungarian and German translations of the QOLRAD, but only for one domain in the Spanish version and for no domains in the Afrikaans and Polish versions of the QOLRAD. It is not clear why the test-retest reliability was low in these patient populations, although it may be an indication that the frequency, severity and impact of abdominal symptoms were not as stable on a week-to-week basis in these patients as was estimated by the investigators. No test-retest was performed in Italy, so it was not possible to assess test-retest reliability for the Italian version of the QOLRAD. Previous international studies in patients with GERD have yielded acceptable test-retest reliability for the German (ICC: 0.70–0.84), Spanish (ICC: 0.79–0.85) and Afrikaans (ICC: 0.71–0.82) versions of the QOLRAD, but only for some domains of the Polish version (ICC: 0.51–74) of the QOLRAD [30-35]. In the GSRS, test-retest reliability was acceptable in the Indigestion and Constipation domains in most language versions. However, reliability was lower in the Abdominal pain domains in all countries assessed. As with internal consistency, the generally low test-retest reliability in the Abdominal pain domain is most likely due to the GSRS Abdominal pain domain containing only two items, as well as the complexity of measuring this variable symptom. It is also likely that at least some of the patients deemed stable by the physicians experienced small changes in their symptoms that were picked up by the PROs.
In terms of construct validity, all GSRS domains correlated with all QOLRAD domains. Correlation was significant for the relevant GSRS Abdominal pain domain and most or all QOLRAD domains in the majority of the different language versions. The GSRS Abdominal pain domain also correlated significantly with the relevant SF-36 Bodily pain domain in all language versions. All QOLRAD domains correlated significantly with the majority of SF-36 domains in most language versions. Both the GSRS and the QOLRAD were able to differentiate between patients whose health status differed with regard to frequency and severity of symptoms, thereby confirming the known-groups validity of the instruments. Known-groups validity has also been confirmed for several translations of the GSRS and the QOLRAD in patients with GERD [30-32,35]. Confirmatory factor analysis has been shown to support the validity of the Scandinavian language versions of the GSRS and the QOLRAD in patients with GERD [41]. Future studies could use this approach to further assess the validity of the additional language versions presented here.
The relevant GSRS Abdominal pain domain correlated with the HAD anxiety score in most of the language versions assessed, and most of the QOLRAD domains correlated with the HAD anxiety score in all language versions assessed. Dyspepsia tends to be associated with anxiety and depression [58]. Previous reports are somewhat contradictory on the role of psychological morbidity in dyspepsia symptoms and healthcare-seeking behaviour [1,58], suggesting that additional research is required on this potentially important aspect of the disorder.
When comparing the frequency and severity of patient-reported symptoms with those assessed by the physician, correlations were only low to moderate. These discrepancies in patient-reported and observer-reported symptom status have been well-documented [59-61]. Our view is that the physician examination needs to be balanced with the patient-reported symptom status when deciding on patient management and outcomes [39,62].
In summary, all language versions of the QOLRAD showed good internal consistency and reliability, although only the German and Hungarian translations were able to demonstrate acceptable test-retest reliability. In general, all language versions of the GSRS were reliable and showed good internal consistency for the Indigestion, Diarrhoea and Constipation domains, but not for the Abdominal pain domain. Overall, the test-retest reliability of the GSRS was not acceptable.
Conclusion
In conclusion, the German, Hungarian, Italian, Polish, Spanish and Afrikaans versions of GSRS and QOLRAD are reliable and show good internal consistency for use in clinical trials. However, the test-retest reliability was low in most language versions. By evaluating symptoms and their impact on patients' lives, both these instruments help physicians interpret clinical benefits as outcomes of significance to patients. We believe that these results will further facilitate the use of these instruments in the clinical setting.
Abbreviations
DSSI, Dyspepsia Symptom Severity Index
EDC, electronic data capture
ePRO, electronic patient-reported outcomes
FDA, Food and Drug Administration
GERD, gastroesophageal reflux disease
GSRS, Gastrointestinal Symptom Rating Scale
HAD, Hospital Anxiety and Depression scale
ICC, intraclass correlation coefficient
NDI, Nepean Dypepsia Index
PRO, patient-reported outcomes
QOLRAD, Quality of Life in Reflux and Dyspepsia questionnaire
SD, standard deviation
SF-36, 36-item Short-Form Health Survey
SODA, Severity of Dyspepsia Assessment
Competing interests
All authors participated in the present study supported by AstraZeneca R&D, Mölndal, Sweden. Károly Kulich, Jonas Carlsson and Katarina Halling are employees of AstraZeneca R&D, Mölndal, Sweden. Ingela Wiklund was an employee of AstraZeneca R&D, Mölndal, Sweden at the time of the study. Ahmed Madisch, Franco Pacini, Jose Piqué, Jaroslaw Regula, Christoffel Johannes van Rensburg and László Újszászy have no additional financial or other relationships to disclose.
Authors' contributions
All authors contributed to the study design, data analyses and interpretation of results. Ahmed Madisch, Franco Pacini, Jose Piqué, Jaroslaw Regula, Christoffel Johannes van Rensburg and László Újszászy coordinated the patient recruitment and data collection. Jonas Carlson provided statistical support. All authors provided comments and revisions on a first version of the manuscript drafted by Károly Kulich, Katarina Halling and Ingela Wiklund. All authors have seen and approved the final version of the manuscript.
Acknowledgements
This study was funded by AstraZeneca R&D, Mölndal, Sweden. Dr Anja Becher provided editorial assistance funded by AstraZeneca. The authors would like to acknowledge the investigators who participated in the recruitment of patients.
Germany: A Madisch, S Miehlke, Medical Department I, Technical University of Dresden, Dresden; P Malfertheiner, Otto-von-Guericke University, Magdeburg; K Ziegler, Private clinic, Berlin; E Bayerdörffer, Department of Internal Medicine, University Hospital of Marburg, Marburg; J Labenz, Department of Medicine, Jung-Stilling Hospital, Siegen.
Hungary: L Újszászy, Semmelweis Hospital, Internal Medicine, Miskolc; TG Tóth, Szent János Hospital, Internal Medicine, Budapest; L Bárány, Megyei Jogú Város Hospital, Internal Medicine, Nagykanizsa.
Italy: L Capurso, Divisione di Gastroenterologia, Azienda Complesso Ospedaliero San Filippo Neri, Roma; L Cipolletta, Gastroenterologia Ospedale Agostino Maresca Torre del Greco; M D'Ayala, Valva Servizio di Gastroenterologia Ospedale SS. Annunziata, Taranto; L Dughera, Servizio Endoscopia e Motilità, Presidio Ospedaliero Molinette, Torino; R Galeazzi, Divisione di Gastroenterologia Ospedale Regionale Umberto I, Ancona; M Rizzetto, Responsabile U.O. di Gastroenterologia, Dipartimento Apparato Digerente e Nutrizione Clinica Presidio, Ospedaliero Le Molinette, Torino; A Russo, Cattedra di Gastroenterologia Policlinico, Catania; A Saggioro, Gastroenterologia Ospedale Umberto I, Mestre Venezia; V Savarino, D.I.M.I. Università di Genova, Genova; A Zambelli, Servizio di Gastroenterologia ed Endoscopia Digestiva Ospedale Maggiore Crema.
Poland: J Regula, Centrum Onkologii im. Marii Sklodowskiej-Curie, Klinika Gastroenterologii CMKP, Warszawa; J Stasiewicz, Samodzielny Publiczny ZOZ, Wojewodzki Szpital Zespolny im. J. Sniadeckego III Oddzial Chorob Wewnetrznychi Gastroenterologii z Pracownia Endoskopowa Bialystok; B Jasinski, Szpital Nr 2 im. Zachorskiego Sosnowiec.
Spain: JM Baena, CS Carles Ribas, Barcelona; R Carrillo, CS Florida Sur L'Hospitalet de Llobregat Barcelona; G Martínez, CAP Les Corts Barcelona; JJ Mascort, CS Florida Sur L'Hospitalet de Llobregat Barcelona; X Puigdengolas, CS Florida Sur L'Hospitalet de Llobregat Barcelona; JJ Valdepérez, CS Actur Sur Zaragoza; R. Altisent, CS Actur Sur Zaragoza; F Muñoz, CS Puente Segovia Madrid; JA Ferrús, CS La Paz Madrid; S Sofos, CS Motril Este Granada; S Fernández, CL Velez de Benaudalla Granada; JM Saniger, CS Punte Segovia Madrid; S Sitjar, CAP Les Corts Barcelona; X Otero, CAP Les Corts Barcelona; J Ortiz, CAP Les Corts Barcelona; A Anguita, CAP Les Corts Barcelona; J Aracil CS El Cristo Oviedo; M Prieto, CS Vallobín Oviedo; S Tranche, CS El Cristo Oviedo.
South Africa: CJ Van Rensburg, E Wilken, CF Kruger, JEC Botha, Gastroenterology Unit, Tygerberg Hospital, South Africa.
  • Talley NJ, Boyce P, Jones M. Dyspepsia and health care seeking in a community: How important are psychological factors? Dig Dis Sci. 1998;43:1016–1022. doi: 10.1023/A:1018878717715. [PubMed] [Cross Ref]
  • Talley NJ, Colin-Jones D, Koch KL, Koch M, Nyren O, Stangellini V. Functional dyspepsia: a classification with guidelines for diagnosis and management. Gastroenterol Int. 1991;4:145–160.
  • Colin-Jones D, Bloom BS, Bodemar G, Crean G, Freston J, Gugler R. Management of dyspepsia: report of a working party. Lancet. 1988;1:576–579. [PubMed]
  • Chiba N, Bernard L, O'Brien BJ, Goeree R, Hunt RH. A Canadian physician survey of dyspepsia management. Can J Gastroenterol. 1998;12:83–90. [PubMed]
  • Bytzer P, Talley NJ. Dyspepsia. Ann Intern Med. 2001;134:815–822. [PubMed]
  • Grainger SL, Klass HJ, Rake MO, Williams JG. Prevalence of dyspepsia: the epidemiology of overlapping symptoms. Postgrad Med J. 1994;70:154–161. [PMC free article] [PubMed]
  • Malfertheiner P. Current concepts in dyspepsia: a world perspective. Eur J Gastroenterol Hepatol. 1999;11 Suppl 1:S25–9. [PubMed]
  • Talley NJ, Stanghellini V, Heading RC, Koch KL, Malagelada JR, Tytgat GN. Functional gastroduodenal disorders. Gut. 1999;45 Suppl 2:II37–42. [PMC free article] [PubMed]
  • Agreus L. Natural history of dyspepsia. Gut. 2002;50 Suppl 4:iv2–9. [PMC free article] [PubMed]
  • Vakil N, Veldhuyzen van Zanten S, Kahrilas P, Dent J, Jones R. The Montreal definition and classification of Gastro-esophageal Reflux Disease (GERD) - a global evidence-based consensus. Am J Gastroenterol. 2006;101:1900–1920. doi: 10.1111/j.1572-0241.2006.00630.x. [PubMed] [Cross Ref]
  • Heading RC. Prevalence of upper gastrointestinal symptoms in the general population: a systematic review. Scand J Gastroenterol Suppl. 1999;231:3–8. doi: 10.1080/003655299750025200. [PubMed] [Cross Ref]
  • Gear MW, Barnes RJ. Endoscopic studies of dyspepsia in a general practice. Br Med J. 1980;280:1136–1137. [PMC free article] [PubMed]
  • van Bommel MJ, Numans ME, de Wit NJ, Stalman WA. Consultations and referrals for dyspepsia in general practice--a one year database survey. Postgrad Med J. 2001;77:514–518. doi: 10.1136/pmj.77.910.514. [PMC free article] [PubMed] [Cross Ref]
  • Heikkinen M, Pikkarainen P, Takala J, Julkunen R. General practitioners' approach to dyspepsia. Survey of consultation frequencies, treatment, and investigations. Scand J Gastroenterol. 1996;31:648–653. doi: 10.3109/00365529609009144. [PubMed] [Cross Ref]
  • Westbrook JI, McIntosh J, Talley NJ. Factors associated with consulting medical or non-medical practitioners for dyspepsia: an Australian population-based study. Aliment Pharmacol Ther. 2000;14:1581–1588. doi: 10.1046/j.1365-2036.2000.00878.x. [PubMed] [Cross Ref]
  • Talley NJ, Weaver AL, Zinsmeister AR. Impact of functional dyspepsia on quality of life. Dig Dis Sci. 1995;40:584–589. doi: 10.1007/BF02064375. [PubMed] [Cross Ref]
  • Talley NJ. Quality of life in functional dyspepsia. Scand J Gastroenterol Suppl. 1996;221:21–22. doi: 10.3109/00365529609095547. [PubMed] [Cross Ref]
  • Wiklund I, Glise H, Jerndal P, Carlsson J, Talley NJ. Does endoscopy have a positive impact on quality of life in dyspepsia? Gastrointest Endosc. 1998;47:449–454. doi: 10.1016/S0016-5107(98)70243-3. [PubMed] [Cross Ref]
  • Meineche-Schmidt V, Talley NJ, Pap A, Kordecki H, Schmid V, Ohlsson L, Wahlqvist P, Wiklund I, Bolling-Sternevald E. Impact of functional dyspepsia on quality of life and health care consumption after cessation of antisecretory treatment. A multicentre 3-month follow-up study. Scand J Gastroenterol. 1999;34:566–574. doi: 10.1080/003655299750026010. [PubMed] [Cross Ref]
  • Mones J, Adan A, Segu JL, Lopez JS, Artes M, Guerrero T. Quality of life in functional dyspepsia. Dig Dis Sci. 2002;47:20–26. doi: 10.1023/A:1013294815971. [PubMed] [Cross Ref]
  • El-Serag HB, Talley NJ. Health-related quality of life in functional dyspepsia. Aliment Pharmacol Ther. 2003;18:387–393. doi: 10.1046/j.1365-2036.2003.01706.x. [PubMed] [Cross Ref]
  • Symonds T, Berzon R, Marquis P, Rummans TA. The clinical significance of quality-of-life results: practical considerations for specific audiences. Mayo Clin Proc. 2002;77:572–583. [PubMed]
  • Veldhuyzen van Zanten S. Problems in measurement of quality of life in dyspepsia trials. Eur J Surg Suppl. 1998:77–80. doi: 10.1080/11024159850191490. [PubMed] [Cross Ref]
  • Rabeneck L, Cook KF, Wristers K, Souchek J, Menke T, Wray NP. SODA (severity of dyspepsia assessment): a new effective outcome measure for dyspepsia-related health. J Clin Epidemiol. 2001;54:755–765. doi: 10.1016/S0895-4356(00)00365-6. [PubMed] [Cross Ref]
  • Talley NJ, Haque M, Wyeth JW, Stace NH, Tytgat GN, Stanghellini V, Holtmann G, Verlinden M, Jones M. Development of a new dyspepsia impact scale: the Nepean Dyspepsia Index. Aliment Pharmacol Ther. 1999;13:225–235. doi: 10.1046/j.1365-2036.1999.00445.x. [PubMed] [Cross Ref]
  • Leidy NK, Farup C, Rentz AM, Ganoczy D, Koch KL. Patient-based assessment in dyspepsia: development and validation of Dyspepsia Symptom Severity Index (DSSI) Dig Dis Sci. 2000;45:1172–1179. doi: 10.1023/A:1005558204440. [PubMed] [Cross Ref]
  • Lohr K. Assessing health status and quality-of-life instruments: Attributes and review criteria. Qual Life Res. 2002;11:193–205. doi: 10.1023/A:1015291021312. [PubMed] [Cross Ref]
  • Food_and_Drug_Administration Patient-reported outcome measures: Use in medical product development to support labeling claims http://www.fda.gov/cder/guidance/5460dft.htm
  • Leidy NK, Revicki DA, Genesté B. Recommendations for Evaluating the Validity of Quality of Life Claims for Labeling and Promotion. Value in Health. 1999;2:113–127. doi: 10.1046/j.1524-4733.1999.02210.x. [PubMed] [Cross Ref]
  • Kulich KR, Malfertheiner P, Madisch A, Labenz J, Bayerdorffer E, Miehlke S, Carlsson J, Wiklund IK. Psychometric validation of the German translation of the Gastrointestinal Symptom Rating Scale (GSRS) and Quality of Life in Reflux and Dyspepsia (QOLRAD) questionnaire in patients with reflux disease. Health Qual Life Outcomes. 2003;1:62. doi: 10.1186/1477-7525-1-62. [PMC free article] [PubMed] [Cross Ref]
  • Kulich KR, Calabrese C, Pacini F, Vigneri S, Carlsson J, Wiklund IK. Psychometric validation of the Italian translation of the Gastrointestinal Symptom-Rating Scale and Quality of Life in Reflux and Dyspepsia Questionnaire in patients with gastro-oesophageal reflux disease. Clin Drug Invest. 2004;24:205–215. doi: 10.2165/00044011-200424040-00002. [PubMed] [Cross Ref]
  • van Rensburg CJ, Kulich KR, Carlsson J, Wiklund IK. Psychometric validation of the Afrikaans translation of two patient-reported outcomes instruments for reflux disease. S Afr Rev Gastroenterol. 2006;4:5–9.
  • Kulich KR, Regula J, Stasiewicz J, Jasinski B, Carlsson J, Wiklund I. [Psychometric validation of the Polish translation of the Gastrointestinal Symptom Rating Scale (GSRS) and Quality of Life in Reflux and Dyspepsia (QOLRAD) questionnaire in patients with reflux disease] Pol Arch Med Wewn. 2005;113:241–249. [PubMed]
  • Kulich KR, Pique JM, Vegazo O, Jimenez J, Zapardiel J, Carlsson J, Wiklund I. [Psychometric validation of translation to Spanish of the gastrointestinal symptoms rating scale (GSRS) and quality of life in reflux and dyspepsia (QOLRAD) in patients with gastroesophageal reflux disease]. Revista Clinica Espanola. 2005;205:588–595. [PubMed]
  • Kulich RK, Ujszaszy L, Toth GT, Barany L, Carlsson J, Wiklund I. [Psychometric validation of the Hungarian translation of the gastrointestinal symptom rating scale (GSRS) and quality of life in reflux and dyspepsia (QOLRAD) questionnaire in patients with reflux disease] Orv Hetil. 2004;145:723-9–739-44. [PubMed]
  • Drummond HE, Ghosh S, Ferguson A, Brackenridge D, Tiplady B. Electronic quality of life questionnaires: a comparison of pen-based electronic questionnaires with conventional paper in a gastrointestinal study. Qual Life Res. 1995;4:21–26. doi: 10.1007/BF00434379. [PubMed] [Cross Ref]
  • Dimenas E, Glise H, Hallerback B, Hernqvist H, Svedlund J, Wiklund I. Well-being and gastrointestinal symptoms among patients referred to endoscopy owing to suspected duodenal ulcer. Scand J Gastroenterol. 1995;30:1046–1052. doi: 10.3109/00365529509101605. [PubMed] [Cross Ref]
  • Dimenas E, Carlsson G, Glise H, Israelsson B, Wiklund I. Relevance of norm values as part of the documentation of quality of life instruments for use in upper gastrointestinal disease. Scand J Gastroenterol Suppl. 1996;221:8–13. doi: 10.3109/00365529609095544. [PubMed] [Cross Ref]
  • Wiklund IK, Junghard O, Grace E, Talley NJ, Kamm M, Veldhuyzen van Zanten S, Pare P, Chiba N, Leddin DS, Bigard MA, Colin R, Schoenfeld P. Quality of Life in Reflux and Dyspepsia patients. Psychometric documentation of a new disease-specific questionnaire (QOLRAD) Eur J Surg Suppl. 1998;583:41–49. doi: 10.1080/11024159850191238. [PubMed] [Cross Ref]
  • Talley NJ, Fullerton S, Junghard O, Wiklund I. Quality of life in patients with endoscopy-negative heartburn: reliability and sensitivity of disease-specific instruments. Am J Gastroenterol. 2001;96:1998–2004. doi: 10.1111/j.1572-0241.2001.03932.x. [PubMed] [Cross Ref]
  • Kulich KR, Wiklund I, Junghard O. Factor structure of the Quality of Life in Reflux and Dyspepsia (QOLRAD) questionnaire evaluated in patients with heartburn predominant reflux disease. Qual Life Res. 2003;12:699–708. doi: 10.1023/A:1025192100450. [PubMed] [Cross Ref]
  • Ware JE, Kosinski M, Gandek B, Aaronson NK, Apolone G, Bech P, Brazier J, Bullinger M, Kaasa S, Leplege A, Prieto L, Sullivan M. The factor structure of the SF-36 Health Survey in 10 countries: results from the IQOLA Project. International Quality of Life Assessment. J Clin Epidemiol. 1998;51:1159–1165. doi: 10.1016/S0895-4356(98)00107-3. [PubMed] [Cross Ref]
  • Ware JE, Gandek B, Kosinski M, Aaronson NK, Apolone G, Brazier J, Bullinger M, Kaasa S, Leplege A, Prieto L, Sullivan M, Thunedborg K. The equivalence of SF-36 summary health scores estimated using standard and country-specific algorithms in 10 countries: results from the IQOLA Project. International Quality of Life Assessment. J Clin Epidemiol. 1998;51:1167–1170. doi: 10.1016/S0895-4356(98)00108-5. [PubMed] [Cross Ref]
  • Ware JE. SF-36 Health Survey: Manual and Interpretation Guide. Boston, New England Medical Centre; 1993.
  • Moorey S, Greer S, Watson M, Gorman C, Rowden L, Tunmore R, Robertson B, Bliss J. The factor structure and factor stability of the hospital anxiety and depression scale in patients with cancer. Br J Psychiatry. 1991;158:255–259. [PubMed]
  • Herrmann C. International experiences with the Hospital Anxiety and Depression Scale--a review of validation data and clinical results. J Psychosom Res. 1997;42:17–41. doi: 10.1016/S0022-3999(96)00216-4. [PubMed] [Cross Ref]
  • Chassany O, Sagnier P, Marquis P, Fullerton S, Aaronson N. Patient-reported outcomes: The example of health-related quality of life – A European guidance document for the improved integration of health-related quality of life assessment in the drug regulatory process. Drug Inf J. 2002;36:209–238.
  • Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297–334. doi: 10.1007/BF02310555. [Cross Ref]
  • Fitzpatrick R, Davey C, Buxton MJ, Jones DR. Evaluating patient-based outcome measures for use in clinical trials. Health Technology Assessment (Winchester, England) 1998;2:i–74. [PubMed]
  • Hinkle DE, Jurs SG, Wiersma W. Applied statistics for the behavioural sciences. 2nd. Boston, Houghton Mifflin; 1988. p. 682.
  • Fayers PM, Machin D. Quality of life: assessment, analysis and interpretation. Chichester, Wiley; 2000.
  • Kerlinger FN, Lee HB. Foundations of behavioural research. 4th. Fort Worth, Harcourt College Publishers; 2000.
  • SAS_Institute_INC SAS version 8.02. Cary, ; 2001.
  • Bonferroni CE. Il Calcolo delle assicurazioni su gruppi di teste. In Studii in onore del prof. S. O. Carboni, Roma. Sci Econ Comm. 1936. pp. 1–62.
  • Halling K, Långström G, Wiklund I. Handling missing data in quality of life questionnaires: Experience from clinical trials. 6th Annual Conference of the International Society for Quality of Life Research, Barcelona, Spain. 1999.
  • Crawley J, Frank L, Joshua-Gotlib S, Flynn J, Frank S, Wiklund I. Measuring change in quality of life in response to Helicobacter pylori eradication in peptic ulcer disease: the QOLRAD. Dig Dis Sci. 2001;46:571–580. doi: 10.1023/A:1005655317121. [PubMed] [Cross Ref]
  • Wahlqvist P, Carlsson J, Stalhammar NO, Wiklund I. Validity of a Work Productivity and Activity Impairment questionnaire for patients with symptoms of Gastro-Esophageal Reflux Disease (WPAI-GERD)--results from a cross-sectional study. Value Health. 2002;5:106–113. doi: 10.1046/j.1524-4733.2002.52101.x. [PubMed] [Cross Ref]
  • Locke GR, 3rd, Weaver AL, Melton LJ, 3rd, Talley NJ. Psychosocial factors are linked to functional gastrointestinal disorders: a population based nested case-control study. Am J Gastroenterol. 2004;99:350–357. doi: 10.1111/j.1572-0241.2004.04043.x. [PubMed] [Cross Ref]
  • Enns MW, Larsen DK, Cox BJ. Discrepancies between self and observer ratings of depression. The relationship to demographic, clinical and personality variables. J Affect Disord. 2000;60:33–41. doi: 10.1016/S0165-0327(99)00156-1. [PubMed] [Cross Ref]
  • McColl E, Junghard O, Wiklund I, Revicki D. Assessing symptoms in gastroesophageal reflux disease: how well do clinicians' assessments agree with those of their patients? Am J Gastroenterol. 2005;100:11–18. doi: 10.1111/j.1572-0241.2005.40945.x. [PubMed] [Cross Ref]
  • Wiklund I, Comerford MB, Dimenas E. The relationship between exercise tolerance and quality of life in angina pectoris. Clin Cardiol. 1991;14:204–208. [PubMed]
  • Slevin ML, Plant H, Lynch D, Drinkwater J, Gregory WM. Who should measure quality of life, the doctor or the patient? Br J Cancer. 1988;57:109–112. [PMC free article] [PubMed]
Articles from Health and Quality of Life Outcomes are provided here courtesy of
BioMed Central