|Home | About | Journals | Submit | Contact Us | Français|
Contributors: ME, SF, and JD designed the protocol for the study. CS performed the literature searches under the supervision of ME. Data were extracted by CS and DT. ME and CS analysed the data. All authors helped to write the paper. ME, JD, and SF are the guarantors for the study.
To examine the frequency and quality of reporting on quality of life in randomised controlled trials.
Search of the Cochrane Controlled Trials Register 1980 to 1997 to identify trials from all disciplines, from oncology, and from cardiovascular medicine that reported on quality of life. Assessment of abstracts from articles published from 1993 to 1996. Assessment of a sample of full reports with a standardised instrument.
Prevalence of reporting on quality of life. Conditions and interventions studied in trials reporting on quality of life. Quality of reporting on quality of life.
During 1980-97 reporting on quality of life increased from 0.63% to 4.2% for trials from all disciplines, from 1.5% to 8.2% for cancer trials, and from 0.34% to 3.6% for cardiovascular trials. Of 364 abstracts, 65% reported on drug interventions. Of a sample of 67 full reports, authors of 48 (72%) used 62 established quality of life instruments. In 15 reports (22%) authors developed their own measures, and in 2 (3%) methods were unclear. Response rates were given in 38 (57%), and complete reporting on all items and scales occurred in 31 (46%).
Less than 5% of all randomised controlled trials reported on quality of life, and this proportion was below 10% even for cancer trials. A plethora of instruments was used in different studies, and the reporting of methods and results was often inadequate. Standards for the measurement and reporting of quality of life in clinical trials research need to be developed.
Quality of life has become an accepted end point in clinical research trials in recent years, as interest in patients’ experiences and preferences has grown.1 Quality of life is of particular importance in trials comparing treatments with similar or no impact on disease progression and survival. However, the term quality of life is often used vaguely and without clear definition.2–4 This is not surprising, considering the broad nature of a concept that includes physical functioning (ability to carry out activities of daily living such as self care and walking around), psychological functioning (emotional and mental wellbeing), social functioning (relationships with others and participation in social activities), and perception of health status, pain, and overall satisfaction with life.5
The lack of a clear definition of quality of life is reflected in the many instruments that have been proposed to measure it. Generic measures (such as the sickness impact profile6 or the short form health survey SF-367) broadly assess physical, mental, and social health and can be used to compare conditions and treatments. Measures specific to illnesses can supplement generic measures or can be used independently.8 Other methods include measures focusing on a single aspect such as pain or anxiety and individualised measures, in which patients themselves define and rate the most important aspects of their quality of life.9
Inadequate reporting of randomised controlled trials is common and hampers the appraisal of the validity and generalisability of results.10–13 In an attempt to remedy this, consolidated standards of reporting trials (CONSORT) have been developed.14 To our knowledge, however, the quality of reporting on quality of life outcomes in trials has not been systematically assessed so far. In addition, it is unclear as to what extent the growth in attention to quality of life is reflected in an increasing number of reports from clinical trials. We addressed these questions in a bibliographic study based on the Cochrane Controlled Trials Register.
We searched the Cochrane Controlled Trials Register on The Cochrane Library (issue 3, 1998)15 to identify randomised controlled trials published in any language between 1980 and 1997 that reported on quality of life. We entered the keyword “random*” to identify randomised controlled trials, the wild card (*) ensuring that all abstracts containing words with the stem “random” (such as randomly, randomised, randomisation) were included. We identified trials that reported on quality of life by combining MeSH and free text terms “quality of life.” For each year, we calculated the proportion of reports mentioning quality of life. We repeated the analysis for cancer trials, using the search strategy of the Cochrane Cancer Network for this purpose. We also examined cardiovascular trials: we exploded all subheadings of the MeSH term “cardiovascular-diseases” and combined the results with free text searches using a total of 28 key words (details of the search strategy are available on the BMJ website).
We examined the abstracts of reports published from 1993 to 1996. We had identified these in a search of an earlier issue of the Cochrane Controlled Trials Register (issue 4, 1997) using the same keywords. Abstracts from articles that reported the results of randomised controlled trials and included at least one outcome that we considered to be related to quality of life were assessed by CS and checked by JD. The following information was recorded: language of article, condition studied, and type of intervention.
A sample of reports from the database of abstracts was assessed in more detail. Using a random number generator, we selected 20% of the trials reported in English. CS and DT independently examined the full reports of these using a standardised data sheet. The data sheet covered the following items: conditions and interventions studied, primary and secondary end points, sample sizes, type and documentation of the quality of life measures used, response rates, and the presentation and direction of quality of life results. If not explicitly stated by the authors, the primary end point was defined as the one that was given prominence in the report. The results relating to quality of life and other end points were also extracted. Finally, CS and DT rated trial quality using the score described by Jadad et al.16 The rate of agreement between the two observers was generally high (median 80%, range 58-100%). Discrepancies were resolved in discussions with ME.
The keyword “random*” identified 92364 records, which shrank to 1939 (2.1%) when “quality of life” was added. There were 10896 cancer trials, of which 443 (4.1%) reported on quality of life, and 18398 cardiovascular trials, of which 393 (2.1%) reported on quality of life. The proportion of trials reporting on quality of life increased over time, from 0.63% in 1980 to 4.2% in 1997 for all trials, from 1.5% to 8.2% for cancer trials, and from 0.34% to 3.6% for cardiovascular trials (figure).
We found 492 reports that were published during 1993-6, of which we excluded 128 (26%) for not being a randomised controlled trial (n=59), not reporting on quality of life (n=12), duplicate entry (n=27), and not being a journal publication (n=30). We examined the abstracts of the remaining 364 reports: the most commonly studied conditions were cancer (29%) and cardiovascular disease (26%), with other diseases occurring in less than 10% of studies (table (table1).1). The most common interventions were drugs (65%) and models of care (11%) (table (table2).2). Most of the reports (93%) were published in English language journals.
We selected 67 English language reports at random and examined them in detail (references are available on the BMJ’s website). They were published in 52 journals. Ten journals published more than one report, and three (Circulation, Lancet, and New England Journal of Medicine) published more than two reports. The distribution of conditions and interventions studied was similar to that in the database of abstracts, with 30% of reports describing cancer trials, 25% describing cardiovascular conditions, and 64% reporting on drug interventions.
—Forty eight of the trials used at least one of the 62 established instruments shown in the box, and most (54) quoted a reference that described the methodology. In 15 trials the authors used instruments or indicators that they had developed for their study. Most of these were questionnaires, and many used visual analogue scales. In a few cases authors constructed composite indices. One of these indices, for example, was described as “the sum of asthenia + unaided dressing + visits to the shops, etc + outside activities + number of consecutive days spent without leaving home + number of days with a body temperature of 38.5°C or less.”17 In two studies it was unclear how quality of life was measured.
|1 Nottingham health profile (NHP) (6)||1 Psychological general well being index (PGWBI) (6)||1 EORTC QLQ C-30 (4)|
|2 Quality of wellbeing index (QWBI) (3)||2 Lehman QOL interview (2)||2 Spitzer QOL index (4)|
|3 Karnosky performance status (KPS) (2)||3 State trait anxiety inventory (STAI) (2)||3 Functional living with cancer index (FLIC) (3)|
|4 Activities of daily living questionnaire (2)||4 Mental health index (2)||4 Minnesota living with heart failure questionnaire (2)|
|5 Short form 36 (SF-36) (2)||5 Beck depression inventory (BDI) (2)||5 Chronic respiratory disease index questionnaire (2)|
|6 Sickness impact profile (SIP) (2)||6 Hospital anxiety and depression scale (HADS) (2)||6 Angina pectoris QOLQ|
|7 QOLQ adapted from the Dartmouth COOP chart||7 Bf-S wellbeing scale||7 Heart condition assessment|
|8 Functional status questionnaire||8 Connecticut department of mental health QOL survey||8 New York Heart Association functional scale (NYHA)|
|9 LASA general wellbeing scale||9 Psychological adjustment to illness scale (PAIS)||9 Chronic heart failure questionnaire|
|10 Social network questionnaire||10 Hamilton anxiety scale (HAS)||10 Dyspnoea fatigue rating scale|
|11 Functional activities questionnaire (FAQ)||11 Social adjustment scale (SAS)||11 Rhinoconjunctivitis QOLQ|
|12 EuroQol||12 Center for epidemiologic studies depression scale (CES-D)||12 Scott-Huskisson VAS for pain|
|13 Self perceived health status scale (SPHS)||13 Montgomery and Asberg depression scale (MADRS)||13 Skin problem list|
|14 General health questionnaire GHQ-30||14 Body cathexis list||14 The side effects of symptom distress index|
|15 Sheehan disability scale (SDS)||15 Self esteem scale||15 The sexual symptom distress scale|
|16 McMaster health index||16 Inventory list on association with others||16 Sleep dysfunction scale|
|17 Revised symptom checklist (SCL90-R)||17 The Petty questionnaire|
|18 Goteborg QOL instrument (GQL)||18 Subjective chemo IQ (SCI)|
|19 Short comprehensive assessment and referral evaluation (short-CARE)||19 South West Oncology Group (SWOG) questionnaire|
|20 The wellbeing at work or daily routine scale||20 Rotterdam symptom checklist|
|21 The life events index||21 Morrow assessment of nausea and emesis (MANE)|
|22 Physical self maintenance scale (PSMS)||22 The linear analogue self assessment scale|
|23 Modified hamburger visual analogue scales|
|24 Perceived QOL scale (PQOLS)|
No of trials given in parentheses if instrument was used in more than one trial.
—Quality of life was the principal end point in 23 reports. Other end points were symptoms (n=12), disease progression or survival (n=8), exercise capacity (n=5), blood pressure (n=3), pulmonary function (n=2), and 14 other end points (one study each).
—The response rate for quality of life end points was given in 38 of the studies, with response rates ranging from 51% to 100%. In 46 reports the authors clearly stated that the patients provided information on quality of life, whereas in the remaining articles it was unclear to what extent the information originated from patients, carers, or relatives. Complete reporting of all items and scales occurred in 31 trials. Probability values were presented in 52, but only 11 provided confidence intervals. The mean Jadad quality score was 2.68, corresponding to 54% of the maximum score of 5.
Quality of reporting on quality of life was higher in the trials with quality of life as the principal end point (table 3). These trials were significantly more likely to have used an established instrument, to state respondents, to give response rates, to report on all items and scales, and to present absolute differences. Jadad scores tended to be lower for these trials, although this did not reach significance (P=0.15).
Overall, 39 articles reported significantly (P<0.05) more favourable outcomes in terms of quality of life for one or more of the experimental groups. The proportion that did so was slightly higher among the trials with quality of life as the principal end point (16 (70%) v 23 (52%), P=0.17). Sample sizes were larger in studies with quality of life as the principal end point (table 3).
Quality of life is widely accepted as an important end point in clinical trials.1,3,5 The prevalence and quality of reporting on quality of life in clinical research is unclear, however. We examined trends in the reporting of quality of life data in randomised controlled trials during 1980-97 and assessed the quality of such reporting in more detail for 1993-6. Based on the Cochrane Controlled Trials Register, the most comprehensive source of trials available, our results indicate that the prevalence of reporting on quality of life remains trivial. Overall, the proportion of trials reporting on quality of life increased from less than 1% in 1980 to about 4% in 1997. A higher proportion might be expected for cancer considering the nature of the disease and the fact that a number of key funding bodies recently introduced policies to promote the assessment of quality of life in randomised controlled trials.18 However, even among these the proportion reporting on quality of life was less than 10% in 1997. Furthermore, these low levels of reporting of quality of life may be an overestimate as almost a quarter of papers retrieved in our abstract search were considered to be irrelevant.
One reason why relatively few trialists embark on measuring quality of life may be because of methodological difficulties. Of the 67 studies sampled for detailed examination, 48 used 62 different pre-existing instruments and a further 15 studies reported new measures, with few following the methods proposed for the development and testing of instruments.8,19 The need for both generic and condition specific instruments means that a range of measures is required, but it is implausible that some 40 different measures of generic or psychological wellbeing could be justified in 48 trials. Critical analysis of these instruments was beyond the scope of our study, but the dimensions of quality of life assessed as well as the levels of validity and reliability of the instruments varied considerably. Such variations will obstruct comparisons between studies.
Gill and Feinstein examined 75 articles with “quality of life” in their titles.2 Selection was independent of study design, and most studies were published during 1987-91. They focused on the investigators’ definition of quality of life, their reasons for choosing a given instrument, and the importance given to the patients’ own rating of quality of life. Gill and Feinstein showed that only 15% of investigators defined quality of life, only 36% gave reasons for their choice of instrument, and only 13% invited patients to contribute personal assessments. We focused on reporting of quality of life outcomes and found that similar problems apply in clinical trials. For example, not only were patients generally not asked to supplement the questionnaire based data with personal responses, in about 30% of studies it remained unclear whether patients had contributed any information at all. Clearly, some investigators continue to believe that health professionals can make a valid assessment of their patients’ quality of life, in spite of evidence to the contrary.20
Response rates are critical for reporting quality of life, as differential non-response can introduce serious bias. The fact that response rates were unreported in almost a half of the trials must be a matter of concern. Our results indicate that selective reporting of items and scales may also be a problem. We could be assured that investigators had reported all the information collected in less than half of the articles. Selective reporting of favourable or statistically significant results may thus have occurred in a number of articles.
We used the Jadad score16 to examine the quality of reporting of aspects of trial methodology that have been shown to be related to the validity of randomised controlled trials.21 With a mean of 54% of the maximum quality score, our sample of randomised controlled trials is of similarly low quality when compared with randomised controlled trials published in leading English language journals or trials published in languages other than English.11,22 Interestingly, the Jadad score tended to be higher among the reports with quality of life as a secondary end point. Reports focusing on quality of life were more likely to use established instruments, to make clear whether the patient’s view had been sought, and to offer complete reporting of items and scores. There may, however, have been some disregard for the conventions of trial design and reporting among those primarily interested in quality of life.
Our comprehensive analysis of the literature shows that, although it is increasing, the reporting on quality of life end points remains uncommon in randomised controlled trials, and the quality of reporting is often poor. An initiative similar to CONSORT14 that involves trialists, specialists in the measurement of quality of life, and journal editors is required to develop standards of assessing and reporting quality of life in clinical trials. Decades after the invention of the randomised controlled trial, such an initiative may finally lead to trials assessing end points that really matter to patients.
The department of social medicine at University of Bristol is part of the MRC Health Services Research Collaboration.
Funding: This work was partially funded by the University of Bristol.
Competing interests: None declared.