|Home | About | Journals | Submit | Contact Us | Français|
To analyze randomized controlled trials (RCTs) on pulmonary rehabilitation (PR) to determine whether the patients who complete PR form a representative subset of the chronic obstructive pulmonary disease (COPD) target population and to discuss what impact this may have for the generalizability and implementation of PR in practice.
A review of 26 RCTs included in a Cochrane Review 2007. We analyzed the selection at three different levels: 1) sampling; 2) inclusion and exclusion; 3) and dropout.
Of 26 studies only 3 (12%) described the sampling as the number of patients contacted. In these studies 28% completed PR. In all we found, that 75% of the patients suitable for PR programs were omitted due to sampling exclusion and dropout. Most of the study populations are not representative of the target population.
The RCTs selected for the Cochrane review gave sparse information about the sampling procedure. The demand for high internal validity in studies on PR reduced their external validity. The patients completing PR programs in RCTs were not drawn from a representative subset of the target population. The ability to draw conclusions relevant to clinical practice from the results of the RCTs on PR is impaired.
The primary goal of pulmonary rehabilitation (PR) is to restore the patients to the highest possible level of independent function, and the target population are patients with stable chronic obstructive pulmonary disease (COPD).1–6 PR evidently seems to benefit the patients in terms of quality of life, functional capacity, symptom relief, and reductions in exacerbation of the condition and in the number of days in hospital. PR is therefore recommended in all influential guidelines based on grade A.2,3,6,7
The concept of PR rests on a large number of randomized controlled trials (RCTs) and is defined as “an evidence-based, multidisciplinary, and comprehensive intervention for patients with chronic respiratory diseases who are symptomatic, and often have decreased daily life activities. Integrated into the individualized treatment of the patient, pulmonary rehabilitation is designed to reduce symptoms, optimize functional status, increase participation, and reduce health care costs through stabilizing or reversing systemic manifestations of the disease”.4 Patients must accordingly exercise with a certain intensity, duration and frequency, and they must be well-informed about their disease through interventions such as patient education, together with being taught self-help and coping strategies. In addition, the patients must perform medication and breathing techniques correctly, and they must stop smoking.
Since the 1980s, at least 600 controlled trials on COPD and rehabilitation have been published. During this period, the PR concept has expanded, specific components have been developed, and techniques have steadily become more comprehensive and sophisticated.8–10
PR is recommended in Denmark and it has therefore been suggested that any COPD patient who is registered with severe dyspnea measured with Medical Research Council Dyspnea Scale MRC11grade ≥3 should be offered PR.12 However, a report from the Danish National Indicator Project13 documents that approximately only 60% (CI:59–62) of patients registered with MRC grade ≥3 are offered rehabilitation. Also there is no documentation of number of patients that complete a program or the effects of the rehabilitation. This shows that we do not know who actually completes PR nor who will benefit from it, when it is to be implemented in clinical practice.12
In general, COPD patients have extensive rehabilitation needs because their disease imposes major restrictions on their everyday life;3,4,6 however, poor adherence is common in daily practice as some patients fail to attend programs and others drop out.3,4,6,7,14–18 Our experience is that in order to optimize the resources used on PR, patients are selected so that those who are deemed to have the ability and motivation to complete a PR program are more likely to be chosen for participation than patients with poor motivation. This may entail understandable, but ethically inappropriate inequality in access to health care.
The effect of PR is well documented in RCTs and its rationale can therefore hardly be questioned as RCTs are considered to represent the most scientifically rigorous method of hypothesis testing in order to provide the best evidence.7,19 RCTs must satisfy strict quality criteria and explicit standards regarding patient selection. However, RCTs on PR often fail to adequately discuss their external validity, ie, the ability to “produce unbiased inferences regarding the target population”.19 We may therefore justifiably question whether the reported effects can indeed be generalized to the target population.
Selection in RCTs may take place at three different levels: 1) sampling; 2) inclusion and exclusion; and 3) dropout. At the sampling level, a number of COPD patients are selected among all subjects within a particular population.19 The selection criteria used prior to randomization, ie, when some patients are contacted for screening, and others are not, need to be explicit. At the inclusion/exclusion level, criteria are defined to establish the study population and to homogenize the intervention group and the control group. This level faces the risk that the patients included differ in certain aspects from those who are not included.
At these first two levels, selection is a matter of the investigator’s choice and ideally, information about all patients who are not included must be registered in order to optimize the external validity.
At the third and final level, dropout may contribute to a weakening of the internal validity; ie, dropout may cause, that the observed differences between the compared groups, may not be attributed only to the hypothesized effect under investigation.19
The challenges involved in achieving high internal and external validity makes selection of patients for PR a pertinent issue. Patients, clinicians and decision-makers need clear messages about the evidence of PR to accept its widespread application and to ensure that scarce resources can be used to good effect. Dropout is usually well-described in RCTs on PR, but information on selection performed during sampling, ie, before randomization, remains sparse. There would therefore seem to be some room for strengthening the discussion of the validity of RCTs in general, and of their external validity in particular by examining pre-randomization selection issues at greater depth.
The aim of the present study is to analyze RCTs on PR to determine whether the patients who complete PR form a representative subset of the COPD target population and to discuss what impact this may have for the generalizability and implementation of PR in practice.
A literature review of the RCTs originally included in the Cochrane review7 published in 2007 endevoured “to establish the influence and the effect size of pulmonary rehabilitation on health related quality of life, functional capacity in patients with COPD”. The review stated that “Rehabilitation relieves dyspnea and fatigue, improves emotional function and enhances patients’ sense of control over their condition”. This review was chosen for this present analysis because it strived to comprehensively identify and synthesize all the literature on PR, and it is in general, well-reputed and often cited.
The Cochrane review included a total of 31 RCTs of which 26 full-text English language versions were examined. Five studies were not examined; three studies were only available as English abstracts as the articles were published in Spanish,20 French,21 and Chinese.22 Boxall23 was not used in the form, ie, congress abstract, in which it was used in the Cochrane review, it was instead published as an article in 2006.24 Chlumsky 200125 was not found. Casaburi26 was not included in this analysis as its focus on testosterone supplementation was deemed irrelevant to the present purpose.
The 26 RCTs were analyzed with regard to their description of sampling, inclusion and exclusion criteria, and dropout illustrated in Figure 1.
Correlation analyses were performed to examine possible associations between selection criteria, disease events, the rehabilitation program, and the number of patients left out.
The results of the analysis are described in three sections corresponding to the three levels: 1. sampling, 2. inclusion and exclusion, and 3. dropout.
Only three (12%) out of 26 studies described the number of patients contacted (Table 1).27–29 In these studies, a total of 322 patients were contacted and out of those 151 (47%) were left out without being screened.
Details regarding the circumstances under which the studies were carried out revealed that Jones and colleagues28 searched computerized records to identify regular attenders at their clinic. Bendstrup and colleagues27 invited patients who were chosen from hospital records, however, the total number of relevant records was not described. Ringbaek and colleagues29 contacted the patients with moderate COPD, who were recruited from an outpatient clinic during a six-month period, however, the total number of relevant records was not described.
In brief, the size of the total COPD population, which is relevant for the external validity of the studies reviewed, is largely unknown and only a few studies were explicit about the characteristics of the populations from which the study populations were drawn. Nearly half of the patients contacted were not offered screening, and only one third of the patients contacted actually completed the PR program.
Information regarding the number of patients screened and the number of patients de-selected is mainly due to inclusion and exclusion criteria, which was stated in the three above-mentioned studies (Table 1) and in five other studies (Table 2).18,33,36,39,40
In all, eight studies screened a total of 1,040 patients of whom 406 (39%) were de-selected before randomization, and the de-selection percentage ranged from 8%–64%.
Reasons for leaving out some patients were, eg, those who lived too far away or whose social circumstances affected their ability to complete the program.18 Only patients who had the ability to travel independently to a physiotherapy practice and the motivation to improve self-care were chosen for participation.33
A study recruited patients by means of announcements and some were screened over the telephone, however, neither the exact number of responders nor the number of telephone screenings were reported.36
In general only stable patients were randomized. The majority of the studies (18 out of 26) were not explicit regarding the number of patients deselected due to exclusion criteria.
The correlation between the proportion of patients left out from screening to randomization and the total number of patients screened showed a correlation, the Spearman’s correlation coefficient (0.643) tended to be significant (P = 0.086).
This indicates that the higher the number of patients screened the stricter the screening procedures were. There was no statistically significant correlation between the number of patients left out from screening to randomization and the number of diagnosis-specific inclusion and exclusion criteria (P = 0.9) nor the number of non-disease-specific inclusion and exclusion criteria (P = 0.3).
The exclusion criteria varied. Up to 6 pulmonary disease-specific exclusion criteria were used in the studies, eg, lung function, dyspnea, arterial blood gases, oxygen saturation, and smoking. One to seven non-pulmonary disease-specific criteria were used in the studies, eg, ischemic heart disease, cognitive impairment, musculoskeletal disorders, social circumstances, transport difficulties, or language barriers.18,33
To summarize, the wide range (6%–64%) of patients left out from screening to randomization could not be explained by the number of explicit exclusion criteria. The analysis showed that a total of 8 (31%)18,27–29,33,36,39,40 of the 26 articles documented the sampling procedure from screening to randomization. The rate of patients completing a PR program out of the total number of patients screened ranged from 32% to 100%.
Dropout was described in all 26 studies (Table 3).18,24,27–34, 36–46,48–52 Dropout ranged from 0% to 36% in the intervention groups (mean = 17[CI: 12–22])) and from 0% to 54% in the control groups (mean = 15 [CI:9–22]). We found no differences in dropout between the intervention groups and the control groups (P = 0.4), and the correlation analysis showed a statistically significant, positive correlation (Spearman 0.72, P = 0.00).
The reasons for dropout can roughly be divided into two categories: “disease-related reasons,” eg, exacerbations, illness and death, and “other reasons”, eg, lack of time, motivation or cooperation (ie, patients did not want to participate, travelling difficulties, the PR-programme was to hard). In 17 out of 26 studies, “other reasons” for dropout were described.
None of the studies discussed the possibility that dropout may cause misclassification,53 or the direction of possibly biased estimates.
Altogether we found, when the size of the unknown total population was ignored, that on average, three quarters of the patients most likely suitable for PR seemed to have been de-selected, probably in a biased way, due to sampling, exclusion criteria, and dropout. None of the studies discussed generalizability and applicability.
The present study aimed to determine whether the patients who complete PR form a representative subset of the target population. This study details aspects of patient selection for RCTs based on the sampling procedures described in the RCTs on PR included in a Cochrane review. The main result of the study is that most RCT study populations are not sufficiently representative of the COPD target population. This seriously affects the external validity of these studies and may inhibit the implementation and effects of PR in clinical practice.
The discussion is divided into four parts. The first part concerns the target population and the following three parts discuss the different levels of selection as illustrated in Figure 1.
For the sake of generalizability the study population must be drawn from a representative subset of the target population,53 which, for PR, would comprise patients diagnosed with stable COPD. However, the target population is not easily determined as the COPD prevalence is generally difficult to estimate.54–56 Firstly, population-based estimates of COPD prevalence are complicated by the variety of tools and definitions used to describe COPD.56 Secondly, COPD terminology is inconsistent and widely accepted diagnostic standards are lacking, therefore COPD coding is insufficient and COPD data often inaccurate.54 Thirdly, the method by which prevalence is estimated (expert opinion, patient reporting, symptom reporting or measurement by spirometry) influences the reported prevalence estimates.55 Furthermore patients who participate in RCTs on PR are selected among patients already in clinical practice, eg, clinic, hospital or by GP, and not amongst patients identified from prevalence studies. For these reasons the exact target population in RCTs remains more or less unknown. None of the studies in the Cochrane review discussed whether the study population was a representative subset of the target population.
Most studies failed to describe which patient were selected and those who were not; so information is lacking on the number of patients left out and their characteristics. Only three (12%) of the included studies were explicit about who were contacted.27–29 We have estimated that almost half (47%) of the patients were left out before screening.
Of particular concern is that none of the studies were explicit about the selection of cases and the bias this may cause. We therefore cannot be sure that the study populations were representative subsets of the target populations studied in relation to the effect of PR. This obviously leaves some severe difficulties in generalizing the findings, both in terms of capability to complete, and on the effects of participating in PR.
Representativeness should be ensured by randomly selecting the study population so that, ideally, all patients relevant for a study would have the same chance of enrolment. This requires that the investigator controls the target population and pays attention to any difference between the patients who were selected and those who were not.53 During this selection process, patients who, eg, were deemed not to have the ability to complete the programs, who lived too far away, or who had difficult social circumstances18 would be at risk of being left out. Such left out patients would likely differ from those who were included in relation to important variables, eg, incriminating psychosocial situation.
Only in 8 of the 26 studies was number of patients deselected during screening explicit (Tables 1 and and22).18,27–29,33,36,39,40 Approximately one third of the patients were left out. No correlation was found between the number of patients lost from screening to randomization and the number of inclusion and exclusion criteria; therefore the number of patients left out cannot be directly explained with reference to these criteria. The study populations must be pathophysiologically uniform to optimize study power and therefore strict exclusions criteria must be used. This premise is obvious but there might still be a risk that nonexplicit criteria are responsible for the number of patients left out. This complicates the implementation of PR, as it is not known whether the findings can be applied to the population which is to benefit from PR.
We assume that the reviewed RCTs a priori recruited the most motivated patients to maximize completion and compliance with the intervention.18 Our analysis revealed that those who declined the intervention stated a variety of reasons, eg, skepticism as to their ability to attend and to adhere to study requirements, problems of transportation. Including smokers in PR programs is often conditional on their participation in a smoking cessation program, and smokers are therefore less likely to participate.17
In summary there is a risk that patients selected for participation differed from the deselected patients in relation to social factors, co-morbidity and their general health status. The selection may not have been truly random, and therefore skewed by explicit as well as non-explicit choices. Implicit criteria hinder the possibility that effects, as documented in RCTs, are to be reached, when PR is implemented in the target population.
In accordance with the Helsinki declaration, dropout was described in all 26 studies. Some dropout should be expected in PR due to the natural history of COPD and therefore selection is usually carried out to reduce dropout.
Our analyses showed huge variation in dropout rates (ranging from 0% to approximately 50%). Moreover, a strong positive correlation was observed between dropout in the intervention groups and in the control groups, which means that only a small part of the dropout can be explained by the intervention. Contrary to expectation, the number of explicit exclusion criteria and the duration of the intervention did not explain the wide variation in dropout.
The patients’ own views and experiences may influence dropout, (eg, if the patients think that the programs are either, too hard, too demanding, too difficult or too easy, unlikely to be helpful, and a waste of time), in which case they will be more likely to drop out. If the patients do not feel comfortable or unsafe and not at ease with the health professionals, or with the other patients, they may also tend to drop out. Whereas the support of family and peers may enhance patient adherence. Besides, the patients’ perception of their illness and its management may have an influence.17,57,58 If the rehabilitation program makes common sense, in relation to the patients’ personal beliefs about their illness, completion may be more likely to be achieved. Empathic understanding and the practitioner relationship may also have some impact.
In general, the differences between completers and dropouts are unknown and it is difficult to determine, how dropout has affected the results.
The question raised in this study is whether the findings of the RCTs and the recommendations that emanate from the Cochrane review7 apply to the entire PR target population. Our analyses show uncertainty regarding the representativeness of the COPD patients who completed the PR programs.
If PR, based on RCTs, is only applicable to a limited subset of the target population, it may lead to social inequality. Patients with severe disease may be too physically impaired to actively participate in and benefit from a program. Patients with a mild disease and minimal limitations may not benefit from the program because of a lack in perceived need and motivation.59 Patients at different stages of COPD have different needs, as the management of mild and moderate COPD involves the avoidance of risk factors in order to prevent the progression of the disease and pharmacotherapy to control symptoms. Severe and very severe COPD require integration of several different disciplines, a variety of treatment approaches, and continued patient support.1
PR is a recommended standard of care that encourages patients to undertake their own health care and to become less dependent on health professionals and expensive medical resources. PR focuses on reducing disability from the disease despite the severity. For these reasons, it is problematic that the evidence for PR rests on studies where the study population is not representative, eg, that COPD patients with co-morbidity and different social circumstances are excluded.
In conclusion, the RCTs selected for the Cochrane review comprised information about the included patients and dropouts, though not about the sampling procedure. The internal validity was assessed in order to examine the relationship between the intervention and the treatment effect. The demand for high internal validity in studies of the effect of PR reduced their external validity, which was not assessed.
The patients completing PR programs in RCTs were not drawn from a representative subset of the target population. A number of criteria which were not explicit were used during the sampling. The studies did not meet the ideal demands for representativeness, which should be obtained by random sampling, and for equality of opportunity to participate for all patients. This impairs our ability to draw general conclusions relevant to clinical practice from the results of the RCTs on PR.
To strengthen the external validity from studies on PR, there is an extensive need for explicitness in all levels of patient selection. Studies on PR in nonselected target populations drawn from prevalence studies are needed in order to determine the number of completers and the effects gained. In addition, studies focusing on potential differences between COPD patients who complete, drop out or de-select rehabilitation are relevant when discussing how PR can be implemented to a larger proportion of the target population.
The authors have no conflicts of interest relevant to the outcome of this work.