Several disease-specific HRQoL instruments have been developed over the past 2 decades to better understand and quantify how COPD affects patients' lives, and to evaluate the effectiveness of treatments designed to manage this often fatal disease.^2–15 Of these, the Chronic Respiratory Disease Questionnaire (CRQ) has consistently provided the most responsive measurements of change in individual patients' HRQoL management.^16–20 In an effort to understand the changes over time that this instrument captures among patients with chronic airway limitations, the developers defined minimal, moderate, and large differences in the CRQ domains.²¹ They defined a minimal clinically important difference (MCID) as

Their research design, consisting of an evaluation of the mean domain change within patients anchored on a global assessment of change and a consensus panel of persons familiar with the patients and the instrument, has been replicated by other investigators seeking to understand meaningful intra-individual differences in HRQoL measures.^22–24

This approach, however, has been criticized on several grounds. The patients' global change ratings were completely subjective, were based on a single item, and were not evaluated with regard to test-retest reliability.²⁵ In addition, the change levels (minimal, moderate, and large) were defined somewhat arbitrarily and were based upon an untested assumption that increases and decreases were symmetric, i.e., that a given level of change (e.g., minimal) can be identified by the same benchmark whether it is an improvement or a decline. Other investigators have questioned the small developmental samples in this study that required the pooling of COPD patients measured by the CRQ with cardiac patients involved in a similar evaluation who were measured by the Chronic Heart Failure Questionnaire (CHQ).^26,27

The results from the original CRQ consensus panel have also been challenged.^26,27 That panel was from a single clinical site, and the published report did not specify the clinical expertise of panel members. In addition, although the CRQ and CHQ instruments are very similar, the results of the consensus panel also involved pooling changes for both the CRQ and CHQ instruments, which assumes that the same change standards should apply to both instruments and both conditions. Recognizing that there were few “clinical” aspects to this investigation, the developers of the CRQ change standards subsequently substituted the term “minimal important difference” for the MCID.²⁸

Despite the many problematic aspects of the CRQ developmental studies, quantifying clinically important increments in HRQoL measures remains an important issue if these instruments are to be routinely and comfortably used by clinicians to enhance their estimation of a COPD patient's disease severity and assessment of the impact on patient care, and ultimately, to improve patient outcomes. Other investigations that tie HRQoL change scores to external evaluations (anchors) of change, and distribution-based studies that use statistical parameters (like the standard deviation) to evaluate individual and group HRQoL changes have appeared in the literature as researchers and practitioners continued to seek standards for assessing important HRQoL differences in COPD-specific measures.²⁹ However, no clinically important change standards have been established for generic HRQoL measures, such as the Medical Outcomes Study Short-Form 36-Item Health Survey (SF-36),³⁰ the most widely-used HRQoL instrument in the world³¹ and one that is frequently used with COPD patients. Although no formal assessments of MCIDs have been performed for the SF-36, studies of cross-sectional differences between clinically defined patient groups have suggested a 3- to 5-point change on any SF-36 scale as clinically important.³²

This report attempts to fill these gaps in the literature by detailing the highly structured process and the resulting clinically important change thresholds for COPD patients established by an expert panel of physicians for the CRQ and the SF-36. Using the consensus development process advanced by researchers at the RAND Corporation, we empanelled 9 physicians from across North America. Each participating physician was experienced in using the CRQ or the SF-36, in addition to other HRQoL questionnaires. The goal of the panel was to reach consensus on how much change in each domain of these 2 instruments signified small, moderate, and large clinically important differences (CIDs), either improvements or declines, for individuals with COPD.

To construct these patient scenarios, we identified 15 patients enrolled in the above-mentioned study²⁶ who had 2 clinical encounters (office visits, etc.) on the day or very close to the day of their HRQoL telephone interviews, which were scheduled every 6 months regardless of any clinic visit schedules. After examining the HRQoL scores of these patients, we focused on the 9 patients who had changes in most of their HRQoL domains between visits. For these patients, we pulled the medical charts to obtain the visit-specific discharge notes and orders. Complete medical chart data were found in a timely manner for 6 of these patients. Blinded to the patients' HRQoL data, 2 physicians (WMT and KK) reviewed these 6 medical charts and reported lung disease and comorbid diagnoses, current medications, history, physical examination results, treatment plans and changes in treatment for the 2 clinical encounters in questions for each patient. The clinical report for each patient was presented to the panelists at the beginning of the patient scenario, followed by the patient's CRQ and SF-36 domain scores from the telephone interviews on or near the date of the clinical encounters, as well as the change score for each domain. The 6 retrospective change scenarios are presented in Appendix A, available on the Journal's web page (http://www.blackwellscience.com/jgi).

Employing a 2-round Delphi process,³⁷ panelists completed 2 tasks using these materials in Round 1. First, they read and reviewed the selected literature and the actual patient scenarios and indicated what they believed constitutes a minimal, moderate, or large clinically important change (improvement or decline) in HRQoL within each domain of the CRQ and the SF-36. Second, they indicated additional pertinent literature that should be included in the next mailing to all panelists.

The panel then discussed the course they would adopt for assigning final values to each scale. It was unanimously agreed that the CIDs should be symmetric so that positive and negative changes in the same category (e.g., a small decline or a small improvement) would uniformly be of the same magnitude. It was also unanimously agreed that the increases from small to moderate and from moderate to large should be arithmetic, increasing incrementally by the value of a small change. Furthermore, there was general agreement that it was usually best to err on the side of accepting a smaller change as important rather than overlooking an important change in an individual patient. Finally, it was noted that, ideally, the smallest CID should be greater than the intrinsic variability of a domain or scale.

The group began with the dyspnea dimension of the CRQ. They first reviewed the results of their earlier Delphi rounds and information about the statistical properties of the scale in several samples of COPD patients. They also considered the number of items in the domain (i.e., 5) and the number of those items that would have to change by the minimal amount to be considered clinically important. For all CRQ dimensions, 1 point represents the smallest possible score change on any item in the domain. The panel came to adopt the term state change to describe this minimal movement on a single item. After extensive discussion, the panel reached a consensus for the dyspnea dimension: 3 points represented a small CID, 6 points a moderate CID, and 9 points a large CID. The group then proceeded to apply this process to determine the CIDs for the remaining 3 dimensions of the CRQ (Table 1). For several of the dimensions, however, the panel deliberated with considerable discussion before achieving a consensus.

Our panel's levels for detecting CIDs on the CRQ were on average slightly higher than previously investigated change levels for the CRQ based on patient-perceived differences. In the 1989 study by Jaeschke et al., an average change per item of 0.5 was considered a minimal (small) important difference in the dyspnea, fatigue, and emotional function dimensions of the CRQ.²¹ When multiplied by the number of items in each dimension, this equates to small CID thresholds of 2.5, 2.0, and 3.5, respectively, in dyspnea, fatigue, and emotional function. In contrast, this expert panel reached consensus on small CID thresholds of 3, 2, and 5, respectively, which are attainable integer change scores for individuals. However, similar to the Jaeschke et al. results, the panel incremented these small CIDs by multipliers of 2 and 3 to set the moderate and large CID standards.²¹ The panel also set CID levels for the mastery dimension, which had been excluded from the Jaeschke et al. investigation because of the lack of a similar domain in the simultaneously studied CHQ. When we applied these new standards for the evaluation of CRQ dimensional change scores among 393 outpatients with COPD in the a previous HRQoL Study²⁶ and compared the classifications (no change, small, moderate, or large) with the cutpoints reported by Jaeschke et al., 51 (13%) of these outpatients were classified differently in the dyspnea dimension, while 107 (27%) had different classifications in the emotional function domain.

Determining CIDs for the SF-36 scales proved to be a more challenging task for the panel that was complicated by the instrument's differing scale increments (i.e., the numeric value of a state change). Nonetheless, the panel adopted an informed and practical approach by considering the possible score changes that can result on each SF-36 scale (noted in the state change column of Table 2). They then compared these state changes to their own experiences with COPD patients, as well as to distribution-based methods that have been used to interpret HRQoL change, and agreed on the reported levels. The magnitude of these change levels (≥8.3 points) should shed light on the widely held but poorly substantiated belief that a 3- to 5-point change on an SF-36 scale indicates a clinically important intra-individual difference. This belief stems from a 1989 report of SF-20³⁹ cross-sectional data collected from a large study of patients with and without chronic health conditions.³² In those data, the predicted average on the health perceptions scale (mean = 69.1) for the hypertension group (N = 2,708) was 3.5 points lower and statistically different (P < .001) from the respective scale average (mean = 72.6) for those participants with no self-reported chronic conditions (N = 2,595), after controlling for age, gender, education, and income. The “significance” of this cross-sectional difference, however, was driven by very large sample sizes, and was not determined using data that assessed patient change over time.

It is important to note that we did not provide definitions of clinically important differences or the magnitudes of change (small, moderate, and large) to the panelists. Instead, we hoped that the physician panelists, each experienced in the use of these measures, would come to their own understanding of these terms. Although prior work by Jaeschke et al. attempted to define a minimally clinically important difference, that definition has some shortcomings.²¹ First, it directly speaks to score differences that are perceived beneficial to the patient, but does not address detrimental changes. Even if one considers an appropriate interpretation of this definition to include avoiding deterioration as a benefit, the patient with no perceived change is not properly recognized or classified. Second, even when the clinician and the patient agree that an important improvement or decline in HRQoL has occurred, this may not necessarily “mandate a change in the patient's management,” especially if such a change would be detrimental to the care of the patient,²⁴ or if all clinical options have been exhausted. We were pleased that our panelists, who are experienced in both providing clinical care for and performing research among patients with COPD as well as being experienced in the use of these HRQoL measures, came to their own interpretations of CIDs and the magnitude of those differences. Moreover, their interpretations converged when the panelists quantified these changes in the CRQ and SF-36 domains.

Despite the extensive efforts we undertook to obtain reliable and accurate results in this study, potential shortcomings remain and warrant mention. First, we employed only a single panel of physicians. Studies by RAND with several physician panels demonstrated substantial heterogeneity in their assessments of the appropriateness of different medical procedures.⁴⁰ However, the panel employed in the present study was comprised only of physicians who had substantial clinical and research experience with the task at hand, while this was not the case in the RAND studies. Second, it is possible that panelists were biased by the results of prior research and might have arrived at different conclusions had they been unfamiliar with that literature. Using experienced physicians, such exposure to earlier studies was unavoidable. Moreover, the widely varying approaches employed by different panelists at the start of the process strongly suggest that few were biased in this fashion. Third, when there are limited clinical data, clinicians may require a greater change in the data that are available before judging that a clinical change has occurred. That is, when clinical information is minimal, the recognition of clinical change will be conservative. And finally, although we made every effort to maintain the integrity of the consensus process, the possibility remains that some members of the panel succumbed to well-known influences of group process.⁴¹ Coupled with this, there was no measure of the reliability of the panelists' judgments about clinically significant change. While the expert panel process is designed to add validity to such judgments, we cannot report a measure of actual agreement.