PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Med Care. Author manuscript; available in PMC 2013 June 14.
Published in final edited form as:
PMCID: PMC3682647
NIHMSID: NIHMS460746

Use of Patient-Reported Outcomes in Randomized, Double-Blind, Placebo-Controlled Clinical Trials

Abstract

Background

To optimize the use of patient-reported outcomes (PROs) in clinical research, it is first necessary to review the current use of these outcomes in clinical trials to determine under what circumstances they are most useful, and to reveal current limitations.

Purpose

To investigate current patterns of use of PROs in clinical trials.

Research Design

We conducted a systematic literature review of all double-blind, placebo-controlled, randomized clinical trials using 1 or more PROs as a study outcome from 2004-2006. Data were abstracted and analyzed with descriptive statistics and logistic regression to characterize the use of PROs in clinical trials.

Results

The 180 clinical trials that met the study inclusion criteria employed 173 unique instruments to measure a total of 466 PROs. Most PRO measurements were obtained using relatively few PRO instruments, with one-third of PRO instruments applied in more than 1 trial. In multivariable analysis, tests of statistical significance were more often reported for PROs used as primary trial outcomes. Statistically significant PRO outcomes (P < .05) were more likely among disease-specific PROs compared with general PROs, PROs with a discussion of minimally important difference, and larger trials.

Conclusion

PRO instruments may be improved through efforts to provide centralized electronic administration, cross-validation, and standardized interpretation of clinically relevant outcomes. The majority of PROs used in current clinical trials come from relatively few, commonly used disease-specific PRO instruments within major therapeutic areas.

Keywords: Clinical Trials as Topic, Quality Indicators, Health Care, Quality of Life, Treatment Outcome

Introduction

Patient-reported outcomes (PROs) are increasingly recognized as valuable clinical research end points, and several reviews of their use in clinical trials have been conducted to provide information on current PRO usage, determine under what circumstances they are most useful, and reveal current limitations. Previous analyses of PRO usage have been relatively focused, concentrating on drug approvals,1 labeling claims,2 specific diseases, or single PROs.3 In 2002, Garrat and colleagues4 demonstrated the increasing pace of PRO use in clinical trials. PRO usage has also been examined in disease-specific contexts, including cancer5, 6 and rheumatoid arthritis.7

We sought to provide a general perspective on how PROs have been used in recent randomized, double-blind, placebo-controlled clinical trials across all therapeutic areas to better assess how PROs can best contribute to clinical research.

Methods

The goal of the present work was to characterize PRO instrument use across a wide spectrum of diseases in clinical trials. This study was limited to randomized, double-blind, placebo controlled trials (the gold standard in guiding clinical practice) to provide a generalized survey of PRO usage across multiple therapeutic areas.

We conducted a MEDLINE search of all English-language articles published between 2004 and 2006 using the terms “double-blind,” “placebo,” and “multicenter,” along with one of the following phrases: “quality of life,” “health-related quality of life,” “patient reported outcome,” “patient reported,” “patient identified,” “patient completed measure,” “proxy reported,” “proxy identified,” “proxy completed measure,” “survey,” or “questionnaire.” All randomized, double-blind, placebo-controlled trials reporting one or more PROs as a primary or secondary end point were included in the analysis.

Data Abstraction

Data abstracted from PROs included the following: (1) PRO instrument name; (2) general vs. disease-specific instrument; (3) statistical significance of measure results; (4) therapeutic area being studied; (5) target population; (6) method of administration; (7) language(s); (8) frequency of administration; (9) description of a minimally important difference (MID); (10) description of a prespecified hypothesis; and (11) whether the PRO was a primary end point.

At most, 6 PRO measures were abstracted from each study. Subscales were abstracted if fewer than six PROs were used. A PRO was considered to have reported significance testing or a significant result if any portion or subscale reported it. Application of a PRO instrument within a trial was counted at most once, regardless of the number of subscales abstracted. We defined PRO measure types as general or disease-specific using previous definitions.8, 9 Disease-specific PRO measures assess the special concerns of diagnostic groups9 and only apply to a single or limited spectrum of disease. General PRO measures cover general physical, mental, or emotional well-being and can be applied to different populations across different interventions (i.e., SF-36, HAQ, EuroQOL).9 Instruments that provided either a global assessment or a measure of pain were combined into a composite category labeled “Global Assessment” or “Pain,” respectively. A full list of PROs, including general vs. disease-specific categorization and composite categories, is available online (see Spreadsheet, Supplemental Digital Content 1).

Statistical Analysis

Multivariable logit models were used to assess factors associated with (1) reporting of statistical significance tests and (2) a statistically significant (P < .05) outcome among reported outcomes. Models included PRO as a primary vs. secondary outcome, specific vs. general PRO measure type, therapeutic area of the trial, the presence of a prespecified hypothesis or MID, and trial size (logged). SAS version 9.1 (SAS Institute Inc, Cary, North Carolina) was used for regression analyses.

Results

We included 180 articles in the analysis. The most common therapeutic areas were rheumatology, oncology, neurology, psychiatry, and allergy and asthma. Most studies were conducted in English (83%), French (14%), German (13%), and/or Spanish (9%).

A total of 173 PRO instruments were used (see Supplemental Digital Content), with the average trial using 2.6 instruments. The modal number of instruments varied among cardiovascular trials (single PRO), trials in oncology, allergy and asthma, and pediatrics (2 PROs), and trials in psychiatry and rheumatology (3 PROs). Only 2% of all trials used more than 5 distinct PRO instruments.

The 173 PRO instruments were applied 466 times among the 180 trials, counting applications of multiple subscales or trial arms for an instrument as a single application. PROs were most often applied via single-item scales (38%), surveys (32%), or questionnaires (21%), with others obtained via diaries (9%) or interviews (1%). No measures were reported to have been conducted by phone or electronically. Timing of PRO measurements varied, and 27% of PRO measurements were administered in conjunction with study events instead of at predetermined times (e.g., follow-up).

Relatively few PRO instruments accounted for a disproportionate share of PRO use, particularly among general PRO measures (Table 1). Of all 173 PRO instruments, 58 were used in more than 1 trial. Eight general PRO instruments were used in more than 1 trial, and the 4 most commonly used general instruments (i.e., SF-36, patient global assessments, HAQ, and EuroQol) provided 80% of all general assessments of patient well-being. Fifty disease-specific PRO instruments were used in more than 1 trial and were used to obtain 71% of all disease-specific PRO instrument measurements. The large majority of trials (94%) employed at least 1 disease-specific PRO instrument (Table 2). Just over half of trials (52%) used a general PRO instrument. The most frequently used PROs were disease-specific instruments in therapeutic areas other than psychiatry and neurology.

Table 1
Number of Trials Using Selected Patient-Reported Outcome Measuresa
Table 2
Number of Articles Using Each PRO Subtype by Therapeutic Area

Of 466 PRO measurements analyzed, 126 (27%) were general measures and 340 (73%) were disease-specific measures (Table 3). Of the 466 PROs measurements, 160 (34%) served as primary outcomes. Eleven percent of trials used a PRO as the primary outcome. Few reported PRO measurements were accompanied by a description or mention of a MID (11%) or prespecified hypothesis (5%).

Table 3
Statistical Significance of Patient-Reported Outcomes by Therapeutic Area

The proportion of PRO measurements that were significant, insignificant, or not reported is shown by outcome type in Table 3. Disease-specific measures were more likely than general measures to result in a statistically significant finding (χ2 = 8.9; P < .01). All PRO measurements employed in cardiovascular studies were accompanied by reports of statistical significance, whereas in the other therapeutic areas statistical significance testing for roughly a fifth of PRO measurements was not reported. Among those PRO measurements that were accompanied by reports of statistical significance tests, oncology and pediatrics had the lowest proportion of statistically significant results, whereas allergy and asthma and rheumatology studies had the highest (χ2 = 15.1; P = .02).

In multivariable analysis, increased PRO significance reporting was associated with use of the PRO as a primary end point (Table 4). Lower rates of PRO significance reporting were associated with larger trials, oncology and psychiatry trials, and trials with a prespecified hypothesis. Of reported PROs, significant outcomes were more associated with a discussion of MID, use a of a disease-specific PRO instrument, and larger trials. PRO measures were less likely to result in a significant outcome when used in an oncology trial (all P < .05).

Table 4
Logistic Regression of Factors Associated with Reporting of Statistical Significance Reporting of Patient-Reported Outcomes by Trial or PRO Characteristic

Discussion

In this systematic review of recent double-blind, placebo-controlled, randomized clinical trials, there was wide variability in the therapeutic areas studied, trial size, and trial length, suggesting that the use of PROs is not restricted to a particular subset of clinical research.

The most common therapeutic areas that were found to use PROs in clinical trials were consistent with previous studies of PROs in recent U.S. and European drug approvals,1,2,10 PRO measure development1 and recent specialty-specific reviews of the use of PROs.5-7

Simple scales, questionnaires, and surveys accounted for 90% of measure administration. No instruments were administered electronically, revealing at least one avenue by which PROs use in clinical trials might be improved. Jones and colleagues11 also documented a lack of electronically administered PROs in recent trials, despite their potential for reducing practical limitations in PRO administration while producing equivalent outcomes.12 Web sites such as PatientViewpoint are currently under development that will enable electronic collection of PROs that can be integrated with electronic medical records in the future.13 Seventeen percent of studies were conducted in a language other than English, highlighting the importance of validating PRO measures in multiple languages and cultures.

The use of PROs differed greatly between therapeutic areas. Rheumatology, psychiatry, and allergy and asthma trials overall reported the highest rates of statistically significant outcomes. This might be expected from fields in which standard composite clinical outcome measures incorporate PROs and must use reliable PROs with known end points. In contrast, PROs in oncology studies were often not reported and less likely to be significant when they were reported. A recent review of PROs in oncology trials found that PROs may provide prognostic information apart from traditional objective outcomes,14 suggesting that efforts to improve their performance in oncology are worth undertaking.

A striking finding of the analysis was that the number of unique PRO instruments (173) used in trials approached the total number of trials (180). Despite the large number of PRO instruments used, PRO measurements were concentrated among relatively few instruments, as has been observed in labeling claims.10 Efforts focused on improving the use of these commonly used instruments may provide a more efficient route of advancing PRO use than efforts equally diffused among all known PROs, the majority of which may only be used in a single trial. Examples of such efforts might include attempts to create centralized repositories, perform cross-validation, or provide standardization in regulatory approval processes.

The diversity of PROs might help to explain practical difficulties in reporting of both MIDs and prespecified hypotheses, which were not discussed in the large majority of trials. FDA guidance on the use of PROs in labeling claims has evolved from draft15 to final guidance,16 with the final guidance arguing against the use of MIDs in favor of responder definitions and pretrial establishment of primary outcomes.

Disease-specific measures were more often statistically significant than general measures and were independently associated with significant PRO outcomes in multivariable analysis. This may help explain observations from previous studies that have found that disease-specific PROs are used more often in FDA labeling claims.2

The Patient-Reported Outcome Measurement Information System (PROMIS) Network, a cooperative network funded by the National Institutes of Health, is working to provide a centralized resource that can aid in consistent application, interpretation, and validation of PROs between studies As of 2010, PROMIS item banks had compiled PRO items from close to five hundred English-language questionnaires.17 Although this resource has previously focused on the use of generic rather than disease-specific instruments, we suggest here that incorporation of commonly used disease-specific measures may be an efficient means of covering current PRO use in randomized controlled clinical trials.

Limitations

It is possible that some randomized, double-blind, placebo-controlled, clinical trials using PROs as outcomes were missed during the literature review. To mitigate potential intercoder variability, cross-validation was performed on the first 5 articles. A maximum of 6 PROs were extracted per trial; however, only 2% of trials had 6 or more PROs, suggesting that this approach was adequate to capture the majority of PRO use. Within a trial, application of a PRO instrument was not counted more than once in order to minimize artifacts of extracting subscales from some PROs and not others.

Only placebo-controlled trials were included in the analysis. We reviewed 20 studies that included PRO search terms but were excluded from the analysis by lack of randomized controlled trial search terms and found that excluded studies were almost exclusively observational or single-arm studies. Of 20 randomized controlled trials excluded from this analysis by the lack of PRO search terms, roughly a third were found to use PROs, suggesting that the present study provided a subset of all randomized controlled trials using PROs. Whether these excluded studies are biased by underreporting of statistically nonsignificant results or PROs used as secondary end points remains an area of future research.

Only published trials were considered in this study, which may have resulted in positive result publication bias. Reported statistical significance testing of PROs may be particularly at risk for publication bias, which is greater in studies with flexibility in designs, definitions, and outcomes.18 The effect of publication bias on the use and reporting of PRO instruments warrants investigation.

Conclusion

This study documents the importance of PROs in clinical research and highlights areas for improvement that could contribute to this important facet of the clinical research enterprise. Focusing efforts to improve the use of PROs in commonly used disease-specific instruments may be more efficient that diffusing efforts over a large number of infrequently used PRO instruments, most of which may never be used outside a single trial.

Supplementary Material

Acknowledgments

Supported by grant U01AR052186 from the National Institute of Arthritis and Musculoskeletal and Skin Diseases. The content of this manuscript is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Arthritis and Musculoskeletal and Skin Diseases or the National Institutes of Health.

Footnotes

Complete Author Information

Michaela A. Dinan, BS, Duke Clinical Research Institute, PO Box 17969, Durham, NC 27715; telephone: 919-668-8101; fax: 919-668-7124; michaela.dinan/at/duke.edu.

Kate L. Compton, BA, Duke Clinical Research Institute, PO Box 17969, Durham, NC 27715; telephone: 919-668-8101; fax: 919-668-7124; katebarr/at/email.unc.edu.

Jatinder K. Dhillon, BSPH, Duke Clinical Research Institute, PO Box 17969, Durham, NC 27715; telephone: 919-668-8101; fax: 919-668-7124; jdhillon/at/email.unc.edu.

Bradley G. Hammill, MS, Duke Clinical Research Institute, PO Box 17969, Durham, NC 27715; telephone: 919-668-8101; fax: 919-668-7124; brad.hammill/at/duke.edu.

Esi Morgan DeWitt, MD, Division of Rheumatology, Cincinnati Children’s Hospital Medical Center, MLC 4010, Cincinnati, OH 45229; telephone: 513-636-7212; fax: 513-636-4116; esi.morgan-dewitt/at/cchmc.org

Kevin P. Weinfurt, PhD, Duke Clinical Research Institute, PO Box 17969, Durham, NC 27715; telephone: 919-668-8101; fax: 919-668-7124; kevin.weinfurt/at/duke.edu.

Kevin A. Schulman, MD, Duke Clinical Research Institute, PO Box 17969, Durham, NC 27715; telephone: 919-668-8101; fax: 919-668-7124; kevin.schulman/at/duke.edu.

References

1. Szende A, Leidy NK, Revicki D. Health-related quality of life and other patient-reported outcomes in the European centralized drug regulatory process: a review of guidance documents and performed authorizations of medicinal products 1995 to 2003. Value Health. 2005;8:534–48. [PubMed]
2. Willke RJ, Burke LB, Erickson P. Measuring treatment impact: a review of patient-reported outcomes and other efficacy endpoints in approved product labels. Clin Trials. 2004;25:535–52. [PubMed]
3. Contopoulos-Ioannidis DG, Karvouni A, Kouri I, Ioannidis JPA. Reporting and interpretation of SF-36 outcomes in randomised trials: systematic review. BMJ. 2009;339:a3006. [PMC free article] [PubMed]
4. Garratt A, Schmidt L, Mackintosh A, Fitzpatrick R. Quality of life measurement: bibliographic study of patient assessed health outcome measures. BMJ. 2002;324:1417. [PMC free article] [PubMed]
5. Lipscomb J, Gotay CC, Snyder CF. Patient-reported outcomes in cancer: a review of recent research and policy initiatives. CA Cancer J Clin. 2007 Sep-Oct;57(5):278–300. [PubMed]
6. Gondek K, Sagnier PP, Gilchrist K, Woolley JM. Current status of patient-reported outcomes in industry-sponsored oncology clinical trials and product labels. J Clin Oncol. 2007 Nov 10;25(32):5087–93. [PubMed]
7. Kalyoncu U, Dougados M, Daurès JP, Gossec L. Reporting of patient-reported outcomes in recent trials in rheumatoid arthritis: a systematic literature review. Ann Rheum Dis. 2008 Mar 28; [PubMed]
8. Lubeck DP. Patient-reported outcomes and their role in the assessment of rheumatoid arthritis. Pharmacoeconomics. 2004;22(2 Suppl 1):27–38. [PubMed]
9. Patrick DL, Deyo RA. Generic and disease-specific measures in assessing health status and quality of life. Med Care. 1989;27(3 Suppl):S217–32. [PubMed]
10. Erickson P, Willke R, Burke L. A concept taxonomy and instrument hierarchy: tools for establishing and evaluating the conceptual framework of a patient-reported outcome (PRO) instrument as applied to product labeling claims. Value Health. 2009;12(8):1158–1167. [PubMed]
11. Jones JB, Snyder CF, Wu AW. Issues in the design of Internet-based systems for collecting patient-reported outcomes. Qual Life Res. 2007 Oct;16(8):1407–17. [PubMed]
12. Gwaltney CJ, Shields AL, Shiffman S. Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health. 2008 Mar-Apr;11(2):322–33. [PubMed]
13. Snyder CF, Jensen R, Courtin SO, Wu AW. PatientViewpoint: a website for patient-reported outcomes assessment. Qual Life Res. 2009 Sep;18(7):793–800. Epub 2009 Jun 21. [PMC free article] [PubMed]
14. Gotay CC, Kawamoto CT, Bottomley A, Efficace F. The prognostic significance of patient-reported outcomes in cancer clinical trials. J Clin Oncol. 2008 Mar 10;26(8):1355–63. [PubMed]
15. Food and Drug Administration [May 15, 2010];Guidance for Industry—Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Draft Guidance. 2006 Feb; http://www.fdanews.com/ext/files/06d-0044-gdl0001.pdf. [PMC free article] [PubMed]
16. Food and Drug Administration [May 29, 2010];Guidance for Industry—Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Final Guidance. 2009 Dec; http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf.
17. [Accessed October 19, 2010];PROMIS Home page. Available at: http://www.nihpromis.org.
18. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005 Aug;2(8):e124. Epub 2005 Aug 30. [PMC free article] [PubMed]