Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
J Oncol Pract. Author manuscript; available in PMC 2017 July 26.
Published in final edited form as:
PMCID: PMC5527827

Patient-Reported Outcome Performance Measures in Oncology

Patient-reported outcomes (PROs) such as symptoms, quality of life, and functional status are commonly measured in cancer clinical trials,1,2 increasingly in comparative effectiveness research,3,4 and in routine clinical care for symptom screening and to enhance communication.5,6 There is emerging interest in integrating PROs into the assessment of care quality.7 Historically, patient-reported experience measures, also called satisfaction measures, have been included in performance measurement programs, for example, using the Consumer Assessment of Healthcare Providers and Systems assessment questionnaires. These measures ask patients about their experiences with providers and care delivery processes, but do not ask patients about their symptoms, functioning, or well-being. PROs reflect how people feel, but generally have not been part of performance evaluation.

Last year, the National Quality Forum (NQF), the major US organization that reviews and endorses quality metrics, assembled an expert panel to develop standards around the development of patient-reported outcome performance measures (PRO-PMs).8 A resulting white paper described a pathway for developing such measures toward NQF endorsement,6 and was endorsed by the International Society for Quality of Life Research.9 Recommended steps include:

  • Start with the population or health care context of interest to identify a performance concern or knowledge gap—for example, it is not well understood how effectively patients’ nausea is controlled after administration of emetogenic chemotherapy in the community.
  • Determine which outcomes are meaningful to patients and are changeable/actionable by providers in that context— for example, nausea control.
  • Develop or identify robust PRO measures for those outcomes—for example, nausea questions from the National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) or from the Multinational Association of Supportive Care in Cancer.
  • Determine the PRO-PM—for example, the proportion of patients receiving moderately or highly emetogenic chemotherapy who report nausea that is moderate in severity or worse during the 7 days after chemotherapy.
  • Conduct pilot tests in actual clinics—for example, administration of measures to a small number of patients to assess feasibility, responsiveness, score cutoffs, case mix adjustment strategies, and/or necessary sample sizes.
  • Standardize interpretation and reporting guidelines—for example, specify how to share results with providers, and what thresholds to use for categorizing providers on the basis of outcomes. Identify potential interventions to improve performance, such as site educational sessions.

Building on this model, the American Society of Clinical Oncology (ASCO) Quality of Care Committee formed a work-group to explore the development of PRO-PMs specific to oncology for integration in ASCO quality programs.

The workgroup determined two different approaches to PRO-PMs: first, outcome measurement (eg, the proportion of patients with a particular symptomatic toxicity); and second, process measurement (eg, the proportion of patients in a clinic whose self-reported symptoms were collected and reviewed at visits). Recognizing that this is a nascent field, the workgroup suggested implementation of both types of measures, but with an initial focus on outcome measures.

Two priority areas were identified through consensus. These included pain assessment among patients with bone metastases, and postchemotherapy nausea. Consistent with the NQF approach, criteria for selection of these outcomes included prevalence of the problems, known underdetection by providers despite substantial efforts, actionability, and appropriateness for patient self-reporting. Notably, clinical practice guidelines already exist in both of these areas,1012 with evidence that guideline adherence improves patient outcomes.13,14 Both areas are well recognized as priorities by cancer care providers. The workgroup conducted a review of existing instruments in both areas, cataloging their characteristics and psychometric properties.

For pain assessment, the workgroup achieved consensus that the “worst pain” item from the well-established Brief Pain Inventory is the most appropriate tool for PRO measurement.15 This item is a simple question that asks patients to report their worst pain severity over the previous 24 hours using a numerical rating scale ranging from 0 to 10. There was consensus that the PRO-PM should assess the proportion of patients with radiographically detected metastatic disease in a given practice with worst pain ≥ 4 (a score threshold associated with clinically meaningful pain that interferes with daily activities16). Recent evidence reports substantial numbers of patients with pain above this threshold in routine oncology practice who might benefit from modified analgesic regimens.17

The second PRO-PM is the proportion of patients receiving moderately or highly emetogenic systemic cancer treatment (on the basis of ASCO and Multinational Association of Supportive Care in Cancer guideline criteria) who experience moderate or worse nausea within a week. The workgroup selected the National Cancer Institute’s PRO-CTCAE nausea items to serve as the assessment instrument.10,11 These two items ask patients to rate their worst nausea severity and frequency of nausea during the last 7 days.

Initial pilot testing will explore the feasibility of implementing these PRO-PMs and will generate information about performance of the items, necessary sample sizes, case mix adjustment, and risk adjustment strategies to produce meaningful data. The method of collecting the patient-reported information to optimize response rates is not yet established (eg, paper surveys, automated telephone interactive voice response systems, tablet computers, Web access), and it is unknown whether medical chart information about supportive medications (eg, analgesics, antiemetics) at the patient level would add information to the PRO-PM. These issues will begin to be addressed during planned ASCO pilot testing.

Beyond these initial measures, the workgroup is interested in developing oncology-specific PRO-PMs for additional symptoms or constellations of symptoms as well as functional status. More broadly, there are opportunities to use PRO measures that assess decision quality and understanding of goals of care. Some of these areas are encompassed in the development program for the new Cancer–Consumer Assessment of Healthcare Providers and Systems measurement system, which is being tested elsewhere but is of direct pertinence to ASCO and oncology practice.18

Use of PROs in performance evaluation is closely related to a growing interest in integrating PROs into electronic health records systems and patient portals as a part of routine clinical care. An increasing number of practices are including PROs in the workflow for care processes, particularly those addressing chronic illnesses that impact how patients feel and function.5 Evidence demonstrates that patient reporting can improve communication, satisfaction, and symptom management.19,20 Several large initiatives are already using patient reporting, including a province-wide initiative in Ontario that administers electronic symptom questionnaires to outpatients with cancer21; a Minnesota program collecting patient-reported depression scores by primary care and psychiatric practices across the state22; administration of a symptom and functional status questionnaire to patients enrolled in Medicare Advantage plans23; and a program for universal reporting of symptoms and functional status by patients in the United Kingdom after selected elective surgeries.24

In summary, patient self-reporting affords the opportunity to better understand the impact of care processes on how patients feel. Optimizing how patients feel is a goal of good oncology practice, and therefore is appropriate for measurement to assess quality. Use of PRO-PMs in oncology is just beginning but is likely to become more common as PROs are increasingly integrated into electronic health records, registries, and routine practice workflow.5,25 It is hoped that using these measures to understand and improve the patient experience will lead to more patient-centered care and to better quality overall.


We thank all members of the American Society of Clinical Oncology (ASCO) Patient-Reported Outcomes Workgroup beyond the listed authors. Members include: Cynthia Chuahan, Michael Danso, MD, Amylou Dueck, PhD, Joseph Jacobson, MD, Jennifer Malin, MD, and Matthias Weiss, MD. We also thank Carolyn Hendricks, MD, chair of the ASCO Quality of Care Committee, for her contributions.


Presented at the American Society of Clinical Oncology Quality Care Symposium, San Diego, CA, November 1–2, 2013.

Authors’ Disclosures of Potential Conflicts of Interest

The authors indicated no potential conflicts of interest.

Author Contributions

Conception and design: Ethan Basch, Monika K. Krzyzanowska

Administrative support: Kristen McNiff, Rebecca Brown, Suzanne E. Maddux

Collection and assembly of data: All authors

Data analysis and interpretation: Ethan Basch, Claire F. Snyder, Thomas M. Atkinson, William Wood, Albert W. Wu, Monika K. Krzyzanowska

Manuscript writing: All authors

Final approval of manuscript: All authors


1. Brundage M, Blazeby J, Revicki D, et al. Patient-reported outcomes in randomized clinical trials: Development of ISOQOL reporting standards. Qual Life Res. 2013;22:1161–1175. [PMC free article] [PubMed]
2. US Food and Drug Administration. Guidance for industry: Patient-reported outcome measures—Use in medical development to support labeling claims. Silver Spring, MD: US Food and Drug Administration; Dec, 2009.
3. Wu AW, Snyder C, Clancy CM, et al. Adding the patient perspective to comparative effectiveness research. Health Aff (Millwood) 2010;29:1863–1871. [PubMed]
4. Methodology Committee of the Patient-Centered Outcomes Research Institute (PCORI) Methodological standards and patient-centeredness in comparative effectiveness research: The PCORI perspective. JAMA. 2012;307:1636–1640. [PubMed]
5. Jensen RE, Snyder CF, Abernethy AP, et al. Review of electronic patient-reported outcomes systems used in cancer clinical care. J Oncol Pract. [epub ahead of print on December 3, 2013]
6. Snyder CF, Aaronson NK, Choucair AK, et al. Implementing patient-reported outcomes assessment in clinical practice: A review of the options and considerations. Qual Life Res. 2012;21:1305–1314. [PubMed]
7. Basch E, Torda P, Adams K. Standards for patient-reported outcomes-based performance measures. JAMA. 2013;310:139–140. [PubMed]
8. National Quality Forum. Patient-reported outcomes in performance measurement. Washington, DC: National Quality Forum; Jan 10, 2013.
9. Basch E, Wu A, Moinpour CM, et al. Steps for assuring rigor and adequate patient representation when using patient-reported outcome performance measures (PRO-PMs) Plymouth Meeting, PA: National Quality Measures Clearinghouse; 2013. pp. 1–5.
10. Swarm RA, Abernethy AP, Anghelescu DL, et al. Adult cancer pain. J Natl Compr Canc Netw. 2013;11:992–1022. [PubMed]
11. Basch E, Prestrud AA, Hesketh PJ, et al. Antiemetics: American Society of Clinical Oncology clinical practice guideline update. J Clin Oncol. 2011;29:4189–4198. [PMC free article] [PubMed]
12. Kris MG, Urba SG, Schwartzberg LS. Clinical roundtable monograph: Treatment of chemotherapy-induced nausea and vomiting—A post-MASCC 2010 discussion. Clin Adv Hematol Oncol. 2011;9(suppl):1–15. [PubMed]
13. Gilmore JW, Peacock NW, Gu A, et al. Antiemetic guideline consistency and incidence of chemotherapy-induced nausea and vomiting in US community oncology practice: INSPIRE study. J Oncol Pract. 2014;10:68–74. [PubMed]
14. Mearis M, Shega JW, Knoebel RW. Does adherence to National Comprehensive Cancer Network guidelines improve pain-related outcomes? An evaluation of inpatient cancer pain management at an academic medical center. J Pain Symptom Manage. [epub ahead of print on January 16, 2014] [PubMed]
15. Shi Q, Wang XS, Mendoza TR, et al. Assessing persistent cancer pain: A comparison of current pain ratings and pain recalled from the past week. J Pain Symptom Manage. 2009;37:168–174. [PMC free article] [PubMed]
16. Serlin RC, Mendoza TR, Nakamura Y, et al. When is cancer pain mild, moderate or severe? Grading pain severity by its interference with function. Pain. 1995;61:277–284. [PubMed]
17. Fisch MJ, Lee JW, Weiss M, et al. Prospective, observational study of pain and analgesic prescribing in medical oncology outpatients with breast, colorectal, lung, or prostate cancer. J Clin Oncol. 2012;30:1980–1988. [PMC free article] [PubMed]
18. American Institutes for Research. Assessing the quality of cancer care.
19. Detmar SB, Muller MJ, Schornagel JH, et al. Health-related quality-of-life assessments and patient-physician communication: A randomized controlled trial. JAMA. 2002;288:3027–3034. [PubMed]
20. Chen J, Ou L, Hollis SJ. A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res. 2013;13:211. [PMC free article] [PubMed]
21. Pereira J, Green E, Molloy S, et al. Population-based standardized symptom screening: Cancer Care Ontario’s Edmonton Symptom Assessment System and performance status initiatives. J Oncol Pract. doi: 10.1200/JOP.2014.001390. [PubMed] [Cross Ref]
22. Minnesota Community Measurement. Minnesota HealthScores.
23. Jones N, 3rd, Jones SL, Miller NA. The Medicare Health Outcomes Survey program: Overview, context, and near-term prospects. Health Qual Life Outcomes. 2004;2:33–43. [PMC free article] [PubMed]
24. Devlin NJ, Appleby J. Getting the most out of PROMS. London, United Kingdom: King’s Fund; 2010.
25. Bennett AV, Jensen RE, Basch E. Electronic patient-reported outcome systems in oncology clinical practice. CA Cancer J Clin. 2012;62:337–347. [PubMed]