Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
J Support Oncol. Author manuscript; available in PMC 2013 July 1.
Published in final edited form as:
J Support Oncol. 2012 Jul-Aug; 10(4): 143–148.
Published online 2012 May 18. doi:  10.1016/j.suponc.2012.02.002
PMCID: PMC3384764


Elizabeth F. Hughes, RN, BSN, CHPN,a Albert W. Wu, MD, MPH,corresponding authorb Michael A. Carducci, MD,corresponding authorc and Claire F. Snyder, PhDb,c,d,*


There is increased interest in using patient-reported outcome (PRO) measures in routine clinical practice to improve patient management. The effectiveness of this intervention may be facilitated by providing suggestions to clinicians on how to address issues identified by the PROs. We sought to develop recommendations for clinicians on how to respond to issues covered by common cancer PRO questionnaires, including functional problems (eg, physical, social, emotional), symptoms (eg, diarrhea, pain), and needs (eg, patient care and support, information). The recommendations would be incorporated into a Web-based system for PRO assessment and reporting in use at our large, academic cancer center. To develop the recommendations, we conducted a multiphase, multidisciplinary, consensus process. We reviewed the literature and conducted one-on-one interviews with experts from various disciplines. Experts included medical oncologists, radiation oncologists, nurses, an internist, a palliative care specialist, an outcomes researcher, a chaplain, a social worker, and patient advocates. These interviews elicited the experts’ recommendations for addressing problems in common PRO domains. Finally, we held a panel meeting attended by all the experts to attain consensus on the recommendations. The final consensus suggestions recommend further assessment of the problem as a first step. Treatment suggestions range from medication adjustments to lifestyle modifications, to referrals to other disciplines. Further research will test whether clinicians find these suggestions useful for patient management.

Keywords: patient-reported outcomes, clinical practice, guidelines, palliative care, symptom management

Cancer patients face a variety of challenges related to their functioning and well-being, and the routine collection of patient-reported outcome (PRO) questionnaires to help clinicians identify and address these issues has been a recent research focus.1 In contrast to the collection of PROs as part of research studies, in which data are aggregated and analyzed without reporting an individual's responses to his or her clinician, the use of PROs in clinical practice involves providing the PRO results for that patient to his or her clinical team. Research investigating the use of PROs in clinical practice has consistently shown improvements in patient-clinician communication2-4 and, in some cases, improvements in patient management and outcomes.5

While initial efforts to collect and use PROs in clinical practice were constrained by logistical challenges, recent technological innovations have largely addressed these barriers.6 Over the past 6 years, an international multidisciplinary team has been working to develop the PatientViewpoint website ( PatientViewpoint is designed for clinicians to assign PRO questionnaires for patients to complete at regular intervals. The patients’ scores are summarized graphically and reported back to the patient and his or her clinical team. In the reports, domains with scores that are either poor in absolute terms or that represent a significant worsening from the previous time point are highlighted to get the clinician's attention. PatientViewpoint is currently being pilot tested with breast and prostate cancer patients at the Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins University.8

One of the barriers to clinicians’ use of PROs in clinical practice is that they may not know how to respond to issues identified by the questionnaires. PROs can assess a variety of topics, ranging from symptoms such as nausea and vomiting to measures of functioning and well-being such as the ability to participate in hobbies. While responding to symptoms is a core focus in clinical training, addressing problems with functioning and well-being is less so. Providing clinicians with guidelines on how to address issues identified by PRO questionnaires may facilitate improvements in care.9,10 Therefore, as part of the development of PatientViewpoint, we sought to include suggestions that clinicians could access when they review the patients’ scores.


In this project, we undertook a multistage, multidisciplinary consensus process to develop suggestions for responding to potential problems identified by PRO questionnaires. Final recommendations are to be incorporated in the PatientViewpoint website, allowing clinicians to click “What can I do?” for domains highlighted as being potentially problematic. Our goal was that the recommendations be brief enough to appear in a pop-up box.

To develop the recommendations, we (1) reviewed the literature for the relevant PRO domains, (2) had individual meetings with experts representing multiple disciplines, and (3) held a consensus development meeting attended by the entire multidisciplinary panel. The PRO questionnaires addressed by this exercise were 6 domains from the Patient-Reported Outcomes Measurement Information System (PROMIS) (physical function, pain, satisfaction with social roles, fatigue, anxiety, depression),11 the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Core 30 (EORTC-QLQ-C30),12 and the Supportive Care Needs Survey–Short Form (SCNS-SF).13,14 As the 3 questionnaires overlap with one another in the topics covered, we mapped the domains across the 3 questionnaires to produce a list of 20 domains: (1) anorexia, (2) anxiety, (3) constipation, (4) depressed mood, (5) diarrhea, (6) dyspnea, (7) fatigue, (8) nausea and vomiting, (9) pain, (10) insomnia, (11) cognitive function, (12) emotional function, (13) financial problems, (14) physical function, (15) role function, (16) social function, (17) sexual function, (18) overall quality of life (QOL), (19) patient care and support needs, and (20) health system and information needs. These domains are commonly covered by cancer PRO questionnaires. While efforts were made to develop recommendations that were generally applicable across cancers, particular emphasis was given to breast and prostate cancers, as we are currently testing PatientViewpoint in these populations.

Literature Review

We conducted a targeted MEDLINE search using key words (eg, neoplasms, breast, prostate, cancer, interventions, supportive care, needs, palliative care, symptom management, QOL) within each domain. Key palliative care textbooks,15,16 websites,17 and their reference sections were also hand searched for additional articles. The literature was evaluated based on the strength of evidence and/or whether the focus of interventions was on QOL. The recommendations for responding to each of the PRO domains of interest were summarized.

Consulting With Individual Experts

The results from the literature review were supplemented by input from interviews with experts representing a variety of disciplines: cancer outcomes research, internal medicine, palliative care, medical oncology (both breast and prostate), radiation oncology (both breast and prostate), oncology social work, psychiatric liaison nursing, triage nursing, clergy, and patient advocacy (both breast and prostate).

During these one-on-one, in-person interviews, the interviewer (Elizabeth Hughes [EH]) provided each interviewee with a brief synopsis of the research study goals and definitions of the PRO domains of interest. Interviewees had the opportunity to review the questionnaire content. With each issue, the interviewee was then asked to think about his/her patients in the clinic. The interviewer probed the expert's perspective on the potential causes for the issue and the expert's best practice(s) for addressing the problem. In responding, we asked the experts to consider not only what they might do to respond themselves, but also how other members of the care team could be involved.

Consulting With a Multidisciplinary Panel

After all the individual interviews were completed, a document was generated that summarized the key recommendations for each domain from each expert's interview, as well as from the literature. We then held a consensus meeting, which was attended by all multidisciplinary experts. At the beginning of the meeting, the goals for the session were reviewed. We also demonstrated the PatientViewpoint website so that all attendees would be oriented regarding how the final results would be applied in practice. We provided all meeting attendees with the detailed summary document so that they could draw on the suggestions from the literature and the expert input from the other panel members, as well as their own recommendations.

During the panel meeting, each PRO domain was discussed individually. First, a draft consensus statement, based on a synthesis of the results from the literature review and individual interviews, was projected on a screen. The meeting participants could then recommend additions, deletions, or modifications to the draft, while referring to the detailed summary document and comments made by other panel members. All panel members had the opportunity to comment on, and suggest refinements to, the draft consensus statement for each domain, with the meeting facilitator (Claire Snyder [CS]) editing the draft in real time on the projected screen. When there were no further comments or suggestions from the panel, the statement was considered finalized, and we moved on to the next domain. The 20 domains were covered during a single 2-hour meeting.


In general, the recommendations that were abstracted from the literature began with an assessment and evaluation of the problem. Treatment suggestions ranged from medication adjustments (eg, opiates, antiemetics) to lifestyle modifications (eg, sleep hygiene), to referrals to other disciplines (eg, social work, psychiatric liaison nurse).

The one-on-one interviews with our experts provided additional recommendations and treatment approaches. While in general the recommendations that emerged from our expert interviews were similar to the those identified in the literature, the experts also made suggestions based on the specific resources available at the Johns Hopkins Cancer Center. The recommendations also reflected the particular expert's background and training; clinicians, for example, were more likely to focus on medical treatment while other experts might focus more on social support, normalizing, and exploring meaning. The experts acknowledged that other members of the multidisciplinary team may be better suited to deal with certain problems. For example, oncologists recommended referrals to social work for some issues (eg, emotional function), and social workers referred back to the patient's physician for others (eg, diarrhea).

The results from the literature review and one-on-one expert interviews were summarized and developed into proposed final recommendations that were available for discussion during the consensus panel meeting. At the panel meeting, the experts modified the proposed recommendations to develop the final consensus recommendations (Table 1). For example, although specific medications were often included in the proposed recommendations based on the literature and interviews, the panel felt it was best to recommend “pharmacologic therapy” generally without naming particular medications. During the panel meeting, a key topic was the experts’ perspectives on why problems would exist with the domains; thus, all of the final recommendations begin with an assessment and evaluation of the problem (eg, its history, acute versus chronic nature, impact on patient QOL). The discussions also highlighted the different approaches taken by the various disciplines to address the identified problems (eg, prescribing medications versus exploring the meaning of issues to patients). The resulting consensus statements sought to incorporate this variety of perspectives, rather than to suggest that there was one “right” approach. Thus, the final recommendations provide a range of suggestions for addressing each of the PRO domains.

Table 1
Final Consensus Recommendations


Clinicians who are presented with their patients’ PRO assessments may fail to act on them because they are uncertain about the most effective action to take. The usefulness of PRO assessments in routine care may thus be facilitated by providing guidance on how to address the issues identified by the questionnaires.9,10 In this project, a multidisciplinary team undertook a consensus process to develop recommendations for addressing problems in common PRO domains. The group achieved consensus with little difficulty. The resulting suggestions have been incorporated in the PatientViewpoint web system, which collects PROs and links the results with the electronic medical record. When viewing the patient results, clinicians can click on the “What can I do?” link to review these recommendations.

The final recommendations emphasize the need to assess and evaluate the history and nature of the specific issue that was identified by the questionnaire, as well as the support systems the patient has available. The patient reports serve primarily as screening tests that are the initial step in a multiphase evaluation. It is critical for clinicians to conduct their own assessments to follow up on those domain scores that were indicated as a potential problem by the PRO, because the estimates offered by individual PRO scores are less precise than aggregate estimates.18 For cases in which the clinicians’ evaluations support the existence of a patient problem, they may consider suggestions covering a full range of responses, from pharmacologic treatments to lifestyle modifications, to referrals to other members of the multidisciplinary team. In several instances, the recommendations focused on listening to the patient and normalizing the patient's experience, since in some cases, these approaches may be the only intervention possible.

The content of the recommendations highlights the importance of multidisciplinary care. In our consensus development process, we obtained input from experts with a variety of backgrounds, and these different perspectives are reflected in the final suggestions. Rather than being competitive, the experts were interested in learning about the approaches offered by other disciplines. That said, some members of the expert panel were more vocal during the consensus development meeting than were others.

These recommendations were developed for our large academic medical center with extensive multidisciplinary resources available. Other settings may not have easy access to social work, chaplains, and other members of the multidisciplinary team. Another limitation of this project is that patients frequently experience a cluster of symptoms, whereas these recommendations were developed for discrete issues. Also, we sought to develop recommendations brief enough to fit into a pop-up box built into the PatientViewpoint website, even while acknowledging that many of the issues assessed by PROs are complex and multifactorial. At the same time, brief suggestions may be more useful than lengthy documents to practicing clinicians.

The suggestions developed as part of this project are meant to complement, rather than replace, the in-depth expertise of the multidisciplinary care team. The panel sought to develop recommendations that respected the expertise of the team by not being too directive, while at the same time not being overly generic and, therefore, of limited use. Of note, clicking on the “What can I do?” link to access the recommendations is optional when clinicians review the PatientViewpoint score reports, so clinicians are not required to consider these recommendations.

Other challenges are likely to emerge during the implementation of the results. A mechanism needs to be put in place to ensure that the recommendations remain up to date and reflect current state-of-the-science care. While consensus recommendations were developed for many of the domains included in commonly used cancer PRO questionnaires, there are other domains included in other PRO questionnaires, particularly regarding disease-specific issues (eg, urinary function). A process for developing recommendations for these additional topics is required. Finally, while we understand that the use of PatientViewpoint extends beyond breast and prostate cancer, and thus tried to make the recommendations generally relevant across cancers, modifications may be required.

Despite these limitations and challenges, the multistage, multidisciplinary, consensus process that was undertaken to develop these recommendations for use in our institution is expected to reinforce the use of PROs as part of routine care for our cancer patients. Ongoing research is investigating the usefulness of these recommendations to clinicians, and the results of these studies will inform future refinements and improvements.


Funding/Support: This research was funded by a Mentored Research Scholar Grant from the American Cancer Society (MRSG-08-011-01-CPPB). Development of the PatientViewpoint has been supported by the National Cancer Institute (1R21CA134805-01A1; 1R21CA113223-01). The funding source had no involvement in study design; data collection, analysis, or interpretation; writing; or decision to submit the paper for publication. We appreciate the participation of the members of the multidisciplinary panel in this effort.


European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Core 30
patient-reported outcome
Patient-Reported Outcomes Measurement Information System
quality of life
Supportive Care Needs Survey–Short Form


Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflict of Interest: The authors report no conflicts of interest related to the subject matter in this manuscript.

Conflict of Interest: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.


1. Aaronson NK, Snyder C. Using patient-reported outcomes in clinical practice: proceedings of an International Society for Quality of Life Research conference. Qual Life Res. 2008;17(10):1295. [PubMed]
2. Greenhalgh J, Meadows K. The effectiveness of the use of patient-based measures of health in routine practice in improving the process and outcomes of patient care: a literature review. J Eval Clin Pract. 1999;5(4):401–416. [PubMed]
3. Valderas JM, Kotzeva A, Espallarques M, et al. The impact of measuring patient-reported outcomes in clinical practice: a systematic review of the literature. Qual Life Res. 2008;17(2):179–193. [PubMed]
4. Marshall S, Haywood K, Fitzpatrick R. Impact of patient-reported outcome measures on routine practice: a structured review. J Eval Clin Prac. 2006;12(5):559–568. [PubMed]
5. Velikova G, Booth L, Smith AB, et al. Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial. J Clin Oncol. 2004;22(4):714–724. [PubMed]
6. Rose M, Bezjak A. Logistics of collecting patient-reported outcomes (PROs) in clinical practice: an overview and practical examples. Qual Life Res. 2009;18(1):125–136. [PubMed]
7. Snyder CF, Jensen R, Courtin SO, Wu AW, the Website for Outpatient QOL Assessment Research Network PatientViewpoint: a website for patient-reported outcomes assessment. Qual Life Res. 2009;18(7):793–800. [PMC free article] [PubMed]
8. Snyder C, Blackford A, Wolff A, Carducci M, Herman J, Wu A. PatientViewpoint Scientific Advisory Board. Feasibility and value of PatientViewpoint: a web system for patient-reported outcomes (PRO) assessment in clinical practice.. Paper presented at: 18th Annual Conference of the International Society for Quality of Life Research; Denver, CO. October 26-29, 2011.
9. Rubenstein LV, McCoy JM, Cope DW, et al. Improving patient quality of life with feedback to physicians about functional status. J Gen Intern Med. 1995;10(11):607–614. [PubMed]
10. Rosenbloom SK, Victorson DE, Hahn EA, Peterman AH, Cella D. Assessment is not enough: a randomized controlled trial of the effects of HRQL assessment on quality of life and satisfaction in oncology clinical practice. Psychooncology. 2007;16(12):1069–1079. [PubMed]
11. National Institutes of Health [August 30, 2011];PROMIS: dynamic tools to measure health outcomes from the patient perspective.
12. Aaronson NK, Ahmedzai S, Bergman B, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst. 1993;85(5):365–376. [PubMed]
13. Bonevski B, Sanson-Fisher R, Girgis A, Burton L, Cook P, Boyes A. Evaluation of an instrument to assess the needs of patients with cancer. Supportive Care Review Group. Cancer. 2000;88(1):217–225. [PubMed]
14. Sanson-Fisher R, Girgis A, Boyes A, Bonevski B, Burton L, Cook P. The unmet supportive care needs of patients with cancer. Supportive Care Review Group. Cancer. 2000;88(1):226–237. [PubMed]
15. Hanks G, Cherny NI, Christakis NA, Fallon M, Kaasa S, Portenoy RK, editors. Oxford Textbook of Palliative Medicine. 4th ed. Oxford University Press Inc; New York, NY: 2010. p. 1666.
16. Lipman AG, Jackson KC, Tyler LS, editors. Evidence Based Symptom Control in Palliative Care: Systematic Reviews and Validated Clinical Practice Guidelines for 15 Common Problems in Patients With Life Limiting Disease. Informa Healthcare USA, Inc; New York, NY: 2009. p. 228.
17. National Comprehensive Cancer Network [September 1, 2011];NCCN Guidelines for Supportive Care: Distress Management.
18. McHorney CA, Tarlov AR. Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res. 1995;4(4):293–307. [PubMed]