Search tips
Search criteria 


Logo of jopHomeThis ArticleASCO JOPSearchSubmitASCO JOP Homepage
J Oncol Pract. 2011 November; 7(6): 366–370.
PMCID: PMC3219462

Palliative Care in the Outpatient Oncology Setting: Evaluation of a Practical Set of Referral Criteria

Paul A. Glare, MBBS, MM, FRACP, FACP, Deborah Semple, RN, Stacy M. Stabler, MD, MPH, and Leonard B. Saltz, MD



The National Comprehensive Cancer Network's (NCCN) palliative care (PC) guidelines recommend screening all patients for PC issues at the first visit with their oncologist and at subsequent, clinically relevant times. Criteria for referral to a PC specialist are also part of the guideline. We evaluated the feasibility of screening for PC needs in outpatients attending one GI oncology practice specializing in colorectal cancer and neuroendocrine tumors.


All patients attending clinic during a 3-week period were screened by the office practice nurse (OPN). A one-page screening tool based on the NCCN guidelines was developed, giving a score from 0 to 13. The OPN also evaluated the NCCN referral criteria.


There were 152 clinic visits by 125 patients during the surveillance period; 119 were screened. Median age was 61 years; half were male. Eighty percent had colorectal cancer, and two thirds had advanced disease. Screening took approximately 3 to 5 minutes per patient. Depending on the PC definition used, between 7% and 17% of patients screened positive; all met the NCCN referral criteria. Psychosocial distress was commoner than physical symptoms. The maximum screening score was 8. A cut point of 5 had the best predictive value for specialist referral, and would result in 13% patients having PC consultations.


Screening identified 7% to 17% of patients in this busy clinic as having PC issues and 13% who might benefit from specialist referral. But even this simple method adds considerably to the clinic workload, so the benefits of implementing routine screening need to be carefully evaluated.


Palliative care aims to relieve suffering and improve the quality of life for patients with advanced cancer and their families. Unlike hospice, palliative care is provided as part of acute care, simultaneous with all other treatments. Which cancer patients would benefit from referral to a palliative care specialist? In the past, referral has been the prerogative of the attending oncologist. More recently, other approaches have been proposed.1 Early blanket referral to palliative care has been shown to improve outcomes, at least in ambulatory patients with lung cancer.2 But having every single patient with advanced cancer see a palliative care specialist may not be efficient or effective. A better approach would be to base referral on an assessment of need. This is still a challenging task for palliative care.3,4 Assessment can rely on clinical judgment or utilize a tool,5 However, the tools that are available aim to provide a comprehensive evaluation,6 but are not suitable for rapid screening of large numbers of patients in a busy outpatient oncology setting.

Criteria have also been proposed to identify patients with more complex palliative care needs for whom referral to a specialist provider is recommended.7,8 As yet there are no data regarding the validity of these criteria, the numbers of patients meeting them, or the benefits of following this approach. We recently completed a quality improvement project on the GI oncology floor at Memorial Hospital which showed that the majority of hospitalized patients with advanced GI malignancies met the National Comprehensive Cancer Network (NCCN) criteria for referral to a palliative care specialist.9 The aim of this pilot project was to explore the implementation of the NCCN screening and referral criteria in an outpatient GI oncology practice.


Population and Setting

All patients attending one GI oncologist's outpatient clinic at a Memorial Sloan-Kettering Cancer Center outpatient facility in Manhattan on six consecutive clinics (two clinic days per week) in late February/early March 2011 were screened. Patients were classified as palliative care cases or not according to a narrow and a broad definition of palliative care. The narrow definition required patients to have metastatic or locally advanced cancer, a limited prognosis, an active source of suffering (pain, other physical symptoms, or psychosocial distress). The broad definition required only metastatic or locally advanced cancer, a limited prognosis, or an active source of suffering.

Study Measures

The NCCN's palliative care screening items and referral criteria were used.7 To operationalize them, the screening items were converted into a scored tool. This screening tool was based on an existing palliative care screening tool,10 adapted for use in the ambulatory oncology setting. The adapted tool is a five-item checklist, with a possible score ranging from zero to 13 (Table 1). Screening was performed by the office practice nurse (OPN), a registered nurse who follows patients between visits as well as in the office, so she is familiar with their cases and concerns. To complete screening, the OPN relied on her knowledge of each patient's situation and used the information available to her in the clinic as part of standard care (eg, home medicine reconciliation list, review of systems). She did not need to interview the patients or access their medical records to screen them. She did occasionally consult with the oncologist when specific information such as performance status or extent of disease needed to be clarified or confirmed.

Table 1.
Five-Item Palliative Care Screening Tool

The NCCN's palliative care referral criteria were also operationalized by converting them into a 24-item checklist. The checklist was also completed by the OPN, but only in the subset of patients who met the broad definition of a palliative care case stated above.

This was a pilot project to explore the feasibility and potential for screening for palliative care needs in the ambulatory setting. It was an extended cross-sectional survey using a convenience sample. No identifying data were collected, and the patients did not self-report any data. The results of screening by the nurse were not shared with the oncologist, and were not used to influence any clinical decisions regarding patient care. Therefore, no institutional review board or privacy board approval was deemed necessary.


The clinic population is described in terms of the extent of disease, the prognosis, and the presence of palliative care needs. Simple epidemiological measures were calculated to evaluate the screening tool. These included the sensitivity (SN), specificity (SP), positive predictive value (PPV), and negative predictive value (NPV) for identifying which patients would meet the broad definition of palliative care.


Clinical and Demographic Characteristics of Patients in the Clinics

There were 152 visits made by 125 patients. Their median age was 61 years (range, 24 to 88 years), and 64 (51%) were male. Eighty percent had colorectal cancers; the remainder had carcinoid/neuroendocrine tumors, small bowel cancers, or other GI malignancies. The median time since diagnosis was 20 months (range, 3 months to 12 years). There were 10 (8%) new visits; the remainder had been followed by the oncologist for a median of 7 months, and more than 2 years in 40% of patients.

Disease status included 42 patients (34%) with no or limited disease, who were undergoing adjuvant treatment or were under expectant observation. Eighty three (66%) had locally advanced or metastatic disease. Of these, 53 (64%) were receiving chemotherapy. The goal of treatment was palliative in 80%, neoadjuvant in 14%, and phase I trials in 9%. The remaining 30 patients (36%) with advanced disease were not receiving chemotherapy; they were on expectant observation or best supportive care, or had their treatment on hold.


One hundred nineteen patients (95%) were screened at least once. All of the patients with limited or no disease were screened, but six of the 83 with advanced disease were missed. Screening took approximately 3 to 5 minutes per patient, adding up to 1 to 2 hours of nursing time per clinic. Thirty seven (31%) of patients had early-stage or no disease, a good prognosis, and no bothersome symptoms or psychosocial distress. Seventy-seven (65%) had metastatic disease, but the performance status was good (Eastern Cooperative Oncology Group [ECOG] 0-2) in 90%. The number of patients with indicators of a prognosis less than 12 months was small; 10% were ECOG 3 (none were ECOG 4), and another 10% had a life-limiting complication of cancer (typically liver failure, bowel obstruction, or cachexia) or serious comorbidities (eg, congestive heart failure, AIDS). Only 10% to 15% had difficult symptom control or moderate-to-severe psychosocial distress, including one patient with early-stage disease.

As shown in Figure 1, patients were categorized as palliative care cases or not. Cases with locally advanced or metastatic disease were either narrowly defined (green box, n = 7) or broadly defined (green and blue boxes, n = 19). There were five other patients who had serious comorbid diseases or uncontrolled symptoms/psychosocial distress, but because they were without advanced disease they were not included as palliative care cases under these definitions.

Figure 1.
Classification of 119 ambulatory oncology patients according to extent of disease, prognosis, and palliative care needs. Patients eligible for palliative care had to have advanced disease and were either narrowly defined (green; n = 7) or broadly defined ...

Screening Tool Scores

The distribution of scores is shown in Appendix Table A1 (online only). Broadly defined palliative care cases (n = 19) had scores of 4 or higher. Narrowly defined cases (n = 7) had scores of 5 or higher. The SN, SP, PPV, and NPV of scores of 4, 5 and 6 are shown in Appendix Table A2 (online only). A score of 4 had the best SN and SP, but a score of 5 had the best predictive values. Using a cut point of 5 or higher, 13% of the patients would be referred to a palliative care specialist.

Palliative Care Problems

Sixteen patients (13%) were identified as having specific palliative care needs, including two patients who did not have advanced disease. Distress related to the cancer diagnosis or therapy was the most common need identified (n = 11), followed by concerns about course of disease and decision making (n = 3). Only two patients had poorly controlled symptoms (pain in one, protracted chemotherapy-induced emesis in another). No patient or family requested a palliative care consult, and the nurse did not think that the oncologist needed assistance with decision making or determining goals of care with any patient. Only one of nine new patients who were screened had a score greater than or equal to 5. None of the nine had any specific palliative care issues.

The NCCN Palliative Care Referral Criteria

Evaluating patients by the criteria took more of the clinic nurse's time, but assessment was still completed in 85% of the intended subset (see Methods section). All were judged to meet at least one criterion for specialist referral. Only eight of the 24 possible criteria were selected by the nurse (Table 2).

Table 2.
Positive Criteria for Referral to Specialist Palliative Care Provider


Implementation of the NCCN screening guideline in this outpatient GI clinic resulted in 7% to 17% of patients being identified as needing to have palliative care incorporated into their comprehensive cancer care plan. Furthermore, it was the OPN's judgment that all of them met the NCCN criteria for referral to a specialist palliative care provider.

Screening for palliative care need was feasible in this busy specialist, predominantly colorectal, outpatient practice, where approximately two thirds of patients had advanced disease and three quarters were receiving some form of chemotherapy. Screening every patient added substantially to nursing time, up to 2 hours over the course of the day. This extra time is exacerbated when the 24 items on the NCCN referral criteria also have to be evaluated. If screening were formally implemented, limiting it to patients with metastatic cancer would save substantial time. Following the NCCN recommendation to screen the patient at the first visit and then only at “clinically relevant times subsequently” would also reduce the number to be screened in each clinic. First visits are easy to identify, but deciding which patients are at a clinically relevant time point is not so straightforward. Changes in extent of disease, goals of treatment, performance status, or symptom burden have been identified as possible triggers.11 Admission to the hospital or the first office visit after release would be other opportunities. An argument could be made for rescreening stable patients on a time-contingent basis, for example, every 6 months. Screening items 1 (disease status), 3 (complications of cancer), and 4 (major comorbid conditions) could be prepopulated from an electronic medical record, and the patient could self-report item 2 (performance status) and many of the elements of item 5 (palliative care problems).

Reducing screening to a one-step process would also be highly desirable for the busy outpatient setting. Using the simple five-item screening tool alone and applying a cut point of 5 would result in 13% of the practice being referred, with no inappropriate referrals; only the small proportion of patients with palliative care needs (mainly those without advanced disease) would be missed in this way.

For patients with severe distress, early referral to a specialist palliative care provider team is recommended.12 The literature on outpatient palliative care is limited. Several studies have described the experience of providing these clinics,1316 and a small number report outcomes.2,17,18 The best example is the study which showed that early referral of all patients with newly diagnosed advanced lung cancer improved outcomes, including survival.2 It is not known whether these results would apply to other diseases, such as colorectal cancer, where the treatment options are greater and the median survival better than in lung cancer. However, the authors of the lung cancer study did not classify patients according to the level of need, and it is not known whether the type of palliative care provided could have been incorporated into general oncology care without specialist referral. The data from this screening project suggests that blanket referral of all patients with metastatic cancer to specialist palliative care service is both unnecessary and difficult to sustain.

Distress associated with the diagnosis or its treatment, including limited treatment options, was the most common affirmative response on the referral criteria. This may be problematic when an oncologist equates palliative care with pain management and end-of-life care. Discussions of illness understanding, coping, and decision making are part of a palliative care consultation,15 but recognition of moderate-to-severe distress is more likely to lead to referral to a mental health professional.19

This was a proof of concept study to evaluate the feasibility of implementing screening in the clinic. The definitions used to classify 7% to 17% of patients as palliative care cases were arbitrary and based on the results of screening. Because we did not have institutional review board permission to review the charts, we do not know if the nurse captured the screening and referral criteria accurately. Nor could we investigate associations between palliative care needs and clinical or demographic parameters.

It is challenging to establish the validity of a palliative care screening tool by the usual techniques. Because palliative care is not a disease entity, there is no gold standard to “diagnose” a patient with complex palliative care needs that require specialist referral. To prove whether screening accurately identifies patients with palliative care needs would require all patients to complete a symptom assessment schedule and a distress scale, as well as have a formal consultation with a specialist palliative care provider, with prespecified definitions of what constitutes a specialist-level case or not. Because a panel of national experts developed the NCCN palliative care guidelines on which the tool is based, it at least has content validity. The reliability of screening and the appropriate times to screen need evaluating. If shown to be reliable and valid, the broad dissemination and implementation of screening could be evaluated. Outcomes of screening and the appropriate integration of palliative care into general oncologic care could then be tested, as well the outcomes of early referral to a palliative care specialist in more challenging cases.

Ambulatory patients with advanced cancer frequently experience significant physical symptoms and psychosocial distress. Through better integration and education, oncologists and supportive/palliative care specialists can work together to minimize the burden of progressive cancer. Simple tools for identifying the approximately 10% to 15% patients in this practice—including new patients—with palliative care needs would be helpful, as they could easily be missed during the course of a busy clinic. But even simple screening is time intensive, and more efficient ways to capture this important information are needed.


Figure A1.

An external file that holds a picture, illustration, etc.
Object name is jop0061128080001.jpg

Relationship between palliative care and place of death.

Authors' Disclosures of Potential Conflicts of Interest

The authors indicated no potential conflicts of interest.

Author Contributions

Conception and design: Paul A. Glare

Provision of study materials or patients: Leonard B. Saltz

Collection and assembly of data: Deborah Semple

Data analysis and interpretation: Paul A. Glare, Stacy M. Stabler, Leonard B. Saltz

Manuscript writing: All authors

Final approval of manuscript: All authors


1. Higginson IJ. The global palliative care lottery: Can we overcome it? J Palliat Med. 2011;14:384–385. [PubMed]
2. Temel JS, Greer JA, Muzikansky A, et al. Early palliative care for patients with metastatic non-small-cell lung cancer. N Engl J Med. 2010;363:733–742. [PubMed]
3. Higginson IJ, Hart S, Koffman J, et al. Needs assessments in palliative care: An appraisal of definitions and approaches used. J Pain Symptom Manage. 2007;33:500–505. [PubMed]
4. Franks PJ, Salisbury C, Bosanquet N, et al. The level of need for palliative care: A systematic review of the literature. Palliat Med. 2000;14:93–104. [PubMed]
5. Dawes RM, Faust D, Meehl PE. Clinical versus actuarial judgment. Science. 1989;243:1668–1674. [PubMed]
6. Emanuel LL, Alpert HR, Emanuel EE. Concise screening questions for clinical assessments of terminal care: The needs near the end-of-life care screening tool. J Palliat Med. 2001;4:465–474. [PubMed]
7. Levy MH, Back A, Benedetti C, et al. NCCN clinical practice guidelines in oncology: Palliative care. J Natl Compr Cancer Netw. 2009;7:436–473. [PubMed]
8. Weissman DE, Meier DE. Identifying patients in need of a palliative care assessment in the hospital setting: A consensus report from the Center to Advance Palliative Care. J Palliat Med. 2011;14:17–23. [PubMed]
9. Glare PA, Plakovic MK, Griffo Y, et al. Fast-tracking palliative care consults: Pilot implementation of National Comprehensive Cancer Network (NCCN) palliative care screening and referral guideline. J Clin Oncol. 2011;29(suppl) abstr e19636.
10. Meier DE, Sieger CE. New York, NY: Center to Advance Palliative Care; 2004. A Guide to Building a Hospital Based Palliative Care Program.
11. Sanft TB, Von Roenn JH. Palliative care across the continuum of cancer care. J Natl Compr Canc Netw. 2009;7:481–487. [PubMed]
12. Bruera E, Hui D. Integrating supportive and palliative care in the trajectory of cancer: Establishing goals and models of care. J Clin Oncol. 2010;28:4013–4017. [PubMed]
13. Zimmermann C, Seccareccia D, Clarke A, et al. Bringing palliative care to a Canadian cancer center: The palliative care program at Princess Margaret Hospital. Support Care Cancer. 2006;14:982–987. [PubMed]
14. Muir JC, Daly F, Davis MS, et al. Integrating palliative care into the outpatient, private practice oncology setting. J Pain Symptom Manage. 2010;40:126–135. [PubMed]
15. Jacobsen J, Jackson V, Dahlin C, et al. Components of early outpatient palliative care consultation in patients with metastatic nonsmall cell lung cancer. J Palliat Med. 2011;14:459–464. [PubMed]
16. Fadul N, Elsayem A, Palmer JL, et al. Supportive versus palliative care: What's in a name?: A survey of medical oncologists and midlevel providers at a comprehensive cancer center. Cancer. 2009;115:2013–2021. [PubMed]
17. Follwell M, Burman D, Le LW, et al. Phase II study of an outpatient palliative care intervention in patients with metastatic cancer. J Clin Oncol. 2009;27:206–213. [PubMed]
18. Yennurajalingam S, Urbauer DL, Casper KL, et al. Impact of a palliative care consultation team on cancer-related symptoms in advanced cancer patients referred to an outpatient supportive care clinic. J Pain Symptom Manage. (epub ahead of print on August 24, 2010) [PubMed]
19. Jacobsen PB, Ransom S. Implementation of NCCN distress management guidelines by member institutions. J Natl Compr Canc Netw. 2007;5:99–103. [PubMed]

Articles from Journal of Oncology Practice are provided here courtesy of American Society of Clinical Oncology