Search tips
Search criteria 


Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
Palliat Support Care. Author manuscript; available in PMC 2013 June 18.
Published in final edited form as:
PMCID: PMC3685415

The Project ENABLE II Randomized Controlled Trial to Improve Palliative Care for Rural Patients with Advanced Cancer: Baseline Findings, Methodological Challenges, and Solutions

Marie Bakitas, DNSc, ARNP, FAAN,1,2,3 Kathleen Doyle Lyons, ScD, OTR,4 Mark T. Hegel, PhD,4 Stefan Balan, MD,5 Kathleen N. Barnett, MA, APRN, BC-PCM,4 Frances C. Brokaw, MD, MS,2,6 Ira R. Byock, MD,1,2 Jay G. Hull, PhD,7 Zhongze Li, MS,8 Elizabeth McKinstry, MS,4 Janette L. Seville, PhD,4 and Tim A. Ahles, PhD9



There is a paucity of randomized controlled trials (RCTs) to evaluate models of palliative care. Although interventions vary, all have faced a variety of methodological challenges including: adequate recruitment, missing data, and contamination of the control group. We describe the ENABLE II intervention, methods, and sample baseline characteristics to increase intervention and methodological transparency, and to describe our solutions to selected methodological issues.


Half of the participants recruited from our rural, US comprehensive cancer center and affiliated clinics were randomly assigned to a phone-based, nurse-led educational, care coordination palliative care intervention model. Intervention services were provided to half of the participants weekly for the first month and then monthly until death, including a bereavement follow-up call to the caregiver. The other half of the participants were assigned to care as usual. Symptoms, QOL, mood, and functional status were assessed every 3 months until death.


Baseline data of 279 participants were similar to normative samples. Solutions to methodological challenges of recruitment, missing data, and "usual care" control group contamination are described.


It is feasible to overcome many of the methodological challenges to conducting a rigorous palliative care RCT.

Keywords: Palliative Care, Randomized Controlled Trial, Advanced Cancer, Methodological Issues, Intervention Study, Rural


Palliative care can be defined as holistic care that is directed at relieving physical, emotional, and spiritual suffering and improving quality of life of patients with life-limiting illnesses and their families (World Health Organization, 2003). A palliative approach promotes identification of individual values and preferences for care, open communication between patients and their care providers, control of distressing symptoms, and family involvement. Hope is maintained and unnecessary suffering is avoided. As simply stated by Teno, in a palliative approach one “hopes for the best, but prepares for the worst” (Teno, 2001).

Ideally, models of palliative care should be evidence-based (McQuay & Moore, 1994) however there are few well-designed, prospective clinical trials of effective palliative care models (Foley & Gelband, 2001; Krouse et al., 2004; Zimmermann, Riechelmann, Krzyzanowska, Rodin, & Tannock, 2008). An abundant literature has chronicled the challenges of designing and conducting rigorous research in seriously-ill patients. Methodological challenges include identifying and recruiting a sample that reflects the heterogeneous seriously-ill population that can be served by palliative care (Bakitas, Lyons, Dixon, & Ahles, 2006; Cook, Finlay, & Butler-Keating, 2002; Grande, 2000; McMillan & Weitzsner, 2003; McWhinney, Bass, & Donner, 1994), finding tools that adequately measure targeted outcomes (Mularski et al., 2007), and analytical strategies that account for the attrition that can be expected when studying persons near the end of life (Jordhoy et al., 1999; McWhinney, Bass, & Donner, 1994; Pickering, 2002). Few interventions are standardized and occur in the context of other medical interventions (McMillan & Weitzsner, 2003; Rinck et al., 1997). Lack of detailed description of the intervention creates difficulty in interpreting results or replicating promising interventions (Conn, Cooper, Ruppar, & Russell, 2008; Zimmermann, Riechelmann, Krzyzanowska, Rodin, & Tannock, 2008).

The formidable challenges of conducting palliative care research call for creativity and rigor (McMillan & Weitzsner, 2003; McQuay & Moore, 1994; Rinck et al., 1997). Some authors have provided guidance on conducting palliative care descriptive studies (Hinds, Burghen, & Pritchard, 2007; Hopkinson, Wright, & Corner, 2005; Steinhauser et al., 2006; Temel et al., 2007) and randomized controlled trials (RCTs) (Currow, Abernethy, Shelby-James, & Phillips, 2006; Jordhoy et al., 1999; McMillan et al., 2005; Northouse et al., 2006). The purpose of this paper is to add to the dialogue about conducting a rigorous palliative care RCT. We do this by first providing a detailed description of the ENABLE II intervention, methods, and baseline sample characteristics and then describing the methodological challenges (recruitment, missing data, and control group contamination) and our solutions in hopes of assisting other researchers planning similar trials.

The ENABLE II Intervention

The ENABLE II RCT is a follow-up study to the prior demonstration project, Project ENABLE (Bakitas et al., 2008; Bakitas et al., 2004; Whedon, 2001). The acronym stands for “Educate, Nurture, Advise, Before Life Ends.” The original demonstration project was one of The Robert Wood Johnson Foundation’s Promoting Excellence in End-of Life Care innovative cancer center-hospice collaborations to address deficiencies in end-of-life care (Byock, Twohig, Merriman, & Collins, 2006). ENABLE II foundational principles, derived from the successes and lessons of ENABLE, were based on the concepts of early detection/identification and crisis prevention and are consistent with the World Health Organization (WHO) (Sepulveda, Yoshida, & Ullrich, 2002) and other national organization (American Cancer Society, 2007; Field & Cassel, 1997; Foley & Gelband, 2001; Lynn & Adamson, 2003; National Comprehensive Cancer Network, 2006) recommendations for early integration of palliative care. The intervention was proactive. It was provided soon after diagnosis-- and concurrent with anti-cancer treatment, thereby “upstreaming” principles of hospice and palliative care. Patients were identified and invited to enroll shortly after a new diagnosis of advanced stage or recurrent cancer. Advance practice palliative care nurse specialists educated participants about key palliative care principles and crisis prevention via practice in problem solving/decision-making skills, symptom management, communication, advance care planning. Concurrent with anti-cancer treatment, they coordinated appropriate referrals to improve the patient and caregiver’s quality of living with advanced cancer and the patient’s end of life experience. Referrals and services generally increased as illness progressed. The intervention was primarily conducted by telephone in order to be accessible to a rural population that was anticipated to become more dependent as illness progresses. The approach was essentially a “both-and” (as opposed to “either-or”) model in which there would be a smooth transition from mostly anti-cancer treatment to mostly palliative care (Byock, 2000). Models with some similar features have been called integrated (Temel et al., 2007), concurrent or simultaneous care (Byock, Twohig, Merriman, & Collins, 2006; Meyers & Linder, 2003; Pitorak, Armour, & Sivec, 2003).


Study Design

Project ENABLE II was a prospective, randomized controlled trial of an educational and care management palliative care intervention for persons with advanced cancer and a caregiver compared to care as usual. The study protocol was approved by the institutional review board of Dartmouth College. Enrollment began in November 2003 and ended in May 2007; caregiver after death interviews are still being conducted.

Study Sites

Participants were recruited from the oncology clinics of our National Cancer Institute-designated comprehensive cancer center and affiliated outreach clinics, and the academically affiliated Veterans Affairs Medical Center (VAMC). Eligibility criteria are listed in Table 1.

Table 1
Eligibility Criteria*

Participant Identification and Recruitment

Research assistants (RAs) at the cancer center and the VAMC attended weekly gastrointestinal (GI), genitourinary (GU), breast, and thoracic cancer management meetings (tumor boards) in which newly diagnosed patients were discussed. The RAs also reviewed clinicians’ clinic schedules to identify potentially eligible patients. The clinician then approached the patient to obtain permission for the RA to provide them with more information about the study. Consent was obtained from the patient either in person or by telephone. If the patient was willing to participate, they were asked if there was someone involved in their care that they would be willing to have enrolled as well. The participant was then randomized to the intervention, or the usual care control condition.

From affiliated outreach clinics, following discussion of the study with clinic staff, the main study site was informed of a potentially interested patient. Informed consent documents were mailed to interested patients, and the RA telephoned the patient to review study information. Once a signed consent form and baseline questionnaires were returned, the patient was enrolled and randomized.


After completion of the baseline assessment, participants (but not clinicians) were informed of their treatment assignment. Participants were randomized equally into either the intervention or usual care group using computer-generated random numbers. There were separate randomization schemes for the cancer center and the VAMC participants (in order to ensure an equal distribution of patients in intervention and control groups from each of these primary sites). Participants from all other sites randomized according to the cancer center scheme as large numbers of participants were not anticipated. Randomization was blocked using random block sizes, and was also stratified by diagnosis (lung, breast, and GI and GU cancers) to control for differential effects of treatment regimens and disease course.

Intervention Design

Palliative Nurse Educator. Participants were assigned to one of two nurse educators (KB and EM), both advanced practice nurses with palliative care expertise, who delivered the bulk of the intervention-specific care. The nurse educator contacted the intervention participants either by telephone or during a routine clinic visit within one week after mailing an educational manual entitled, “Charting your Course: An Intervention for People and Families Living with Cancer”.

The manual was constructed by the study team from materials developed during ENABLE I (Bakitas et al., 2004; Skalla, Bakitas, Furstenberg, Ahles, & Henderson, 2004) and from publicly available resources (e.g., NCI, ACS). The manual contained four modules: 1) problem solving, 2) communication and social support, 3) symptom management, 4) advance care planning and unfinished business, and an appendix listing supportive care resources. During the initial call, the nurse introduced herself, determined a convenient schedule for the subsequent four weekly phone sessions, and initiated a therapeutic alliance with the patient, encouraging active participation in their oncology treatment and care. Caregivers also were invited and encouraged to participate in these sessions.

The nurse educator then contacted the participant weekly for the first four weeks to review each module in the manual. Session 1 included an overview of the intervention, history taking, and rapport building, in addition to the educational content on the problem-solving technique (Hegel, Barrett, & Oxman, 2000). Each session began with the nurse administering the Distress Thermometer (Roth et al., 1998) an 11-point rating scale (0–10) of distress. If the participant provided a rating greater than 3, the nurse explored the sources of distress and identified if the participant would like to apply the problem solving approach to address the issues. They then covered the assigned module for that session. On average, Session 1 (introduction and problem solving) lasted 41 minutes and sessions 2–4 each lasted 30 minutes.

After completion of the four structured sessions the nurse phoned the participant at least monthly. Follow up calls also began by administering the Distress Thermometer and were followed by problem solving for a distressing situation if indicated. The nurse educators also triaged medical complaints and offered to help arrange care and services as needed, including palliative and hospice care. Monthly contacts continued as long as the participant was alive. In later stages of illness the nurse educator may have communicated primarily with the participant’s caregiver. If their living arrangement changed (e.g., they moved to a skilled nursing facility) the nurse attempted to follow them there as well. Average length of follow up contacts was 12 minutes. The vast majority of follow up contacts were by telephone (91%) and the remainder were accomplished during routine clinic visits or inpatient hospital stays.

Shared Medical Appointments (SMA). Intervention participants and their caregiver were invited to attend monthly group medical appointments led by a palliative care physician (FB) and nurse practitioner (MB). The purpose of these appointments were to allow participants and their family to ask questions about their medical problems and/or related issues (e.g., insurance coverage, social services, and rehabilitation services) in a forum allowing more in-depth discussion than normally feasible during a typical clinic visit (Noffsinger, 2000). In addition, participants had the opportunity to learn from the experiences of other group members. Participants could attend in person or by toll-free conference call. The phone-in option was added in response to recognition that many participants had difficulty traveling long distances for multiple appointments outside of usual oncology treatment days. Participants reported that they enjoyed and appreciated the opportunity to attend by telephone, and group process was not disrupted by the conference call format. The SMAs were attended by 18% of intervention participants for an average of 2.7 sessions each.

Care as Usual. Participants assigned to care as usual were allowed to use all oncology, palliative care and other medical center services without restrictions. The cancer center site has a consultative interdisciplinary palliative care team (PCT) comprised of a physician and nurse practitioners. The team provided care for both inpatient and outpatient populations. Oncologists could refer patients for assessment by this team for symptom and supportive care while receiving anti-cancer treatments. Patients and family members were often followed through death and bereavement. From 2003–2005, the team expanded to include additional physicians, nurse practitioners, and a dedicated social worker, chaplain, volunteer coordinator/volunteers and administrative staff. During the last year of study enrollment, automatic PCT consultation at the time of diagnosis became a routine part of the clinical pathways for advanced lung and GI malignancies.

The VAMC site also had an Advanced Illness Care Committee (AICC) which provided consultation to oncology staff for inpatients with life threatening illness. In 2006, a multidisciplinary Palliative Care Consult Team was developed, comprised of representatives from nursing, social work, chaplaincy, medicine, nutrition, and mental health services. The team provides a coordinated program of palliative and supportive care for inpatients or outpatients in the final stages of a terminal illness and their families. Consultations include assessments and recommendations related to prognosis, pain and symptom management, goals of care and associated treatment decisions, advance care planning, psychosocial, spiritual and other issues, family meetings, and referrals to hospice and other VAMC and community services. The team conducts palliative care rounds weekly. All VAMC patients with a diagnosis of metastatic cancer are routinely referred for services as part of their clinical pathways.

Interventionist Training and Evaluation of Intervention Fidelity

The nurse educators were trained in the problem-solving method (Hegel, Barrett, & Oxman, 2000) during a two day orientation involving oral and written educational materials. The trainer (JS) and nurse educators met on a routine basis after participant enrollment began to discuss difficult situations and appropriate management. Additionally, the nurse educators participated in bi-weekly team meetings which included the investigators and study staff. These meetings focused on study management and review of complicated cases. With participant consent, telephone sessions were audio-taped to permit checking for intervention quality and treatment fidelity. Treatment fidelity was assessed by the trainer for 20% of randomly chosen audio-taped sessions using a checklist of essential components of the intervention. The nurse educators also met monthly with the trainer for feedback and supervision regarding their fidelity to the intervention model. Ninety eight percent of reviewed sessions met treatment fidelity criteria.

The physician and nurse practitioner who conducted the shared medical appointments were trained in this method by an expert in this approach (Ferguson, 2003). The training consisted of 2 half-day sessions and readings (Ferguson, 2003). A fidelity checklist was developed and 26% of all sessions were attended by a trainer who evaluated them against these specific criteria. The physician and nurse practitioner met criteria for fidelity for 100% of reviewed sessions.

Baseline Assessment and Outcome Measures

Baseline and follow up questionnaires assessed functional status, symptom intensity, mood status, and quality of life upon enrollment. Follow up questionnaires were mailed one month after enrollment/baseline assessment, and then every three months until the participant died or the study ended. Shortly after participant death, the research staff made a bereavement follow-up phone call and scheduled an after death interview with the caregiver, to be conducted within three to six months. We also conducted semi-structured interviews of a sub-sample of intervention and control participants and caregivers, as well as oncology clinicians, to obtain in-depth data regarding their experience with the intervention or usual cancer care.

Functional Status: Karnofsky Performance Scale (KPS)

The KPS is a ten-point clinician-rated scale designed to measure a person’s ability to perform daily tasks (Karnofsky & Burchenal, 1949). Higher scores indicate higher performance ability. The KPS has strong interrater reliability (r=.89) (Schag, Heinrich, & Ganz, 1984). Construct validity has been demonstrated by close associations with other standard measures of daily function (Buccheri, Ferrigno, & Tamburini, 1996; Schag, Heinrich, & Ganz, 1984). The KPS has also been shown to accurately predict survival time (Buccheri, Ferrigno, & Tamburini, 1996). In this study, the clinician-documented KPS closest to the time of the self-report measures was extracted from the medical record by the RA.

Symptom Intensity: Edmonton Symptom Assessment Scale (ESAS)

The ESAS is a validated, reliable instrument that has been used in palliative care settings (Bruera, 1996; Bruera, Kuehn, Miller, Selmser, & Macmillan, 1991).The ESAS assesses nine symptoms rated by severity on visual analogue scales for each symptom (10-cm line) (Bruera, 1996; Bruera, Kuehn, Miller, Selmser, & Macmillan, 1991). These include pain, activity, nausea, depression, anxiety, drowsiness, appetite, sense of well-being, and shortness of breath. The sum of responses to these nine symptoms, in millimeters, is the ESAS total score. Because the current study uses an optical scanning method for data collection the scales were redesigned as numerical visual analogue scales with discrete check boxes (0–10). Thus, for comparison purposes, the scores reported here were converted to an equivalent of the original ESAS scoring method by multiplying by ten. For example, an ESAS total score of 30 in our study was multiplied by ten to yield a converted score of 300 so that it would be comparable to ESAS scores reported from other studies.

Mood Status: Center for Epidemiological Study- Depression Scale (CES-D)

The CES-D is a 20-item measure of depressive symptoms that is widely used in epidemiological studies of depression.(Radloff, 1977) Participants are asked to rate how frequently they have experienced each symptom on a four point scale ranging from “Rarely or none of the time” to “Most or all of the time.” The CES-D has been widely studied and has strong data supporting its validity and reliability (Okun, Stein, Buman, & Silver, 1996). A score of 16 or higher indicating a clinically significant level of depressive symptoms (Radloff, 1977).

Quality of Life: The Functional Assessment of Chronic Illness Therapy-Palliative Care (FACIT-Pal)

The FACIT-Pal consists of the FACT-G (Functional Assessment of Cancer Therapy-General), a general measure of quality of life, and the palliative care subscale (Pal), which assesses issues specifically relevant to palliative care (Brady et al., 1997; Lyons et al., in press). The FACT-G is a 27-item questionnaire which provides a total score as well as four subscale scores: physical, social/family, emotional, and functional well-being. Lower scores indicate lower quality of life. The FACIT-Pal includes 19 additional concerns relevant for persons at end of life. Evidence supports the reliability, validity, and sensitivity of the instrument and its ability to detect change over time (Cella et al., 1993).

Results of Baseline Assessments

Seven hundred and eighty-five patients were screened between November, 2003 and May, 2007. Consent was obtained for 322, a 48% response rate. Those not enrolling declined due to lack of interest (43%), too much effort required (19%), not feeling they needed the intervention (13%), being too ill (9%), and a variety of other reasons (16%). Following consent 279 (87%) participants returned baseline questionnaires. Participants were not informed of their group assignment until after they returned the baseline assessment.

Demographic and Clinical Characteristics

Table 2 compares participant demographic and other baseline characteristics to normative data when available. A higher percentage of men than national norms likely reflects recruitment of 25% of participants from the VAMC. The relatively small number of widowed participants is also likely due to over sampling of males. Education level (86% with high school degree/equivalent or greater) was similar to national norms (93%). Only 1% of ENABLE II participants were of minority ethnicity, compared with 18% of older Americans nationwide, which is consistent with the racial/ethnic characteristics of the population from which the study sample was drawn (approximately 4% racial/ethnic minority). Fifty-seven percent of participants resided in rural areas1, which is substantially higher than the national distribution of 20%, but is consistent with our rural setting in Northern New England. The majority of participants had primary cancers other than breast cancer, partially due to the primarily male sample.

Table 2
Demographic characteristics (N=279)*

Baseline Advance Directives, Palliative and Hospice Referral and Hospitalization Rates

At the time of enrollment approximately one-half of participants had already established a living will and durable power of attorney, but few had established “do not resuscitate” status (Table 3). Few patients had been referred to hospice, but about one-quarter had already been referred to the palliative care team. Because a minority of participants (n =13) were recruited from the four sites from which we did not have access to medical records, we summarized patient-reported rates of hospital and intensive care unit admissions, and emergency room visits. Future analyses will compare chart review and patient report for participants that have data from both sources.

Table 3
Baseline Rates of Advance Directives Completion, Palliative or Hospice Referral and Hospitalization*

Comparison of Function, Symptom Intensity, Mood, and Quality of Life Ratings with Other Samples

Table 4 lists our sample’s baseline scores on our study’s main outcome measures and scores reported in the general population, advanced stage outpatient and general inpatient cancer or inpatient palliative care samples (when available) for comparison. On the KPS, a functional status score of 100 is considered “normal, no complaints, no evidence of disease”. Our sample’s median KPS of 80 approximated that of a mixed outpatient and a mixed inpatient cancer sample (Chang, Hwang, & Feuerman, 2000). Although at baseline our sample had a similar KPS range (40–100), the median score of 80 (indicating “Normal activity with effort; some signs or symptoms of disease”) was higher than that of a hospitalized lung cancer sample (N=536) with a median KPS of 70 signifying an inability “to carry on normal activity or do active work” (Buccheri, Ferrigno, & Tamburini, 1996).

Table 4
Baseline Assessment Measures with comparisons to existing norms in the literature.

Symptom intensity, as measured by the ESAS total score (mean 284.3±151.1) and range were higher than the original outpatient and inpatient validation cancer samples with mixed diagnoses (Chang, Hwang, & Feuerman, 2000). However our sample’s ESAS score was lower than that seen in a sample at end of life newly admitted to a palliative care unit (mean = 410±95) (Bruera, Kuehn, Miller, Selmser, & Macmillan, 1991). The maximum transformed ESAS score is 900 indicating the most severe rating on nine common symptoms of advancing illness (Bruera, Kuehn, Miller, Selmser, & Macmillan, 1991).

Depressive symptoms, as measured by the CES-D, although elevated (mean = 13±8.7) is below the score generally regarded as indicative of a depressed mood of clinical significance (CES-D = 16) (Okun, Stein, Buman, & Silver, 1996; Roberts & Vernon, 1983), but is higher than that seen in the general population (mean = 8.5±8.1) (Radloff, 1977). This score was also somewhat higher than a similar mixed advanced diagnosis outpatient sample of community and rural cancer patients (Given, Given, & Stommel, 2002); although that sample also did not reach a score indicative of a clinically depressed mood.

Quality of life, as measured by the FACT-G total score (mean = 75.2±15.4) is understandably lower than norms for similar age groups from the general population (age 60–69, mean = 86.4±13.1; age > 70, mean = 83.3±17.3; Lower scores signify lower quality of life) (Holzner et al., 2004). Quality of life for the ENABLE II sample is more comparable to an original validation sample for the FACT-G with stage IV disease (mean = 80.7±15.0) (Cella et al., 1993). (Only results for the FACT-G are compared here due to the lack of normative comparison data for the FACIT-Pal at this time.)


We have provided a detailed description of the ENABLE II intervention, study design and sample baseline characteristics of our prospective, palliative care RCT in order to identify how we addressed some of the common methodological issues that have challenged palliative care researchers (Conn, Cooper, Ruppar, & Russell, 2008; Davies, Chekryn Reimer, Brown, & Martens, 1995; Dean & McClement, 2002; Grande, 2000; Hopkinson, Wright, & Corner, 2005; Hudson, Aranda, & McMurray, 2001; Jordhoy et al., 1999; Karim, 2000; Mazzocato, Sweeney, & Bruera, 2001; Rinck et al., 1997; Steinhauser et al., 2006). We were able to develop a sampling plan and recruitment strategy to meet recruitment goals and maximize external validity; develop an analysis plan that would account for non-random missing data occurring when patients are seriously ill and dying, and develop a mechanism to identify possible contamination of the usual care group (internal validity threat of “history”) to help us understand the impact (if any) our study outcomes.

Sampling and Recruitment of Subjects for Palliative Care Studies. We have previously described potential sampling and recruitment issues relative to a palliative care RCT (Bakitas, Lyons, Dixon, & Ahles, 2006), so these points are only briefly summarized in the context of our study results. In order to maximize external validity, our challenge was to recruit a sample of participants’ representative of the cancer center population that might be served by palliative care. First, we chose four of the most prevalent types of advanced cancers and focused the intervention on issues germane for anyone with life-limiting cancer (i.e., the need to problem-solve, communicate effectively, manage symptoms, and engage in advance care planning). Second, the intervention was designed for patients who would likely live at least two months (in order to experience all components of the intervention), but die within one year (and, therefore, be experiencing the need for advance care planning, and symptom management education that was part of the program). Because clinician estimates of survival can be biased (Lassauniere & Vinant, 1992; Oxenham, 1998), we did not use estimated survival as criteria for eligibility. Instead, we specified the stages and prognostic cues that would be most predictive of death within two months to one year. Third, we chose broad eligibility criteria; our major exclusion criterion was for persons with psychiatric co-morbidity that would demand more tailored services than provided by our model. Finally, we employed two RAs who dedicated more than half of their time to consulting with the clinicians to obtain and then follow through on referrals. Having dedicated RAs, who attended disease management tumor boards and advocated for the proactive ENABLE II intervention, avoided much of the “gate-keeping” that is often lamented in literature on end-of-life research (Cook, Finlay, & Butler-Keating, 2002; Ewing et al., 2004; Jordhoy et al., 1999; McMillan & Weitzsner, 2003; Steinhauser et al., 2006). These strategies helped us to meet our goal of recruiting nearly 50% of eligible patients.

The baseline results on our main clinical measures when compared with other cancer samples (Table 4) also affirm that we were able to recruit a sample representative of patients with new diagnoses of advanced cancer. As of December 2007, our average duration of involvement in the study was 320±331 days, suggesting we also met our target of recruiting a population of patients with life expectancy between 2 months to one year.

Measurement and Analysis Challenges in Seriously Ill Subjects. The first challenge in this area involved ensuring that we were adequately measuring the outcomes of interest (Mularski et al., 2007), using instruments that could be easily completed by patients. Pilot tests of the quality of life and symptom intensity measures in our demonstration project provided us with ample evidence that the chosen tools were understood by patients, were feasible to complete over time, and included items reflecting the experiences of the target population. The second measurement challenge involved minimizing missing data. Because we were working with participants who were living with serious illness, we endeavored to make it as easy as possible to complete the surveys, including offering to read them over the telephone or during clinic visits if participants preferred.

Even if participants completed each assessment point, individuals have different numbers of assessment points because of differences in time from enrollment to death. To address this issue we will analyze data using linear mixed modeling (LMM). Linear mixed modeling has several advantages over traditional repeated measures analysis of variance (ANOVA) in its treatment of (a) missing data, (b) data collected at varying points in time, (c) data with unusual error structure, as well as (d) statistical power. Specifically, data do not need to be collected at particular time-points for every participant; data collected at any point in time can contribute to statistical estimates of effects. As a consequence, missing data is much less of an issue for LMM than for traditional repeated measures ANOVA both in estimating effects and statistical power. In addition, because of the assumptions underlying the statistical approach of LMM, the researcher is not restricted to a few options in the treatment of error variance. Instead, LMM allows one to identify the error covariance structure most appropriate to the observed data prior to conducting tests of the principal hypotheses (Singer & Willett, 2003).

Although LMM has multiple advantages over ANOVA, it is not a panacea. In particular, LMM, and indeed any analysis conducted on an intent-to-treat basis, is strongly wedded to the assumption that any missing data are missing in random fashion. As a consequence every effort is made to obtain the most complete data possible from all participants (e.g., reminder phone calls from research assistants and offers to read the questions orally if desired). Although LMM will be adopted as the principal data analytic strategy, a sensitivity analysis will be conducted examining the effects of different strategies for handling missing data on the conclusions.

Our principal hypotheses are that individuals assigned to the ENABLE intervention will show higher quality of life on a variety of measures relative to those assigned to usual care. The study plan dictates that we first analyze for differences between intervention and control groups. However, given the richness of the data set, it will also be examined with respect to (a) relative mortality using survival analysis, (b) potential interrelations among the dependent variables using factor analysis and structural equation modeling, and (c) possible discrete types of end-stage experience (e.g., slow decline - death; quick decline- poor functioning maintenance period - death; high functioning maintenance period - quick decline - death) using latent class analysis.

Contamination of Usual Care Group (History). Changes to the standard of “usual care” over the course of an RCT may create threats to the internal validity of the study due to history (e.g. events occurring concurrently with treatment could cause the observed effect) and maturation (naturally occurring changes over time could be confused with a treatment effect) (Shadish, Cook, & Campbell, 2002). At both our cancer center and the VAMC, the ENABLE RCT and a clinical Palliative Care Consult Team (PCT) evolved simultaneously. Because palliative care services were growing and expanding as part of usual cancer care, it was not considered ethical to withhold such services from patients who wished to enroll in the ENABLE RCT. Recruitment to the RCT was consistent with projections even as the PCT expanded and thrived, with significant growth in the availability and integration of multidisciplinary palliative care clinical services. Additionally, specialty palliative care educational programs and grand rounds have been provided regularly to oncology clinicians throughout the region.

In the last year of enrollment, emerging evidence suggested that aspects of our intervention were now being incorporated into “usual care”. For example, PCT referral at the time of diagnosis (a main component of the ENABLE RCT) was integrated into the disease management pathways of patients with pancreatic and lung cancer. ENABLE II recruiters and intervention nurses identified confusion among both participants and oncology clinicians, regarding the roles and interventions offered through ENABLE II versus the PCT. Specifically, when some patients were invited to participate in ENABLE II they commented that they were already involved, when in fact they were receiving services from the PCT. Similarly PCT patients (who are also enrolled in ENABLE) assumed that PCT clinicians had access to their study questionnaire responses. Some oncology clinicians referred patients to either ENABLE II or the PCT, believing that a single referral provided access to both the research and clinical endeavors. Hence, we designed a qualitative supplement interview study to explore intervention and usual care participants’ and clinicians’ beliefs about the standard of “usual care” for patients with advanced cancer at our cancer center (Morgan, 1998; Sandelowski, 1996). These interview data will provide important context within which to understand the results of our study.


Project ENABLE II draws on methods used in traditional clinical trials, new models of enhanced palliative care coordination, and sophisticated study design, sampling, and data analytic procedures, to contribute toward the development of an evidence-based approach to the field of palliative care. While we hope that this study will demonstrate the effectiveness of a new model of palliative care for advanced cancer that can be applied in a range of oncology settings, we also hope that sharing this experience will further the dialogue with other palliative care researchers dealing with the challenges of conducting this type of research.


The authors thank the study participants and oncology clinicians from the Section of Hematology/Oncology and VA Medical Centers, and our outreach clinics for their cooperation with the conduct of this study. We would like to especially acknowledge Robert Ferguson, PhD for SMA facilitator training, and the diligent recruitment and data collection and management efforts of Daphne Ellis, AS, Julie Wolf, RN, Luann E. Graves B.S.MT, Susan Horrigan and Linda Eickhoff. Preparation of this manuscript was supported by grants from the the National Cancer Institute (R01 CA101704) and NIH/NINR (T32NR008346).

Funding Considerations: This study is supported by the National Cancer Institute, Bethesda, MD; R01CA101704


Informed Consent: The Dartmouth College Institutional Review Board approved this study.

Conflict of Interest: None

1To determine the percentage of participants living in rural areas, we identified the Rural Urban Commuting Area (RUCA) code for each participant’s zip code. We collapsed the ten-tiered RUCA system into a four-tiered system with the following levels: (1) urban core areas (built up areas with greater than 50,000 persons); (2) suburban areas (areas with high commuting relationships to urban core areas); (3) large town areas (areas with populations between 10,000 and 49,999 persons); and (4) small town and isolated rural areas (areas with less than 10,000 persons).

Washington State Department of Health (2006). Guidelines for using rural-urban classification systems for public health assessment. Retrieved May 16, 2007. Access: Last updated: 1/12/2006.


  • American Cancer Society. Cancer Facts and Figures 2007. Atlanta, GA: American Cancer Society; 2007.
  • Bakitas M, Ahles T, Skalla K, Brokaw F, Byock I, Hanscom B, et al. Proxy perspectives on end-of-life care. Cancer. 2008
  • Bakitas M, Lyons KD, Dixon J, Ahles T. Palliative Care Program Effectiveness Research: Developing rigor in sampling design, conduct and reporting. Journal of Pain and Symptom Management. 2006;31(3):270–284. [PubMed]
  • Bakitas M, Stevens M, Ahles T, Kirn M, Skalla K, Kane N, et al. Project ENABLE: A palliative care demonstration project for advanced cancer patients in three settings. Journal of Palliative Medicine. 2004;7(2):363–372. [PubMed]
  • Brady MJ, Cella DF, Mo F, Bonomi AE, Tulsky DS, Lloyd SR, et al. Reliability and validity of the functional assessment of cancer therapy - Breast Quality-of-Life instrument. Journal of Clinical Oncology. 1997;15(3):974–986. [PubMed]
  • Bruera E. Assessing quality of life in palliative care. Cancer treatment and research. 1996;100:3–12.
  • Bruera E, Kuehn N, Miller MJ, Selmser P, Macmillan K. The Edmonton Symptom Assessment System (ESAS): A simple method for the assessment of palliative care patients. Journal of Palliative Care. 1991;7:6–9. [PubMed]
  • Buccheri G, Ferrigno D, Tamburini M. Karnofsky and ECOG performance status scoring in lung cancer: a prospective, longitudinal study of 536 patients from a single institution. Eur J Cancer. 1996;32A:1135–1141. [PubMed]
  • Byock I. Completing the continuum of cancer care: integrating life-prolongation and palliation. CA Cancer J Clin. 2000;50(2):123–132. [PubMed]
  • Byock I, Twohig JS, Merriman M, Collins K. Promoting excellence in end-of-life care: a report on innovative models of palliative care. J Palliat Med. 2006;9(1):137–151. [PubMed]
  • Cella DF, Tulsky DS, Gray G, Sarafian B, Linn E, Bonomi A, et al. The Functional Assessment of Cancer Therapy Scale: Development and validation of the general measure. Journal of Clinical Oncology. 1993;11:570–579. [PubMed]
  • Chang VT, Hwang SS, Feuerman M. Validation of the Edmonton Symptom Assessment Scale. Cancer. 2000;88:2164–2171. [PubMed]
  • Conn VS, Cooper PS, Ruppar TM, Russell CL. Searching for the intervention in intervention research reports. J Nurs Scholarsh. 2008;40(1):52–59. [PubMed]
  • Cook AM, Finlay IG, Butler-Keating RJ. Recruiting into palliative care trials: lessons learnt from a feasibility study. Palliat Med. 2002;16(2):163–165. [PubMed]
  • Currow DC, Abernethy AP, Shelby-James TM, Phillips PA. The impact of conducting a regional palliative care clinical study. Palliat Med. 2006;20(8):735–743. [PubMed]
  • Davies B, Chekryn Reimer J, Brown P, Martens N. Challenges of conducting research in palliative care. Omega. 1995;31(4):263–273. [PubMed]
  • Dean RA, McClement SE. Palliative care research: Methodological and ethical challenges. International Journal of Palliative Nursing. 2002;8(8):376–380. [PubMed]
  • Ewing G, Rogers M, Barclay S, McCabe J, Martin A, Todd C. Recruiting patients into a primary care based study of palliative care: why is it so difficult? Palliat Med. 2004;18(5):452–459. [PubMed]
  • Ferguson R. Group Shared Medical Appointments (SMAs) for Project Enable II: Facilitator Training Manual. Lebanon, NH: Dartmouth-Hitchcock Medical Center; 2003.
  • Field MJ, Cassel CK. Approaching Death: Improving Care at the End of Life. Washington, D.C: National Academy Press; 1997.
  • Foley KM, Gelband H. Improving Palliative Care for Cancer. Washington, DC: Institute of Medicine and National Research Council; 2001. [PubMed]
  • Given BA, Given CW, Stommel M. Depression and functional status as predictors of death among cancer patients. Cancer. 2002;94:2719–2727. [PubMed]
  • Grande G. Why are trials in palliative care so difficult? Palliative Medicine. 2000;14:69–74. [PubMed]
  • Hegel M, Barrett J, Oxman T. Training therapists in problem-solving treatment of depressive disorders in primary care: Lessons learned from the "treatment effectiveness project". Family Systems, & Health. 2000;18:359–407.
  • Hinds PS, Burghen EA, Pritchard M. Conducting end-of-life studies in pediatric oncology. West J Nurs Res. 2007;29(4):448–465. [PubMed]
  • Holzner B, Kemmler G, Cella D, De Paoli C, Meraner V, Kopp M, et al. Normative data for functional assessment of cancer therapy--general scale and its use for the interpretation of quality of life scores in cancer survivors. Acta Oncol. 2004;43:153–160. [PubMed]
  • Hopkinson JB, Wright DN, Corner JL. Seeking new methodology for palliative care research: challenging assumptions about studying people who are approaching the end of life. Palliat Med. 2005;19(7):532–537. [PubMed]
  • Hudson P, Aranda S, McMurray N. Randomized controlled trials in palliative care: overcoming the obstacles. International Journal of Palliative Nursing. 2001;7(9):427–434. [PubMed]
  • Jordhoy MS, Kaasa S, Fayers PM, Ovreness T, Underland G, Ahlner-Elmqvist M. Challenges in palliative care research; recruitment, attrition and compliance: experience from a randomized controlled trial. Palliative Medicine. 1999;13:299–310. [PubMed]
  • Karim K. Conducting research involving palliative patients. Nursing Standard. 2000;15(2):34–36. [PubMed]
  • Karnofsky D, Burchenal J. The clinical evaluation of chemotherapeutic agents in cancer. In: MacLeod C, editor. Evaluation of Chemotherapeutic Agents. New York: Columbia University Press; 1949.
  • Krouse RS, Rosenfeld KE, Grant M, Aziz N, Byock I, Sloan J, et al. Palliative care research: issues and opportunities. Cancer Epidemiol Biomarkers Prev. 2004;13(3):337–339. [PubMed]
  • Lassauniere J, Vinant P. Prognostic factors, survival, and advanced cancer. Journal of Palliative Care. 1992;8(4):52–54. [PubMed]
  • Lynn J, Adamson D. Living well at the end of life: Adapting health care to serious chronic illness in old age. Santa Monica, CA: 2003.
  • Lyons KD, Bakitas M, Hegel MT, Hanscom B, Hull J, Ahles T. Reliability and Validity of the Functional Assessment of Chronic Illness Therapy-Palliative Care (FACIT-Pal) Scale. Journal of Pain & Symptom Management. (in press). [PMC free article] [PubMed]
  • Mazzocato C, Sweeney C, Bruera E. Clinical research in palliative care: patient populations, symptoms, interventions and endpoints. Palliative Medicine. 2001;15:163–168. [PubMed]
  • McMillan SC, Small BJ, Weitzsner MA, Schonwetter R, Tittle M, Moody L, et al. Impact of coping skills intervention with family caregivers of hospitce patients with cancer: A randomized clinical trial. Cancer. 2005;106(1):214–222. [PubMed]
  • McMillan SC, Weitzsner MA. Methodologic issues in collecting data from debilitated patients with cancer near the end of life. Oncology Nursing Forum. 2003;30(1):123–129. [PubMed]
  • McQuay H, Moore A. Need for rigorous assessment of palliative care. British Medical Journal. 1994;309(6965):1315–1316. [PMC free article] [PubMed]
  • McWhinney I, Bass M, Donner A. Evaluation of a palliative care service: Problems and pitfalls. British Medical Journal. 1994;309(6965):1340–1342. [PMC free article] [PubMed]
  • Meyers F, Linder J. Simultaneous care: Disease treatment and palliative care throughout illness. Journal of Clinical Oncology. 2003;21(7):1412–1415. [PubMed]
  • Morgan DL. Practical strategies for combining qualitative and quantitative methods: Applications to health research. Qualitative Health Research. 1998;8(3):362–376. [PubMed]
  • Mularski RA, Rosenfeld K, Coons S, Dueck A, Cella D, Feuer D, et al. Measuring outcomes in randomized prospective trials in palliative care. Journal of Pain & Symptom Management. 2007;34(1S):S7–S19. [PubMed]
  • National Comprehensive Cancer Network. Palliative Care V.I. 2006, NCCN Clinical Practice Guidelines in Oncology. New York: National Comprehensive Cancer Network; 2006.
  • Noffsinger E. Understanding today's group visit models. Group Practice Journal. 2000;49(2):46–58.
  • Northouse LL, Rosset T, Phillips L, Mood D, Schafenacker A, Kershaw T. Research with families facing cancer: the challenges of accrual and retention. Res Nurs Health. 2006;29(3):199–211. [PubMed]
  • Okun A, Stein R, Buman L, Silver E. Content validity of the Psychiatric Symptom Index, CES-depression Scale, and State-Trait Anxiety Inventory from the perspective of DSM-IV. Psychological Reports. 1996;79(3 Pt 1):1059–1069. [PubMed]
  • Oxenham D. Accuracy of prediction of survival by different professional groups in a hospice. Palliative Medicine. 1998;12:117–118. [PubMed]
  • Pickering RM. Statistical aspects of measurement in palliative care. Palliat Med. 2002;16(4):359–364. [PubMed]
  • Pitorak E, Armour M, Sivec H. Project Safe Conduct integrates palliative goals into comprehensive cancer center. Journal of Palliative Medicine. 2003;6(4):645–655. [PubMed]
  • Radloff L. The CES-D Scale: A self report depression scale for research in the general population. Applied Psychological Measurement. 1977;1:385–401.
  • Rinck G, van den Bos G, Kleijnen J, de Haes H, Schade E, Veenhof C. Methodologic issues in effectiveness research on palliative cancer care: A systematic review. Journal of Clinical Oncology. 1997;15(4):1697–1707. [PubMed]
  • Roberts R, Vernon S. The Center for Epidemiologic Studies Depression Scale: its use in a community sample. American Journal of Psychiatry. 1983;140:41–46. [PubMed]
  • Roth AJ, Kornblith AB, Batel-Copel L, Peabody E, Scher HI, Holland JC. Rapid screening for psychologic distress in men with prostate carcinoma. Cancer. 1998;82(10):1904–1908. [PubMed]
  • Sandelowski M. Using qualitative methods in intervention studies. Research in Nursing and Health. 1996;19:359–364. [PubMed]
  • Schag CC, Heinrich RL, Ganz PA. Karnofsky performance status revisited: reliability, validity, and guidelines. J Clin Oncol. 1984;2:187–193. [PubMed]
  • Sepulveda C, Yoshida A, Ullrich T. Palliative care: The World Health Organization's global perspective. Journal of Pain & Symptom Management. 2002;24:91–96. [PubMed]
  • Shadish W, Cook TD, Campbell DT. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston: Houghton Mifflin Company; 2002.
  • Singer J, Willett J. Applied longitudinal data analysis: Modeling change and event occurrence. New York: Oxford University Press; 2003.
  • Skalla K, Bakitas M, Furstenberg C, Ahles T, Henderson J. Patients' need for information about cancer therapy. Oncology Nursing Forum. 2004;31(2):313–320. [PubMed]
  • Steinhauser KE, Clipp EC, Hays JC, Olsen M, Arnold R, Christakis NA, et al. Identifying, recruiting, and retaining seriously-ill patients and their caregivers in longitudinal research. Palliat Med. 2006;20(8):745–754. [PubMed]
  • Temel J, Jackson V, Billings A, Dahlin C, Block S, Buss M, et al. Phase II Study: Integrated palliative care in newly diagnosed advanced non-small cell lung cancer patients. Journal of Clinical Oncology. 2007;25(17):2377–2382. [PubMed]
  • Teno JM. Quality of care and quality indicators for end-of-life cancer care: Hope for the best, yet prepare for the worst. In: Foley KM, Gelband H, editors. Improving Palliative Care for Cancer. Washington, D.C: National Academy Press; 2001. pp. 96–131.
  • Whedon M. Revisiting the road not taken: Integrating palliative care into oncology nursing. Clinical Journal of Oncology Nursing. 2001;6(1):27–33. [PubMed]
  • World Health Organization. Palliative Care: What is it. [Retrieved December, 2005];2003 from
  • Zimmermann C, Riechelmann R, Krzyzanowska M, Rodin GC, Tannock I. Effectiveness of Specialized Palliative Care: A Systematic Review. JAMA. 2008;299(14):1698–1709. [PubMed]