|Home | About | Journals | Submit | Contact Us | Français|
There are few randomized controlled trials of the effectiveness of palliative care.
To determine the effect of a palliative care intervention on quality of life (QOL), symptom intensity, mood, and resource utilization.
Randomized controlled trial (November 2003-May 2008) of 322 patients with advanced cancer and an identified caregiver in a rural, NCI-designated comprehensive cancer center (the Norris Cotton Cancer Center, Lebanon, NH) and affiliated outreach clinics and Veteran’s Affairs Medical Center (White River Junction, VT).
A multi-component, psycho-educational, palliative care intervention (Project ENABLE) conducted by an advanced practice nurse consisting of 4 weekly educational sessions and monthly follow-up until death or study completion.
(1) The Functional Assessment of Chronic Illness Therapy-Palliative (range: 0 to 184; higher scores indicate better QOL), (2) Edmonton Symptom Assessment Scale (range: 0 to 900; higher scores indicate greater symptom intensity), (3) Center for Epidemiological Studies-Depression (range: 0 to 60; higher scores indicate more depressive symptoms), completed at baseline, 1 month and every 3 months until death or study completion, (4) days in hospital, intensive care unit (ICU), and emergency department visits recorded in the medical record.
322 participants with gastrointestinal (41%), lung (36%), genitourinary (12%), and breast (10%) cancer were randomized. Estimated treatment effects (intervention minus usual care) for all subjects were 4.6 (P = .02) for QOL, −27.8 (P = .06) for symptom intensity, and −1.8 (P = .02) for depressed mood. Estimated average treatment effects in the sample of participants who died during the study were 8.6 (P = .02) for QOL, −24.2 (P = .24) for symptom intensity, and −2.7 (P = .03) for depressed mood. Days in hospital, intensive care unit, and emergency department visits were not different between groups.
Compared to participants receiving usual oncology care, participants receiving a palliative care intervention addressing physical, psychosocial, and care coordination provided concurrently with oncology care had higher QOL and mood; comparisons of symptom intensity and days in hospital, ICU, and emergency department visits were not statistically significant.
clinicaltrials.gov Identifier: NCT00253383
Fifty percent of persons with cancer are not cured of their disease; 1 however, with improved treatment even patients with advanced disease may live for years. Providing palliative care concurrent with oncology treatment has been proposed to improve quality of life (QOL) for patients with advanced cancer. 2–8 The National Consensus Project Clinical Practice Guidelines for Quality Palliative Care recommends palliative care referral at the time of a life-threatening diagnosis and other core elements including: multidimensional assessment to identify, prevent, and alleviate suffering, interdisciplinary team evaluation and treatment in selected cases, effective communication skills and assistance with medical decision-making, skill in care of dying and bereaved, continuity of care, equitable access, and commitment to continued improvement and excellence.9 However the evidence supporting many of these recommendations is sparse.6
We translated effective strategies from the literature10, 11 and our prior work, including a 3 year palliative care demonstration project, Project ENABLE (Educate, Nurture, Advise, Before Life Ends), 12–17 into a palliative care multi-component intervention that was consistent with guideline9 essential core elements. Specifically, the intervention included a palliative care advanced practice nurse-administered, phone-based, intensive curriculum and on-going assessment and coaching in problem-solving, advance care planning, family and health care team communication strategies, symptom management and crisis prevention, and timely referral to palliative care and hospice resources. We hypothesized that patients exposed to this palliative care intervention soon after a new diagnosis of an advanced cancer would become informed, active participants in their care and would experience improved QOL, symptom relief, mood, and lower resource use over the illness including at the very end-of-life (EOL) compared to care as usual. Therefore the goals of this study were to determine whether a palliative care intervention, that was introduced at the time of a new diagnosis of an advanced stage cancer, could influence QOL, symptom intensity, mood, and resource utilization. We also examined caregiver outcomes (e.g. caregiver burden, perceptions of EOL care, and grief), but a full discussion of these results is beyond the scope of this paper.
Project ENABLE II was a randomized controlled trial of a palliative care intervention compared to care as usual for persons newly diagnosed with advanced cancer. The primary endpoints were patient-reported QOL, symptom intensity, and resource utilization. Mood was a secondary outcome. Enrollment began in November 2003 and ended in May 2007. Data collection of patient-reported outcomes closed December 31, 2007; outcomes that could be monitored via chart review (e.g. resource utilization and vital status) were collected through May 1, 2008. The study protocol and data and safety monitoring plan were approved by the institutional review boards of the Norris Cotton Cancer Center (NCCC)/Dartmouth College and the Veterans Administration Medical Center (VAMC), White River Junction, VT and registered in the National Cancer Institute’s (NCI) PDQ database (clinicaltrials.gov; Identifier: NCT00253383). All patient and caregiver participants signed a document confirming their informed consent.
Patients identified at NCCC tumor boards with a life-limiting cancer (prognosis of approximately 1 year) were eligible if they were within 8–12 weeks of a new diagnosis of gastrointestinal (unresectable stage III or IV), lung (stage IIIB or IV non-small cell or extensive small cell), genitourinary (stage IV), or breast (stage IV and visceral crisis, lung or liver metastasis, ER −, Her 2 neu +) cancer. Patients with impaired cognition (< 17 on a modified Mini Mental State Exam), 18 an Axis I psychiatric disorder (schizophrenia, bipolar disorder), or active substance use were excluded. Patients were asked to select a caregiver to participate in the study. Patients who did not select a caregiver were not excluded from the study.
Patients and their caregiver were randomly assigned to the ENABLE intervention or usual care using a stratified randomization scheme developed for each of the two primary sites (NCCC or VAMC). The schemes were stratified by disease and blocked within strata (block lengths of 2 and 4 varied randomly). Research assistants notified the participant of group allocation when the baseline assessment was returned. Referring clinicians were not informed of nor formally blinded to participant assignment.
The intervention has been described in detail elsewhere. 19 Briefly, the intervention, based on the Chronic Care Model,20–24 used a case management, educational approach to encourage patient activation, self-management, and empowerment. We refined and converted the in-person and group strategies used in our prior studies and demonstration project 12–14 to a manualized, phone-based format to improve access to palliative care in our rural population. One of two advanced practice nurses (APN) with palliative care specialty training conducted 4 initial structured educational and problem-solving sessions and at least monthly telephone follow up until the participant died or the study ended. APN caseloads were balanced by diagnosis and gender.
The APN began all contacts with an overall assessment by administering the Distress Thermometer (DT), an 11-point rating scale (0–10) of distress recommended by the National Comprehensive Cancer Network guidelines.25, 26 In addition to an overall intensity rating, the DT identified sources of distress in 5 areas: 1) Practical Problems (e.g., work/school); 2) Family Problems; 3) Emotional Problems; 4) Spiritual / Religious Concerns; and 5) Physical Problems. If distress intensity was rated greater than 3, the APN explored the sources of distress and identified if the participant would like to apply the problem solving approach to address the issues. They then covered the assigned module for that session. The education manual entitled, “Charting your Course: An Intervention for People and Families Living with Cancer”, developed during ENABLE I, 13, 17, 27 contained four modules: 1) problem solving, 2) communication and social support, 3) symptom management, 4) advance care planning and unfinished business, and an appendix listing supportive care resources (available from the authors or on the web at http://www.cancer.dartmouth.edu/palliative/index.shtml). On average, Session 1 (introduction and problem solving) lasted 41 minutes and sessions 2–4 each lasted 30 minutes. Following the 4 formal sessions the APN was readily available by phone and also telephoned the participant (or their caregiver) at least monthly (until the participants’ death) to follow up on active issues and assess the need for referral to appropriate care resources (e.g. palliative care team, hospice, etc.). When concerns were identified participants were encouraged to contact the oncology or palliative care clinical teams (if they had received a palliative care team consultation). However, with the participant’s permission the APN would, at times, contact the appropriate clinical team about issues requiring attention (e.g., unrelieved pain) or referrals to community resources (e.g., spiritual counselor). The clinical teams were responsible for all medical decisions including medication and inpatient care management; however, the APN, in consultation with the team, could facilitate referrals to ancillary resources.
Additionally, intervention participants and their caregiver were invited to attend monthly group medical appointments28, 29 led by a certified palliative care physician and nurse practitioner. These appointments allowed participants and caregivers to ask questions about medical problems or related issues (e.g. symptom management, insurance, social services) and to have more in-depth discussions than is practical during typical clinic visits. 29
Training of study interventionists in problem-solving and group medical appointments was provided by one of the study team psychologists (J.S.). Initial training took approximately 20 hours for the two nurse interventionists and 12 hours for the nurse practitioner and physician SMA facilitators. Training methods included didactic presentations, written treatment manuals, and role playing with feedback (all training materials are available on request from the authors). Thereafter the study team, including the palliative care-certified nurse practitioner and physician, psychologists, and other team members met bi-weekly to review the APNs’ audio-taped educational sessions and to provide feedback on difficult patient management issues.
Participants assigned to usual care were allowed to use all oncology and supportive services without restrictions including referral to the institutions’ interdisciplinary palliative care service. The VAMC site had an Advanced Illness Coordinated Care Program which provided consultation to oncology staff for inpatients with life-limiting illness.
Participants completed baseline questionnaires upon enrollment. Follow-up questionnaires were mailed one month after baseline and every three months until the participant died or study completion (12/31/07). QOL was measured by the Functional Assessment of Chronic Illness Therapy- Palliative Care (FACIT-Pal). 30,31 This 46-item tool measures physical, emotional, social, and functional well-being in addition to concerns relevant to persons with life-threatening illness (e.g., feeling peaceful, reconciling with others). Scores range from 0–184; higher scores indicate better quality of life. Cronbach’s alpha for our sample was .80. Symptom intensity was measured by a modified Edmonton Symptom Assessment Scale (ESAS). 32, 33 The ESAS assessed nine symptoms (pain, activity, nausea, depression, anxiety, drowsiness, appetite, sense of well-being, and shortness of breath) using numerical visual analogue scales with discrete check boxes (0–10). Scores were multiplied by 10 to allow comparisons with other studies that used a 100cm line to calculate symptom intensity; hence consistent with other studies, our scores range from 0–900; higher scores indicate greater symptom intensity. Cronbach’s alpha in our sample was .80. Mood was measured by the Center for Epidemiological Study- Depression Scale (CES-D). The CES-D is an established 20-item measure. 34,35 Scores range from 0 to 60; a score of 16 or higher generally indicates a clinically significant level of depressed mood. 34 Cronbach’s alpha in our sample was .84. Chart review outcomes of resource use (days in hospital, ICU, ED visits) and vital status were collected by chart review until death or 5/01/08.
The original target sample size of 400 was chosen to provide 80% power to detect treatment effects of at least 0.35 standard deviations for FACIT-Pal-Total Score, ESAS-Total Score, and CES-D based on a t-test comparing the treatment groups with respect to the last observed value with a two-sided alpha of 0.01. However, at the date of planned closure of enrollment (5/1/07), the final sample size was 322 due to slightly slower accrual than anticipated.
Our primary outcome measures were QOL (FACIT-Pal), symptom intensity (ESAS) and resource utilization; mood (CES-D) was a secondary outcome. For QOL, symptom intensity and mood, we conducted two sets of longitudinal, intention-to-treat analyses for all participants with baseline and one or more follow-up assessments using repeated measures analysis of covariance to examine (a) the impact of the intervention on the total sample in the year after enrollment and (b) the impact of the intervention on the sample of participants who died.
In the first set of analyses, we proceeded forward in time from enrollment. Age and baseline outcome data were used as adjusting variables. Adjusted means were estimated for the intention-to-treat groups including all assessment data. For these analyses, we applied a mixed effects model for repeated measures to the longitudinal data using random subject effects to account for correlation between repeated outcome measurements on the same individual. Confidence intervals and p values were formed for the overall average effects.
The second set of analyses was restricted to the sample of participants who had died during the study (as of the final chart review on 5/1/08) and had completed baseline and one or more additional assessments. We applied the same intention-to-treat longitudinal model to estimate the mean overall treatment effect for this subsample using the three assessments prior to death.
Mean, median, and maximum values were calculated for chart review data on days in hospital, ICU, and emergency visits at baseline and the sums of the total days/visits over the length of enrollment. Groups were compared using Wilcoxon rank sum test (See Table 1).
For all analyses, we examined the baseline covariates that were predictive of missing data, and found that both treatment and the baseline outcomes were statistically significant predictors. We then included these as adjusting variables in our analyses to meet the conditions for “missing at random”.36 Two-sided p-values <.05 are described as statistically significant. All calculations were performed using SAS (version 9.1).
We did an exploratory, post hoc analysis of survival. We used a log-rank test to compare Kaplan-Meier survival curves for the two groups. We used Cox proportional-hazards regression modeling with a time-dependent indicator of time less than one year to estimate and compare the hazard ratios (HR) for intervention versus usual care groups before and after one year from enrollment. We examined these timeframes because our intervention was designed for patients with a projected survival of approximately 1 year. The effect of adjusting for anticancer treatment and site was examined by including terms for chemotherapy and radiation therapy in the Cox model. Because neither site nor anticancer treatment was statistically significant; we did not adjust the analyses for these variables.
Of 1222 patients screened between November 2003 and May 2007, 681 were eligible and were approached and 322 were enrolled (47% participation rate) (Fig. 1). Following consent, participants were randomly assigned to receive usual care (N=161) or the intervention (N=161). Subsequently, 27 usual care participants dropped out (8 died; 19 withdrew) and 16 intervention participants dropped out (3 died; 13 withdrew); 134 usual care and 145 intervention participants were analyzed for patient-reported outcomes. Table 1 shows no statistically significant differences at baseline between intervention and usual care groups for demographic and clinical characteristics, the use of chemotherapy or radiation anticancer treatments, advance directives, palliative care or hospice referral, days in hospital or intensive care unit (ICU), or emergency department visits. Our sample included slightly more men than is typical of the general population due to the predominant male population at our VAMC recruitment site. Over the course of the study there was no statistically significant difference between the groups relative to the number of participants who received parenteral chemotherapy (usual care=116/161 [72%] vs intervention 119/161 [74%] P = 0.80 [Fisher exact test]) or radiation therapy (usual care = 34/161 [21%] vs. intervention 36/161 [22%] P = .89 [Fisher exact test]).
Of the 681 eligible patients there were no statistically significant differences between participants (n=322) and non-participants (n=359) relative to age, gender, Karnofsky Performance Scale score (KPS), referral to hospice, or ICU days in the prior 3 months. The reasons given by the 359 who declined were 43% (n=156) not interested, 19% (n=67) “too much work”, 13% (n=46) “did not need it”, 9% (n=33) “too busy”, 9% (n=33) “too ill” and 7% (n=24) no reason provided.
There were no statistically significant differences at baseline between the groups for the three patient-reported outcomes (Table 1). Longitudinal intention-to-treat analyses for the total sample revealed higher QOL (4.6 [Standard Error [SE], 2] P = .02 [Fig, 2, Panel A]), a trend toward lower symptom intensity (−27.8 [SE, 15] P = .06 [Fig. 2, Panel B]), and lower depressed mood (−1.8 [SE, 0.81] P = .02 [Fig. 2, Panel C]) in the intervention compared to the usual care group
Longitudinal analyses for the subset of participants who died during the study revealed a similar pattern of effects: higher QOL (8.6 [SE, 3.6] P = .02 [Fig. 3, Panel A]) and no differences in symptom intensity (−24.2 [SE, 20.5] P = .24 [Fig. 3, Panel B]) and lower depressed mood (−2.7 [SE, 1.23] P = .03 [Fig. 3, Panel C]), in the intervention group relative to the usual care group.
There were no statistically significant differences between intervention and usual care groups in hospital days (6.6 vs. 6.5; P = .14) or ICU days (.06 vs.06; P = 1) or emergency department visits (.86 vs. .63; P = .53) (as of the final chart review 5/1/08).
Post-hoc, exploratory analyses demonstrated no statistically significant differences in survival between the groups (Fig. 4). Intervention median survival was 14 months (95% CI, 10.6–18.4) and usual care median survival was 8.5 months (95% CI, 7–11.1; P = .14 [log rank test]). After a mean follow-up of 14.6±12.8 months (median 10.7), there were 112 deaths in the intervention group and 119 in the usual care group. Forty-nine (30.4%) intervention and 42 (26.1%) usual care participants were alive at our final chart review (5/01/08) and were censored. When the model was adjusted for the use of chemotherapy and/ or radiation, all results were similar. Relative to the usual care group, a Cox proportional hazards model estimate demonstrated that there was a reduced relative risk of death (hazard ratio [HR], .67 [95% CI, .496-.906] P = .009) in the intervention group during the first year of the study and a greater relative risk after one year, (HR, 1.56 [95% CI, .908–2.655] P = 0.11)
This study shows that integration of a palliative care intervention concurrent with anti-cancer treatments demonstrated higher QOL (measured by an instrument designed for this specific population),19, 30 lower depressed mood, and limited impact on symptom intensity in intervention participants relative to those receiving usual cancer care. The intervention had no apparent effect on the use of hospital and ICU days, ED visits, or anticancer treatment as the proportions of intervention and usual care participants receiving these therapies were similar. To our knowledge, this is the first adequately powered RCT designed to test a palliative care intervention concurrent with oncology treatment as has been recommended by international guidelines and consensus recommendations. 2–7, 37, 38
A systematic review of specialized palliative care identified 22 trials (16 from the United States) between 1984–2007 with a median sample size of 204, half exclusively with cancer patients.6 It suggested that evidence for the effectiveness of this care was sparse and limited by methodological shortcomings including control group contamination, recruitment, attrition, and adherence issues. Our trial addressed these issues and has contributed to the growing evidence that palliative care may improve two of the main targets of care at EOL--QOL and mood. 4 In our study, intervention participants’ higher QOL and lower depressed mood may be attributed to improved psychosocial and emotional well-being. Mood is a strong determinant of the experience of QOL and suffering, despite a mounting burden of physical symptoms.39
There is no universally accepted definition of the magnitude of difference in QOL scores that is considered “clinically meaningful” or “clinically important”.40–43 Differences between groups of 4% for “improvement” or 9% for “worsening” have been cited as clinically meaningful differences using the FACT40 and we found such between group differences in our scores. Others have recommended using a distribution-based approach to compare 2 subgroups relative to the standard deviation[SD] or standard error[SE]. Differences of 0.5–1 SD or SE are considered statistically significant for most health-related QOL instruments.41, 42 In our study, QOL and mood scores demonstrated at a greater than 1 SE difference between groups. By these benchmarks, group differences for QOL and mood achieved clinical significance in addition to statistical significance.
The results did not demonstrate a group difference in symptom intensity as measured by the ESAS. In a systematic review of palliative care effectiveness, which included 14 studies that measured symptom intensity using a variety of scales, only one demonstrated improvement of one of the targeted symptoms (dyspnea). 6 In that study,44 the palliative care physician contacted the patients’ primary care physician directly with symptom management recommendations. It is possible that an intervention focused primarily on patient empowerment is not robust enough to achieve improved symptom management. Alternatively, the mean ESAS scores (scale range 0–900); were essentially in the 200s (equivalent to a rating of ‘2’ on a 0 to 10 scale) for both groups; therefore there may be little room for improvement since usual care participants also reported relatively low symptom intensity scores for patients with advanced cancer.45 It may be unrealistic to expect to reduce symptoms further in the setting of progressive disease. Finally, it is possible that symptoms were intermittently improved but the ESAS tool or our data collection schedule may not have been sensitive enough to accurately portray the dynamic, multidimensional symptom experience of this sample.38 However, despite persistent or rising symptom intensity, improved QOL and mood are still high priority patient-centered goals.46–48 When little can be done to reverse or halt the disease, the preservation of emotional well being is perhaps paramount.
The ENABLE intervention was designed to educate and provide on-going support to patients (from enrollment/diagnosis to death) with life-limiting cancers and their caregivers about symptom management, advance care planning, treatment decision-making, and communication. Beyond education, we hoped to “activate” patients by coaching them to enhance their coping and problem-solving skills over the illness trajectory.14, 49 The intervention emphasized the importance of patients taking an active role in openly communicating with family and the oncology team regarding their values, priorities, and treatment preferences. There was particular emphasis on communicating during times when anti-cancer treatments were less likely to halt disease progression or alleviate symptoms.50 Such communication has recently been demonstrated to be associated with improved QOL, reduced use of aggressive treatments at the end of life, and increased length of hospice stays.51 Unlike other studies that were specifically designed to evaluate costs,52, 53 our intervention did not demonstrate reduced use of hospital, ICU, or ED resources compared with usual care. However, data collection via chart review may have missed participants’ use of resources. Use of databases that may more comprehensively capture costs (e.g. Medicare) would address such limitations.54, 55
This study introduced a palliative care intervention concurrent with anti-cancer treatments early in the cancer trajectory to overcome prevalent patterns of late referral to hospice and palliative care. 56–58 Our findings add to the growing body of evidence confirming oncologists’ and patients’ concerns about palliative care shortening survival or hastening death are unfounded.59–62 Connor et al. 63 in their retrospective study of 4493 Medicare beneficiaries (most with cancer), who died with and without hospice care, suggested that there may be a survival advantage to palliative care.
Oncology palliative care may lead to positive outcomes by a number of mechanisms. First our intervention may have led to increased social support,64 patient activation (self-advocacy), or more coordinated and improved medical care. These factors may in turn lead to improved clinical outcomes.22, 24, 49 Second, meta-analyses in the United States 65 and Europe 66 of over 10,000 cancer patients in clinical trials that measured QOL demonstrated a strong association between higher QOL and longer survival. Third, palliative and hospice care have been associated with less aggressive cancer care, such as reduced use of chemotherapy in the days before death and reduced inappropriate use of hospital and intensive care resources in terminal patients-factors that may influence patients’QOL.67 Finally, Nelson et al.68 proposed a biobehavioral paradigm whereby interventions that enhance QOL may positively influence the psychoneuroimmune axis and improve physiological clinical outcomes. Identifying mechanisms of intervention effect on QOL are an important future area of research.
A number of limitations are worthy of note. First, consistent with the paucity of racial and ethnic diversity in this rural New England region from which our sample was drawn, we had limited ethnic and racial representation and therefore recognize the need to replicate this study with more diverse populations. Second, it is important to note that our intervention was primarily conducted by telephone; a strategy that has shown promise in delivery of psychotherapy69 and in encouraging screening behaviors.70 It is possible that a more robust effect, particularly in reducing symptom intensity, may have been seen with in-person interactions, as demonstrated in an outpatient palliative care intervention,71 versus our telephone-based program. However, in-person consultation was often not feasible for our debilitated, rural population, many of whom live more than an hour’s drive from the cancer center. In light of this, it is encouraging that we were able to maintain QOL and reduce depressive symptoms through telephone consultation. Further research is needed to explore optimal care delivery systems in this population.
Institute of Medicine reports, 2, 3 the National Consensus Project for Quality Palliative Care, 9 other consensus panels, 72, 73 and oncology professional societies, 74 agree that comprehensive cancer care must incorporate more than state-of-the-art disease-modifying treatment. Comprehensive, high quality cancer care includes interdisciplinary attention to improving physical, psychological, social, spiritual and existential concerns, for the patient and their family. This study provides additional evidence that early introduction of a palliative care intervention, concurrent with disease-modifying treatments, improves patient-reported QOL and mood.
Funding/Support: Dr. Bakitas is a recipient of a Department of Defense, Clinical Nurse Researcher award, an American Cancer Society Doctoral Scholarship, and a post-doctoral fellowship, Yale School of Nursing, NIH/NINR (T32NR008346). This study was supported by National Cancer Institute R01 CA101704.
Role of the sponsors: The sponsors had no role in the design, conduct of the study, collection, management, analysis, or interpretation of the data; and preparation, review, or approval of the manuscript.
We thank the study participants and oncology clinicians from the Section of Hematology/Oncology, outreach clinics, and VAMC for their cooperation with the conduct of this study (all without payment). Robert Ferguson, PhD, of Eastern Maine Medical Center, was a funded co-investigator and provided facilitator training and monitored fidelity of group medical appointments while at Dartmouth College as Assistant Professor of Psychiatry. Daphne Ellis, AS, Julie Wolf, RN, Luann E. Graves B.S.MT, CCRP, and Linda Eickhoff, M.S., all of Dartmouth College, provided diligent recruitment, data collection and data management as funded research coordinators.
Financial Disclosures: None Reported
Author Contributions:Dr. Bakitas had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Bakitas, Seville, Ahles,
Acquisition of data: Bakitas, Lyons, Hegel, Balan, Brokaw, Ahles,
Analysis and interpretation of data: Bakitas, Lyons, Hegel, Brokaw, Hull, Li, Tosteson, Byock, Ahles
Drafting of the manuscript: Bakitas, Lyons, Hegel, Brokaw, Hull, Tosteson, Ahles,
Critical revision of the manuscript for important intellectual content: Bakitas, Lyons, Hegel, Balan, Brokaw, Seville, Hull, Li, Tosteson, Byock, Ahles
Statistical analysis: Bakitas, Hull, Li, Tosteson,
Obtained funding: Bakitas, Ahles
Administrative, technical, or material support: Bakitas, Lyons, Hegel, Balan, Brokaw, Li, Tosteson, Byock
Study supervision: Bakitas, Lyons, Seville, Tosteson, Ahles