|Home | About | Journals | Submit | Contact Us | Français|
Objective To develop an information model for automating evaluation of concordance between patient preferences and end-of-life care.
Methods We modeled and validated 15 end-of-life care preference option domains, to which we mapped preferences recorded in standardized advance care planning documents and 232 end-of-life care events defined by procedure and medication codes. Patient preferences and end-of-life care events were available in electronic health records. Data from Kaiser Permanente Southern California modeling and testing populations were evaluated for concordance between patients’ preferences and the end-of-life care events they experienced.
Results The information model successfully assessed concordance between patient preferences and end-of-life care events. Among 388 expired patients in the modeling population, 4164 care events occurred, 4100 (98%) of which were preference-concordant, and 64 (2%) of which were preference-discordant. Including end-of-life care events that did not occur increased the number of observations to 6029; 99% were preference-concordant. At the level of individuals, 72% (278) of patients experienced only preference-concordant care events, 13% (50) experienced at least one preference-discordant care event, and 15% (60) experienced no preference-related care events.
Discussion Model limitations pertain to assumptions that are required to match advance care planning documents with patient preference options and exclusion of preferred care that did not occur. Further research is required to apply the model to larger populations and to investigate the need for additional preference options.
Conclusion An information model for automating the assessment of the concordance between patients’ advance care planning preferences and the end-of-life care they received was effective in a small population and has the potential to assess population-level preference-concordance on an ongoing basis.
The goal of advance care planning is to allow patients to maintain their autonomy should they become incapable of making care decisions on their own behalf, ensuring that patients can be provided with patient-centered care even when they cannot express their values and preferences or advocate for themselves.1 To achieve this goal, advance care planning activities must be documented in patients’ health records and be able to be readily retrieved in any care setting.
In addition, it is essential that healthcare providers adhere to patients’ values and goals. However, there is conflicting evidence about the extent to which providers do so. Estimates of overall concordance of patient preferences with the end-of-life care they actually receive range from 41-86%.2–6 Patients who prefer life-extending treatments were more likely to receive them,7–9 but more than half of those preferring such treatments received only comfort measures.8 Conversely, among inpatients, 70% preferred comfort care, but 63% of these inpatients received at least one life-extending treatment, and 80% had a “do not resuscitate” (DNR) order listed in their medical record, but 18% of these patients received cardiopulmonary resuscitation (CPR).10 Older patients preferring life-extending treatments were less likely to receive them than younger patients, and black patients were less likely to experience preference-concordant care than white patients, indicating that systematic biases may exist.11,12
Previous assessments of concordance between patients’ end-of-life preferences and the care they receive share several limitations. Many such studies rely on patient interviews to ascertain their preferences or previous documentation of their preferences. Few studies examine the relationship between end-of-life care and advance care planning documents in health records, which may include advance directives, physician orders for life-sustaining treatments (POLSTs) and code status (such as DNR orders), and powers of attorney.5,9,13,14 These documents provide varying degrees of specificity regarding patient preferences for end-of-life care (see Supplementary Appendix Table 1).
Although a few studies have assessed administrative indicators of care (e.g., emergency department visits, hospitalizations),15–17 patient treatment near the end of life is assessed primarily by chart review and postmortem caregiver interviews.3,4,7,12,18–27 Postmortem interviews with caregivers may be subject to response bias.28,29 In addition, these time-consuming methods, by their nature, constrain sample sizes, which, in existing reports, range from a few dozen to approximately 1500 individuals.2–10,30,31 One exception is a single population-based study examining the association of a POLST with place of death among nearly 18000 decedents.13
Population-based assessments of concordance between documented patient preferences and end-of-life care events are essential for identifying and ameliorating system-level issues in end-of-life care that may be represented by preference-discordant events. Such assessments require the ability to identify and extract discrete data on patient preferences and the care they receive, which are stored across unrelated databases in an electronic health record (EHR), and evaluate them within an organizing framework that enables comparing patient preferences and received care. We report on the development of information models to represent the domains of advance care planning preferences and end-of-life care and their combined use, to assess concordance between patient preferences and received care. Our goal was to develop a method for identifying episodes and rates of discordant care from administrative data systems that can be used for quality improvement.
Information models were developed for advance care planning preference options and end-of-life care events at Kaiser Permanente, which, with 10 million members in seven regions, is among the largest not-for-profit integrated healthcare delivery systems in the United States. An integrated, comprehensive EHR, KP HealthConnectTM, is available in all Kaiser Permanente regions and care settings and captures all care events that occur within the Kaiser Permanente system. To facilitate recording advance care planning preferences and retrieving patient preference documents, Kaiser Permanente Southern California implemented a Care Directives Activity tab in their EHR. This tab provides a single, consistent location for entering and retrieving scanned end-of-life care preference documents.32
We developed the information model based on a population of 421 patients, aged 65 years or older, who passed away during or after an inpatient event at a single Kaiser Permanente Southern California medical center during the study period, between January 1 and September 30, 2013. Of these patients, who were hospitalized 740 times during the observation period, 388 (92%) had end-of-life care preferences documented in the EHR in standardized advance care planning documents and were included in the modeling population.
We tested the information model on 393 patients from the same medical center, who were also aged 65 years or older, had an emergency room visit or inpatient event requiring admission to the intensive care unit between January 1 and September 30, 2013, and were not included in the modeling population. Of these patients, who were hospitalized 605 times during the observation period, 354 (90%) had end-of-life care preferences documented in the EHR and were included in the testing population. Of these 354 patients, 60 patients were excluded because they were also part of the modeling population; the final testing population comprised 294 patients.
To systematically assess advance care planning preferences in the study population, we first converted the patients’ end-of-life care preference document information into discrete data. We collected all scanned end-of-life care preference documents for the modeling and testing populations, which included Kaiser Permanente and external advanced directive forms, POLSTs, and powers of attorney. Code status orders, including DNR orders, were available in the EHR.
There were 1099 end-of-life care preference documents for the modeling population: 842 (77%) code status orders, of which 456 (54%) were DNR orders and 386 (46%) were orders for full resuscitation; 160 (15%) POLSTs; 76 (7%) advance directives; and 21 (2%) powers of attorney that were not part of advance directives. There were 435 end-of-life care preference documents for the testing population: 391 code status orders, of which 86 (22%) were DNR orders and 305 (78%) were orders for full resuscitation; 29 (7%) POLSTs; 14 (3%) advance directives; and 1 (0.2%) power of attorney that was not part of an advance directive.
We entered the data from the scanned end-of-life care preference documents into a spreadsheet, for each document type, for each population. To define the domain of advance care planning preferences, we pooled patient preference data across populations and preference document types and identified 15 options that collectively captured patient preferences. We excluded a few preferences with nuances that could not be generalized in the model (eg, avoiding and continuing to enjoy a quality life, perpetual masses, etc.).
Next, we developed a scheme for coding documented patient preferences for the 15 advance care planning options. We began by examining the state-specific POLST, which included three major domains with a total of eight preference options:
The POLST options did not represent the 15 advance care planning preferences in sufficient detail (Figure 1). The three POLST treatment options were related to 13 end-of-life care preferences, so more granular mapping was required. Advance directive preferences and code status orders overlapped POLST preferences and also required more granular mapping. Consequently, we created dichotomous codes for the 15 care preferences (Table 1), using clinical judgment to translate any vague language about patient preferences in advance care planning documents into discrete values.
The end-of-life care preference options documented in POLSTs, advance directives, and code status orders were then mapped to the coding scheme. Two POLST-documented preferences, CPR and artificial nutrition, could be mapped directly. Coding preferences for end-of-life care intensity (comfort care only, partial treatment, or full treatment) required the following assumptions. We inferred that the partial treatment preferences both included preferences for dialysis unless otherwise specified and precluded preferences for cardioversion or defibrillation, because these procedures would be explicitly requested along with CPR. We used clinical judgment to infer care preferences related to antibiotics and intravenous fluids that differed from the POLST language. For instance, some patients may benefit from intravenous fluids at the end of life, and antibiotics may be appropriate as a comfort measure in some cases, eg, to address a cough or an oozing wound. Therefore, we inferred that patients with comfort-only end-of-life care preferences preferred both treatments. In addition, we inferred that patients preferred to die at home unless they specified treatment preferences. Specifically, we inferred that patients with comfort-only goals preferred to die at home and that patients with partial and full treatment goals preferred to receive more intensive treatment that increased the likelihood of dying in the hospital (Table 2).
Advance directive-documented care preferences could be coded by following the preference document’s language and POLST assumptions. Mapping code status orders to the 15 preference options required slightly modified assumptions (Table 2). All code status order preferences defaulted to “yes,” unless they explicitly stated otherwise, with the exception of the DNR preference, which precluded preferences for CPR and defibrillation.
To define the domain of end-of-life care, we collected all EHR-recorded medical care events for the modeling and testing populations, consisting of procedure and medication codes, and any additional claims for procedures or medications that were not captured in the EHR. During the observation period for the modeling population, more than 1.65 million individual care events took place, which reflected multiple occurrences of 395 events.
We used a two-step process, relying on clinical judgment to identify procedures and medications indicating care events relevant to advance care planning. Two nurse informaticists reviewed the list of 395 care events and assessed the relevance of each to one of the 15 preference options, determining that some were irrelevant to end-of-life care. This step reduced the list of care events to 261. A physician with board certification in both palliative and geriatric medicine (S.W.) further refined the list, again using clinical judgment and consulting with physician colleagues and clinical resources for additional advice. The final list included 232 end-of-life care events. The number of procedures and medication orders per advance care planning preference option varied widely. Table 3 contains examples of procedures and medications potentially indicating end-of-life care; a complete list of procedures and associated codes are available in Supplementary Appendix Tables 2 and 3.
A documented code for any procedure or medication within the clinical description indicated that an end-of-life care event occurred, and it was assigned a value of 1. If no procedures or medications within the clinical description were documented, the end-of-life care event was assigned a value of 0.
The two domains were combined into a single model juxtaposing end-of-life care events with patients’ documented preferences. All patient-level document and end-of-life care event data were used to create new datasets consisting of binary values for the 15 care preferences and related end-of-life care events. Each event was considered in relation to the patient’s most recently documented advance care planning preference.
Figure 2 illustrates the relationship between codes for documented patient preferences and end-of-life care events for a hypothetical patient and concordance between them, which is indicated by a value of 1, for preference-concordant care, and a value of 0, for preference-discordant care. Preference-concordant care occurred when patients experienced end-of life care events they preferred or did not experience end-of-life care events they did not prefer. Preference-discordant care occurred when patients experienced end-of-life care events they did not prefer. The information model did not support assessing preference-discordance in which patients did not experience the end-of-life care events they preferred, because we could not ascertain whether such events did not occur because because they were not medically necessary or not appropriate.
We measured event-level concordance by tabulating all end-of-life care events that were documented as occurring and for which there was a recorded patient preference or nonpreference and all end-of-life care events that were not documented as occurring and for which there was a recorded patient nonpreference. We assessed patient-level concordance by tabulating, for each patient, all end-of-life care events documented as occurring and for which there was a recorded patient preference or nonpreference and end-of-life care events not documented as occurring and for which there was a recorded patient nonpreference. Concordance evaluation was automated in SAS 9.2, which enabled us to perform millions of concordance assessments for the study population at the patient and event levels for the 15 advance care planning preferences and related end-of-life care events.
At the level of care events, 4164 end-of-life care events occurred among the 388 patients in the modeling population and were concordant with preferences in 4100 (98%) instances and discordant in 64 (2%) instances. Including observations for patients’ nonpreferred care events that did not occur increased the number of concordance observations to 6029, of which 99% were preference-concordant. We excluded 7182 care events that did not occur and for which patients had documented end-of-life care preferences (Figure 3).
At the patient level, 72% (278) of patients in the modeling population experienced only preference-concordant care events, 13% (50) experienced at least one preference-discordant care event, and 15% (60) experienced no recorded preference-related care events.
At the event level, 38 of the 294 patients in the testing population experienced end-of-life care events, totaling 968 events. End-of-life care events that occurred were concordant with the patient’s preferences in 964 (99%) instances and discordant with the patient’s preferences in 4 (<0.01%) instances. Including observations for nonpreferred end-of-life care events that did not take place increased the number of concordance observations to 1125, of which 99% were preference-concordant. We excluded 1087 end-of-life care events that did not occur and for which patients had documented preferences (Figure 4).
At the patient level, 90% (34) of patients in the testing population experienced only preference-concordant care events, 5% (2) experienced at least one preference-discordant care event, and 5% (2) experienced no recorded end-of-life care events.
We successfully modeled the domains of advance care planning preferences and end-of-life care events and combined the models to assess the concordance between patients’ preferences and the care they actually received. The information model provides a framework for conducting efficient assessments of the degree to which end-of-life care is patient-centered. We also tested the model in a separate population, with similar results.
The information model has several limitations. As we noted earlier, we made assumptions to map some information contained in advanced directives and code status orders to the 15 advance care planning preferences, but we did not attempt to validate our assumptions. We aimed to make conservative assumptions and consulted with colleague physicians about the extent to which we succeeded in that aim, but a different set of assumptions could result in different findings. Some procedures and medications that we selected as being indicative of end-of-life care events are also used in care before the end of life (eg, such as elective external cardioversion and intravenous administration of adenosine and sodium bicarbonate; see Table 3). We captured an unknown number of procedures and medications that were used before the end of life without regard to documented patient end-of-life care preferences, the net effect of which would have been to increase the proportion of preference-discordant care. As a result, the estimated rate of preference-discordant end-of-life care reported herein is conservative.
Although the information model we describe is broadly applicable, the ability to implement it is contingent on an integrated EHR that provides current advance care planning preference documents and comprehensively captures end-of-life care events as discrete data. The availability of documented advance care planning preferences was enhanced by the Care Directives Activity tab in the Kaiser Permanente Southern California EHR, because, in general, locating advance care planning preferences within the EHR is challenging.33 The degree to which the model can be meaningfully applied is contingent on provider-patient advance care planning conversations and whether or not patients completed related end-of-life care preference documents, preferably in an electronic format, but estimates of the proportion of the older adult population with completed advance care planning documents vary widely.34,35
The primary purpose of this report is to describe automating the process of assessing concordance, not to present concordance results for comparison with other reports. We are currently applying the model to assess concordance in larger populations and will report those findings separately. However, characteristics of the model itself and of the modeling and testing populations likely contributed to the concordance rates reported here. First, we found event-level rates of preference-concordant care that are higher than such concordance rates reported elsewhere in the literature.2–6 Existing reports assess concordance at the level of patients, rather than events; we were unable to identify benchmarks for event-level concordance between patient preferences and the end-of-life care patients received. At the patient level, concordance was within the previously reported range.2–6
No recorded end-of-life care events occurred for 15% of the patients in the modeling population and 5% of the patients in the testing population. These portions of the two populations included patients who died after hospital discharge, those who preferred care events that were not medically necessary or appropriate, and those who did not experience preferred end-of-life care events that were medically necessary and appropriate. The last category represents preference-discordant care; however, because we were unable to distinguish between the three categories, the model excluded all preferred care events that did not occur. Such instances collectively accounted for 54% of all the end-of-life care events in the modeling population. Had we been able to clearly identify preferred end-of-life care events that were medically necessary and appropriate but did not occur, the proportion of event-level, preference-concordant care would be lower than what we report here.
In addition, although patients in the modeling and testing populations experienced different rates of preference-concordant care, the two populations were not intended to be used to enable concordance comparisons. The modeling population included 388 patients who died, all of whom experienced end-of-life care events. In contrast, among members of the testing population, who were admitted to the intensive care unit but did not necessarily die during the study period, only 38 (12%) experienced end-of-life care events.
An inherent limitation of using advance care planning documentation to assess patients’ end-of-life care preferences is the possibility of discordance between end-of-life care preference documentation and a patients’ actual preferences. Updating documents as patients’ conditions change is imperative; our project did not assess the currency of documentation; the model assessed only care that was delivered in the inpatient setting.
Our report’s strengths include the fact that, to the best of our knowledge, it is the first to assess automated assessment of advanced care planning preferences and end-of-life care using administrative data. The information model we present can be tailored for application to various setting- or population-specific patient preference and care domains and assumptions, supporting performance improvement by trending end-of-life care concordance over time.
An information model for automating the assessment of the concordance between patients’ advance care planning preferences and the end-of-life care they received was effective in a relatively small population consisting of patients who had died. It provides a foundational framework for assessing population-level concordance between patient preferences and end-of-life care on an ongoing, timely, and efficient basis, identifying system-level issues and biases and improving patient-centered care near the end of life.
M.T., S.W., D.M., M.H.K., and T.G. were involved in the conception and design of the study. M.T. acquired the data that were analyzed. M.T., S.W., D.M., M.H.K., and T.G. analyzed and interpreted the data. M.T. drafted the manuscript. S.W., D.M., M.H.K., and T.G. revised the manuscript for important intellectual content. All authors approved the final version of the manuscript. All authors had access to all of the data in the study and can take responsibility for the integrity of the data and the accuracy of the data analysis. M.T. is the guarantor.
This study was supported in its entirety by Kaiser Permanente.
The Kaiser Permanent Southern California Institutional Review Board approved this study.
Supplementary material is available online at http://jamia.oxfordjournals.org/.