|Home | About | Journals | Submit | Contact Us | Français|
To estimate the relative importance of organizational‐, procedural‐, and interpersonal‐level features of health care delivery systems from the patient perspective.
We designed four discrete choice experiments (DCEs) to measure patient preferences for 21 health system attributes. Participants were recruited through the online patient portal of a large health system. We analyzed the DCE data using random effects logit models.
DCEs were performed in which respondents were provided with descriptions of alternative scenarios and asked to indicate which scenario they prefer. Respondents were randomly assigned to one of the three possible health scenarios (current health, new lung cancer diagnosis, or diabetes) and asked to complete 15 choice tasks. Each choice task included an annual out‐of‐pocket cost attribute.
A total of 3,900 respondents completed the survey. The out‐of‐pocket cost attribute was considered the most important across the four different DCEs. Following the cost attribute, trust and respect, multidisciplinary care, and shared decision making were judged as most important. The relative importance of out‐of‐pocket cost was consistently lower in the hypothetical context of a new lung cancer diagnosis compared with diabetes or the patient's current health.
This study demonstrates the complexity of patient decision making processes regarding features of health care delivery systems. Our findings suggest the importance of these features may change as a function of an individual's medical conditions.
Delivering optimal patient‐centered care may require restructuring the current health care system. Plans to design new health care delivery systems or to improve current systems should consider the preferences of patients at the organizational, process, and interpersonal levels. Historically, health system reforms have been conceptualized in a top‐down manner by governments and health care administrators with little regard to patients’ preferences (Wismar and Busse 2002). Patients often have little choice among health care systems. Their options are typically restricted to the health systems with which their health plans have established payment contracts. Thus, the ability to study revealed preferences for features of health care systems using observational data is limited.
Stated preference methods, such as discrete choice experiments (DCEs), provide a means to elicit information about which features of health care systems patients would most highly value if they had the opportunity to choose (Louviere, Hensher, and Swait 2000; Ryan and Farrar 2000; Bridges 2003; Bridges et al. 2008). Grounded in random utility theory, which posits that individuals act rationally to maximize their own utility (McFadden 1974), stated preference methods were developed in marketing and psychology to investigate trade‐offs between crucial attributes of products to understand purchasing behavior (McFadden 1974; Thurstone 1974; Ryan and Hughes 1997; Ryan and Gerard 2003). The theory defines the value of a product as the sum of an individual's utilities for the product's attributes. Therefore, the value of any product or service, real or hypothetical, can be estimated as a function of its underlying features (Lancaster 1966, 1971). In DCEs, respondents must choose between two or more options (i.e., “choice sets” in DCE nomenclature) that are characterized by varying levels of the options’ relevant features (i.e., attributes). For example, a patient may be asked to choose between two possible drug treatments that vary with regard to the risks of side effects and positive outcomes.
We designed and executed a DCE to characterize patients’ values for specific features of health care delivery systems by identifying how varying attributes at the organizational, procedural, and interpersonal levels influences patients’ choices and how these choices may change as a function of health status. Our goal was to provide quantitative information about the relative importance of each attribute so that patients’ priorities could be considered in the design (or modification) of health care delivery systems. We followed a detailed checklist developed by experts in the field (Bridges et al. 2011) throughout the design and analysis of the DCE.
The design and analytic methods applied in this DCE were motivated by our research question, which was to discern which organizational‐, procedural‐, and interpersonal‐level features of a health care delivery system are most important to patients and whether their importance differs when patients are confronted with different medical conditions. Although our focus was on measuring patient preferences, our aim was to inform individuals involved in designing and managing patient‐centered health care delivery systems.
Guided by existing conceptual frameworks, we developed a list of patient‐relevant attributes of health care delivery systems. These existing frameworks included the chronic care model described by Wagner (1998), which captures all relevant aspects of delivery systems needed to improve health care for persons with chronic illness, and a framework for primary care organizations described by Hogg and colleagues (2008). This framework also makes the important distinction between structural and performance domains and uses further distinctions within each domain. Our preliminary list of attributes also aligned well with the recently published modification of the World Health Organization's Innovative Care for Chronic Conditions framework (Oni et al. 2014).
To refine the list of all decision‐relevant attributes and corresponding levels within each attribute, we conducted key informant interviews (n = 9) and focus groups (n = 20). After developing the list, we completed cognitive interviews (n = 9) with individuals as they were reading the attributes and their levels. We then made revisions to the attributes to improve the validity and reliability of each item. Consistent with the models described above, these processes yielded a conceptual framework representative of preference‐sensitive health care delivery systems incorporating individual‐level, procedural‐level, and organizational‐level features containing 21 attributes (Appendix Figure S1).
Discrete choice tasks with 21 attributes would be too cognitively burdensome for individuals to complete. Ryan and Gerard (2003) suggest including 4–8 attributes per choice task. To reduce the number of attributes in each choice task to a manageable level, we deconstructed the conceptual framework into four single DCEs.
Each of the attributes and their corresponding levels were described using simple, patient‐friendly language (Appendix S1). We chose the levels to represent realistic possibilities in contemporary health systems but limited them to three levels to allow respondents to discriminate between them. After developing the four DCEs, we conducted pretest interviews with nine individuals who demonstrated that they understood the choice tasks and made logical trade‐offs.
To evaluate whether patients’ priorities for various characteristics of health care delivery systems may be influenced by their medical conditions, we randomly assigned each respondent to complete the DCEs for one of the three scenarios. In one scenario, respondents were asked to consider their current health status (i.e., status quo). In the other two scenarios, respondents were asked to imagine how they would make choices if they were recently diagnosed with diabetes mellitus, a chronic but manageable disease, or recently diagnosed with lung cancer, an acute diagnosis that is incurable. Therefore, the last two scenarios were explained as hypothetical situations.
The choice scenarios presented within a DCE are based on an experimental design that combines the attributes and levels. To generate this experimental design, we used Sawtooth Software (Sawtooth Software, Inc, Orem, Utah). To reduce respondent burden, we limited the number of unique choice tasks to 14 and required that the first task choice was repeated at the end of the experiment to control for consistency between answers. Thus, each respondent had to complete 15 choice tasks in total, as he or she was randomly assigned to 1 of the DCEs. We chose a balanced overlapping design that created 140 choice sets to maximize D‐efficiency. We divided the choice sets into 10 questionnaire versions, each consisting of the 14 unique choice tasks. The experimental design was identical for each of the four different DCEs. We replicated these questionnaire versions for each of the three patient scenarios (i.e., status quo, diabetes, lung cancer). We randomly assigned respondents to one questionnaire version, and patients completed the questionnaire according to the patient scenario to which they were assigned. An example choice set is shown in Appendix Figure S2.
We designed the survey to be an online, self‐administered survey. In addition to the DCE, we included survey items to elicit sociodemographic information (e.g., age, sex, employment status, and income), patient experience with health care delivery programs, and participants’ health‐related information (e.g., self‐rated health, chronic diseases, and hospitalizations). Based on pretesting, the survey was expected to take 35 minutes to complete. Potential survey respondents included all individuals with access to the Duke HealthView platform (now called Duke MyChart), an online portal that offers individuals personalized and secure access to view their medical records and test results, schedule appointments, request prescriptions, and communicate with medical care teams. It is available to patients who registered with the Duke University Health System. Participants were informed about the study on the HealthView website and invited to take part. An interested person could access the survey via a link in the information box.
Eligible participants were 18 years or older. Before completing the survey, each respondent provided informed consent online. The institutional review board of the Duke University Health System approved the study.
Sample size calculations represent a challenge in DCEs (Bridges et al. 2011). Minimum sample size depends on a number of criteria, including the question format, the complexity of the choice task, the desired precision of the results, and the need to conduct subgroup analyses (Louviere, Hensher, and Swait 2000). The minimum sample size according to the Orme's calculation (based on two alternatives per choice set, 14 total choice sets for each participant, and three levels for each attribute) was 54, which was the lowest possible sample size for main effects estimation for each hypothetical medical condition scenario (i.e., status quo, diabetes, lung cancer).
We applied random effects logit models to analyze the data from the four DCEs. The dependent variables represent the respondents’ preferred options for the choice tasks presented. The independent variables were the attributes and levels within each content block. We applied effects coding to calculate coefficients for each level (Bech and Gyrd‐Hansen 2005). The “best” level by content was coded positive, and the “worst” level was used as the reference level. Thus, the effect coding coefficients show the difference from the grand mean. It is likely that the middle category of a three‐level attribute will be close to the grand mean and, in this way, less likely to significantly differ from the grand mean.
By calculating a coefficient for each level of each attribute, linearity assumptions relating the levels of the attribute to the odds of being chosen could be tested and confirmed. The random effects logit models were also calculated separately for the three medical condition scenarios sets (i.e., status quo, diabetes, and lung cancer). We did not specify hypotheses about the relative importance for specific attributes across the three scenarios; thus, these analyses were exploratory.
The coefficients for each attribute level were estimated with the random effects logit models. These coefficients were exponentiated and reported as odds ratios. Odds ratios greater than 1 represent positive utilities, wherein respondents assign greater importance to the attribute level. Conversely, odds ratios less than 1 represent negative utilities and lower probability of a respondent choosing an alternative when this attribute level is shown (relative to the average).
In a DCE, utility is measured on an arbitrary scale, and the overall importance of each attribute is conditional upon the range of attribute levels chosen for the experiment. We computed the mean relative importance of each attribute by computing the difference between the highest and lowest coefficients for levels of the attribute. Then, we normalized the scale by assigning the most important attribute 10 units and measured the other attributes’ importance relative to this change. The mean relative importance score for each factor expresses an improvement from the worst level to the best level for an attribute on a scale from 0 to 10.
Because coefficients and utilities do not have a certain 0 point (Lancsar, Louviere, and Flynn 2007; Louviere, Hensher, and Swait 2000), we used the “out‐of pocket costs” attribute (which was identical in all DCE versions) to transform the estimates to a uniform scale. Inclusion of the price proxy for different health care delivery programs ($500 per year; $1,000 per year; $2,000 per year; $3,000 per year) allowed estimation of the marginal rate of substitution between the costs and the other attributes as they express these “part‐worth” values (Ryan and Hughes 1997; Gerard et al. 2008). The theory of welfare economics allows the estimation of willingness to pay based on patients’ willingness to trade between alternatives as long as a cost attribute is included (Lancsar, Louviere, and Flynn 2007; Louviere et al. 2007). By including the “out‐of‐pocket cost” attribute as a price proxy in each of the four DCEs, we are able to estimate the money value of each attribute, of combinations of attributes, or even the whole choice set (Ryan and Hughes 1997; Gerard et al. 2008; Telser, Becker, and Zweifel 2008; Becker et al. 2008), translating “intangible” values into assessable values (Schöffski, Glaser, and Schulenburg 1998).
In this approach, estimates of willingness to pay are restricted to quantifying the monetary value of changes in hypothetical health care delivery systems. From a practical perspective, when including a cost attribute within a DCE, it is important to display acceptable and realistic level spans. Otherwise, participants may be unwilling to trade, may give biased responses, or may even refuse to answer (Drummond et al. 2005; Sculpher et al. 2005). For example, participants may engage in “protest choosing,” in which they trade only against the cost attribute and do not trade between all available attributes (Ryan and Hughes 1997; Gerard et al. 2008). Also, the inclusion of a “zero cost” attribute could have led patients to use a simplifying heuristic in which they always select the scenario with zero cost without considering other features. Therefore, we decided not to include a zero cost level. In addition, willingness to pay is nearly always positively associated with income, so it is necessary to adjust for income in the estimation. Despite this adjustment, there may be other factors, such as household assets or education status that could differentially affect the impact of income on willingness to pay across participants.
Following standard consumer theory, the marginal rate of substitution between attributes can be obtained by calculating the ratio of the partial derivatives of each attribute. Willingness to pay represents the mean maximum monetary equivalent of an improvement in a single attribute, estimated by dividing the marginal utility of a change in attribute levels by the marginal utility of a change in the cost attribute. Therefore, willingness to pay can be calculated as the difference between the utility provided by a specified outcome at the lowest level (or initial level) and the utility provided after the change to the highest level, divided by the marginal utility of income. (Sometimes referred to as “implicit price” and usually represented by the coefficient of the payment attribute, this is also the utility provided by $1.00.) Hicksian compensating variation measures a change in expected utility due to a change in the level of provision in the attribute or attributes by weighting this change by the marginal utility of income. The distribution of welfare effects can be reported by bootstrap techniques (Krinsky and Robb 1986). The present study represents a cross‐sectional study, and therefore no interpretation regarding possible discounting effects is possible.
Our study included 3,900 individuals who provided consent and completed the survey between April and August 2011. Table 1 summarizes the characteristics of the study sample. The majority of the participants were women (65.5 percent), were married (62.3 percent), and had at least a college or university degree (64.1 percent). There was a broad distribution of patients with regard to self‐rated health status: 42.5 percent rated their health as “excellent” or “very good,” 32.6 percent rated their health as “good,” and 24.5 percent rated their health as “fair” or “poor.”
Table 2 reports the odds ratios and corresponding 95 percent confidence intervals for all attribute levels, as well as the differences in beta coefficients (log odds) between the worst and best levels for each attribute and the estimated willingness to pay to switch from the worst to the best level within each attribute. Figures 1, ,2,2, ,3,3, and and44 show the beta coefficients generated with the random effects logit models that represent the relative importance between the levels within an attribute with regard to respondents’ choices about hypothetical health care delivery systems. Across all four DCEs, the ordering of the point estimates were consistent with expectations with higher coefficients indicating a greater likelihood to select an alternative when more desirable attribute levels were shown and lower coefficients for less desirable attribute levels. Moreover, respondents clearly discriminated between the three levels for nearly all attributes.
The absolute values of the coefficients and the difference in importance weights for the “out‐of‐pocket costs” attributes were the greatest across all DCEs, indicating that costs to the patient had the strongest influence on the alternatives selected in the DCE.
Within DCE 1, the attribute “trust and respect” (relative importance, 9.6) was nearly equal in importance to the respondents as “out‐of‐pocket costs” (relative importance, 10.0). The attribute “attention to personal situation” (relative importance, 7.0) was the third most important attribute in the first DCE block. The remaining three attributes were similar and considered less important: patient education, accessibility to accurate health information, and care transition between institutions (relative importance range, 3.6–4.0).
After out‐of‐pocket costs, the “shared decision making” attribute was important to respondents within DCE 2 (relative importance, 8.3). The accessibility that a patient has to his or her own medical records was the next most important attribute (relative importance, 4.6), followed by the frequency with which a provider proactively follows up with the patient about his or her health. Over the range of waiting 3–14 days for an appointment, patients did not prioritize waiting time as particularly important, nor did they perceive ease or difficulty navigating the facility as being as important as the other attributes.
Choices made within DCE 3 revealed that multidisciplinary care (relative importance, 9.2) was nearly equal in importance to the range in annual out‐of‐pocket costs of $2,500 (relative importance, 10) when making choices about hypothetical health care delivery systems. Respondents also revealed that the experience of the provider, ranging from 1 to 10 years (relative importance, 6.3), and attentiveness of the care provider (relative importance, 4.3) were also moderately important. Respondents’ choices were less influenced by the ease with which they could access information about provider performance or friendliness and helpfulness of the staff (relative importance for both, 2.6).
There was relatively little discrimination between the attributes examined in DCE 4 relative to annual out‐of‐pocket costs. The relative importance values ranged from 4.8 to 5.0 for the attributes clinical information exchange, case management, treatment guidelines, and medical devices and furnishings. Across the levels of travel time included in the DCE ranging from 20 to 60 minutes, there was little influence of travel time on respondents’ choices about health care delivery systems (relative importance, 2.9).
In our examination of whether choices differed systematically between respondents who were instructed to assume that they had recently been diagnosed with diabetes or lung cancer versus those who were instructed to make choices for their current health state, we found that respondents’ mean relative importance estimates for the out‐of‐pocket cost attribute (based on the level difference estimation between the highest and lowest levels) were lowest in the lung cancer scenario across each of the four DCEs (Appendix Figure S3). Conversely, the relative importance of the out‐of‐pocket cost attribute was generally the highest among respondents considering a new diabetes diagnosis. Relative importance weights for other attributes were similar across the health state scenarios in the four different DCEs.
The Institute of Medicine report “Crossing the Quality Chasm” (2001) emphasized that health decisions should be customized based on patients’ needs and values. In health policy terms, this customization refers to services “closely congruent with, and responsive to patients’ wants, needs and preferences” (Laine and Davidoff 1996). While many publications about new health care delivery approaches concentrate on quality and structural or financial aspects, our study represents one of the relatively few studies that have investigated how health care delivery systems can be better designed to align with patient preferences (Ellrodt et al. 1997; Wensing and Elwyn 2003; Amelung 2012).
DCEs provide researchers with a method to examine the importance that patients place on different features of health care delivery systems and, more important, the trade‐offs that they are willing to make across those features (de Bekker Grob, Ryan, and Gerard 2010). The results can assist policy makers and health care institutions in decision making processes and prioritizing efforts to center delivery systems on patients by offering an additional source of information. The presented data could also be used to train health care providers to instill trust and respect when communicating with their patients and to engage in shared decision making. Furthermore, communication or decision aids could be developed so that providers can better account for patients’ personal situations and concerns about out‐of‐pocket costs.
This “patient evidence” or “patient perspective” can enhance clinical practice. Medical care should be organized to facilitate multidisciplinary care. To the extent that patients’ personal situations are considered, there may be greater need for supportive care services such as financial, social, and mental health counseling and those needs could be better handled with integrated, multidisciplinary care.
Our results revealed that out‐of‐pocket cost was the most important attribute across the four different DCEs. This finding is in agreement with other large studies and confirms that controlling costs is valuable not only from the experts’ point of view but also from patients’ perspectives (Schoen et al. 2010). Following the cost attribute, trust and respect, multidisciplinary care, and shared decision making were ranked highly, followed by attention to personal situation and experience of the provider. These findings indicate that patients value engagement and dialog with their health care providers, underscoring a desire for collaborative, patient‐centered care. When taken together with the importance of out‐of‐pocket costs, it is notable that provisions of these specific features are not particularly costly to a health care system. Factors that are significant cost drivers in the current health care marketplace were generally considered less important, including the availability of electronic health records (clinical information exchange and patient's health records), travel time to care providers, and condition of medical equipment and furnishings.
The similarity in relative importance weights for the attributes included in the DCE block examining organizational features may indicate that patients had a difficult time choosing between the importance of access to the patient's health information, case management, adherence to treatment guidelines, and the appearance of medical devices and the facility. Likewise, it might be conceivable that the attributes “patient education” and “accurate health information” could be considered to be overlapping. The intention was to address distinct concepts. The “patient education” attribute was intended to represent the extent to which physicians and other health care workers provided information to patients to assist with their understanding of their condition and decisions to be made. Conversely, “accurate health information” was designed to represent the ease with which providers could access information about their condition and health care decisions. Therefore, “patient education” represented passive receipt of information from the system, while “accurate health information” represented the extent to which patients could find “accurate, reliable, and timely health and medical information.” This information was given within the detailed attribute explanations (Appendix Table S1). Alternatively, the similarity in importance weights may indicate that patients value these attributes equally. If these organizational attributes were included in a DCE along with patient‐centered attributes, their relative weighting may change. This question could be answered empirically by conducting another DCE in which the highest‐rated attributes were combined in the alternative scenarios presented.
Although respondents appeared to assign greater importance to attributes that center on provider–patient interactions and less on organizational features, considering the provider perspective is critical with regard to prioritizing resource allocation within a delivery system. While showing trust and respect and attention to a patient's personal situation should be a priority for any medical provider, medical delivery systems should be designed and managed to instill and support these behaviors across their staff. Furthermore, even though respondents did not weight some of the organizational attributes as highly as others, organizations must carefully consider the operational infrastructure needed to support the provision of multidisciplinary care, care transition, case management, and adherence to medical guidelines.
The findings from this study may also indicate that the respondents were savvy medical consumers. Within DCE 1, the least valued attributes were patient education and accurate health information, and within DCE 3, information about performance was lowly valued. Although these features were not valued as highly as the other attributes (e.g., trust and respect, attention to personal situation), the respondents may be indicating that this type of information can be obtained from other sources.
The importance of understanding patient preferences about health delivery systems goes beyond the mantra of patient‐centered care. With the advent of accountable care organizations and other innovative payment models, there are potential short‐term economic consequences associated with falling short of patients’ expectations. With some payment models, shared savings will be tied to a score that includes patient‐rated experiences with health care. Previous studies have reported that features like timely care, appointments and information, communication with doctors, helpful and respectful office staff, patient education, and shared decision making influence patients’ experiences. Furthermore, our results support the design and implementation of innovative health care delivery systems that utilize patient engagement and patient‐centered care as a core strategy (Berwick 2011). Our study not only confirms that these features are important but provides information about their relative importance that can assist in prioritization of efforts aimed at improving health care delivery from a patient's perspective.
Respondents to the survey represented a convenience sample of individuals who voluntarily accessed an online portal. This opt‐in nature of the sample reduces the generalizability of the results and should be taken into consideration. On the whole, the respondents were well‐educated. They also represented a relatively broad range of self‐rated health status. The extent to which respondents’ health status differs from the general population of patients treated by the health system is unknown. While they may not represent the sickest patients, individuals with more health system contacts may be more likely to visit the online portal than individuals with fewer contacts. Although available data representing all individuals treated within the health system were limited, we observed that men represented a similar proportion of our sample (40.9 percent) relative to the Duke patient population (39.9 percent), but a larger proportion reported being married in our sample (58.2 percent) than in the Duke patient population (42.3 percent).
Readers should understand that willingness to pay is a measurement technique and differs from contingent valuation studies in which respondents are directly asked for prices or costs (Ryan, Gerard, and Amaya‐Amaya 2008). In our DCE, the annual out‐of‐pocket cost attribute ranged from $500 to $3,000 per year. We did not include a no‐cost level, and respondents did not have an option to choose neither scenario. Thus, the lowest amount a respondent could choose was equivalent to $500 in out‐of‐pocket costs per year, and respondents may have traded between the scenarios to obtain the best available value for money. Nevertheless, redesigned and more coordinated health care delivery programs cannot reasonably be implemented at no cost. As Americans increasingly face greater out‐of‐pocket costs for health care services, health care systems will need to better understand patient preferences and their willingness to pay for various health care delivery options through studies such as ours. Across respondents, we observed relatively homogeneous preferences for lower out‐of‐pocket costs. We did not observe this consistency for other attributes. It is not clear whether this pattern could be attributed to respondents’ inherent understanding of out‐of‐pocket costs relative to other health care system features. A similar phenomenon exists with regard to public goods financed through taxation. Individuals may highly value various public goods, such as a reliable judicial system or national security, but maintain a negative view on paying taxes. Thus, we believe the cost levels in our survey were realistic, but there may have been cases in which respondents traded only against the cost attribute instead of considering all available attributes (Ryan and Hughes 1997; Gerard et al. 2008).
The findings of this study provide guidance to policy makers, health care managers, and providers in designing new or redesigning existing health delivery systems to increase the extent to which they align with patients’ stated preferences about features they most value. A consistent finding from this study is the importance of patients’ out‐of‐pocket costs as well as the extent to which they are valued in making personalized medical decisions. In prioritizing these features, the perspectives of other stakeholders must be considered in context of the costs required to provide them and how preferences of individuals may change as a function of one's health status.
Appendix SA1: Author Matrix.
Table S1. Attributes and Levels Used in the Discrete Choice Experiment.
Figure S1. Conceptual Framework for Patient‐Centered Health Care Delivery Systems.
Figure S2. Example of Choice Set for DCE 1 Representing a Status Quo Scenario.
Figure S3. Mean Relative Importance of Out‐of‐Pocket Cost Attribute Across All Four DCEs for Three Health State Scenarios.
Joint Acknowledgment/Disclosure Statement: This work was supported internally by the Duke Clinical Research Institute.
Harkness Fellowship during the time of the study.