|Home | About | Journals | Submit | Contact Us | Français|
Dissatisfaction is an important threat to high-quality care. The aim of this study was to identify factors independently associated with dissatisfaction with critical care.
Prospectively collected observational cohort study.
Nine intensive care units at a tertiary care university hospital in the United States.
Four hundred forty-nine family members of adult intensive care unit patients who completed the Family Satisfaction with Care in the Intensive Care Unit instrument.
Four family-and patient-related factors ascertainable at intensive care unit admission independently predicted low overall satisfaction: living in the same city as the hospital, disagreement within the family regarding care, having a cardiac comorbidity but being hospitalized in a noncardiac-care intensive care unit, and living in a different household than the patient. When three or more risk factors were present, 63% (95% confidence interval 48%–78%) of families were dissatisfied. Among factors ascertained at the end of the intensive care unit stay, dissatisfaction with six items was independently associated with overall dissatisfaction: 1) perceived competence of nurses (odds ratio for dissatisfaction = 5.9, 95% confidence interval 2.3–15.2); 2) concern and caring by intensive care unit staff (odds ratio 5.0, 95% confidence interval 1.9–12.6); 3) completeness of information (odds ratio 4.4, 95% confidence interval 2.4–8.1); 4) dissatisfaction with the decision-making process (odds ratio 3.0, 95% confidence interval 1.6– 5.6); 5) atmosphere of the intensive care unit (odds ratio 2.6, 95% confidence interval 1.4–4.8); and 6) atmosphere of the waiting room (odds ratio 2.7, 95% confidence interval 1.2–6.0).
Specific factors ascertainable at intensive care unit admission identify families at high risk of dissatisfaction with care. Other discrete aspects of the patient/family experience that develop during the intensive care unit stay are also strongly associated with dissatisfaction with the critical care experience. These results may provide insight into the design of future evidence-based strategies to improve satisfaction with the intensive care unit experience.
The Institute of Medicine defines high-quality care as safe, timely, efficient, effective, equitable, and patient centered (1). In the intensive care unit (ICU), patient-centered care includes family-centered care (2–4) since patients are frequently unable to participate in their care due to sedation, delirium, and severity of illness. Measuring the quality of patient-and family-centered care is complex, but many authorities view satisfaction with care as a central outcome measure in this domain (4–6). Although increased attention has recently turned to improving the patient and family centeredness of ICU care, most data suggest that significant opportunities for improvement remain both in the United States (7) and abroad (8, 9).
Three areas of this burgeoning subject of inquiry are particularly notable. First, although survivorship has been called the defining problem of 21st century critical care (10), most research aimed at improving the ICU experience has focused on end-of-life care (5, 11–18) rather than the broader population of the critically ill, most of whom survive. Second, a great deal of research has focused on issues relating to individual patient-provider interactions, such as family conferences (9, 11, 12, 14, 15, 18–20), and less has focused on system-level issues that may affect the experience of critical illness but are not part of the individual-level encounter (3, 21–23). Third, no tools exist to guide providers in predicting which families are at high risk of dissatisfaction, or that guide researchers and quality improvement teams in selecting the most important factors on which to focus limited improvement resources.
We therefore sought to understand how patient, family, ICU, and provider characteristics were related to the risk of less-than-ideal satisfaction with critical care. We were interested in three primary questions. First, what factors–ascertainable at ICU admission–independently predicted dissatisfaction with the future ICU experience? This clinical prediction rule might be useful in the design of patient-or family-specific interventions to prevent dissatisfaction. Second, we wanted to understand whether patients’ clinical characteristics (both interventions and outcomes) were associated with the risk of dissatisfaction. Finally, we were interested in understanding what aspects of families’ perceptions of specific aspects of the ICU experience were most strongly correlated associated with overall dissatisfaction. These analyses might be useful to ICU directors and administrators in choosing improvement priorities. They might also be hypothesis generating, helping researchers choose promising targets for novel interventions. Together, these analyses might form the basis for the design of future customized interventions–based on patient-level risks and ICU-specific factors–that aim to improve the critical care experience.
This is a prospective observational cohort study conducted in a university tertiary care medical center with nine adult ICUs, 77 adult ICU beds, and approximately 500 total beds. We included all adult patients at least 18 yrs of age who: 1) stayed in the ICU for >6 hrs from April 2008 to November 2009; and 2) whose proxy or spokesperson completed a Family Satisfaction with Care in the ICU (FS-ICU) instrument within 3 days after discharge from the ICU, or who responded to a mailed survey if the patient had died. Family members were selected based on documentation as next of kin or healthcare proxy status. We excluded three (0.7%) patients for whom the primary outcome variable (“overall satisfaction”) was missing.
This study was approved by the Committee on Clinical Investigations/Institutional Review Board of Beth Israel Deaconess Medical Center with a waiver of informed consent.
The FS-ICU is a well-validated instrument designed to measure satisfaction with the critical care experience. It was rigorously derived (24) and validated in multicenter (8), international (4), and cross-cultural (25) cohorts. Detailed descriptions of the instrument and its characteristics have previously been published (4, 24).
As part of a quality improvement effort, our medical center began administering the long-form (34-question) FS-ICU in April 2008. We identified patients discharged or transferred from the ICU and administered the survey in person or by telephone to available family members. To ensure minimization of bias by the surveyor, interviewers were formally trained by a PhD methodologist. Only families who could be contacted during the interviewer’s hours of work (approximately 20 hrs per week) were interviewed. For all nonsurvivors, we mailed surveys to patients’ next of kin at 7 wks after the patient’s death. For nonresponders, a reminder letter was mailed 4 wks later. We included all respondents in the analysis.
The primary outcome variable of this study was less-than-complete overall satisfaction with critical care, which for brevity we refer to as “low overall satisfaction” or “dissatisfaction.” We ascertained this based on family members’ responses to a specific FS-ICU item that asks family members to rate “overall satisfaction with your experience in the ICU” and includes a 5-point Likert-type response scale that ranges from “completely satisfied” to “very dissatisfied” (4, 24). Responses for this item were strongly skewed toward complete satisfaction. We therefore dichotomized the outcome variable into those who were completely satisfied, defined as “high satisfaction” (n = 292; 65% [95% confidence interval (CI) 61%–69%]) and those who were not, defined as “low satisfaction” (n = 157; 35% [95% CI 31%–39%]). Although any cut point is somewhat arbitrary, we believed that this represented the best balance between competing factors. In addition, this sort of dichotomization is familiar to patients, families, clinicians, and administrators, since it is the approach that the Centers for Medicare and Medicaid Services has taken with public reporting of patient satisfaction (26).
We collected several factors present at ICU arrival as well as factors that developed during the course of the ICU stay, including patient-related factors (demographics, diagnosis, and comorbidities), ICU characteristics, and family-related factors. We collected clinical factors related to the ICU stay, including major procedures, treatment, and outcome, as well as the FS-ICU responses. In brief, the FS-ICU includes 34 questions concerning: 1) satisfaction with care, and 2) satisfaction with decision making. The responses to these questions were also generally very skewed; we converted them into binary variables for further assessment.
We calculated unadjusted odds ratios (ORs) to represent the strength of association between each variable and low satisfaction. We used Fisher’s exact test or the Wilcoxon’s rank-sum test for unadjusted tests of difference.
We built three primary models. In the first, which we refer to as the “predictive model,” we restricted candidate predictor variables to those ascertainable at ICU admission. In the second, which we refer to as the “clinical model,” we included major ICU interventions (e.g., mechanical ventilation, new dialysis, etc). In the third, we included family-reported aspects of the ICU experience that are only ascertainable after the ICU experience is underway or complete (e.g., satisfaction with concern and caring of ICU staff). Since these cannot be ascertained before overall satisfaction is measured, and are measured on the same instrument, we kept these separate from the other baseline and clinical factors. We performed model creation and variable selection as follows (27, 28). First, missing data among predictor variables were approached using multiple imputation. We constructed an augmented data set by performing 100 imputations, using a Monte Carlo Markov Chain approach with 500 burn-in iterations and 100 iterations between imputations to create samples with no missing data. Then, to prevent overfitting and reduce the risk that we would select variables for our model that were only related to outcome by chance alone, we performed a bootstrap resampling procedure. Using random resampling with replacement from the augmented, imputed dataset, we constructed 100 separate bootstrap samples. Then, within each sample, we modeled the relationship of predictor variables to overall dissatisfaction by performing logistic regression, using forward stepwise selection for variable selection. We retained variables in the model if their regression coefficients were significant in >50 of the 100 bootstrap samples at the α = 0.05 level. For final coefficient estimates, we repeated the imputation step (again using a Monte Carlo Markov Chain approach with 100 iterations), performed logistic regression on each imputation, and combined parameter estimates correctly accounting for variances using SAS PROC MIANALYZE. Finally, we conducted model validation using random resampling with replacement (bootstrapping) (29), with 1000 bootstrap samples. Variables that did not maintain significance in this validation were removed from the final model.
Of 657 family members asked to participate in the FS-ICU survey, 452 (69%) participated (330 of 332 in person, 107 of 310 by mail, and 12 of 12 by telephone interview). Three family members (0.7%) did not provide a rating of overall satisfaction, and so were excluded from analysis, leaving 449 (99.3%) in the final cohort. In 54% of cases the spouse, and in 25% of cases the adult child, answered the FS-ICU questionnaire. Family members had a median age of 58 yrs (interquartile range 49–69) and most (71%) were women (Table 1).
The median age of patients was 68 yrs (interquartile range 54–78) and 192 (43%) were women. The most common primary diagnoses are presented in Table 1. One third of patients received vasopressors; 57% were mechanically ventilated. The median length of ICU stay was 3 days (interquartile range 2–7) and the ICU mortality rate in respondents was 24%. These represent a group of patients more ill than our average ICU population, partly related to our sampling strategy, which provided more opportunities for families of nonsurvivors to respond than families of survivors. However, when stratified by survival status, there were also some important differences. Among survivors, there were no significant differences in terms of age (62.6 vs. 62.8, p = .9 for responders vs. nonresponders) or gender (58% male vs. 56% male, p = .5), but ICU length of stay was longer (5.6 days vs. 3.4 days, p < .0001) and case mix was higher (4.0 vs. 3.0, p < .0001). These difference were less apparent among nonsurvivors (age: 71.4 vs. 69.8 yrs, p = .4; gender [57% male vs. 52% male], ICU length of stay [6.3 vs. 5.9 days, p = .04], and case mix [3.20 vs. 3.23, p = .09]), where only ICU length of stay was significantly different.
We found that several factors ascertainable at ICU admission predicted dissatisfaction with the future ICU experience.
Univariate analysis of patient and family characteristics (Table 1) identified several factors associated with overall dissatisfaction. Being an adult child of the patient was associated with low satisfaction (OR 1.13, 95% CI 1.01–1.26) compared to other categories of relatives. Younger family members were more likely to be dissatisfied (OR 1.2, 95% CI 1.04–1.19, per decade), as were those who lived in a different household than the patient (OR 1.81, 95% CI 1.21–2.72) and those who reported that they lived in the same city as the hospital (OR 2.33, 95% CI 1.37–3.96). Finally, being a family member of a patient with a cardiac comorbidity was associated with lower satisfaction, but only when the patient was hospitalized in a different ICU than the coronary care unit (OR 1.70, 95% CI 1.09–2.65).
In this model, we included only factors ascertainable at ICU arrival as candidate predictors. This model identified four independent predictive factors for dissatisfaction with ICU care: 1) disagreement within the family regarding care, 2) the family member living in a different household than the patient, 3) the family member living in the same city as the hospital, and (4) having a family member with cardiac comorbidity hospitalized in a different ICU than the critical care unit (Table 2). We used these four factors to create a simple, integer-based clinical prediction rule that assigned one point for each of the factors. The area under the receiver operating characteristic curve for this rule was 0.64 (95% CI 0.59–0.69). As shown in Figure 1, the risk of low overall satisfaction increased as the number of points in this rule increased, ranging from a risk of 23% when no predictor was present to 63% when three or more predictors were present.
We found that clinical interventions and events had relatively little impact on family member satisfaction with the ICU experience. In unadjusted and adjusted analyses, only the need for mechanical ventilation was associated with overall satisfaction with care (Tables 3 and and4).4). Families of patients who were mechanically ventilated were 32% less likely to be dissatisfied with care (p = .008). When adjusted for the time-of-admission factors (Table 2), mechanical ventilation remained associated with a lower risk of dissatisfaction (adjusted OR 0.58, 95% CI 0.40–0.86). Unlike previous reports, we did not find that families of patients who died were more satisfied with care (30). In addition, ICU length of stay was unassociated with the risk of dissatisfaction.
A number of specific elements of the ICU experience, as determined by the FS-ICU, were strongly associated with overall dissatisfaction with care.
In unadjusted analyses, low satisfaction with many FS-ICU items was strongly associated with low overall satisfaction. The covariates with the strongest association were concern and caring given to patients by ICU staff, coordination of care, concern and caring of family members by ICU staff, competence of nurses, ease of getting information, honesty of information, and completeness of information (Table 5).
This model identified low satisfaction with nursing competence, concern and caring given to patients by ICU staff, and completeness of information as the independent factors most strongly associated with low overall satisfaction, followed by dissatisfaction with the atmosphere of the waiting room, atmosphere of the ICU, decision-making experience, pain control, and time to address concerns and questions when making decisions as additional independently associated factors. In the bootstrap validation procedure, neither pain control nor time to address concerns and questions remained significant, and these were therefore not included in the final model (Table 6).
We found that specific baseline factors were strongly associated with the risk of dissatisfaction with ICU care, and that these factors could be combined into a simple rule that identified high-risk patients and families. In addition, we found that specific aspects of the patient and family experience that developed during the ICU stay were strongly associated with the quantitative risk of low satisfaction with the critical care experience.
Improving satisfaction with ICU care is a vital priority for critical care medicine. Patient centeredness is a core aspect of healthcare quality as defined by the Institute of Medicine (1)–a goal in and of itself–and patient and family satisfaction has been clearly articulated as a quality measure for ICUs (31–33). Furthermore, the ICU experience negatively impacts the emotional and psychological well-being of family members of critically ill patients (34–38), and problems in the patient experience of inpatient care are also strongly associated with increased adverse events and medical errors (39). For all of these reasons, tools are urgently needed to facilitate researchers’ and quality improvement teams’ efforts to develop and test interventions in this critical arena.
In our predictive model, we found that disagreement among the family predicted dissatisfaction. These conflicts are relatively common (40) and may have a prolonged impact on families (16). Early identification of families in conflict, especially when other risk factors are present, may be a promising path to new proactive interventions to prevent dissatisfaction before it occurs. We also found that the location of residence (in the same city as the hospital or not) and whether the family member lived with the patient were both important factors. Others have found also that issues outside the healthcare system (41), such as the perception of community (42), strongly influence reports of satisfaction with the healthcare experience. Location of residence could reflect community issues, that patients traveling further distances have specifically chosen the hospital, or a difference between urban and more rural dwellers, all of which others have found to be correlated with satisfaction in other settings (43, 44).
Second, we sought to understand which components of the entire ICU experience–including major invasive ICU interventions, patient outcome, and families’ perceptions of specific aspects of experience–were most strongly correlated with dissatisfaction. Although powerful, this type of analysis carries with it an important limitation: because perceptions of specific components of the ICU experience are obtained after the ICU stay, this type of analysis is fundamentally hypothesis generating and we should be cautious in causal inference. Still, these findings may help facilitate the future design of multifactorial interventions to improve the experience of critical care in clinical trials. They may also be useful to administrators and ICU directors as they select larger-scale improvements (e.g., waiting room improvements) for local tests of change. We found that, among clinical variables, only mechanical ventilation was associated with satisfaction.
Among the FS-ICU items, we found that the single factor most strongly associated with dissatisfaction was perceived nursing competence. This is in line with recent studies suggesting that factors related to nursing care are strongly associated with overall satisfaction (8, 9, 24, 45). We also found that both concern and caring for patients by ICU staff and completeness of information were very highly correlated with overall satisfaction. This validates previous findings that many family members would prefer more time with caregivers for information transfer and for addressing the emotional distress caused by the ICU experience (9, 46–50). A survey of family members of patients dying in the ICU found that key determinants of family satisfaction were good communication, good decision making, and respect and compassion shown to both the patient and the family (51). Our study validates these findings. Notably, however, this was true both for survivors and nonsurvivors in our cohort.
Finally, we found that two environmental factors–the ICU waiting room and ICU atmosphere–were independently associated with dissatisfaction. Although it is not surprising that suboptimal waiting rooms and ICU environments would be associated with dissatisfaction, it does seem remarkable that these were more strongly associated with dissatisfaction than the vast majority of other factors ascertained with the FS-ICU. This should sound a warning bell for administrators and ICU directors: an important component of satisfaction with critical care relates directly to the physical environment and capital investment, something that cannot be controlled in the individual provider-patient encounter.
Our work has several important strengths. First, we measured satisfaction using a meticulously developed and validated instrument, the FS-ICU (4, 24). Second, we included both survivors and nonsurvivors, not just patients facing end-of-life care. This is important because most ICU patients survive, but posttraumatic stress disorder and other psychiatric problems may cause debilitating symptoms for months or years after the ICU encounter (10). In fact, ICU survivorship has been called the defining challenge of critical care for this century (10), and families may even be at higher risk than patients for problems with posttraumatic stress after an ICU stay (34). We did assess the relationship of survival status to satisfaction, but it was not a significant predictor of dissatisfaction in either unadjusted or adjusted analyses. Third, we did not focus on factors associated with dissatisfaction that were isolated to specific periods of a patient’s ICU experience (such as a family conference), but instead sought potential factors in patients’ clinical status, family members’ characteristics, and hospital/ICU systems (such as the ICU waiting room), as well as in communication-related issues. Fourth, we utilized robust analytic techniques, such as using multiple imputation and bootstrap resampling, to minimize the impact of item nonresponse and lessen the risk of model overfitting. These techniques are superior to the commonly used complete-case analysis or mean imputation, resulting in less biased coefficient estimates (27, 52–54). We also assessed for and incorporated issues of colinearity and variable interactions in our models, something always recommend but often omitted (55) in multivariable analyses.
Our study also has several key limitations. First, it remains a single-center study, and single-center patient populations may not generalize to other settings. To mitigate this possibility, we took particular care to avoid overfitting our models by using bootstrap resampling techniques. However, our results should still be validated in other centers before widespread use. Second, we chose to dichotomize the outcome variable and covariates because of the skewed nature of the data and to improve clinical interpretability. This means that the “low satisfaction” category may be heterogeneous, which might bias our study toward the null. On balance, however, we felt that the results from these type of “best possible performance” analyses are significantly more clinically interpretable than coefficients from an ordinal logistic regression, and that they are increasingly familiar to providers and administrators because they mirror the approach that the Center for Medicare and Medicaid Services has taken with publicly reported data (26). Third, our predictive model (which includes only factors ascertainable at ICU arrival) had only modest discrimination, with an area under the curve of 0.64. However, this model still provided clinically important risk stratification, categorizing substantial fractions of patients into low-and high-risk categories. For example, about one in four patients could be categorized into the lowest risk group, and about one in ten into the highest risk group, which had an observed incidence of overall dissatisfaction of >60%. Finally, our respondents represent a patient population somewhat sicker than our average ICU patient. Although this may simply reflect that it is difficult to sample short-stay patients with good outcomes, selection bias remains possible and it remains unclear if our findings are also true for the nonresponders.
To our knowledge, ours is the first attempt to create a clinical prediction rule for dissatisfaction with critical care. In addition, our study provides a potential model to understand which factors developing during the ICU stay represent the most important targets for intervention to improve satisfaction. These results represent an important first step in a potentially novel approach to improving satisfaction with the critical care experience: risk-targeted, individually customized interventions. Early identification of risk constellations may help to better allocate resources toward these at-risk patients and families. Such a risk-factor-targeted strategy has been successfully used with other multifactorial syndromes (56, 57), and it represents an important potential future avenue of research that is enabled by our study.
Specific factors ascertainable at ICU admission identify families at high risk of dissatisfaction with care, and families with multiple risk factors are at especially high risk. Aspects of the patient/family experience that develop during the ICU stay are also strongly associated with dissatisfaction with the critical care experience. Although some of these factors relate to communication, others relate to family perceptions of technical aspects of care, and others to the built environment. These results may provide insight into the design of future evidence-based strategies to improve satisfaction with the ICU experience.
Supported, in part, by the Swiss National Foundation (SNF PBBSP3-128266 to Dr. Hunziker) and from the University of Basel, Switzerland. Dr. Marcantonio is supported by a Midcareer Investigator Award in Patient-Oriented Research from the National Institute on Aging [K24 AG035075]. Dr. Howell is supported by the Physician Faculty Scholars Program of the Robert Wood Johnson Foundation (#66350).
The authors have not disclosed any potential conflicts of interest.