PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Crit Care Med. Author manuscript; available in PMC 2013 May 8.
Published in final edited form as:
PMCID: PMC3647885
NIHMSID: NIHMS459255

Predictors and correlates of dissatisfaction with intensive care

Abstract

Objective

Dissatisfaction is an important threat to high-quality care. The aim of this study was to identify factors independently associated with dissatisfaction with critical care.

Design

Prospectively collected observational cohort study.

Setting

Nine intensive care units at a tertiary care university hospital in the United States.

Participants

Four hundred forty-nine family members of adult intensive care unit patients who completed the Family Satisfaction with Care in the Intensive Care Unit instrument.

Intervention

None.

Measurements and Main Results

Four family-and patient-related factors ascertainable at intensive care unit admission independently predicted low overall satisfaction: living in the same city as the hospital, disagreement within the family regarding care, having a cardiac comorbidity but being hospitalized in a noncardiac-care intensive care unit, and living in a different household than the patient. When three or more risk factors were present, 63% (95% confidence interval 48%–78%) of families were dissatisfied. Among factors ascertained at the end of the intensive care unit stay, dissatisfaction with six items was independently associated with overall dissatisfaction: 1) perceived competence of nurses (odds ratio for dissatisfaction = 5.9, 95% confidence interval 2.3–15.2); 2) concern and caring by intensive care unit staff (odds ratio 5.0, 95% confidence interval 1.9–12.6); 3) completeness of information (odds ratio 4.4, 95% confidence interval 2.4–8.1); 4) dissatisfaction with the decision-making process (odds ratio 3.0, 95% confidence interval 1.6– 5.6); 5) atmosphere of the intensive care unit (odds ratio 2.6, 95% confidence interval 1.4–4.8); and 6) atmosphere of the waiting room (odds ratio 2.7, 95% confidence interval 1.2–6.0).

Conclusion

Specific factors ascertainable at intensive care unit admission identify families at high risk of dissatisfaction with care. Other discrete aspects of the patient/family experience that develop during the intensive care unit stay are also strongly associated with dissatisfaction with the critical care experience. These results may provide insight into the design of future evidence-based strategies to improve satisfaction with the intensive care unit experience.

Keywords: critical care, family satisfaction, intensive care, quality of health care

The Institute of Medicine defines high-quality care as safe, timely, efficient, effective, equitable, and patient centered (1). In the intensive care unit (ICU), patient-centered care includes family-centered care (24) since patients are frequently unable to participate in their care due to sedation, delirium, and severity of illness. Measuring the quality of patient-and family-centered care is complex, but many authorities view satisfaction with care as a central outcome measure in this domain (46). Although increased attention has recently turned to improving the patient and family centeredness of ICU care, most data suggest that significant opportunities for improvement remain both in the United States (7) and abroad (8, 9).

Three areas of this burgeoning subject of inquiry are particularly notable. First, although survivorship has been called the defining problem of 21st century critical care (10), most research aimed at improving the ICU experience has focused on end-of-life care (5, 1118) rather than the broader population of the critically ill, most of whom survive. Second, a great deal of research has focused on issues relating to individual patient-provider interactions, such as family conferences (9, 11, 12, 14, 15, 1820), and less has focused on system-level issues that may affect the experience of critical illness but are not part of the individual-level encounter (3, 2123). Third, no tools exist to guide providers in predicting which families are at high risk of dissatisfaction, or that guide researchers and quality improvement teams in selecting the most important factors on which to focus limited improvement resources.

We therefore sought to understand how patient, family, ICU, and provider characteristics were related to the risk of less-than-ideal satisfaction with critical care. We were interested in three primary questions. First, what factors–ascertainable at ICU admission–independently predicted dissatisfaction with the future ICU experience? This clinical prediction rule might be useful in the design of patient-or family-specific interventions to prevent dissatisfaction. Second, we wanted to understand whether patients’ clinical characteristics (both interventions and outcomes) were associated with the risk of dissatisfaction. Finally, we were interested in understanding what aspects of families’ perceptions of specific aspects of the ICU experience were most strongly correlated associated with overall dissatisfaction. These analyses might be useful to ICU directors and administrators in choosing improvement priorities. They might also be hypothesis generating, helping researchers choose promising targets for novel interventions. Together, these analyses might form the basis for the design of future customized interventions–based on patient-level risks and ICU-specific factors–that aim to improve the critical care experience.

MATERIALS AND METHODS

Design, Setting, and Patients

This is a prospective observational cohort study conducted in a university tertiary care medical center with nine adult ICUs, 77 adult ICU beds, and approximately 500 total beds. We included all adult patients at least 18 yrs of age who: 1) stayed in the ICU for >6 hrs from April 2008 to November 2009; and 2) whose proxy or spokesperson completed a Family Satisfaction with Care in the ICU (FS-ICU) instrument within 3 days after discharge from the ICU, or who responded to a mailed survey if the patient had died. Family members were selected based on documentation as next of kin or healthcare proxy status. We excluded three (0.7%) patients for whom the primary outcome variable (“overall satisfaction”) was missing.

This study was approved by the Committee on Clinical Investigations/Institutional Review Board of Beth Israel Deaconess Medical Center with a waiver of informed consent.

Survey Instrument and Procedure for Data Collection

The FS-ICU is a well-validated instrument designed to measure satisfaction with the critical care experience. It was rigorously derived (24) and validated in multicenter (8), international (4), and cross-cultural (25) cohorts. Detailed descriptions of the instrument and its characteristics have previously been published (4, 24).

As part of a quality improvement effort, our medical center began administering the long-form (34-question) FS-ICU in April 2008. We identified patients discharged or transferred from the ICU and administered the survey in person or by telephone to available family members. To ensure minimization of bias by the surveyor, interviewers were formally trained by a PhD methodologist. Only families who could be contacted during the interviewer’s hours of work (approximately 20 hrs per week) were interviewed. For all nonsurvivors, we mailed surveys to patients’ next of kin at 7 wks after the patient’s death. For nonresponders, a reminder letter was mailed 4 wks later. We included all respondents in the analysis.

Outcomes and Covariates

Primary Outcome

The primary outcome variable of this study was less-than-complete overall satisfaction with critical care, which for brevity we refer to as “low overall satisfaction” or “dissatisfaction.” We ascertained this based on family members’ responses to a specific FS-ICU item that asks family members to rate “overall satisfaction with your experience in the ICU” and includes a 5-point Likert-type response scale that ranges from “completely satisfied” to “very dissatisfied” (4, 24). Responses for this item were strongly skewed toward complete satisfaction. We therefore dichotomized the outcome variable into those who were completely satisfied, defined as “high satisfaction” (n = 292; 65% [95% confidence interval (CI) 61%–69%]) and those who were not, defined as “low satisfaction” (n = 157; 35% [95% CI 31%–39%]). Although any cut point is somewhat arbitrary, we believed that this represented the best balance between competing factors. In addition, this sort of dichotomization is familiar to patients, families, clinicians, and administrators, since it is the approach that the Centers for Medicare and Medicaid Services has taken with public reporting of patient satisfaction (26).

Exposure Variables

We collected several factors present at ICU arrival as well as factors that developed during the course of the ICU stay, including patient-related factors (demographics, diagnosis, and comorbidities), ICU characteristics, and family-related factors. We collected clinical factors related to the ICU stay, including major procedures, treatment, and outcome, as well as the FS-ICU responses. In brief, the FS-ICU includes 34 questions concerning: 1) satisfaction with care, and 2) satisfaction with decision making. The responses to these questions were also generally very skewed; we converted them into binary variables for further assessment.

Statistical Methods

Unadjusted Analyses

We calculated unadjusted odds ratios (ORs) to represent the strength of association between each variable and low satisfaction. We used Fisher’s exact test or the Wilcoxon’s rank-sum test for unadjusted tests of difference.

Multivariable Analyses

We built three primary models. In the first, which we refer to as the “predictive model,” we restricted candidate predictor variables to those ascertainable at ICU admission. In the second, which we refer to as the “clinical model,” we included major ICU interventions (e.g., mechanical ventilation, new dialysis, etc). In the third, we included family-reported aspects of the ICU experience that are only ascertainable after the ICU experience is underway or complete (e.g., satisfaction with concern and caring of ICU staff). Since these cannot be ascertained before overall satisfaction is measured, and are measured on the same instrument, we kept these separate from the other baseline and clinical factors. We performed model creation and variable selection as follows (27, 28). First, missing data among predictor variables were approached using multiple imputation. We constructed an augmented data set by performing 100 imputations, using a Monte Carlo Markov Chain approach with 500 burn-in iterations and 100 iterations between imputations to create samples with no missing data. Then, to prevent overfitting and reduce the risk that we would select variables for our model that were only related to outcome by chance alone, we performed a bootstrap resampling procedure. Using random resampling with replacement from the augmented, imputed dataset, we constructed 100 separate bootstrap samples. Then, within each sample, we modeled the relationship of predictor variables to overall dissatisfaction by performing logistic regression, using forward stepwise selection for variable selection. We retained variables in the model if their regression coefficients were significant in >50 of the 100 bootstrap samples at the α = 0.05 level. For final coefficient estimates, we repeated the imputation step (again using a Monte Carlo Markov Chain approach with 100 iterations), performed logistic regression on each imputation, and combined parameter estimates correctly accounting for variances using SAS PROC MIANALYZE. Finally, we conducted model validation using random resampling with replacement (bootstrapping) (29), with 1000 bootstrap samples. Variables that did not maintain significance in this validation were removed from the final model.

RESULTS

Patient and Family Characteristics

Of 657 family members asked to participate in the FS-ICU survey, 452 (69%) participated (330 of 332 in person, 107 of 310 by mail, and 12 of 12 by telephone interview). Three family members (0.7%) did not provide a rating of overall satisfaction, and so were excluded from analysis, leaving 449 (99.3%) in the final cohort. In 54% of cases the spouse, and in 25% of cases the adult child, answered the FS-ICU questionnaire. Family members had a median age of 58 yrs (interquartile range 49–69) and most (71%) were women (Table 1).

Table 1
Factors ascertainable at intensive care unit arrival: Unadjusted relationships

The median age of patients was 68 yrs (interquartile range 54–78) and 192 (43%) were women. The most common primary diagnoses are presented in Table 1. One third of patients received vasopressors; 57% were mechanically ventilated. The median length of ICU stay was 3 days (interquartile range 2–7) and the ICU mortality rate in respondents was 24%. These represent a group of patients more ill than our average ICU population, partly related to our sampling strategy, which provided more opportunities for families of nonsurvivors to respond than families of survivors. However, when stratified by survival status, there were also some important differences. Among survivors, there were no significant differences in terms of age (62.6 vs. 62.8, p = .9 for responders vs. nonresponders) or gender (58% male vs. 56% male, p = .5), but ICU length of stay was longer (5.6 days vs. 3.4 days, p < .0001) and case mix was higher (4.0 vs. 3.0, p < .0001). These difference were less apparent among nonsurvivors (age: 71.4 vs. 69.8 yrs, p = .4; gender [57% male vs. 52% male], ICU length of stay [6.3 vs. 5.9 days, p = .04], and case mix [3.20 vs. 3.23, p = .09]), where only ICU length of stay was significantly different.

Predicting Dissatisfaction: Factors Ascertainable at ICU Admission

We found that several factors ascertainable at ICU admission predicted dissatisfaction with the future ICU experience.

Unadjusted Analyses

Univariate analysis of patient and family characteristics (Table 1) identified several factors associated with overall dissatisfaction. Being an adult child of the patient was associated with low satisfaction (OR 1.13, 95% CI 1.01–1.26) compared to other categories of relatives. Younger family members were more likely to be dissatisfied (OR 1.2, 95% CI 1.04–1.19, per decade), as were those who lived in a different household than the patient (OR 1.81, 95% CI 1.21–2.72) and those who reported that they lived in the same city as the hospital (OR 2.33, 95% CI 1.37–3.96). Finally, being a family member of a patient with a cardiac comorbidity was associated with lower satisfaction, but only when the patient was hospitalized in a different ICU than the coronary care unit (OR 1.70, 95% CI 1.09–2.65).

Multivariable Model and Performance of Clinical Prediction Rule

In this model, we included only factors ascertainable at ICU arrival as candidate predictors. This model identified four independent predictive factors for dissatisfaction with ICU care: 1) disagreement within the family regarding care, 2) the family member living in a different household than the patient, 3) the family member living in the same city as the hospital, and (4) having a family member with cardiac comorbidity hospitalized in a different ICU than the critical care unit (Table 2). We used these four factors to create a simple, integer-based clinical prediction rule that assigned one point for each of the factors. The area under the receiver operating characteristic curve for this rule was 0.64 (95% CI 0.59–0.69). As shown in Figure 1, the risk of low overall satisfaction increased as the number of points in this rule increased, ranging from a risk of 23% when no predictor was present to 63% when three or more predictors were present.

Figure 1
Predicting and explaining dissatisfaction with intensive care unit (ICU) care. One point is assigned for each factor identified in Table 1, and includes only factors ascertainable at ICU arrival.
Table 2
Independent predictors of dissatisfaction with intensive care unit care, including only factors ascertainable at intensive care unit arrival

Clinical Interventions and Outcomes

We found that clinical interventions and events had relatively little impact on family member satisfaction with the ICU experience. In unadjusted and adjusted analyses, only the need for mechanical ventilation was associated with overall satisfaction with care (Tables 3 and and4).4). Families of patients who were mechanically ventilated were 32% less likely to be dissatisfied with care (p = .008). When adjusted for the time-of-admission factors (Table 2), mechanical ventilation remained associated with a lower risk of dissatisfaction (adjusted OR 0.58, 95% CI 0.40–0.86). Unlike previous reports, we did not find that families of patients who died were more satisfied with care (30). In addition, ICU length of stay was unassociated with the risk of dissatisfaction.

Table 3
Clinical interventions and outcomes: unadjusted analyses
Table 4
Independent association between relationship of clinical interventions and outcomes to dissatisfaction with intensive care unit care

Components of the ICU Experience

A number of specific elements of the ICU experience, as determined by the FS-ICU, were strongly associated with overall dissatisfaction with care.

Unadjusted Analyses

In unadjusted analyses, low satisfaction with many FS-ICU items was strongly associated with low overall satisfaction. The covariates with the strongest association were concern and caring given to patients by ICU staff, coordination of care, concern and caring of family members by ICU staff, competence of nurses, ease of getting information, honesty of information, and completeness of information (Table 5).

Table 5
Elements of the intensive care unit experience as perceived by family members and measured using the Family Satisfaction with Care in the Intensive Care Unit instrument: Unadjusted analyses

Multivariable Analyses

This model identified low satisfaction with nursing competence, concern and caring given to patients by ICU staff, and completeness of information as the independent factors most strongly associated with low overall satisfaction, followed by dissatisfaction with the atmosphere of the waiting room, atmosphere of the ICU, decision-making experience, pain control, and time to address concerns and questions when making decisions as additional independently associated factors. In the bootstrap validation procedure, neither pain control nor time to address concerns and questions remained significant, and these were therefore not included in the final model (Table 6).

Table 6
Independent association between elements of the intensive care unit experience and overall dissatisfaction with care

DISCUSSION

We found that specific baseline factors were strongly associated with the risk of dissatisfaction with ICU care, and that these factors could be combined into a simple rule that identified high-risk patients and families. In addition, we found that specific aspects of the patient and family experience that developed during the ICU stay were strongly associated with the quantitative risk of low satisfaction with the critical care experience.

Improving satisfaction with ICU care is a vital priority for critical care medicine. Patient centeredness is a core aspect of healthcare quality as defined by the Institute of Medicine (1)–a goal in and of itself–and patient and family satisfaction has been clearly articulated as a quality measure for ICUs (3133). Furthermore, the ICU experience negatively impacts the emotional and psychological well-being of family members of critically ill patients (3438), and problems in the patient experience of inpatient care are also strongly associated with increased adverse events and medical errors (39). For all of these reasons, tools are urgently needed to facilitate researchers’ and quality improvement teams’ efforts to develop and test interventions in this critical arena.

In our predictive model, we found that disagreement among the family predicted dissatisfaction. These conflicts are relatively common (40) and may have a prolonged impact on families (16). Early identification of families in conflict, especially when other risk factors are present, may be a promising path to new proactive interventions to prevent dissatisfaction before it occurs. We also found that the location of residence (in the same city as the hospital or not) and whether the family member lived with the patient were both important factors. Others have found also that issues outside the healthcare system (41), such as the perception of community (42), strongly influence reports of satisfaction with the healthcare experience. Location of residence could reflect community issues, that patients traveling further distances have specifically chosen the hospital, or a difference between urban and more rural dwellers, all of which others have found to be correlated with satisfaction in other settings (43, 44).

Second, we sought to understand which components of the entire ICU experience–including major invasive ICU interventions, patient outcome, and families’ perceptions of specific aspects of experience–were most strongly correlated with dissatisfaction. Although powerful, this type of analysis carries with it an important limitation: because perceptions of specific components of the ICU experience are obtained after the ICU stay, this type of analysis is fundamentally hypothesis generating and we should be cautious in causal inference. Still, these findings may help facilitate the future design of multifactorial interventions to improve the experience of critical care in clinical trials. They may also be useful to administrators and ICU directors as they select larger-scale improvements (e.g., waiting room improvements) for local tests of change. We found that, among clinical variables, only mechanical ventilation was associated with satisfaction.

Among the FS-ICU items, we found that the single factor most strongly associated with dissatisfaction was perceived nursing competence. This is in line with recent studies suggesting that factors related to nursing care are strongly associated with overall satisfaction (8, 9, 24, 45). We also found that both concern and caring for patients by ICU staff and completeness of information were very highly correlated with overall satisfaction. This validates previous findings that many family members would prefer more time with caregivers for information transfer and for addressing the emotional distress caused by the ICU experience (9, 4650). A survey of family members of patients dying in the ICU found that key determinants of family satisfaction were good communication, good decision making, and respect and compassion shown to both the patient and the family (51). Our study validates these findings. Notably, however, this was true both for survivors and nonsurvivors in our cohort.

Finally, we found that two environmental factors–the ICU waiting room and ICU atmosphere–were independently associated with dissatisfaction. Although it is not surprising that suboptimal waiting rooms and ICU environments would be associated with dissatisfaction, it does seem remarkable that these were more strongly associated with dissatisfaction than the vast majority of other factors ascertained with the FS-ICU. This should sound a warning bell for administrators and ICU directors: an important component of satisfaction with critical care relates directly to the physical environment and capital investment, something that cannot be controlled in the individual provider-patient encounter.

Our work has several important strengths. First, we measured satisfaction using a meticulously developed and validated instrument, the FS-ICU (4, 24). Second, we included both survivors and nonsurvivors, not just patients facing end-of-life care. This is important because most ICU patients survive, but posttraumatic stress disorder and other psychiatric problems may cause debilitating symptoms for months or years after the ICU encounter (10). In fact, ICU survivorship has been called the defining challenge of critical care for this century (10), and families may even be at higher risk than patients for problems with posttraumatic stress after an ICU stay (34). We did assess the relationship of survival status to satisfaction, but it was not a significant predictor of dissatisfaction in either unadjusted or adjusted analyses. Third, we did not focus on factors associated with dissatisfaction that were isolated to specific periods of a patient’s ICU experience (such as a family conference), but instead sought potential factors in patients’ clinical status, family members’ characteristics, and hospital/ICU systems (such as the ICU waiting room), as well as in communication-related issues. Fourth, we utilized robust analytic techniques, such as using multiple imputation and bootstrap resampling, to minimize the impact of item nonresponse and lessen the risk of model overfitting. These techniques are superior to the commonly used complete-case analysis or mean imputation, resulting in less biased coefficient estimates (27, 5254). We also assessed for and incorporated issues of colinearity and variable interactions in our models, something always recommend but often omitted (55) in multivariable analyses.

Our study also has several key limitations. First, it remains a single-center study, and single-center patient populations may not generalize to other settings. To mitigate this possibility, we took particular care to avoid overfitting our models by using bootstrap resampling techniques. However, our results should still be validated in other centers before widespread use. Second, we chose to dichotomize the outcome variable and covariates because of the skewed nature of the data and to improve clinical interpretability. This means that the “low satisfaction” category may be heterogeneous, which might bias our study toward the null. On balance, however, we felt that the results from these type of “best possible performance” analyses are significantly more clinically interpretable than coefficients from an ordinal logistic regression, and that they are increasingly familiar to providers and administrators because they mirror the approach that the Center for Medicare and Medicaid Services has taken with publicly reported data (26). Third, our predictive model (which includes only factors ascertainable at ICU arrival) had only modest discrimination, with an area under the curve of 0.64. However, this model still provided clinically important risk stratification, categorizing substantial fractions of patients into low-and high-risk categories. For example, about one in four patients could be categorized into the lowest risk group, and about one in ten into the highest risk group, which had an observed incidence of overall dissatisfaction of >60%. Finally, our respondents represent a patient population somewhat sicker than our average ICU patient. Although this may simply reflect that it is difficult to sample short-stay patients with good outcomes, selection bias remains possible and it remains unclear if our findings are also true for the nonresponders.

To our knowledge, ours is the first attempt to create a clinical prediction rule for dissatisfaction with critical care. In addition, our study provides a potential model to understand which factors developing during the ICU stay represent the most important targets for intervention to improve satisfaction. These results represent an important first step in a potentially novel approach to improving satisfaction with the critical care experience: risk-targeted, individually customized interventions. Early identification of risk constellations may help to better allocate resources toward these at-risk patients and families. Such a risk-factor-targeted strategy has been successfully used with other multifactorial syndromes (56, 57), and it represents an important potential future avenue of research that is enabled by our study.

CONCLUSIONS

Specific factors ascertainable at ICU admission identify families at high risk of dissatisfaction with care, and families with multiple risk factors are at especially high risk. Aspects of the patient/family experience that develop during the ICU stay are also strongly associated with dissatisfaction with the critical care experience. Although some of these factors relate to communication, others relate to family perceptions of technical aspects of care, and others to the built environment. These results may provide insight into the design of future evidence-based strategies to improve satisfaction with the ICU experience.

Acknowledgments

Supported, in part, by the Swiss National Foundation (SNF PBBSP3-128266 to Dr. Hunziker) and from the University of Basel, Switzerland. Dr. Marcantonio is supported by a Midcareer Investigator Award in Patient-Oriented Research from the National Institute on Aging [K24 AG035075]. Dr. Howell is supported by the Physician Faculty Scholars Program of the Robert Wood Johnson Foundation (#66350).

Footnotes

The authors have not disclosed any potential conflicts of interest.

REFERENCES

1. Institute of Medicine. Crossing the Quality chasm: A New Health System for the 21st Century. Washington DC: National Academy Press; 2001. [PubMed]
2. Davidson JE, Powers K, Hedayat KM, et al. Clinical practice guidelines for support of the family in the patient-centered intensive care unit: American College of Critical Care Medicine Task Force 2004–2005. Crit Care Med. 2007;35:605–622. [PubMed]
3. Dowling J, Vender J, Guilianelli S, et al. A model of family-centered care and satisfaction predictors: The Critical Care Family Assistance Program. Chest. 2005;128:81S–92S. [PubMed]
4. Wall RJ, Engelberg RA, Downey L, et al. Refinement, scoring, and validation of the Family Satisfaction in the Intensive Care Unit (FS-ICU) survey. Crit Care Med. 2007;35:271–279. [PubMed]
5. Curtis JR, Engelberg RA. Measuring success of interventions to improve the quality of end-of-life care in the intensive care unit. Crit Care Med. 2006;34:S341–S347. [PubMed]
6. Levy M. Including families in quality measurement in critical care. Crit Care Med. 2007;35:324–325. [PubMed]
7. Johnson D, Wilson M, Cavanaugh B, et al. Measuring the ability to meet family needs in an intensive care unit. Crit Care Med. 1998;26:266–271. [PubMed]
8. Heyland DK, Rocker GM, Dodek PM, et al. Family satisfaction with care in the intensive care unit: Results of a multiple center study. Crit Care Med. 2002;30:1413–1418. [PubMed]
9. Azoulay E, Chevret S, Leleu G, et al. Half the families of intensive care unit patients experience inadequate communication with physicians. Crit Care Med. 2000;28:3044–3049. [PubMed]
10. Iwashyna TJ. Survivorship will be the defining challenge of critical care in the 21st century. Ann Intern Med. 2010;153:204–205. [PubMed]
11. Curtis JR, Treece PD, Nielsen EL, et al. Integrating palliative and critical care: Evaluation of a quality-improvement intervention. Am J Respir Crit Care Med. 2008;178:269–275. [PMC free article] [PubMed]
12. Gerstel E, Engelberg RA, Koepsell T, et al. Duration of withdrawal of life support in the intensive care unit and association with family satisfaction. Am J Respir Crit Care Med. 2008;178:798–804. [PMC free article] [PubMed]
13. Gries CJ, Curtis JR, Wall RJ, et al. Family member satisfaction with end-of-life decision making in the ICU. Chest. 2008;133:704–712. [PMC free article] [PubMed]
14. Curtis JR, Engelberg RA, Wenrich MD, et al. Missed opportunities during family conferences about end-of-life care in the intensive care unit. Am J Respir Crit Care Med. 2005;171:844–849. [PubMed]
15. Curtis JR, Patrick DL, Shannon SE, et al. The family conference as a focus to improve communication about end-of-life care in the intensive care unit: Opportunities for improvement. Crit Care Med. 2001;29:N26–N33. [PubMed]
16. Abbott KH, Sago JG, Breen CM, et al. Families looking back: One year after discussion of withdrawal or withholding of life-sustaining support. Crit Care Med. 2001;29:197–201. [PubMed]
17. White DB, Braddock CH, 3rd, Bereknyei S, et al. Toward shared decision making at the end of life in intensive care units: Opportunities for improvement. Arch Intern Med. 2007;167:461–467. [PubMed]
18. Lautrette A, Darmon M, Megarbane B, et al. A communication strategy and brochure for relatives of patients dying in the ICU. N Engl J Med. 2007;356:469–478. [PubMed]
19. Fassier T, Darmon M, Laplace C, et al. One-day quantitative cross-sectional study of family information time in 90 intensive care units in France. Crit Care Med. 2007;35:177–183. [PubMed]
20. Stapleton RD, Engelberg RA, Wenrich MD, et al. Clinician statements and family satisfaction with family conferences in the intensive care unit. Crit Care Med. 2006;34:1679–1685. [PubMed]
21. Garrouste-Orgeas M, Philippart F, Timsit JF, et al. Perceptions of a 24-hour visiting policy in the intensive care unit. Crit Care Med. 2008;36:30–35. [PubMed]
22. Gajic O, Afessa B, Hanson AC, et al. Effect of 24-hour mandatory versus on-demand critical care specialist presence on quality of care and family and provider satisfaction in the intensive care unit of a teaching hospital. Crit Care Med. 2008;36:36–44. [PubMed]
23. Fridh I, Forsberg A, Bergbom I. Family presence and environmental factors at the time of a patient’s death in an ICU. Acta Anaesthesiol Scand. 2007;51:395–401. [PubMed]
24. Heyland DK, Tranmer JE. Kingston General Hospital ICU Research Working Group: Measuring family satisfaction with care in the intensive care unit: The development of a questionnaire and preliminary results. J Crit Care. 2001;16:142–149. [PubMed]
25. Stricker KH, Niemann S, Bugnon S, et al. Family satisfaction in the intensive care unit: Cross-cultural adaptation of a questionnaire. J Crit Care. 2007;22:204–211. [PubMed]
26. Services CoMaM. [Accessed July 1, 2010];CAHPS Hospital Survey (HCAHPS): Quality Assurance Guidelines (version 3.0) Available at: http://www.hcahpsonline.org/files/HCAHPS%20Quality%20Assurance%20Guidelines%20Version%203.0%20Final.pdf.
27. Heymans MW, van Buuren S, Knol DL, et al. Variable selection under multiple imputation using the bootstrap in a prognostic study. BMC Med Res Methodol. 2007;7:33. [PMC free article] [PubMed]
28. Rudolph JL, Jones RN, Levkoff SE, et al. Derivation and validation of a preoperative prediction rule for delirium after cardiac surgery. Circulation. 2009;119:229–236. [PMC free article] [PubMed]
29. Efron B. Estimating the error rate of a prediction rule: Improvement on cross-validation. J Am Stat Assoc. 1983;78:316–331.
30. Wall RJ, Curtis JR, Cooke CR, et al. Family satisfaction in the ICU: Differences between families of survivors and nonsurvivors. Chest. 2007;132:1425–1433. [PubMed]
31. de Vos M, Graafmans W, Keesman E, et al. Quality measurement at intensive care units: Which indicators should we use? J Crit Care. 2007;22:267–274. [PubMed]
32. Berenholtz SM, Dorman T, Ngo K, et al. Qualitative review of intensive care unit quality indicators. J Crit Care. 2002;17:1–12. [PubMed]
33. Clarke EB, Curtis JR, Luce JM, et al. Quality indicators for end-of-life care in the intensive care unit. Crit Care Med. 2003;31:2255–2262. [PubMed]
34. Azoulay E, Pochard F, Kentish-Barnes N, et al. Risk of post-traumatic stress symptoms in family members of intensive care unit patients. Am J Respir Crit Care Med. 2005;171:987–994. [PubMed]
35. Pochard F, Darmon M, Fassier T, et al. Symptoms of anxiety and depression in family members of intensive care unit patients before discharge or death. A prospective multicenter study. J Crit Care. 2005;20:90–96. [PubMed]
36. Pochard F, Azoulay E, Chevret S, et al. Symptoms of anxiety and depression in family members of intensive care unit patients: Ethical hypothesis regarding decision-making capacity. Crit Care Med. 2001;29:1893–1897. [PubMed]
37. Gries CJ, Engelberg RA, Kross EK, et al. Predictors of symptoms of posttraumatic stress and depression in family members after patient death in the ICU. Chest. 2010;137:280–287. [PubMed]
38. Anderson WG, Arnold RM, Angus DC, et al. Posttraumatic stress and complicated grief in family members of patients in the intensive care unit. J Gen Intern Med. 2008;23:1871–1876. [PMC free article] [PubMed]
39. Taylor BB, Marcantonio ER, Pagovich O, et al. Do medical inpatients who report poor service quality experience more adverse events and medical errors? Med Care. 2008;46:224–228. [PubMed]
40. Studdert DM, Mello MM, Burns JP, et al. Conflict in the care of patients with prolonged stay in the ICU: Types, sources, and predictors. Intensive Care Med. 2003;29:1489–1497. [PubMed]
41. Bleich SN, Ozaltin E, Murray CK. How does satisfaction with the health-care system relate to patient experience? Bull World Health Organ. 2009;87:271–278. [PubMed]
42. Ahern MM, Hendryx MS, Siddharthan K. The importance of sense of community on people’s perceptions of their health-care experiences. Med Care. 1996;34:911–923. [PubMed]
43. Nguyen Thi PL, Briançon S, Empereur F, et al. Factors determining inpatient satisfaction with care. Soc Sci Med. 2002;54:493–504. [PubMed]
44. Rahmqvist M, Bara AC. Patient characteristics and quality dimensions related to patient satisfaction. Int J Qual Health Care. 2010;22:86–92. [PubMed]
45. Baker R, Wu AW, Teno JM, et al. Family satisfaction with end-of-life care in seriously ill hospitalized adults. J Am Geriatr Soc. 2000;48:S61–S69. [PubMed]
46. Azoulay E, Pochard F, Chevret S, et al. Meeting the needs of intensive care unit patient families: A multicenter study. Am J Respir Crit Care Med. 2001;163:135–139. [PubMed]
47. Medland JJ, Ferrans CE. Effectiveness of a structured communication program for family members of patients in an ICU. Am J Crit Care. 1998;7:24–29. [PubMed]
48. Atkinson JH, Jr, Stewart N, Gardner D. The family meeting in critical care settings. J Trauma. 1980;20:43–46. [PubMed]
49. Tilden VP, Tolle SW, Garland MJ, et al. Decisions about life-sustaining treatment. Impact of physicians’ behaviors on the family. Arch Intern Med. 1995;155:633–638. [PubMed]
50. Malacrida R, Bettelini CM, Degrate A, et al. Reasons for dissatisfaction: A survey of relatives of intensive care patients who died. Crit Care Med. 1998;26:1187–1193. [PubMed]
51. Heyland DK, Rocker GM, O’Callaghan CJ, et al. Dying in the ICU: Perspectives of family members. Chest. 2003;124:392–397. [PubMed]
52. Wood AM, White IR, Royston P. How should variable selection be performed with multiply imputed data? Stat Med. 2008;27:3227–3246. [PubMed]
53. Austin PC, Tu JV. Automated variable selection methods for logistic regression produced unstable models for predicting acute myocardial infarction mortality. J Clin Epidemiol. 2004;57:1138–1146. [PubMed]
54. Harrell FE, Jr, Lee KL, Mark DB. Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361–387. [PubMed]
55. Ottenbacher KJ, Ottenbacher HR, Tooth L, et al. A review of two journals found that articles using multivariable logistic regression frequently did not report commonly recommended assumptions. J Clin Epidemiol. 2004;57:1147–1152. [PubMed]
56. Inouye SK, Bogardus ST, Jr, Charpentier PA, et al. A multicomponent intervention to prevent delirium in hospitalized older patients. N Engl J Med. 1999;340:669–676. [PubMed]
57. Marcantonio ER, Flacker JM, Wright RJ, et al. Reducing delirium after hip fracture: A randomized trial. J Am Geriatr Soc. 2001;49:516–522. [PubMed]