The CAHPS Hospital Survey is an assessment tool soon to be in use across the country that was designed to be supplemented with additional questions to reflect user-specific needs. This study was undertaken to assess the psychometric properties of the survey and to evaluate benefits of the additional survey items used in the CHART project.
The study findings are generally consistent with previous evaluations of the psychometric properties of the instrument (Keller et al. 2005
; Sofaer et al. 2005
; Arah et al. 2006
). However, the original two-item CAHPS discharge information and physical environment composites had relatively low internal consistency reliabilities. The augmentation of the discharge information composite by two CHART items improved the psychometric properties of the composite as well as the associations with the global rating and the willingness to recommend the hospital. The results of this study also indicated a need for improvement in the two-item CAHPS communication about medicine composite. The hospital-level reliability of both items and the composite were relatively low.
Consistent with findings in other research, the nurse communication composite within the CAHPS Hospital Survey had the strongest association with the ratings and willingness to recommend (Jenkinson et al. 2002
; Sofaer et al. 2005
; Arah et al. 2006
). For example, the regression coefficients for explaining variation in the willingness to recommend the hospital in the CAHPS Hospital Survey was 0.013 for nurse communication compared with 0.005 for the physical environment—the composite that had the second strongest association. Hence, nurse communication is the major driver of bottom-line perceptions of hospital care. When the CHART coordination of care composites are considered along with the CAHPS Hospital Survey, all three had stronger associations with the hospital ratings and willingness to recommend than the physical environment composite, making these the composites with the second strongest associations with overall hospital rating and willingness to recommend.
The strong performance of the CHART coordination of care composites supports the reconsideration of including a coordination of care domain to the CAHPS Hospital Survey. Although there has been significant discussion regarding the inclusion of a coordination of care domain within the CAHPS Hospital Survey, the discussion has been based on a broad concept of coordination of care, much of which may not be visible or detectable by the patient, about which, therefore, the respondent would not be expected to be a knowledgeable informant. It has also been suggested that coordination of care may only be recognized in its absence (Levine, Fowler, and Brown 2005
). Questions that were considered and deleted during the original CAHPS Hospital Survey development asked, for example, “did staff members who cared for you know about your condition without having to ask you?”
The questions used in the CHART coordination of care domain are more specifically focused on elements of coordination that may be directly experienced and understood by the patient (e.g., “scheduled tests were performed on time”). In this study, we chose to examine three different composites in coordination of care: a three-item composite that included three questions that had been clearly mapped to coordination of care in pilot studies, a four-item composite that included the “delay going to room explained” question that mapped well in this analysis, and a five-item composite that included an additional question—“ID band checked”—that also mapped to this domain. Although all of these composites performed well in this analysis, the five-item composite is slightly superior psychometrically.
There is always tension surrounding the inclusion of additional items within any patient survey—the benefit of increased reliability of the survey as questions are added must be balanced against the additional response burden on patients. Thus, the decision to add questions to improve the discharge information domain or add a coordination of care domain must be made in the context of overall patient survey strategies. For instance, some hospitals might already be asking questions beyond the CAHPS Hospital Survey that address areas of their own choosing, and using an expanded CAHPS Hospital Survey may reduce their capacity to address these local issues. However, it will increase the amount of standardized information available if many hospitals agree to use a specific expanded version of the CAHPS Hospital Survey, as has occurred in CHART.
There is also an issue of using proprietary versus public domain questions in a patient survey. The CAHPS Hospital Survey questions are now in the public domain, but the process for moving questions from proprietary to public domain status has not yet been worked out. Furthermore, we cannot state whether the questions we used in this study are superior to similar questions from other sources. For instance, the three-item Care Transitions Measures developed by Coleman, Mahoney, and Parry (2005)
might function as well as the four-item discharge information domain presented here. Further investigation of alternatives to improving the discharge information domain (and the physical environment domain) of the CAHPS Hospital Survey is needed, as is research on how best to measure coordination of care. However, our results suggest that these improvements in patient surveys are feasible and can generate important information.
In summary, the findings of this study provide further support for the reliability and validity of the CAHPS Hospital Survey. Because of its brevity, it is feasible to add items to the CAHPS Hospital Survey to provide a more psychometrically sound assessment of the current domains and to assess domains not represented. The findings of this study illustrate how such additional items can be assessed and that additional items can improve our measurement of patients' hospital experiences.