This study is the first to assess the effect of case-mix adjustment on CHC performance on patient experience measures. We found that statistical adjustments have important, though modest, effects on CHC performance and alter the relative ranking of CHCs. If pay-for-performance is implemented as a quality improvement strategy, differences in rankings would affect how financial incentives are distributed and could have unintended negative consequences for CHCs serving high proportions of ethnic and racial minority patients (Casalino and Elster 2007
). In particular, CHCs under financial strain (Shi et al. 2001
; McAlearney 2002
; Cunningham, Bazzoli, and Katz 2008
;) that perform worse on patient experience measures might avoid seeing patients that are thought to contribute to lower scores. Conversely, CHCs that perform well on experience measures that are not adjusted accurately may not recognize care processes that need improvement. CHC clinicians and leaders might also be skeptical of patient-reported health care experience information if differences in patient mix are not considered in scoring methods.
Our findings differ from previous research among commercially insured patient samples, which suggested that adult respondent characteristics have influences on patient experience measures similar in size and direction for both adult and child surveys (Zhan et al. 2002
). We observed differences between adult and child surveys in the case-mix effects of adjusters, including differences in their explanatory powers and coefficients. In particular, the explanatory powers of health status, Latino ethnicity, and survey language were generally larger in the child sample for report items. If the case-mix effects are not uniform across samples, the coefficients reflect differences in how CHCs serve either adult or child patients (Zaslavsky, Zaborski, and Cleary 2000
). Combining samples would require extensive statistical adjustment for differential effects using interaction terms and scores would still inadequately adjust performance for case-mix effects.
We also found that including race/ethnicity and non-English survey language in our case-mix model (Model 2) had a greater impact on patient experience summaries and rankings than that observed for Model 1. This case-mix effect was even more pronounced in the child sample, where we observed larger explanatory power of race/ethnicity and non-English survey language for reports than in the adult sample. The greater between-CHC variability of race/ethnicity and language was the primary driver of these results, as minority and non-English proficient parents reporting on their child's health care experiences were more clustered at CHCs than other adults. The effects of race/ethnicity or language may also be stronger in the child sample because parents have been found to be less tolerant of perceived problems with their child's primary care experiences and more readily seek alternatives if they perceive quality to be better elsewhere (Berry et al. 2008
Our results are consistent with previous research findings of differences between patient-reported health care experiences and overall ratings (Morales et al. 2001
; Weech-Maldonado et al. 2001
). Low English proficiency has been shown to have a negative association with patients' experiences of physician interactions (Garcia et al. 2004
) and positively associated with patients' global ratings of care (O'Malley et al. 2005
). Similarly, we found that non-English survey language was negatively associated with patient reports on access and physician–patient interactions, but a positive effect on overall ratings. There are many possible explanations for these lower patient-reported experiences. Physicians may not be communicating effectively with low English proficient patients, with our without professionally trained interpreters, and language barriers may also impede timely access to care at some CHCs. The discrepancy between negative reports and positive ratings may stem from the greater influence of expectations on ratings (Morales et al. 2001
; Weech-Maldonado et al. 2001
;), or from extreme reporting tendencies, in which respondents are more likely to select extremely high or extremely low scores (Weech-Maldonado et al. 2008
). More research is needed to determine the nature of the relationship between language proficiency and reporting tendencies.
There is some debate about whether to include race/ethnicity variables in case-mix models. It has been proposed that adjusting for race/ethnicity may actually discourage health care organizations from implementing systems to remove health care disparities (Romano 2000
). Yet different racial and ethnic groups have been consistently shown to report their experiences of health care differently. For example, Asian patients tend to give more negative reports of their health care experiences (Saha and Hickam 2003
; Rodriguez et al. 2008
;). Adjusting for all patient demographics that affect scores ensures that summaries of composite scores for CHCs reflect the scores that CHCs would have been given if they were all serving the same standard population and do not merely represent differences in the populations served (Zaslavsky, Zaborski, and Cleary 2000
). Although a preferable practice may be to report performance measures separately by race/ethnicity (Fiscella et al. 2000
), achieving adequate sample sizes from all major racial and ethnic groups could be very costly. Moreover, racial and ethnic minority groups are significantly clustered in primary care practices with low performance on patient experience measures (Rodriguez et al. 2008
), making stratifying results by race/ethnicity impractical.
Our study results should be considered in light of several important limitations. First, the survey had a low response rate of 32 percent and we do not have information about nonresponders. However, our response rate is similar to that achieved in other similar studies of patient experiences of care (Jha et al. 2008
). Although response rates may be related to patient experiences of care, empirical evidence suggests that adjusting for nonresponse does not improve the precision of performance comparisons (Elliott et al. 2005
). Furthermore, the relationship between response rates and scores on patient experience measures is often a product of differences in case-mix, and case-mix adjustment has been shown to eliminate much of the impact of nonresponse bias (Elliott et al. 2005
; Campbell et al. 2009
Second, our sample was from the State of Washington and our results may not generalize to geographic areas with different demographic distributions. For example, in Washington State, a larger proportion of the state's Latino population is foreign born (41 percent), younger than 18 (38 percent), and of Mexican origin (80 percent) than in other states (Pew Hispanic Center 2009
). Other states serving a different Latino population may observe different patterns of patient experience reporting, since evidence suggests that Mexican Americans are less integrated into primary care (Burnette and Mui 1999
). Nevertheless, our findings highlight the importance of case-mix adjustment across CHCs and the methods could help to guide these efforts in other regions. Third, the data were obtained via telephone interviews, which are known to result in lower response rates and higher overall scores than other survey modes (Elliott et al. 2009
). However, the use of telephone interviews also increases the representativeness of respondents because patients with low literacy levels are less likely to complete mailed surveys (Fowler et al. 2002
). This is a particular concern for CHCs because most primarily serve patients of low socio-economic status.
Finally, recent studies have found that language has a greater effect on patient experiences than race/ethnicity (Weech-Maldonado et al. 2001
). Although we suspect that the effect of race/ethnicity on patient reports and ratings was modified by language, we did not have a large enough sample size per clinic to stratify major racial/ethnic groups by survey language. Yet this is a limitation that many health care organizations face because of small samples for subgroups. Our findings suggest that both English proficiency and race have strong effects on patient experiences and should be included in case-mix adjustments. A recent report from the Institute of Medicine has highlighted the importance of accurately collecting information on patient race, ethnicity, and language preference in order to identify and address disparities in health care quality (Institute of Medicine 2009
). As our findings demonstrate, standardized collection of patient demographics is also critical to accurately comparing CHC performance on patient experience measures.
In conclusion, our findings provide valuable information for managed care organizations that seek to improve patient experiences of primary care at CHCs. When comparing the performance on patient experience measures across CHCs, we recommend adjusting for the case-mix effects of adult self-reported health status or child health status, adult education, age, educational attainment, race/ethnicity, and non-English language. We also recommend analyzing adult and child reports and ratings of health care experiences separately. It is ideal to focus improvement efforts on primary care sites and individual clinicians because they account for most of the explainable variation on patient experience measures (Rodriguez et al. 2009
). CHCs serving diverse patient populations often take pride in delivering culturally competent medical care. Strategies such as providing interpreter services, cultural competency training, recruiting and retaining diverse staff, using community health workers, and culturally competent health promotion have been proposed as ways to improve quality of care and reduce racial and ethnic disparities (Brach and Fraser 2000
; Beach et al. 2005
; Morales et al. 2006
;). However, little empirical research has been conducted to examine how organizational systems and policies aimed at improving cultural competency impact performance on patient experience measures, such as physician–patient communication quality (Paez et al. 2008
). Focused research on improving patient-centeredness in safety-net clinics and CHCs will be important for reducing racial and ethnic disparities in health care quality.