|Home | About | Journals | Submit | Contact Us | Français|
To examine the association between care experiences and parent ratings of care within racial/ethnic/language subgroups.
National Consumer Assessment of Healthcare Providers and Systems Benchmarking Database 3.0 (2003–2006).
111,139 parents of minor Medicaid managed care enrollees.
Cross-sectional observational study predicting “poor” (0–5 on 0–10 scale) parent ratings of personal doctor, specialist, health care, and health plan from care experiences for different parent race/ethnicity/language subgroups (Latino/Spanish, Latino/English, white, and black).
Care experiences had similar associations with the probability of poor parent ratings of care across the four racial/ethnic/language subgroups (p>.20). A one standard deviation improvement in the doctor communication care experience was associated with about half the frequency of poor ratings of care for personal doctor and health care in all subgroups (p<.05). Sensitivity analysis of individual communication items found that failure to provide explanations to children predicted poor ratings of care only among whites, who also weighed the length of physician interaction more heavily than other subgroups.
Communication-based interventions may improve experiences and ratings of care for all subgroups, although implementation of these interventions may need to consider preferences associated with race, ethnicity, and language.
Patient and parent assessments of health care have become increasingly recognized as important dimensions of health care quality (Committee on Quality of Health Care in America, Institute of Medicine 2001). The Consumer Assessment of Healthcare Providers and Systems (CAHPS) includes a family of standardized surveys used throughout the United States to evaluate patient experiences with health plans, providers, hospitals, and other health care entities (AHRQ 2007). CAHPS surveys elucidate how best to meet the needs of patients (Crofton, Lubalin, and Darby 1999). For adults, poor patient evaluations of care are associated with higher utilization of services (Zastowny, Roghmann, and Cafferata 1989), less compliance with medical regimens, poorer medical outcomes (Hall and Dornan 1990; David and Rhee 1998; Alazri and Neal 2003;), and increased frequency of changing providers (Newcomer, Preston, and Harrington 1996; Allen and Rogers 1997; Kerr et al. 1998; Schlesinger, Druss, and Thomas 1999; Haviland, Pincus, and Morales 2003; Lied et al. 2003;). In pediatric populations, high parent-reported quality of care has been associated with less nonurgent emergency department utilization (Brousseau et al. 2007, 2009).
Prior studies have found specific associations between care experiences assessed using CAHPS composite measures and CAHPS global ratings (Morales et al. 2001, 2003; Hargraves, Hays, and Cleary 2003; Elliott et al. 2009). There is also evidence that some care experiences may be more important in predicting ratings of care than others (Morales et al. 2001; Hargraves, Hays, and Cleary 2003; Elliott et al. 2009;). Further, certain patient-specific characteristics, such as visit type and patient demographics, may influence the weight given to various care experiences (Morales et al. 2001; Mosen et al. 2004; Elliott et al. 2009;). Together, these studies indicate the potential utility in tailoring quality improvement interventions for patients with different characteristics and provide the rationale for the approach used here. This is the first detailed large-scale examination of the association between care experiences (as measured by CAHPS composites), and global ratings of care for children of different racial, ethnic, and language groups. It provides understanding of whether the same underlying experiences can be targeted in quality improvement efforts to improve overall experiences, and it informs the extent to which such efforts must be tailored for different subgroups.
These prior studies, as well as many others examining CAHPS scores, have focused on the frequency of positive assessments of care. However, recent studies demonstrate the importance and utility of problem-oriented reporting, focusing on the frequency of negative ratings (Elliott et al. 2007), given that patients assign greater weight to negative information when forming impressions, or making judgments and choices (Ito et al. 1998; Scanlon et al. 2002;). Dissatisfaction with care, as demonstrated by poor ratings of care, has been associated with frequently changing doctors and disenrollment from health plans (Newcomer, Preston, and Harrington 1996; Kerr et al. 1998;) as well as malpractice suits (Fullam et al. 2009).
In this study, we use a problem-oriented approach to examine the relationship between parent experiences of care as measured by CAHPS composites and negative ratings of that care for different racial/ethnic/language subgroups in order to better understand whether future interventions to improve pediatric care experiences and ratings should be differentially tailored for these groups. Because we were most interested in studying differences between whites and other racial/ethnic groups, we chose to conduct this study with a population of children insured by Medicaid to ensure adequate sample sizes for our subgroups of interest.
We analyzed data from the National CAHPS Benchmarking Database 3.0 (NCBD 3.0) Child Medicaid CAHPS Surveys. The NCBD, developed through collaboration among AHRQ, Quality Measurement Advisory Service, Picker Institute, and Westat, compiles and standardizes CAHPS survey data to facilitate health services research with respect to patient care experiences.
The NCBD 3.0 Child Medicaid data consist of survey responses from 207,649 parents of children who were enrolled in U.S. Medicaid managed care plans between 2003 and 2006, 111,139 of whom met criteria for inclusion in this study. Individuals included in the study were those Latinos, blacks, and whites who responded to Child CAHPS 3.0 Medicaid survey and provided information on the child's race/ethnicity/language. The survey respondent was the adult who knew the child best. For convenience, we will refer to that individual as the parent. Data included plans in 26 states. Most (75–92 percent) plans were Health Maintenance Organizations, with Primary Care Case Management plans being the next most common type (5–12 percent). About 1 percent were Point of Service plans, and the rest were classified as “other.” The sampling frame included Medicaid enrollees between 0 and 11 years of age at the time of the survey. The survey is administered in English or Spanish by telephone or mail or a combination of telephone and mail according to the CAHPS protocols (AHRQ 2007). The mean response rate for all plans for the 4-year period was 36 percent, ranging from 12 to 63 percent across plans and years.
The dependent variables in this study were dichotomized global ratings of care for personal doctor, specialist, health care, and health plan. Questions use a 0–10 response scale, with 0 labeled as “worst possible” and 10 as “best possible.” The distribution of these ratings is typically negatively skewed, with as many as 50 percent of responses often being in the top category (Elliott et al. 2007). Following Elliott et al. (2007), we designated a rating of 0–5 rating as “poor.”
Independent variables included the five composite measures of care experiences (getting needed care, timeliness of care, doctor communication, staff helpfulness, and plan customer service; see Appendix SA2 for more details), case mix adjustors, and racial/ethnic/language subgroup indicators. CAHPS composite measures were used to assess care experiences as predictors of poor ratings of care. Composite measures were standardized as z-scores (mean 0, standard deviation 1).
Case mix adjusters included parent respondent age (<18, 18–24, 25–34, 35–44, and >44), gender, educational attainment (less than high school degree, high school degree only, some college, and at least 4-year college degree), parent-rated child health status (excellent, very good, good, fair, or poor), and a chronic disease indicator defined as the parental indication that at least one of the following applied to their child: chronic use or need for prescription medication; above average use or need of medical, mental health, or education services; functional limitations compared with others of the same age; use or need for specialized therapies (e.g., occupational therapy, physical therapy, speech); or undergoing treatment or counseling for emotional, behavioral, or developmental problems (Bethell et al. 2002). These case mix adjusters have been associated with systematic differences in response tendency (Zaslavsky, Zaborski, and Cleary 2000; Elliott et al. 2001; O'Malley et al. 2005;). For the child health status variable, fair and poor responses were collapsed for purposes of adjustment because “poor” responses comprised <1 percent of the total.
Racial/ethnic/language subgroups were defined using parent race/ethnicity and parent survey language in responding to the CAHPS survey (English or Spanish).
Following a common approach (Mays et al. 2003), respondents who characterized themselves as Hispanic/Latino were classified as Latino regardless of race. Respondents who did not answer the ethnicity question, but who did select a race, were assumed to be non-Latino. To ensure adequate sample sizes within each category, we limited ourselves to respondents whose race/ethnicity was Latino, black, or white. Respondents were then further classified by survey language, with Latinos split by survey language into English and Spanish subgroups. Latinos with missing language information were dropped from the analysis (n = 326), as were whites (n = 7) and blacks (n = 811) whose language was not English. This resulted in four primary groups: Latino/Spanish, Latino/English, black, and white.
Respondent age, child age, parent educational attainment, and child health status were compared by racial/ethnic/language subgroup using ordered logistic regression. Logistic regression was used for similar comparisons of respondent gender and child chronic illness status.
A series of four logistic regression models (one for each 0–10 global rating) predicted poor ratings of care from three racial/ethnic/language indicators (relative to a white reference group), the five composite measures of care experiences, and the case mix adjustors. The results of these models are presented in the form of multivariate-adjusted proportions (Graubard and Korn 1999). Because composite measures of care experiences were entered in standardized form (with a standard deviation of one), the percentages displayed correspond to the percentage point change in the probability of rating care as poor associated with a one standard deviation increase in the composite measure.
A second series of four models predicting poor ratings of care added interactions between racial/ethnic/language indicators and composite measures. A Wald test of the 15 interaction degrees of freedom simultaneously provided an omnibus test of whether the association between care experiences and poor ratings of care varied by parent racial/ethnic/language subgroup. Standard errors for all regressions were adjusted for correlation within health plans using the Huber/White correction (White 1980).
Additional sensitivity analyses paralleled the first and the second series of models, but in the place of the five composites substituted the five individual items comprising the communication composite (see Appendix SA2 for individual items).
Table 1 shows the demographic characteristics of the various racial/ethnic/language subgroups, including case mix adjusters. Latino/Spanish, Latino/English, black, and white parents comprised 20, 14, 21, and 45 percent of the sample, respectively. Latino/Spanish parents had the lowest educational attainment of any group (p<.05) with 31 percent having only an elementary school education, as compared with 2 percent of white parents. Consistent with other national estimates for the Medicaid population (Bloom, Cohen, and Freeman 2009), overall child health status ratings were low, with only 36–44 percent of children being rated as having excellent health. Also, consistent with prior findings in this population (Bethell et al. 2002), only 13 percent of Latino/Spanish children screened positively for chronic illness, compared with 22–24 percent of other children.
In our sample, 5.9 percent of parents assigned poor ratings of care to their child's doctor, 8.3 percent to specialists, 6.7 percent to their child's health care, and 7.3 percent to their health plan. Table 2 shows the multivariate-adjusted change in the percent probability of rating care poorly given a one standard deviation improvement in a care experience composite, controlling for other care experiences and case mix adjusters. For each of the four ratings of care, the joint Wald test of interactions was not significant (p>.20 for each, coefficients not shown), consistent with care experiences having similar influence on the probability of poor parent ratings of care across the four racial/ethnic/language subgroups.
Overall, lower mean care experiences for doctor communication were the strongest predictor of rating a child's personal doctor and health care poorly (Table 2). A one standard deviation increase in the mean care experience for doctor communication was associated with a 3.4 percentage point decrease in poor personal doctor ratings (95 percent CI 3.1, 3.7 percent; p<.05) and a 3.5 percentage point decrease in poor health care ratings (95 percent CI 3.3, 3.7; p<.05). Thus, the probability of poor ratings with doctor communication at one standard deviation above the current mean is about one-half of what it is at the mean value of doctor communication.
Poor ratings of specialist were most closely associated with the getting needed care composite and poor ratings of health plan were most strongly associated with the customer service measure. A one standard deviation increase in the mean care experience for getting needed care was associated with a 2.8 percentage point decrease in poor specialist ratings (95 percent CI 2.3, 3.2 percent; p<.05). A one standard deviation increase in the mean customer service composite was associated with a 3.7 percentage point decrease in poor health plan ratings (95 percent CI 3.5, 3.9; p<.05).
Although not the primary focus of these analyses, there were some differences in the overall levels of poor ratings of care by race/ethnicity/language, even after controlling for care experiences. Significantly negative coefficients (p<.05) for Latino/Spanish indicators mean that Latino/Spanish respondents assigned fewer poor care ratings than whites with similar care experiences for all four ratings; Latino/English respondents assigned fewer negative ratings of Health Plan than whites with similar experiences (Table 2).
Because doctor communication had the strongest associations with negative ratings of care for child's personal doctor and health care, additional models were run as sensitivity tests using the five component items of the communication composite in place of the five composite measures. Table 3 presents the results of a series of models (one for each global rating of care) that included the component items of the communication composite, race/ethnicity/language indicators, and case mix adjustors (coefficients for case mix adjustors not shown). The five individual items included in the communication composite were independently associated with the probability of negative ratings for all four ratings of care (p<.05, Table 3) in all but one case (“Doctor explained things in a way child understands” as it relates to Specialist ratings). The largest associations are seen for the items asking about the amount of time the doctor spent with the child and the extent to which the doctor listened to the parent. Associations were weakest in magnitude for the items regarding explaining to parents and explaining to children, with respect being intermediate in value.
Models that tested interactions of communication items with race/ethnicity/language indicators found no significant interactions for Specialist or Health Plan ratings (p>.20 in all instances), providing no evidence that the subgroups differed in the implicit importance of the components of communication assessed by the different items with respect to dissatisfaction with specialists or plans. In contrast, some evidence was found for differences in the importance of these dimensions by race/ethnicity/language for dissatisfaction with personal doctor and health care received. For poor doctor ratings, 4 of 15 interactions (5 items × 3 groups compared with whites) between race/ethnicity/language and communication items were significant (p<.05); 3 of 15 interactions were significant for poor health care ratings (data not shown).
The associations for doctor–parent communication items vary by race/ethnicity/language as follows: for poor ratings of personal doctor, “doctor explained to child” had a significant negative association with ratings for whites (log odds = −0.252, p<.001) but a nonsignificant association (p>.2 in all cases) for Latino/Spanish, Latino/English, and blacks (p<.05 for the white coefficient being lower). “Doctor spent enough time with child” was a significantly weaker predictor (p = .007) of poor doctor ratings for Latino/Spanish (log odds = −0.241, p = .008) than for whites (log odds = −0.559, p<.001), but it mattered for both groups (p = .007 for the white coefficient being lower).
For poor ratings of health care, “doctor spent enough time with child” was a significantly weaker predictor of poor health care ratings for Latino/Spanish (log odds = −0.531, p = <.001) and for blacks (log odds = −0.608, p<.001) than for whites (log odds = −0.848, p<.001; p<.01 for the white coefficient being lower), but it mattered for all groups. Similarly, “doctor respect” was a significantly weaker predictor of poor health care ratings for blacks (log odds = −0.251, p = .003) than for whites (log odds = −0.586, p<.001; p = .002 for the whites coefficient being lower), but it mattered for both groups.
This study represents the first large-scale examination of the association between care experiences and ratings of care for different racial/ethnic/language subgroups. Our tests of variation in the extent to which composite measures varied across racial/ethnic/language subgroups in predicting poor global ratings found that each of several broad domains of experiences was of approximately equal importance across racial/ethnic/language groups. Although we originally hypothesized that the care experiences that were most associated with a particular rating of care might differ across racial/ethnic/language subgroups based on previous research (Kreuter et al. 2003), we instead found that a uniform emphasis on improving the same care experience domains for all racial/ethnic/language subgroups might be appropriate and might lead to similar improvements in CAHPS ratings for all groups. The different racial/ethnic/language subgroups appear to have similar concerns and seem to care about them to a similar extent, as shown by the nonsignificant joint Wald tests for interactions.
We also found that doctor communication was the most important domain for all global ratings of care except for the health plan rating. This finding is consistent with previous studies that have examined the relationship between doctor–patient communication and parental or patient satisfaction (Street 1991, 1992; Williams, Weinman, and Dale 1998; Mangione-Smith et al. 1999; Nobile and Drotar 2003; Beckett et al. 2009). The 3.4 percentage point decrease in poor personal doctor ratings and 3.5 percentage point decrease in poor health care ratings associated with a one standard deviation increase in the mean communication care experience represent a halving of poor care ratings for doctor and health care. Such an improvement in communication thus has the potential to halve the incidence of marked parental dissatisfaction with care incidents, which leads to physician changes, health plan disenrollment (Newcomer, Preston, and Harrington 1996; Kerr et al. 1998;), and/or malpractice suits (Fullam et al. 2009).
Given the fundamental importance of communication, we conducted sensitivity analyses that found a failure to deliver on any of the five dimensions of communication is associated with a higher probability of a negative assessment. Among these five dimensions, failure to listen to parents or spend enough time with the child may be especially consequential. This finding is consistent with studies that have shown that patient satisfaction is higher when the patient perceives longer visit duration (Gross et al. 1998; Lin et al. 2001; Kong et al. 2007;), and it conflicts with current pressures that physicians feel to reduce visit length and increase efficiency (Grumbach et al. 1998; Blendon et al. 2001;). Previous studies have demonstrated that “chatting” or spending time on nonmedical talk (Kikano, Gross, and Stange 1999), providing patients with feedback (Kikano, Gross, and Stange 1999), avoiding the appearance of being rushed (Hardee and Kasper 2008), maintaining eye contact (Hardee and Kasper 2008), and sitting for the visit (Johnson et al. 2007; Hardee and Kasper 2008;) all provide patients with an increased perception of time spent and could be used to address this dimension of the communication composite. However, such communication need not be overly burdensome to providers, as parents who receive moderate-length answers to their questions more often report feeling listened to by the provider than parents given either brief or long answers (Goore et al. 2001).
While spending enough time was important for all groups, it appeared to be less strongly associated with care ratings assigned by Latino/Spanish parents (health care and personal doctor) and by black parents (health care) compared with whites. Thus, interventions focused on improving parent perceptions related to time spent by the provider might be particularly effective for whites. One possible reason that time spent might be less strongly associated with care ratings for Latino/Spanish and black parents is that minorities, more than whites, may be concentrated in practices with severe time constraints. Rodriguez et al. (2008) found that most racial and ethnic minorities were concentrated within physician practices with lower performance on patient experience measures and that disparities for these groups were mainly attributable to this clustering (Rodriguez et al. 2008). More specifically, Reschovsky and O'Malley (2008) found that time spent per patient seen was lower in high-minority practices, suggesting that physicians in these practices may see increased patient volumes. Future research with data that contained practice identifiers might examine the extent to which the present findings of differences in the importance of time spent with the physician persists within, in addition to between, practices. If practices explain these patterns, there may be a need to target communication improvements to resource and time-constrained practices where minorities are clustered. This is especially important in light of new Child Health Insurance Program Reauthorization Act health care legislation that may contribute to an increased burden on already resource-limited practices.
We also found evidence that explaining things to the child reduced the probability of white parents providing a poor rating for their child's personal doctor, but it had no corresponding independent association for any of the other three racial/ethnic language subgroups. This may reflect a preference for direct inclusion of children in the conversation on the part of white parents that is less common among Latino and black parents, consistent with evidence of differences in physician–parent communication style for minority families (Johnson 2004). Dimensions such as trust, respect, authoritativeness, patient involvement, and formality may be valued differently, so that the behaviors that are perceived as constituting good communication may differ.
This study has several limitations. The CAHPS Medicaid surveys have relatively low response rates for the years studied. Although recent metaanalyses suggest that the association between response rates and nonresponse bias within probability samples is generally weak (Groves 2006; Groves and Peytcheva 2008;), nonresponse bias may have played a role in the obtained estimates. For nonresponse bias to have affected the estimates presented here, patterns of nonresponse would have had to differ within racial/ethnic/language subgroups. We were unable to assess the association between care experiences and ratings of care for Asians and other smaller groups in this study because of limited sample size; such groups should be targeted in future research. This was a retrospective observational study, which tested associations and not causation, so there could be other unmeasured explanatory variables that may cause the observed associations. As such, it is not certain that an intervention that resulted in better-reported care experiences would reduce the incidence of poor ratings to the same extent observed here if such unobserved third factors were not also affected by the intervention.
If the observed associations are entirely causal, improving physician communication measures by one standard deviation could halve poor ratings of care for each of the racial/ethnic/language groups studied. Quality improvement interventions targeting communication thus may improve care experiences and satisfaction for all groups and should receive particular emphasis. However, the content of these interventions may need to be tailored for the specific subgroups studied here. There is a paucity of evidence for why there are differences in parental communication preferences, and it is unclear whether the different groups define these dimensions of care or subdimensions of communication in similar ways. To better inform how communication-based quality improvement interventions might best be tailored for these subgroups of parents, future studies, which might include vignette studies (King et al. 2004) and cognitive interviews of parents, should examine individual items within the communication composite to determine whether interpretation of these items varies by racial/ethnic/language subgroup.
Joint Acknowledgment/Disclosure Statement: The authors wish to thank Dale Shaller, CAHPS Database Managing Director, Shaller Consulting Group, Stillwater, MN, for his help in obtaining the data for this study.
The research support was provided by the Ruth L. Kirschstein National Research Service Award (NRSA) Institutional Research Training Grant (T32) (V.W.).
Additional supporting information may be found in the online version of this article:
Appendix SA1: Author Matrix.
Appendix SA2: Parent-Reported Care Experiences in Each Composite.
Please note: Wiley-Blackwell is not responsible for the content or functionality of any supporting materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.