PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Med Care. Author manuscript; available in PMC Sep 1, 2013.
Published in final edited form as:
PMCID: PMC3748811
NIHMSID: NIHMS391989
The Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Cultural Competence (CC) Item Set
Robert Weech-Maldonado, Adam Carle, Beverly Weidmer, Margarita Hurtado, Quyen Ngo-Metzger, and Ron D. Hays
Robert Weech-Maldonado, Department of Health Services Administration, University of Alabama at Birmingham, 1675 University Boulevard, 520 Webb, Birmingham, AL 35294, Phone: (205) 996-5838, Fax: (205) 975-6608, rweech/at/uab.edu;
Background
There is a need for reliable and valid measures of cultural competence from the patient’s perspective.
Objective
This paper evaluates the reliability and validity of the Consumer Assessments of Healthcare Providers and Systems (CAHPS®) Cultural Competence (CC) item set.
Research Design
Using 2008 survey data, we assessed the internal consistency of the CAHPS CC scales using Cronbach alphas, and examined the validity of the measures using exploratory and confirmatory factor analysis, multitrait scaling analysis, and regression analysis.
Subjects
A random stratified sample (based on race/ethnicity and language) of 991 enrollees, less than 65 years old, from two Medicaid managed care plans in California and New York.
Measures
CAHPS CC item set after excluding screener items and ratings.
Results
Confirmatory factor analysis (CFI= 0.98; TLI= 0.98; RMSEA= 0.06) provided support for a seven-factor structure: Doctor Communication-Positive Behaviors; Doctor Communication-Negative Behaviors; Doctor Communication-Health Promotion; Doctor Communication-Alternative Medicine; Shared Decision Making; Equitable Treatment; and Trust. Item--total correlations (corrected for item overlap) for the 7 scales exceeded 0.40. Exploratory factor analysis showed support for one additional factor: Access to Interpreter Services. Internal consistency reliability estimates ranged from 0.58 (Alternative Medicine) to 0.92 (Positive Behaviors), and was 0.70 or higher for four of the eight composites. All composites were positively and significantly associated with the overall doctor rating.
Conclusions
The CAHPS CC 26-item set demonstrates adequate measurement properties, and can be used as a supplemental item set to the CAHPS Clinician and Group Surveys in assessing culturally competent care from the patient’s perspective.
Keywords: CAHPS, cultural competence, patient-centered care, measurement
Among the strategies that have been advocated for reducing racial/ethnic differences in patient experiences is the provision of “culturally competent” care.1, 2 The National Quality Forum (NQF) (p. 2) recently defined cultural competency as the “ongoing capacity of healthcare systems, organizations, and professionals to provide for diverse patient populations high-quality care that is safe, patient and family centered, evidence based, and equitable.”3
Based on the NQF definition, patient-centered care is one of the major elements of cultural competency. The Institute of Medicine (p. 3)4 defined patient-centered care as “care that is respectful and responsive to individual patient preferences, needs, and values…” McWhinney5 described patient-centered care as being able to see through the patient’s eyes. Thus, the patient’s perspective on the care that he or she receives is an essential barometer of culturally competence care.
The Consumer Assessment of Healthcare Providers and Systems (CAHPS®) project has resulted in a set of standardized survey instruments that can be used to collect reliable information from patients about the care they have received. These evaluations provide important information about how well providers meet the needs of the people they serve.6 For example, the CAHPS Clinician and Group (C&G) Surveys assess patients’ experiences with health care providers and staff in doctor’s offices. The CAHPS C&G core survey contains 13 items measuring 3 domains of performance: timeliness of care, provider communication, and staff helpfulness. In addition it has one global rating for provider.7
CAHPS data have been used to assess racial/ethnic and language differences in patient experiences with care.8-12 However, there are concerns that the CAHPS instrument does not fully capture domains of care of particular relevance to diverse populations, such as trust, perceived discrimination, shared decision making, and access to language services.1 To address this gap, the CAHPS team developed an item set to assess aspects of cultural competency not adequately addressed in the existing CAHPS surveys, which could serve as a supplemental item set to the CAHPS Clinician & Group surveys. The purpose of this study is to evaluate the internal consistency reliability and validity of the CAHPS Cultural Competence (CC) item set. The psychometric analysis by survey language (English/Spanish) is reported elsewhere.13
Guided by Bethell et al.’s14 conceptual model of measuring health care quality among diverse populations and a comprehensive literature review on diverse populations in the U.S., we developed a framework for obtaining the patient’s perspective on culturally competent care.1 In this framework, health care is experienced by the patient in the context of interactions with providers within the health care system. Therefore, there are three factors that affect the quality of care for diverse populations: patient factors, provider factors, and health care system factors. The framework focuses on the areas where the three factors overlap (Figure 1). The first two domains reflect interactions between the patient and the provider: 1) Patient-provider communication; 2) Respect for patient preferences/ shared decision-making. The other four domains include patient and provider interactions, but also include interactions with other staff and the health care system overall: 3) Experiences leading to trust or distrust; 4) Experiences of discrimination; 5) Health literacy strategies; and 6) Language services. These six domains are best measured by patient assessments as opposed to organizational or provider assessments. There are other domains of quality care that are important, such as Access (ability to get timely care) or Coordination of Care (between different providers and health care settings.) However, because these domains of quality care have been extensively examined as part of patient-centered care, we chose not to include them within the scope of this study. Furthermore health literacy was not included as one of the domains of CAHPS CC item set because it was the focus of a separate project (CAHPS Health Literacy item set).15
Figure 1
Figure 1
Conceptual Framework of Culturally Competent Care From the Patient’s Perspective
We summarize below our literature review on five of the cultural competence domains. We focused specifically on empirical studies that included diverse populations (African Americans, Hispanics/Latinos, Asians and Pacific Islanders, or American Indians). We also only included studies conducted in the United States, as the healthcare systems and experiences in other countries may be very different and thus, not generalizable to the US.
Patient-Provider Communication
Communication in the medical interaction plays a central role in decisions about subsequent interventions, and can influence patient adherence, satisfaction with care, and health outcomes16. Some racial/ethnic groups and those of lower socioeconomic status are more likely to report poor communication with their physicians17,18 Providers’ non-verbal and interpersonal communication behaviors have been found to be particularly important for diverse patient populations. For example, Hurtado et al.19 found that empathy and establishing rapport (as opposed to just providing health information) were more important to minority patients than to White patients. Similarly, African American, Hispanic, and Asian patients were found to rate the provider’s display of “concern, courtesy, and respect” as the most important factor in the interaction.20, 21 Finally, studies have found that listening and spending adequate time are especially important for Asian22 and Hispanic patients.23
While the CAHPS C&G surveys include provider communication as a core measure, this measure focuses on the behavioral aspects rather than the content of communication. One important content area is communication about complementary and alternative medicine (CAM).1 CAM refers to diverse practices and products that are not currently considered part of conventional medicine.24 An estimated 26 percent of African Americans, 28 percent of Hispanics, 36 percent of non-Hispanic whites, and 43 percent of Asian Americans use CAM.25 It is important for patients to discuss their use of CAM with their medical practitioners. Some therapies, such as herbal or vitamin therapies, may cause adverse events or interfere with medical regimens. Furthermore, knowledge of patients’ CAM practices can provide valuable insight into patients’ values, lifestyles, and health beliefs, which may, in turn; assist practitioners in providing optimum care.26, 27 Yet, in a national survey of U.S. adults, 70 percent of patients who used CAM reported that their providers did not discuss CAM use with them.24, 28
Another content area of particular importance to diverse populations is communication about health promotion. Racial/ethnic minorities suffer disproportionately from chronic diseases, such as diabetes and heart disease.29 As such communication about diet and exercise are particularly important for prevention in promoting a healthy life style among diverse patient populations. Similarly, racism and discrimination places minorities at higher risk for depression and anxiety.30 Yet minorities are less likely to receive treatment for depression than Whites.31 Communication about stress and depression with diverse populations can help reduce disparities in care.
Respect for Patient Preferences/Shared Decision Making
The Institute of Medicine encourages providers to respect patients’ preferences and promote their active participation in clinical decision-making to the extent that patients’ feel comfortable and are willing to take part.4 Patients may participate in their care in a variety of ways, which includes having meaningful dialogue about their preferences, knowing all the available options, and making final decisions about treatment.
A report by the Agency for Healthcare Research and Quality (p. 122)17 noted that “Blacks, Asians, Hispanics, and low income populations are more likely to feel disenfranchised in the decision-making process.” Compared with Whites, African Americans and Asians were more likely to report under-involvement in the health care decision-making process (e.g. 22% of Whites, versus 27% of African Americans and 42% of Asians reported that they were “not as involved as they would like to have been”). Similarly, Hispanics were more likely than non-Hispanic Whites to report feeling disenfranchised (34% vs. 21%), and low-income populations were more likely than higher-income populations to report this (30% vs. 20%).18
Experiences Leading to Trust
Patient’s trust is an important element of the health care encounter. Thom et al.32 found that patients with lower levels of trust were less likely to adhere to their physician’s advice, and were more likely to report not receiving the services they requested or needed. Similarly, patients with lower levels of trust have reported lower levels of satisfaction with the patient-provider relationship.32
Several studies have observed lower levels of patient trust among racial and ethnic minorities.33-36 Using data from a national sample of adults, Hunt et al. (2005)33 reported that African Americans and Latinos were less trusting and less satisfied with their physicians than Whites. Similarly, LaVeist et al.36 found that African Americans were significantly more likely than Whites to report mistrust of the medical system, and those who reported greater mistrust of the medical care system were less satisfied with their care.
Experiences of Discrimination
According to the Institute of Medicine,37 discrimination is differential treatment based on race, ethnicity, gender or other individual attribute. Racial bias or discrimination in the practice and delivery of healthcare may be at least partly responsible for the observed racial and ethnic health disparities.38 Prior research has found that perceived discrimination is associated with various negative health outcomes. Perceived discrimination has also been found to have a negative effect on satisfaction with care.39-41 For example, a study of the California Health Interview survey found that patient perceptions of discrimination based on race or ethnicity were associated with lower ratings of health care quality.40 Similarly, Hausmann et al.41 found that perceived racism was associated with low patient ratings of provider warmth/respectfulness, and ease of communication. Finally, Weech-Maldonado et al. found that Medicaid beneficiaries in Florida who perceived discrimination based on race/ethnicity and insurance reported lower CAHPS scores.42
Language Services
According to the 2000 census, approximately 47 million people in the U.S. speak a language other than English at home and over 21 million are Limited English Proficient (LEP)43 Previous research has shown that non-English patients have worse access to care 44-47 and report lower ratings of care than English-speaking patients.9, 10, 12, 48, 49
The limited supply of bilingual providers has led health care organizations to use interpreter services to bridge language gaps. When examining the impact of language services, it is important to distinguish between professional interpreters and ad-hoc interpreters. Ad-hoc interpreters are individuals whose primary job function is something other than interpretation and includes clinic staff as well as the patient’s family members and friends.50 On the other hand, professional interpreters receive specialized training on medical interpretation and their main function is interpretation.50 This includes in-person interpretation, third party interpretation and remote, third party interpretation using technology (e.g. using telephone or video). Professional interpreters have been shown to reduce barriers to care among LEP patients.51, 52
The CAHPS CC was designed to assess five domains of cultural competence: Patient-provider communication; Respect for patient preferences/ shared decision-making; Experiences leading to trust or distrust; Experiences of discrimination; and Language services. The survey development involved five steps: 1) Evaluating existing CAHPS surveys to identify existing items that addressed the domains of interest; 2) Conducting a literature review in order to identify existing instruments or item sets that had been used in the past to collect data on cultural competency from the patient’s perspective; 3) Placing a Federal Register notice with a call for measures; 4) Reviewing and adapting existing measures in the public domain; and 5) Writing additional survey items as needed for each of the proposed cultural competency domains.
Each of these steps was conducted collaboratively by members of the CAHPS Cultural Comparability Team, a sub-team of the CAHPS Consortium that includes health researchers experienced in survey development and testing, psychometrics, and translation and cultural adaptation of survey measures. The review of the literature and the various CAHPS surveys yielded over 90 survey items related to the domains of interest for the CAHPS CC. This included survey items that could be adapted from other CAHPS Surveys, such as the CAHPS In-Center Hemodialysis Survey and the CAHPS American Indian Survey, as well as various surveys that are in the public domain (for example, the National Health Interview Survey, the California Health Interview Survey, the Child Hospitalization Communication, Quality and Safety Survey, Hablamos Juntos, the Commonwealth Fund’s Survey on Disparities in Quality of Healthcare, and the Primary Care Assessment Survey). Only one measure submission was received in response to the Federal Register Notice. The survey development team mapped each of the items that was identified through the literature and survey review to the domains of interest and adapted measures that best captured the domains of interest by rewording the items so that they would include a 12 –month reference period, the same language in referencing the provider as utilized in other CAHPS measures, and response options that were consistent with the other CAHPS measures.
The CAHPS CC item set was translated into Spanish using the Agency for Healthcare Research and Quality’s (AHRQ) Guidelines for Translating CAHPS Surveys.53 First, two independent American Translators Association’s (ATA) certified translators conducted two forward translations of the survey into Spanish. Translators were instructed to aim for a 6th grade reading level or lower. Second, a committee formed by the two translators and members of the CAHPS Cultural Comparability team reviewed the translations and reconciled the differences. The review committee had representation from different Spanish subgroups in the US, such as Mexican, Puerto Rican and South American.
Next we conducted 18 semi-structured cognitive interviews (9 in Spanish and 9 in English) with scripted probes in Los Angeles, Boston, and Chapel Hill, North Carolina. Recruitment for the cognitive testing was aimed at getting a mix of respondents in terms of age, race/ethnicity, gender, and level of education. Based on the findings of the cognitive interviews, the instrument was revised and a final 44-item set was available for field testing.
Field Test
A field test of the CAHPS CC survey was conducted with a stratified random sample of 6,000 adult (18 years and older) enrollees of two Medicaid managed care plans, one in New York (3,200) and the other in California (2,800) in 2008. We chose New York and California for the field test given the diversity of their population. The stratified sample based on race/ethnicity and preferred language was drawn using the health plans’ administrative data: 1,200 non-Hispanic White English speakers, 1,200 non-Hispanic Black English speakers, 900 Hispanic English speakers, 900 Hispanic Spanish speakers, 900 non-Hispanic Asian English speakers, and 900 non-Hispanic Asians with a preferred language other than English. Health plan enrollees 65 years and older were not included in the sample, since this population is generally dually eligible (Medicare and Medicaid) and their health care experiences may be different than those with Medicaid as their primary insurer.
The survey consisted of a 2-wave mailing with follow-up telephone interview of non-respondents. The first mailing included an English survey and a cover letter in English and Spanish. The letter directed Spanish speakers to call an 800 number to request a copy of the survey materials in Spanish. Four weeks after the initial mailing, a second survey packet was mailed to non-respondents. Telephone follow ups in English and Spanish started 2 weeks after mailing the second survey packet. A monetary incentive of $10 was offered to non-respondents after the second call attempt. A 26% response rate (n=1,380) was achieved with the multi-pronged approach. After excluding individuals that did not have a personal doctor or a doctor visit during the last 12 months, the final analytic sample consisted of 991 respondents.
Measures and Analysis Plan
The analysis focuses on the CAHPS CC 27-item set after excluding screener items and ratings. Psychometric analysis was used to assess the internal consistency reliability and validity of the CAHPS CC scales. First, exploratory factor analysis (EFA) with varimax rotation in both SAS and Mplus was used. 54 The number of factors retained was determined by a combination of criteria: (1) the roots criterion of selecting factors with eigenvalues to be greater than 1; (2) the scree plot to examine the point at which the plot of eigenvalues begins to level off; and (3) the interpretability of factors based on the conceptual framework.55 The language services items were factor analyzed separately, since the items only applied to a small subset of respondents, those with limited English proficiency (LEP) and who had used an interpreter (N= 76). LEP respondents were identified as those for which English was not their primary language and who spoke English not well or not at all.
Second, confirmatory factor analysis (CFA) was conducted of the CAHPS CC scales derived from the exploratory factor analysis using Mplus We used fit index levels for Root Mean Square Error or Approximation (RMSEA), Comparative Fit Index (CFI), and Tucker Lewis Index (TLI) as identified by the literature.56 Mplus accounted for the ordinal nature of the response options.57
Third, multitrait scaling analysis was used to assess item discrimination across scales.58 Multitrait scaling generalizes multitrait-multimethod analytic methods from the trait level to the item level. In doing so, it allows the user to investigate convergent and divergent validity at the item level.
Fourth, we used Cronbach’s alpha to estimate scale internal consistency reliability. Cronbach alphas were estimated for the overall sample.
Finally, the convergent validity of the CAHPS CC scales was assessed. Regression analyses were conducted to examine the associations between the CAHPS CC composite scores and the overall doctor rating item, controlling for race/ethnicity, gender, age, education, self-rated health status, and language of the survey (Spanish vs. English). Separate regression models were run for each CAHPS CC composite. The CAHPS global rating (0-10) for personal doctor, where 0= worst possible doctor and 10= best possible doctor, was linearly transformed to a 0-100 possible range (i.e., multiplied by 10). The CAHPS CC composite scores were calculated by linearly transforming items to a 0-100 possible (with a higher score representing more favorable scores), and then averaging items within each composite. Negatively worded items were rescored to show a positive scale. Self-reported race/ethnicity consisted of non-Hispanic White, Hispanics, non-Hispanic Black, non-Hispanic Asian, and Other. The gender item asked whether the respondent is male or female. Age consisted of five categories: 18-24, 25-34, 35-44, 45-54, and 55-64. Education consisted of five categories: 8th grade or less, some high school but did not graduate, high school graduate or GED, some college or 2-year degree, 4-year college graduate or higher. Self-reported health status asked respondent to rate their overall health: excellent, very good, good, fair, and poor. Finally, Spanish survey is a dummy variable that identifies whether or not the survey was completed or administered in Spanish.
Descriptive statistics for the analytic sample are presented on Table 1. Hispanics were the largest racial/ethnic group (34%) in the sample, while non-Hispanic Whites and Blacks had the smallest representation with 15% each. Approximately 30% of respondents rated their health as fair or poor. The majority of the sample was female (67%) and younger than 44 years old (52%). A significant proportion of respondents (31%) had less than high school graduation or GED. Finally, 12% of respondents completed a Spanish survey.
Table 1
Table 1
Characteristics of Survey Respondents (N=991)
A comparison of respondents and non-respondents based on gender, age, race/ethnicity, primary language, and health plan affiliation showed only a few significant differences. Respondents were more likely to be White (24% versus 20%) and older (39 versus 36 years, and less likely to be Black (18% versus 22%) compared to non-respondents. There were no significant differences between respondents and non-respondents in terms of gender, Hispanic or Asian ethnicity, preferred language, or health plan affiliation.
A seven-factor structure was obtained based on EFA (eigenvalues > 1 and scree plot) and guided by the conceptual framework: Doctor Communication-Positive Behaviors (5 items); Doctor Communication-Negative Behaviors (3 items); Doctor Communication-Health Promotion (4 items); Doctor Communication-Alternative Medicine (2 items); Shared Decision Making (2 items); Equitable Treatment (2 items); and Trust (5 items). CFA (CFI= 0.98; TLI= 0.98; RMSEA= 0.06) provided support for the seven-factor structure: Item--total correlations (corrected for item overlap) were higher than 0.40 for all the multi-item scales (Table 2). The eight domains of the CAHPS CC match closely those of the initial conceptual framework, except that Provider Communication is further divided into four different scales based on both behavioral aspects (positive and negative behaviors) and content (health promotion and alternative medicine) of communication.
Table 2
Table 2
CAHPS Cultural Competence Item-Scale Correlation, Corrected for Item Overlap
Exploratory factor analysis (eigenvalue > 1) of the language services items (4 items) provided support for a one-factor structure for Access to Interpreter Services. Factor loadings were higher than 0.40 except for one item (how often did the interpreter you had most often at this doctor’s office treat you with courtesy and respect). This item was dropped from the Access to Interpreter Services resulting in a 3-item scale: 1) Did you use friends or family members as interpreters because there was no other interpreter available at this doctor’s office?; 2) How often did your visit with this doctor start late because you had to wait for an interpreter?; and 3) Was there any time when you needed an interpreter and did not get one at this doctor’s office?
Means, standard deviations, and Cronbach’s alphas for each of the multi-item scales are shown on Table 3. The lowest CAHPS CC scores were observed for Doctor Communication-Alternative Medicine (Mean= 12.1), Doctor Communication- Health Promotion (Mean= 49.1) and Access to Interpreter Services (Mean= 76). Internal consistency reliability estimates were 0.60 or higher for seven of the eight composites, and was 0.70 or higher for four of the seven composites. The two-item Doctor Communication-Alternative Medicine scale had the lowest internal consistency reliability of 0.58.
Table 3
Table 3
Descriptive Statistics and Internal Consistency Reliability Estimates (Alphas) for the CAHPS Cultural Competence Scales
All CAHPS CC composites were positively and significantly associated with the overall doctor rating (Table 4). The largest unique association was observed for Doctor Communication-Positive Behaviors (B = 0.71) with a 10 point increase (p < 0.001) in this composite resulting in a 7.1 point increase in doctor rating (0-100 scale). The smallest effect was observed for Access to Interpreter Services (B =0.08) with a 10 point increase (p < 0.10) resulting in a 0.8 point increase in doctor rating. The relatively small effect of Access to Interpreter Services may be due to the small sample size for this regression, since this scale was only applicable to those with limited English proficiency (LEP) and who had used an interpreter (n=76).
Table 4
Table 4
Multiple Regression Results of CAHPS CC Composites with Overall Doctor Rating1 (N=991)2
Among the strategies that have been advocated for reducing racial/ethnic differences in patient experiences with care is the provision of “culturally competent” medical care. This study provides support for the psychometric properties of the CAHPS CC item set in general, especially for the English survey. Our results suggest that the 26-item set measures eight separate domains (Doctor Communication-Positive Behaviors (Provider Communication core measure in the CAHPS C&G Survey); Doctor Communication-Negative Behaviors; Doctor Communication-Health Promotion; Doctor Communication-Alternative Medicine; Shared Decision Making; Equitable Treatment; Trust; and Access to Interpreter Services) rather than a single “cultural competence” construct (Appendix 1). Cultural competence is a multi-dimensional concept. Creating individual scales based on these items that correspond to each domain provides a rich set of items that can be used to supplement the CAHPS Health Plan and Clinician and Group survey instruments.59
There is the possibility that the multiple factors we observe may reflect “nuisance” factors rather than substantively meaningful factors. A bi-factor analysis can be used to explore whether a multidimensional model or unidimensional (bi-factor) model best represents the measurement structure of the data. Bifactor models posit that a unidimensional factor (e.g., cultural competence) accounts for the covariance among item responses, but that one or more “nuisance” or “group” factors also exist. Group factors can result from patterns in survey questions, such as common content (e.g., repeated questions about a similar topic) or common methodology (e.g., questions with similar stems). A multidimensional factor structure consists of meaningful and separate factors. If the bifactor model fits better than the multidimensional model, one would generally choose the bifactor model over the multidimensional model. In a separate study by Carle and Weech-Maldonado,60 we use bi-factor analysis to provide support for the multidimensional model and the use of separate CAHPS CC scales.
The lowest CAHPS CC mean scores were observed for Doctor Communication-Alternative Medicine, Doctor Communication- Health Promotion, and Access to Interpreter Services. These are domains of particular relevance to racial/ethnic minorities. Doctor communication on alternative medicine and health promotion is crucial when caring for diverse populations. Racial/ethnic minorities have greater use of alternative medicine, and tend to engage less in health promotion behaviors. Access to interpreter services is important for patient communication particularly as the population becomes more linguistically diverse. Policymakers and health plans should incentivize providers to engage in these types of behaviors.
While our results suggest that all CAHPS CC measures are positively associated with CAHPS doctor ratings, there are three scales that are most strongly associated with doctor ratings: Provider Communication- Positive Behaviors, Provider Communication-Negative Behaviors, and Trust. Health care organizations wanting to improve their CAHPS ratings can implement quality improvement activities to address CAHPS CC domains, with a focus on the behavioral aspects of communication and trust.
The study presents several limitations. First, the study was limited to a Medicaid managed care population. However, a recent study by Stern and colleagues tested the CAHPS CC with an uninsured/underinsured patient population with type 2 diabetes, and provided support for the model reported here.61 Further research is needed testing these measures with other insured populations such as Medicare and commercial. Second, due to resource limitations, the survey was only translated into Spanish. Translation into other languages as well as research examining the measurement equivalence of CAHPS CC in other languages is needed. Third, we did not have any provider identifiers. Therefore, we were not able to assess the inter-rater reliability of the measures at the provider or practice level. Future studies are needed to examine the inter-rater reliability of the CAHPS CC measures, as well as the temporal stability of the measures. Finally, the survey had a relatively low response rate. This may have resulted in potential response bias. Respondents were less likely to be Black, but there were no significant differences between respondents and non-respondents in terms of gender, Hispanic or Asian ethnicity, or preferred language. Despite these limitations, the CAHPS CC item set can serve as a tool to measure culturally competent care from the patient’s perspective.
Acknowledgments
This project has been funded in part by Commonwealth Fund Grant # 20060692. Robert Weech-Maldonado was supported in part by the UAB Center of Excellence in Comparative Effectiveness for Eliminating Disparities (CERED), NIH/NCMHD Grant 3P60MD000502-08S1. Adam Carle was partly supported by 1R15NR010631 (NIH/NINR) and U01 AR057940-02S1 (NIH/NIAMS). Ron Hays was supported by a cooperative agreement from the Agency for Healthcare Research and Quality (U18 HS016980), the national Institute on Aging (P30AG021684), and NCMHD (2P20MD000182). This work was conducted while Quyen Ngo-Metzger was at the University of California, Irvine. The views expressed in this publication are solely the opinions of the authors and do not necessarily reflect the official policies of the U.S. Department of Health and Human Services or the Health Resources and Services Administration, nor does mention of the department or agency names imply endorsement by the U.S. Government.
Appendix 1
CAHPS Cultural Competency’s Domains, Survey Items, and Response Scales*
DomainSurvey ItemResponse Scale

Doctor Communication-Positive Behaviors1. In the last 12 months, how often did this doctor explain things in a way that was easy to understand?Never-Sometimes-Usually-Always

2. In the last 12 months, how often did this doctor listen carefully to you?Never-Sometimes-Usually-Always

3. In the last 12 months, how often did this doctor spend enough time with you?Never-Sometimes-Usually-Always

4. In the last 12 months, how often did this doctor show respect for what you had to say?Never-Sometimes-Usually-Always

5. In the last 12 months, how often did this doctor give you easy to understand instructions about taking care of these health problems or concerns?1Never-Sometimes-Usually-Always

Doctor Communication-Negative Behaviors6. In the last 12 months, how often did this doctor interrupt you when you were talking?Never-Sometimes-Usually-Always

7. In the last 12 months, how often did this doctor talk too fast when talking with you?Never-Sometimes-Usually-Always

8. In the last 12 months, did this doctor ever use a condescending, sarcastic, or rude tone or manner with you?Yes, Definitely
Yes, Somewhat
No

Doctor Communication-Health Promotion9. In the last 12 months, did you and this doctor talk about a healthy diet and healthy eating habits?Yes, Definitely
Yes, Somewhat
No

10. In the last 12 months, did you and this doctor talk about the exercise or physical activity you get?Yes, Definitely
Yes, Somewhat
No

11. In the last 12 months, did you and this doctor talk about things in your life that worry you or cause you stress?Yes, Definitely
Yes, Somewhat
No

12. In the last 12 months, did this doctor ever ask you whether there was a period of time when you felt sad, empty or depressed?Yes
No

Doctor Communication-Alternative Medicine13. In the last 12 months, has this doctor ever asked you if you have used these other people to help with an illness or to stay healthy (for example, acupuncturist or herbalist)?Yes
No

14. In the last 12 months, has this doctor ever asked you if you used natural herbs?Yes
No

Shared Decision Making15. In the last 12 months, did this doctor talk with you about the pros and cons of each choice for your treatment or health care?2Yes
No

16. In the last 12 months, when there was more than one choice for your treatment or health care, did this doctor ask which choice you thought was best for you?Yes
No

Equitable treatment17. In the last 12 months, how often have you been treated unfairly at this doctor’s office because of your race or ethnicity?Never-Sometimes-Usually-Always

18. In the last 12 months, how often have you been treated unfairly at this doctor’s office because of the type of health insurance you have or because you don’t have health insurance?Never-Sometimes-Usually-Always

Trust19. Do you feel you can tell this doctor anything, even things that you might not tell anyone else?Yes
No

20. Do you trust this doctor with your medical care?Yes
No

21. Do you feel this doctor always tells you the truth about your health, even if there is bad news?Yes
No

22. Do you feel this doctor cares as much as you do about your health?Yes
No

23. In the last 12 months, how often did you feel this doctor really cared about you as a person?Never-Sometimes-Usually-Always

Access to Interpreter Services24. In the last 12 months, did you use friends or family members as interpreters because there was no other interpreter available at this doctor’s office?3, 4Yes
No

25. In the last 12 months, how often did your visit with this doctor start late because you had to wait for an interpreter? Do not include friends or family members.Never-Sometimes-Usually-Always

26. In the last 12 months, was there any time when you needed an interpreter and did not get one at this doctor’s office? Do not include friends or family members.Yes
No
*For latest version, please see: Agency for Health Care Research and Quality (AHRQ). CAHPS Cultural Competence Item Set. Available at: https://www.cahps.ahrq.gov/Surveys-Guidance/Item-Sets/Cultural-Competence.aspx.
1Screener item for item 5: In the last 12 months, did you talk with this doctor about any health problems or concerns? Yes, No
2Screener item for items 15 and 16: Choices for your treatment or health care can include choices about medicine, surgery, or other treatment. In the last 12 months, did this doctor tell you there was more than one choice for your treatment or health care? Yes, No
3Screener for items 24 and 25: An interpreter is someone who helps you talk with others who do not speak your language. Interpreters can include friends or family members, staff from the doctor’s office, or telephone interpreters. In the last 12 months, did you ever use an interpreter to help you talk with this doctor? Yes, No (Skip if Response= Yes)
4Screener for item 24: In the last 12 months, how often did you use a friend or family member as an interpreter when you talked with this doctor? Never- Always. (Skip if Response= Never)
Contributor Information
Robert Weech-Maldonado, Department of Health Services Administration, University of Alabama at Birmingham, 1675 University Boulevard, 520 Webb, Birmingham, AL 35294, Phone: (205) 996-5838, Fax: (205) 975-6608, rweech/at/uab.edu.
Adam Carle, University of Cincinnati School of Medicine, 3333 Burnet Avenue, MLC 7014, Cincinnati, OH 45229, Phone: (513) 803-1650, adam.carle.cchmc/at/gmail.com.
Beverly Weidmer, RAND, 1776 Main Street, P.O. Box 2138, Santa Monica, CA 90407-2138, Phone: (310) 393-0411, x6788, Beverly_Weidmer/at/rand.org.
Margarita Hurtado, American Institutes for Research (AIR), 10720 Columbia Pike- Suite 500, Silver Spring, MD 20901, Phone: (301) 592-2215, mhurtado/at/air.org.
Quyen Ngo-Metzger, University of California, Irvine, Department of Medicine, University of California, Irvine School of Medicine, 100 Theory Drive, Suite 110, Irvine, CA 92697-5800, Phone: (301) 443-8894, Qngo-metzger/at/hrsa.gov.
Ron D. Hays, University of California, Los Angeles, Department of Medicine, 911 Broxton Avenue, Room 110, Los Angeles, CA 90024, Voice: (310) 794-2294; drhays/at/ucla.edu.
1. Ngo-Metzger Q, Telfair J, Sorkin D, Weidmer B, Weech-Maldonado R, Hurtado M, Hays RD. Cultural Competency and Quality of Care: Obtaining the Patient’s Perspective. NY: Commonwealth Fund; 2006.
2. Weech-Maldonado R, Dreachslin JL, Dansky KH, et al. Racial/ethnic diversity management and cultural competency: The case of Pennsylvania hospitals. Journal of Healthcare Management. 2002;47:111–126. [PubMed]
3. National Quality Forum. A Comprehensive Framework and Preferred Practices for Measuring and Reporting Cultural Competency. Washington DC: National Quality Forum; 2008.
4. Institute of Medicine (IOM) Crossing the Quality Chasm. In: Kohn L, C J, Donaldson M, editors. Committee on Quality Health Care in America. Washington, D.C.: National Academy Press; 2001.
5. McWhinney I. The Need for a Transformed Clinical Method. In: Stewart M, R D, editors. Communicating with medical patients. London: Sage; 1989. pp. 25–40.
6. Crofton C, Lubalin JS, Darby C. Foreword. Medical Care. 1999;37:MS1–MS9. [PubMed]
7. Agency for Health Care Research and Quality (AHRQ) CAHPS Survey and Tools to Advance Patient-Centered Care. [5/6, 2012];2012 https://www.cahps.ahrq.gov/Surveys-Guidance/CG.aspx.
8. Morales LS, Elliott MN, Weech-Maldonado R, et al. Differences in CAHPS adult survey reports and ratings by race and ethnicity: an analysis of the National CAHPS benchmarking data 1.0. Health Services Research. 2001;36:595–618. [PMC free article] [PubMed]
9. Weech-Maldonado R, Morales LS, Spritzer K, et al. Racial and ethnic differences in parents’ assessments of pediatric care in Medicaid managed care. Health Services Research. 2001;36:575–594. [PMC free article] [PubMed]
10. Weech-Maldonado R, Morales LS, Elliott M, et al. Race/ethnicity, language, and patients’ assessments of care in Medicaid managed care. Health Services Research. 2003;38:789–808. [PMC free article] [PubMed]
11. Weech-Maldonado R, Elliott MN, Morales LS, et al. Health plan effects on patient assessments of Medicaid managed care among racial/ethnic minorities. Journal of General Internal Medicine. 2004;19:136–145. [PMC free article] [PubMed]
12. Weech-Maldonado R, Fongwa MN, Gutierrez P, et al. Language and regional differences in evaluations of Medicare managed care by Hispanics. Health Services Research. 2008;43:552–568. [PMC free article] [PubMed]
13. Carle A, Weech-Maldonado R, Weidmer B, et al. Does the CAHPS Cultural Competence Survey Provide Equivalent Measurement Across English and Spanish Versions? Medical Care. In Press. [PMC free article] [PubMed]
14. Bethell C, Carter K, Latzke B, et al. Measuring and interpreting health care quality across culturally-diverse populations: A focus on consumer-reported indicators of health care quality. Portland: Foundation for Accountability; 2003.
15. Weidmer B, Brach C, Hays RD. Developing a CAHPS Item Set to Address Health Literacy. Medical Care. In Press. [PubMed]
16. Lambert BL, Street RL, Cegala DJ, et al. Provider-patient communication, patient-centered care, and the mangle of practice. Health Communication. 1997;9:27–43.
17. Agency for Health Care Research and Quality (AHRQ) National Healthcare Disparities Report. Rockville: U.S. Department of Health and Human Services; 2003.
18. Collins KS, H D, Doty MM, Ives BL, Edwards JN, Tenney K. Diverse Communities, Common Concerns: Assessing Health Care Quality for Minority Americans. New York: The Commonwealth Fund; 2002.
19. Hurtado M, Frazier K, Levin A, et al. Valued aspects of ambulatory Health Care: Comparing patients and Health Care providers based on office visit accounts. AcademyHealth Annual Meeting; 2005. abstract no. 4214.
20. Murray-García JL, Selby JV, Schmittdiel J, et al. Racial and ethnic differences in a patient survey: patients’ values, ratings, and reports regarding physician primary care performance in a large health maintenance organization. Medical Care. 2000;38:300–310. [PubMed]
21. Nápoles-Springer AM, Santoyo J, Houston K, et al. Patients’ perceptions of cultural factors affecting the quality of their medical encounters. Health Expectations. 2005;8:4–17. [PubMed]
22. Ngo-Metzger Q, Legedza ATR, Phillips RS. Asian Americans’ reports of their health care experiences. Journal of General Internal Medicine. 2004;19:111–119. [PMC free article] [PubMed]
23. Saha S, Arbelaez JJ, Cooper LA. Patient-physician relationships and racial disparities in the quality of health care. American Journal of Public Health. 2003;93:1713–1719. [PubMed]
24. Eisenberg DM, Kessler RC, Foster C, et al. Unconventional medicine in the United States--prevalence, costs, and patterns of use. New England Journal of Medicine. 1993;328:246–252. [PubMed]
25. Barnes PM, Powell-Griner E, McFann K, et al. Complementary and alternative medicine use among adults: United States, 2002. Advance Data. 2004:1–19. [PubMed]
26. Pappas S, Perlman A. Complementary and alternative medicine: the importance of doctor-patient communication. Medical Clinics of North America. 2002;86:1–10. [PubMed]
27. Robinson A, McGrail M. Disclosure of CAM use to medical practitioners: a review of qualitative and quantitative studies. Complementary Therapies in Medicine. 2004;12:90–98. [PubMed]
28. Richardson MA, Straus SE. Complementary and alternative medicine: opportunities and challenges for cancer management and research. Elsevier; 2002. pp. 531–545. [PubMed]
29. Office of Minority Health and Health Disparities (OMHD) Disease Burden & Risk Factors. [5/6, 2012]; https://www.cahps.ahrq.gov/Surveys-Guidance/CG.aspx.
30. Center for Mental Health Services. Mental health: culture, race, and ethnicity—a supplement to Mental Health: A Report of the Surgeon General. Rockville, MD: Substance Abuse and Mental Health Services Administration: Department of Health and Human Services; 2001.
31. Agency for Health Care Research and Quality (AHRQ) National Healthcare Disparities Report 2011. Rockville, MD: Substance Abuse and Mental Health Services Administration: Department of Health and Human Services; 2012.
32. Thom DH, Kravitz RL, Bell RA, et al. Patient trust in the physician: relationship to patient requests. Family Practice. 2002;19:476–483. [PubMed]
33. Hunt KA, Gaba A, Lavizzo-Mourey R. Racial and ethnic disparities and perceptions of health care: does health plan type matter? Health Services Research. 2005;40:551–576. [PMC free article] [PubMed]
34. Schnittker J. Social distance in the clinical encounter: interactional and sociodemographic foundations for mistrust in physicians. Social Psychology Quarterly. 2004;67:217–235.
35. Meredith LS, Siu AL. Variation and quality of self-report health data: Asians and Pacific Islanders compared with other ethnic groups. Medical Care. 1995:1120–1131. [PubMed]
36. LaVeist TA, Nickerson KJ, Bowie JV. Attitudes about racism, medical mistrust, and satisfaction with care among African American and white cardiac patients. Medical Care Research and Review. 2000;57:146–161. [PubMed]
37. Institute of Medicine (IOM) Committee on Understanding and Eliminating Racial and Ethnic Disparities in Health Care. In: Smedley BD, S A, Nelson AR, editors. Unequal treatment: confronting racial and ethnic disparities in healthcare. Washington, DC: The National Academies Press; 2003.
38. King WD, Wong MD, Shapiro MF, et al. Does Racial Concordance Between HIV-positive Patients and Their Physicians Affect the Time to Receipt of Protease Inhibitors? Journal of General Internal Medicine. 2004;19:1146–1153. [PMC free article] [PubMed]
39. Benkert R, Peters RM, Clark R, et al. Effects of perceived racism, cultural mistrust and trust in providers on satisfaction with care. Journal of the National Medical Association. 2006;98:1532–1540. [PMC free article] [PubMed]
40. Sorkin DH, Ngo-Metzger Q, De Alba I. Racial/ethnic discrimination in health care: impact on perceived quality of care. Journal of General Internal Medicine. 2010;25:390–396. [PMC free article] [PubMed]
41. Hausmann LRM, Hannon MJ, Kresevic DM, et al. Impact of Perceived Discrimination in Healthcare on Patient-Provider Communication. Medical Care. 2011;49:626–633. [PMC free article] [PubMed]
42. Weech-Maldonado R, Hall A, Bryant T, et al. The Relationship between Perceived Discrimination and Patient Experiences with Care. Medical Care. In press. [PMC free article] [PubMed]
43. U.S. Census Bureau. Census 2000 Summary File 3, Matrices P19, P20, PCT13, and PCT14. Washington, DC: 2000.
44. Solis JM, Marks G, Garcia M, et al. Acculturation, access to care, and use of preventive services by Hispanics: findings from HHANES 1982-84. American Journal of Public Health. 1990;80:11–19. [PubMed]
45. Stein JA, Fox SA. Language preference as an indicator of mammography use among Hispanic women. Journal of the National Cancer Institute. 1990;82:1715–1716. [PubMed]
46. Timmins CL. The impact of language barriers on the health care of Latinos in the United States: a review of the literature and guidelines for practice. The Journal of Midwifery & Women’s Health. 2002;47:80–96. [PubMed]
47. Kirkman-Liff B, Mondragon D. Language of interview: relevance for research of southwest Hispanics. American Journal of Public Health. 1991;81:1399–1404. [PubMed]
48. Morales LS, Cunningham WE, Brown JA, et al. Are Latinos less satisfied with communication by health care providers? Journal of General Internal Medicine. 1999;14:409–417. [PMC free article] [PubMed]
49. Carrasquillo O, Orav EJ, Brennan TA, et al. Impact of language barriers on patient satisfaction in an emergency department. Journal of General Internal Medicine. 1999;14:82–87. [PubMed]
50. Grant Makers in Health (GIH) the Right Words: Addressing Language and Culture in Providing Health Care. San Francisco, CA: Grantmakers Health; 2003. [PubMed]
51. Jacobs EA, Lauderdale DS, Meltzer D, et al. Impact of Interpreter Services on Delivery of Health Care to Limited–English-proficient Patients. Journal of General Internal Medicine. 2001;16:468–474. [PMC free article] [PubMed]
52. Tocher TM, Larson EB. Do Physicians Spend More Time with Non-English-Speaking Patients? Journal of General Internal Medicine. 1999;14:303–309. [PMC free article] [PubMed]
53. Weidmer B, Hurtado M, Weech-Maldonado R, Ngo-Metzger Q, Bogen K. Guidelines for Translating CAHPS Surveys. Prepared by the CAHPS II Cultural Comparability Team. 2006
54. Muthén LK, Muthén BO. Mplus User’s Guide. Los Angeles: CA: Muthén & Muthén; 2009.
55. Hair JF Jr, Black W, Babin B, et al., editors. Multivariate Data Analysis. Auflage, Upper Saddle River NJ: Pearson Prentice Hall; 2006.
56. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6:1–55.
57. Muthén B. A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika. 1984;49:115–132.
58. Hays RD, Hayashi T. Beyond internal consistency reliability: rationale and user’s guide for multitrait analysis program on the microcomputer. Behavior Research Methods. 1990;22:167–175.
59. Agency for Health Care Research and Quality (AHRQ) CAHPS Cultural Competence Item Set. [5/6, 2012];2011 https://www.cahps.ahrq.gov/Surveys-Guidance/Item-Sets/Cultural-Competence.aspx.
60. Carle A, Weech-Maldonado R. Validly interpreting patients’ reports: Using bifactor and multidimensional models to determine whether surveys and scales measure one or more constructs. Medical Care. In press. [PubMed]
61. Stern RJ, Fernandez A, Jacobs EA, et al. Advances in Measuring Culturally Competent Care: A Confirmatory Factor Analysis of CAHPS-CC in a Safety-Net Population. Medical Care. In press. [PMC free article] [PubMed]