The CCAS criteria and the activities they define serve as a roadmap for the introduction of CC activities into an agency, as well as a marker of progress. The range of scores observed in the study is indicative of the differential progress of the participating agencies. Factor analysis suggests that there are three stages in the process of becoming increasingly culturally competent. In the first stage, corresponding to the first factor, the emphasis is on administrative elements (agency has a commitment, receives community input from a CC committee and conducts staff training). In the second stage, the agency engages in activities designed to better understand and serve their communities (collects data, institutes recruiting/hiring/retention policies and creates translated and easy to read service descriptions and educational materials). The third stage is comprised of activities that are directly associated with clinical care (has interpreters, bilingual, bicultural staff and reviews, adapts and institutes new services). Notably, none of the administrative items, including conducting staff training, contributed to the prediction of reduced service outcomes disparities. In contrast, all of the clinically related items had an impact. Administrative activities, such as showing commitment by having a mission statement, are essential elements of the CC process, but were insufficient in and of themselves to reduce odds ratios. However, it seems self evident that these activities must be in place before activities more closely related to direct care, such as adapting a service to a cultural group, can commence. Surprisingly, training activities were not predictive of disparity reduction. This may be a consequence of the often heard criticisms that existing training curricula are of mixed quality, overly broad and too distant from the clinical process.
Construct validity was demonstrated for several CCAS criteria. Low correlation of the CCAS Total Score and the CCSI total score was expected as some features of CC contained in the CCSI checklist are not reflected in the CCAS. If there is future modification of the scale, some of these items might be usefully introduced as discussed below. The inter-rater reliability of the CCAS is satisfactory, but is substantially improved if two or more raters perform the evaluation. Field experience in the use of the scale suggests that a rater needs considerable interpersonal skills and a determined attitude to obtain the necessary information to complete the CCAS. Multiple raters with varying agency connections and sufficient levels of assertiveness will likely increase the chance of obtaining accurate and comprehensive coverage of the scale items.
Although Hispanics and Whites did not statistically differ in their average outcomes for both service measures, Blacks and Whites did. At the same time, CCAS items demonstrated predictive validity for Hispanics versus Whites, but not for Blacks versus Whites. Though these findings may appear to be counter-intuitive, they are not contradictory. The fact that there are no differences between Hispanics and Whites over all facilities may indicate that in this sample CC is working effectively to eliminate Hispanic disparities. But this does not preclude higher CCAS scores predicting smaller odds ratios. The former is the difference between the average outcome (of a univariate measure) of the Whites and Hispanics, while the latter quantifies the (bivariate) relationship between the odds ratio and the CCAS score across the 27 facilities in terms of a slope in the logistic regression.
The meaning of a health care disparity has been widely discussed (e.g. IOM 2002
; McGuire et al. 2006
; Cook et al. 2009
; Duan et al. 2008
). In these definitions, differences or divergences in service outcome rates are viewed as disparities when they are estimated within stratum in which the groups are comparable with respect to conditions, other than the service itself, that can impact outcome, e.g., health status and access. We included covariates for diagnosis, age and gender in the logistic hierarchical models as proxies for health status. It would have been desirable also to include direct measures of socioeconomic status as they are contributors to access, but there were no relevant variables for which data were systematically available. However, the clinics in the study are part of a widely available public mental health system of the county, so it is likely that the socioeconomic status of clients is fairly homogeneous and that access is not an issue. We obtained odds ratios separately for comparable diagnostic groups and consider these as measures of disparity.
The predictive validity finding that CC scores on criteria related to linguistic capacity of the agency resulted in disparity reduction for Hispanics but not for Blacks is not unexpected. However, linguistic accommodations of a different sort than are required by Hispanics may be needed to enhance the care experience for Blacks. Staff familiar with the language and phrases of the daily lives of Blacks, even to the extent that the vernacular is used in the clinical setting when appropriate, might improve the clinical experience. It is disconcerting, however, that the service related criterion predicted disparity reductions for Hispanics but did not for Blacks. New or adapted services for Blacks may not have been in place in the studied agencies. Identification of the cultural elements that more effectively engage and retain Black clients is currently the subject of considerable research and development. The reversal of odds ratios for Blacks for the engagement outcome with increasing CCAS scores on hiring and retention policies might be spurious or speculatively attributable to hiring practices that increased the availability of Hispanic but not Black staff.
There may be organizational features of an agency that are not assessed in the CCAS that may be of greater relevance to Blacks than the ones now included. It might be valuable to measure the degree to which services are flexibly delivered, appointment times are available and that transportation, child care, and food are provided as needed. On the other hand, there may be no organizational level items that are able to predict disparity reduction for Blacks because the therapeutic alliance between clients and providers is most critical and dominates all other considerations (Whaley 2001
; Cruz and Pincus 2002
For the subset of the patients who had psychotic disorders, no relationship between CCAS scores and disparity reduction was found. This is consistent with the expectation that for this diagnostic group the two outcome measures, engagement and retention, are likely to be positive by nature of the illness. This would tend to substantially dominate CC effects, if any are present. These findings may be viewed as supporting the instrument’s predictive validity.
In interpreting the results of this study, several caveats must be kept in mind. Because the data are naturalistic, diagnoses are not research-based and therefore may have been recorded unsystematically. Indeed, the diagnoses may have been biased as evidenced by the rate of psychotic disorders among Blacks which was twice the rate among Whites and Hispanics. Similar biases have recently been reported (Hampton 2007
). In addition, limitations of the information in the database did not allow refined specification of cultural groups. Lumping clearly distinct cultural groups (e.g., African-American and Afro-Caribbean) into one category (Blacks) could have masked an effect or produced a misleading result because the groups may have heterogeneous service use distributions. Additionally, the outcome measures were based exclusively on patterns of service use rather than clinical or functional outcomes because these were not reported. Finally, the reported results focus on parameters in the models that were statistically different. Each result of a hypothesis test stands as a valid statistical statement, with a statement specific 5% probability of erroneously rejecting the null. However, in aggregate many tests were performed and therefore the chance that at least one of them was falsely positive is greater than 5%.
There is still much to be learned about organizational level practices that successfully promote the delivery of culturally competent mental health services. Future research might profitably focus on identifying scale components that successfully predict disparity reductions for Blacks and for specific diagnostic groups. These in turn would revise and improve the roadmap that describes the path to CC. Study of diverse organizational types, such as child and adolescent clinics, may enhance understanding of differential effects of CC across outpatient settings. Finally, longitudinal studies of culturally competent administrative processes that facilitate changes in the delivery of clinical services with direct positive effects on treatment outcomes are highly desirable. These may guide the process and promote organizational change that would reduce cultural disparities in mental health care.