PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Adv Soc Work. Author manuscript; available in PMC 2010 April 1.
Published in final edited form as:
Adv Soc Work. 2010 April 1; 10(1): 19–38.
PMCID: PMC2829886
NIHMSID: NIHMS152595

Child Welfare Workers’ Connectivity to Resources and Youth’s Receipt of Services

Abstract

Youth involved in the child welfare system are at high risk for mental illness, substance abuse, and other behavioral health issues, which child welfare workers are expected to address through referrals. Child welfare workers (N=27) who participated in Project IMPROVE (Intervention for Multisector Provider Enhancement) reported on services they provided to youth (N=307) in their caseloads. Using survey and administrative data, this paper examines workers’ service actions on behalf of youth. Results were consistent with the Gateway Provider Model and showed that youth received help from a greater variety of service sectors when their workers were able to identify behavioral health problems, and were familiar with and connected to other providers in the community. Improving service delivery to youth in child welfare may be accomplished by training workers in the signs and symptoms of behavioral health problems and familiarizing them with providers in the community.

Keywords: Child welfare workers, behavioral health, connectivity

INTRODUCTION

Child welfare workers are responsible for advancing the safety, permanency, and well-being of youth involved in the child welfare system. Consistent with federal child welfare goals, workers must also help youth obtain treatment and support services (U.S. Department of Health and Human Services, 2003) requiring them to reach across the traditional boundaries of the child welfare system to other human service systems. Because the estimated prevalence of behavioral health conditions is greater for youth in the child welfare system than in the general public assistance sector, workers’ connections with behavioral health services are particularly important (Burns, et al., 2004; Harman, Childs, & Kelleher, 2000; Leslie, Hurlburt, Landsverk, Barth & Slymen, 2004).

Workers in public child welfare agencies are well-positioned to identify behavioral health service needs among youth and connect them to appropriate care in the community, thus serving as Gateway Providers to mental health, substance abuse, and other service sectors (Stiffman, Pescosolido & Cabassa, 2004). This paper examines the predictors of child welfare workers’ service actions for youth with behavioral health problems from the context of the Gateway Provider Model (Stiffman, Pescosolido et al., 2004).

Behavioral Health Problems and Services Use

Youth who experience traumatic and adverse experiences like child abuse and neglect are at high risk for psychiatric disorders, social problems, and general functional impairment (Cerezo & Frias, 1994; Famularo, Kinscherff, & Fenton, 1992; Kaufman, 1991; Kazdin, Moser, Colbus, & Bell, 1985). Youth involved in the child welfare system have high rates of behavioral health problems with estimated prevalences ranging from 50–70% (Burns, et al. 2004; Garland,et al., 2001; Trupin, Tarico, Low, Jemelka, & McClellan, 1993) compared to 20% in the general youth population (Costello et al., 1996; Shaffer, et al., 1996). Behavior problems and conduct disorders are particularly prevalent but depression, anxiety, post-traumatic stress, and substance use disorders are also common (Garland et al., 2001; Halfon, Berkowitz & Klee, 1992; Stiffman, Chen, Elze, Doré & Cheng, 1997).

Youth in the child welfare system use behavioral health services at high rates. State Medicaid programs and child welfare agencies finance a range of behavioral health services that include inpatient, outpatient, and residential care. Studies of mental health service users show that youth in child welfare, particularly those in out-of-home placements, use these behavioral health services at a much higher rate than youth in the community (Farmer et al., 2001; Halfon et al., 1992; Harman et al., 2000).

Youth’s need is a strong predictor of their use of behavioral health services. Although research supports the relationship between need and receipt of behavioral health services, evidence suggests that other factors such as abuse type, placement type, age, and race/ethnicity also influence youth’s receipt of services (Garland, Landsverk, Hough & Ellis-MacLeod, 1996; Hurlburt et al., 2004; Leslie et al., 2000; Walrath, Ybarra, Sheehan, Holden & Burns, 2006). In addition, child welfare workers’ daily decisions and actions on behalf of the youth they serve also have the potential to impact the services youth receive.

Role of Child Welfare Workers in Youth Access to Care

Child welfare workers impact the services, resources, and placements youth receive when involved with the system (Ryan, Garnier, Zyphur, & Zhai, 2005). Specifically, youth’s entry into the behavioral health service sector has been linked to their contact with child welfare services (Leslie, Hurlburt et al., 2005; Stiffman, Chen, Elze, Doré & Cheng, 1997). However, poor assessment of treatment needs and limited access to behavioral health services are barriers in youth’s pathways between the child welfare and behavioral health systems, leaving many with unmet service needs (Leslie, Gordon, et al., 2005; Stiffman, Pescosolido et al., 2004).

Research on systems of care suggests that provider-level actions are important determinants of care regardless of administrative reforms designed to bridge systems (Bickman, 1996; Bickman, Lambert, Andrade & Penaloza, 2000; Bickman, Noser & Summerfelt, 1999). The Gateway Provider Model posits that non-mental health providers play a key role in connecting youth to services (Stiffman, Pescosolido et al., 2004). To ensure that youth receive treatment, gateway providers must be able to recognize behavioral health service needs and be familiar with and connected to the larger service network. Therefore, connectivity, or the degree to which a provider interacts with youth-serving agencies can influence services youth receive (Stiffman, Hadley-Ives et al., 2000; Stiffman et al., 2001).

This paper examines the relationship between worker knowledge and skills and behavioral health service provision to youth. We believe it is the first to explore the applicability of this framework for child welfare workers, while verifying it with state billing data. Based on evidence with providers from multiple disciplines (Stiffman, Hadley-Ives et al., 2000; Stiffman, Pescosolido et al., 2004), we posited that child welfare workers’ connectivity and recognition of behavioral health service needs would be positively associated with service provision (Figure 1).

Figure 1
Selected Elements of the Gateway Provider Model of Service Provision

METHOD

Design

Project IMPROVE (Intervention for Multisector Provider Enhancement) was an intervention study funded through the National Institute of Mental Health (NIMH). Through the intervention, the study examined child welfare worker knowledge of their clients’ needs, their referral and treatment actions on behalf of those clients, and also obtained billing records concerning diagnoses and services provided to these youth (Stiffman, Foster, Hamburg, & Doré, 2003). The analyses for this paper utilized state billing records and baseline quantitative survey data from the larger IMPROVE intervention study.

Sample

Child welfare workers (N=27) in the St. Louis City and County Children’s Division, Missouri Department of Social Services self-selected into the study, providing they had an active case load of at least 15 youth between five and 17 years of age and a job description that included mental health and substance abuse assessment and referral as regular job duties. As part of the intervention, workers were invited to the university for training and data collection. While onsite, workers completed self-administered surveys where they provided data on 307 youth whom they identified only though their public record locator code (the Departmental Client Number or DCN). Both the university’s Institutional Review Board and the Missouri Department of Social Services Privacy Review Board approved the protocol and each worker provided written consent. The review boards waived third party consent for workers’ reports on the youth because youth names were never available to the study, and all other youth data were de-identified prior to the study’s access.

Workers in the sample were predominantly female (85%) and white (66%). All workers possessed a bachelors-level degree with 15% having a masters-level degree. The average worker was employed by the Children’s Division for three years (SD=2.14), with a range from less than one year to seven years.

Measures

Measures consisted of worker reports and state administrative data.

Worker Reports

Workers reported on (1) their background and knowledge of community referral resources (connectivity); (2) youth’s mental health and environmental problems; and (3) services obtained for youth, via self-administered surveys comprised of measures used successfully in several prior studies (Horwitz et al., 2001; Stiffman et al., 2006; Stiffman, Hadley-Ives et al., 2000; Stiffman, Horwitz et al., 2000; Stiffman et al., 2001),

Worker Background

Education and Training

Workers reported the highest level of education they obtained, the total hours of in-service training or continuing education in the past year, and the types of topics covered in the training (e.g. illicit drug use, assessment, neglect).

Connectivity

Provider connectivity is the degree to which an individual worker is connected to other youth-serving agencies (Stiffman et al., 2006; Stiffman, Hadley-Ives et al., 2000; Stiffman et al., 2001). Workers reported their relationships with twenty-eight service categories grouped into four major service domains (Appendix A): six for the health and education domain (e.g., job training, tutoring, public health centers); seven for the inpatient behavioral health care domain (e.g., psychiatric or substance abuse treatment facilities); eight for the outpatient treatment domain (e.g., community-based mental health or substance abuse services); and seven for the ‘Other’ service domain (e.g., religious providers, basic needs).

Workers reported on two types of relationships: familiarity and referrals (for any youth in their caseload, not just those reported on in this study) in the past six months. The data provided three measures: 1) the number of service categories with which workers were familiar (familiarity); 2) the sum of referrals made to service categories (referrals); and 3) an aggregate score of the sum total of both familiarity and referrals (total connectivity).

Worker Reports on Youth

Workers reported on their perceptions of youth’s behavioral health and environmental problems.

Behavioral Health Problems

We measured workers’ perceptions of behavioral health problems using eight questions concerning depression, post traumatic stress, anxiety, alcohol misuse, illicit drug use, behavior problems/conduct disorder, and other (Stiffman et al., 2006; Stiffman, Hadley-Ives et al., 2000; Stiffman et al., 2001). Youth could have between zero and eight behavioral health problems. Behavioral health problems were assigned if the worker perceived the problem to be moderate, serious, or critical.

Environmental Problems

Workers reported on the presence or absence of 14 types of environmental or social problems including school and learning problems, physical health problems, basic need issues, problem peers, violence in the family, family problems, living in a violent neighborhood, exposure to violence in school, gangs, family instability, not living at home, lack of family support, legal problems, and family financial problems (Stiffman et al., 2006; Stiffman, Hadley-Ives et al., 2000; Stiffman et al., 2001).

Service Provision

We collected information about workers’ service actions on behalf of youth, and the services youth obtained.

Service Actions

The number of services that workers provided directly to youth, and the number of services referred were measured (Stiffman, Hadley-Ives et al., 2000, Stiffman et al., 2001). Workers reported whether they acted in any of four ways (took no action; recommended or referred youth to service; personally provided but did not refer; or personally provided and referred) for 11 types of services: parenting/caregiving, counseling/therapy, teaching/alternative/education-related services, substance abuse treatment, self-help group or peer counseling, crisis intervention, inpatient/residential mental health care, medication to control symptoms, psychiatric evaluation, family counseling, and service coordination and referral.

Services Obtained

Workers completed the brief module of the Services Assessment for Children and Adolescents (SACA) which collects information about inpatient, outpatient, school and other services used in the last six months, whether or not they were provided by the child welfare workers (Horwitz et al., 2001; Stiffman, Horwitz et al., 2000).

Administrative Data

Billing records from the Missouri Division of Mental Health and Medicaid were extracted for each youth reported on by the workers. These records were included because they reflect outcomes of child welfare service actions - child welfare workers refer youth to outside service providers who, in turn, bill the Division of Mental Health or Medicaid for services rendered.

We used DCN numbers to merge the records from both sources. The data included information about services billed for the youth between July 1, 2005 and March 8, 2008. Each billing record contained a diagnostic code indicating the type of problem the youth was treated for, and a procedure code indicating the type of service provided.

Diagnostic Codes

The data contained ICD-9 behavioral health diagnostic codes and their descriptions. We grouped the diagnostic codes into the same eight types of problems that workers reported in the survey (depression, post traumatic stress, anxiety, alcohol misuse, illicit drug use, behavior problems/conduct disorder, and other).

Procedure Codes

Procedure codes followed the Common Procedural Terminology (CPT) and Healthcare Common Procedure Coding System (HCPCS) conventions. We obtained descriptions of the CPT and HCPCS codes and grouped them into four main types of services: overnight, outpatient, school-based, and other. Procedure categories (like the diagnostic categories) were grouped to parallel the types of services reported through the SACA.

Analyses

Two forms of analysis were performed: 1) accuracy calculations (the percentage of all cases in which both data sources agreed that the youth had a problem or service or did not have a problem or service) to examine the relationship between provider reports on youth and the billing data; and 2) generalized estimating equations (GEE) with a negative binomial distribution to examine multivariate predictors of service provision. Our data did not satisfy the independence and normality assumptions of general linear regression. First, our data was nested (307 youth clustered by 27 workers) and the reports about youth with the same worker were related observations, and not independent. Second, the dependent variable (services obtained as reported on the SACA) was not normally distributed. As is common when studying services or other counts of rare events, the data took the shape of a negative binomial distribution (Gardner, Mulvey, & Shaw, 1995), where the majority of observations were concentrated at the low end (none or only a few types of services), and only a few observations fell along the upper range (six or more types of services). Therefore, GEE is an appropriate method to analyze the non-normal (negative binomial) distribution of the dependent variable and to account for correlations within worker subgroups (Ballinger, 2004). The model fit was determined by a Pearson chi square test where a lack of significant difference between the observed and expected values indicated a good fit. Analyses were run using SAS 9.1 and Stata SE 9.

RESULTS

We examined information about youth’s demographic characteristics, the relationships between worker reports and administrative data on youth’s behavioral health problems, environmental problems, and services received, and workers’ connectivity to community resources. Then we explored the Gateway Provider Model’s assumptions about predictors of service provision.

Youth Demographic Characteristics (based on worker reports)

The youth were nearly evenly split between males (49%) and females (51%). The age of the youth ranged from five to 20 (median of 13 years). Over two-thirds of youth in the sample were African-American (70.9%), about one-quarter were Caucasian (28.8%), 1.5% Hispanic, and 1.2% were “other.”

Relationship between Worker Reports and Administrative Data

Youth’s behavioral health and environmental problems, and service provision reported by workers and documented in the administrative data were examined separately. Then the two data sources were compared to determine the correspondence between both types of reports.

Behavioral Health Problems

We examined youth’s behavioral health problems by analyzing worker reports and the diagnostic codes that accompanied the billing records in the administrative data. Findings from each data source were then compared (Table 1).

Table 1
Youth Behavioral Health Problems Reported by Workers and Billing Data (N=307)

Worker Reports of Behavioral Health Problems

Workers reported that nearly two-thirds (64%) of youth in each of their caseloads had a moderate to severe behavioral health problem. The two most commonly reported were depression (44%) and behavior problems/conduct disorder (43%), followed by anxiety (33%), post traumatic stress (29%), illicit drug use (5%), suicidality (4%), and alcohol misuse (2%). About 10% of youth had “other” types of behavioral health problems such as developmental delays and personality disorders.

Billing Records on Behavioral Health Problems

Like the worker reports, billing records showed that nearly three quarters (73%) of youth were diagnosed with some type of behavioral health problem. Similarly, the most common diagnoses were behavior problems/conduct disorder (57%) and depression (41%), followed by anxiety (18%), post-traumatic stress (11%), illicit drug use (10%), suicidality (1%), and alcohol misuse (1%). Unlike the worker reports, a much higher percentage of youth had “other” types of behavioral health problems. Approximately 50% of youth were diagnosed with psychotic and personality disorders, developmental delays, or other disorders not otherwise specified.

Relationship between Worker Reports and Billing Records for Behavioral Health Problems

Behavioral health diagnoses in the billing records and worker reports on whether or not youth had a behavioral health problem matched 69% of the time. The highest concordance between worker reports and billing records were for alcohol misuse (97%), suicidality (96%), and illicit drug use (89%), followed by post traumatic stress (72%), behavior problems/conduct disorder (65%), anxiety (63%), depression (62%) and other (52%).

It is unlikely that two data sources will ever be in total concordance, therefore some discordance between worker reports and billing data is to be expected. These data may not agree because of differences in perception of the problem between workers and providers, as well as differing reporting, and billing policies.

Environmental Problems

First we explored worker reports of environmental problems and second we examined the diagnostic codes that accompanied the billing records in the administrative data. Findings from each data source were then compared (Table 2).

Table 2
Youth Environmental Problems Reported by Workers and Billing Data (N=307)

Worker Reports of Environmental Problems

Almost all (98%) of the youth had environmental problems, which is not surprising for public child welfare clients (for example, see Berger, 2004 or Hardin & Koblinski, 1999). Workers most frequently noted the following environmental problems among youth in their caseload: family instability (78%), family problems (78%), not living at home (67%), and school and learning (66%). These were followed by lack of family support (53%), basic need issues (52%), family financial problems (41%), problem peers (38%), violence in the family (30%), physical health problems (14%), violent neighborhoods (13%), violent schools (12%), legal problems (9%), and gangs (7%).

Billing Records for Environmental Problems

Billing records showed much lower rates of environmental problems than worker reports. These results are to be expected since billing records reflect service needs reimbursed by the state which are typically psychiatric or health-related rather than environmental. Almost 21% of youth were diagnosed with some type of environmental problem. About 10% received a diagnostic code for violence in the family, 7% with family problems, 4% with basic needs, 2% with school and learning problems and 1% with exposure to violence in school.

Relationship between Worker Reports and Billing Records on Environmental Problems

The diagnostic codes in the billing records matched worker reports of environmental problems (those that are billable) 65% of the time. Exposure to violence in schools had the highest match rate (83%) followed by violence in the family (65%), basic needs (49%), school and learning problems (35%), and family problems (27%).

Service Provision

We explored three types of service reports: (1) worker reports of personal service actions on behalf of youth, (2) worker reports of any youth services according to the SACA, and (3) procedure codes for services in the billing records. The agreement between the SACA service data and billing records was also calculated (Table 3).

Table 3
Services Provided Reported by Workers and Billing Data

Worker Reports – Service Actions

Workers reported their personal service actions on behalf of the youth. Workers provided and/or referred 84% of youth to services. For the most part, when workers identified youth’s need for service, they responded by either providing the service or through referral. On average, workers did not take action for 12% of youth’s identified service needs.

Most commonly, workers provided or referred youth for counseling/therapy services (67%) followed by service coordination and referral (57%), and education-related services (50%). Other services included family counseling (33%), medications for symptom management (26%), psychiatric evaluation (25%), crisis intervention (21%), self help or peer groups (20%), parenting/care-giving (17%), inpatient mental health care (14%), and substance abuse treatment (7%) (Figure 2).

Figure 2
Service Actions on Behalf of Youth

Worker Reports – SACA

Workers reported that youth and their families received services from an average of four different service sectors within the previous six months. The most commonly received services by youth include incidentals/clothing/transportation (68%), school-based help for problems with behavioral health issues (47%) and recreational/community activities (46%). According to workers, about 38% of youth received help from outpatient facilities such as a mental health clinic, and 23% received overnight help from a facility such as a hospital, group home, or foster home for problems with drugs, alcohol, or emotional problems.

Billing Records of Services

Similar to worker reports, administrative data showed that 88% of youth had a procedure code in the billing records for services. Unlike the SACA data, only 12% had records for incidentals/clothing/transportation. But once again, as with environmental issues, these types of services are seldom billed to Medicaid or the Division of Mental Health. Most youth (88%) had records for outpatient services, 15% had charges for overnight services.

Relationship between Worker Reports and Billing Records of Services

The presence of worker reports and procedure codes in the billing records matched 80% overall. The highest match rate was for overnight care (76%), followed by outpatient services (45%), and incidentals/clothing/transportation (38%).

Connectivity Scores

Workers’ connectivity to referral resources in the community is a key construct examined in this study. Total connectivity scores (familiarity and number of referrals) ranged from 37 to 272. The average was 83 with a median of 64, indicating that the distribution was positively skewed by several extreme outliers. There was no statistically significant relationship between worker education, number of training hours, or type of training and connectivity. Each component of the score is described below and summarized in Table 4.

Table 4
Familiarity, Referrals and Connectivity by Domain

Familiarity

Workers are familiar with at least some organizations in most service domains. All of the workers were familiar with at least one resource in each of the four domains and almost half (47%) were familiar with all 28 service categories. On average, workers were familiar with 26 of the 28, or about 93% of the service categories.

Referrals

Workers made referrals across all four major domains of services in the six months prior to the intervention. The number of total referrals ranged from 15 to 245. The average number of referrals was 57 and the median was 37, indicating large variations in referral patterns. Twenty six percent of referrals were made to outpatient services, 25% to inpatient services, and 19% to health and education services, with the greatest (29%) to “Other” service domains.

Model Exploration

Above, we examined the separate constructs of the Gateway Provider Model which posits that providers’ connectivity to the larger service network and ability to identify behavioral health service needs influences youth’s receipt of behavioral health services. Here we examine the multivariate relationships between those variables and service provision.

Multivariate Predictors of Service Provision

The Gateway Provider Model predicts that provider assessment of behavioral health problems and connectivity determines provision of care. The multivariate model is based only on worker reports of behavioral health problems, connectivity and services; billing records are not included in this analysis because they reflect the billable actions of multiple service providers, not the individual child welfare actions that are the focus of this study. As hypothesized, when all key variables were entered into the equation, workers’ assessment of mental health and workers’ connectivity together contributed to the prediction of SACA reports of the number of sectors of care that served youth. However, assessment of environmental problems was not significantly related to service receipt.

In our final model, where we retained only the significant predictors, workers’ assessment of mental health problems (b=.2923, SE=.0664, p<.0001) and workers’ connectivity (b=.0026, SE=.0009, p=.0048) predicted 37% of the variance (marginal R2) in service receipt. A goodness of fit test demonstrated an adequate model fit, χ2(253)=260.12, p=.37.

We ran a similar analysis to examine whether workers’ assessment of behavioral problems and connectivity predict the number of sectors of care that served youth in the billing records. None of the independent variables significantly predicted the number of sectors of care reported in the billing records.

DISCUSSION

This paper addressed the influence of child welfare workers’ ability to recognize mental health problems and the effect their knowledge of community referral resources had on behavioral health service provision to youth. Youth involved in the Missouri Children’s Division received services from multiple sectors of care, reinforcing the importance of child welfare workers’ role as gateways to care for youth. As expected, child welfare workers’ connectivity and their identification of mental health problems were associated with youth’s receipt of services and explained 37% of the variance in youth’s service receipt. These two worker-level characteristics had more explanatory power than Andersen’s model which is usually found to explain about 20% of the variance in service use (Mechanic, 1979; Phillips, Morrison, Andersen & Aday, 1998).

Youth receive help from a greater variety of service sectors when their gateway providers are able to identify mental health problems and are familiar with and connected to community resources. For example, one way of thinking about these relationships is that if a youth with two types of mental health problems is working with a child welfare worker with a connectivity score of 131, she or he will likely receive care from an average of six different service sectors. However, if the same youth is working with a child welfare worker with a connectivity score of 74, she or he will likely receive care from about five different service sectors (assuming that both child welfare workers similarly identify youth’s mental health problems).

These are important findings because youth in the child welfare system have multiple behavioral health and environmental problems requiring services from multiple service sectors. While more service referrals may not benefit an individual client, the more service sectors involved in their care increases the likelihood that a greater number of service needs are addressed, which is the ultimate goal of the call for inter-agency coordination and reduced fragmentation (Stroul & Friedman, 1986).

Identification of Youth’s Problems

Our data show that workers’ ability to identify mental health problems among youth in their caseload is critical for service delivery. Workers reported that most of the youth had moderate to severe mental health problems. Behavior problems and mood disorders were most common among youth while substance misuse and suicidality were less frequent. Workers identified environmental problems for nearly all of the youth. The majority of these problems were related to families and school. Although workers’ identification of environmental problems was not significantly related to service provision in multivariate analyses, the descriptive information about the types of mental health and environmental problems illustrates the multitude of complex problems that child welfare workers address on behalf of youth.

Worker Response to Youth’s Problems

Workers responded to mental health problems with a variety of services. The service action data suggests that workers address youth’s mental health problems by recommending counseling and therapy, service coordination, and education-related services. Their connections with community agencies that provide these services reveal additional insight about referral practices that guide youth in the child welfare system to the behavioral health service system.

Connectivity

Data show that workers are familiar with nearly all of the service domains and make frequent linkages with other organizations. Workers often connected with general social service agencies in the community to meet youth’s basic needs. However, workers also refer frequently to outpatient and inpatient services specifically designed to address behavioral health needs.

The wide range of reported referrals to and familiarity with youth-serving agencies suggest that workers vary in the way they are connected with the full inter-organizational network of agencies in the region. Neither education, experience, nor training accounted for the variations in connectivity, suggesting that other factors determine workers’ knowledge and connections to referral resources. Understanding these variations in connectivity is important because workers’ connections to resources facilitated youth’s receipt of services. Our results provide an opportunity to advance a practice-oriented research agenda focusing on individual worker characteristics, practices, and skills that help bridge multiple service delivery systems.

Limitations

The primary findings of this study were based on workers’ reports on youth so we were concerned that recall biases may influence the data. To evaluate the potential impact of such a bias, we compared worker reports with state billing records. If there was little concordance between billing records and worker reports, our findings in support of the model might have been spurious. However, the billing records support the general reliability of worker assessments and reports of service provision. Both worker reports and billing data were consistent in reports of mental health problems, particularly conduct disorder/behavior problems and depression. The high correspondence between worker reports of service receipt and billing records bolstered our confidence in the reliability of the SACA data.

The discrepancy between worker reports and the billing records on youth’s environmental problems may be attributed to an underestimation of environmental problems in the billing records in our study because CPT or HCPCS codes do not exist for all of the types of environmental problems measured in the workers’ survey instrument, and the state may not reimburse providers for services related to all of types of environmental problems. The discrepancy does raise questions regarding youth’s access to social services, and how organizations finance the provision of services that address environmental problems among youth in the child welfare system. Our worker reported data indicate that youth obtained services (e.g. transportation and incidentals) that could address some environmental problems (e.g. basic needs and family financial problems). However, we cannot draw any conclusions about whether and how youth-serving agencies in the community address other environmental problems like family violence or instability. This gap suggests that child welfare workers need to be familiar with youth-serving agencies that are able to provide non-billable services in addition to those with formal contracts or the ability to bill public systems for services.

Neither worker self-reports nor billing records are gold standard measures for behavioral health or environmental problem diagnoses and there is no perfect measure for capturing service receipt. However, we can present our data and results here with greater confidence because the billing data and worker reports of youth’s problems and services are fairly consistent.

The study focused on practices within public child welfare offices in St Louis, MO and findings may not be generalizable across other locales. Other factors not included in this study may influence assessment, referral practices and service provision for youth. Studies conducted with the St. Louis Children’s Division (Fedoravicius, McMillen, Rowe, Kagotho & Ware, 2008; Foster & Stiffman, 2009; McMillen, Fedoravicius, Rowe, Zima & Ware, 2007) show that mental health services for youth in the child welfare system are frequently court ordered. The pressure to comply with court demands for documentation and time constraints influence workers’ assessment and referral practices (Fedoravicius et al., 2008; Foster & Stiffman, 2009; Smith & Donovan, 2003).

The small sample size did not allow us to run a structural equation model to test the Gateway Provider Model, the preferred method to examine such relationships. Social desirability and recall biases may have influenced the accuracy of the connectivity data reported. Workers may have felt pressure to appear well-connected or may have had difficulty remembering the number of referrals they made. Finally, the continuity of services was not addressed in the data. We cannot determine whether youth received continuous services from a single provider, or sporadically by many providers. Despite these limitations, the study findings provide support and direction for the growing body of literature on the role of child welfare workers’ knowledge and practices on youth entry into the behavioral health service system.

Future Studies

Given the variation in workers’ connectivity, the next step in this line of research is to examine how workers develop their knowledge of referral resources. If training, experience and education were not associated with connectivity, other factors that were not included in these analyses must be important. Focus groups revealed that interactions at the office shaped workers’ familiarity with other providers and their referral patterns (Foster & Stiffman, 2009). Constructive organizational culture and positive perceptions of organizational climate influence work-place interactions and may have a role in the transfer of practice knowledge (Glisson & Green, 2006; Uzzi & Lancaster, 2003). The Gateway Provider Model also posits that organizational culture and climate are related to connectivity. Exploring and testing these relationships could inform training in child welfare agencies and have implications for cross-sector service delivery to youth served by the system.

CONCLUSION

This study highlights how child welfare workers act as gateway providers and influence youth’s receipt of behavioral health services. The identification of behavioral health problems and connections to service providers who can meet youth’s needs are critical skills and knowledge for child welfare workers. These skills might also be important for social workers and other human service professionals who are responsible for working across system boundaries.

Child welfare workers’ ability to identify behavioral health problems and their knowledge of service providers in the community make a difference. Given the high prevalence of behavioral health problems among youth involved in the child welfare system, it is important to identify and provide clear pathways to needed treatment for youth. By improving providers’ connections with agencies in their community and their ability to identify behavioral health problems, child welfare systems may enhance youth’s service access and receipt and reduce unmet needs for behavioral health services.

Acknowledgments

This research was supported by NIMH grants R34 MH072871, P30 MH068759, T32 MH19960 and K02 MH001797, and NIDA grant R24 DA013572. Special thanks to Jennifer Bellamy for comments on earlier drafts of this paper.

APPENDIX A: Service Domains and Categories

DomainService Category
Health and EducationAny Public Health Clinic with Mental Health Services
Pregnancy related services or similar resources
A school social worker, guidance counselor or school psychologist
Special schools
Job training resource
Educational resources (e.g., tutoring)
Inpatient ResourcesPsychiatric hospital or psychiatric or medical units in a general hospital for emotional or behavioral problems
Drug or Alcohol treatment units
Residential treatment centers
Group or foster homes
Detention center/prison or jails
Emergency shelters for emotional or behavioral problems
Other places like summer treatment programs or boarding schools
Outpatient ResourcesCommunity mental health center or outpatient mental health clinics
Professionals (e.g., psychologist, psychiatrist, social worker or marriage or family counselor) not mentioned
Day treatment programs
Intensive in-home services
Emergency room that treats emotional/behavioral problems
Pediatrician/family doctors for emotional or behavioral problems
Probation or juvenile corrections services
Self-help groups (e.g., AA, 12-step programs, or peer youth counseling program)
OtherCrisis intervention services (e.g., suicide hotlines)
Social services (basic needs)
Religious providers of services (e.g., churches, ministers)
Life skills programs for children/youth
Family or parenting programs
Victim’s programs (domestic violence programs; crime victims’ assistance)
Any other resources which service the drug/alcohol/mental health needs of children/youth

References

  • Ballinger GA. Using generalized estimating equations for longitudinal data analysis. Organizational Research Methods. 2004;7(2):127–150.
  • Bickman L. A continuum of care: More is not always better. American Psychologist. 1996;51(7):689–701. [PubMed]
  • Bickman L, Lambert W, Andrade AR, Penaloza RV. The Fort Bragg continuum of care for children and adolescents: Mental health outcomes over 5 years. Journal of Consulting and Clinical Psychology. 2000;68(4):710–716. [PubMed]
  • Bickman L, Noser K, Summerfelt WT. Long-term effects of a system of care on children and adolescents. Journal of Behavioral Health Services and Research. 1999;26(2):185–202. [PubMed]
  • Burns BJ, Phillips S, Wagner HR, Barth RP, Kolko DJ, Campbell Y, et al. Mental health need and access to mental health services by youths involved with child welfare: A national survey. Journal of the American Academy of Child and Adolescent Psychiatry. 2004;43(8):960–970. [PubMed]
  • Cerezo M, Frias D. Emotional and cognitive adjustment in abused children. Child Abuse and Neglect. 1994;18(11):923–932. [PubMed]
  • Costello EJ, Angold A, Burns BJ, Stangl DK, Tweed DL, Erkanli A, et al. The Great Smoky Mountains Study of Youth. Goals, design, methods, and the prevalence of DSM-III-R disorders. Archives of General Psychiatry. 1996;53(12):1129–1136. [PubMed]
  • Famularo R, Kinscherff R, Fenton T. Psychiatric diagnoses of maltreated children: preliminary findings. Journal of the American Academy of Child and Adolescent Psychiatry. 1992;31(5):863–867. [PubMed]
  • Farmer EMZ, Burns BJ, Chapman MV, Phillips SD, Angold A, Costello EJ. Use of mental health services by youth in contact with social services. Social Service Review. 2001;75(4):605–624.
  • Fedoravicius N, McMillen JC, Rowe J, Kagotho N, Ware NC. Funneling child welfare consumers into and through the mental health system: Assessment, referral, and quality issues. Social Service Review. 2008;82(2):273–290. [PMC free article] [PubMed]
  • Foster KA, Stiffman AR. Child welfare workers’ adoption of decision support technology. The Journal of Technology in Human Services. 2009;27(2):1–21. [PMC free article] [PubMed]
  • Gardner W, Mulvey EP, Shaw EC. Regression analyses of counts and rates: Poisson, overdispersed Poisson, and negative binomial models. Psychological Bulletin. 1995;118(3):392–404. [PubMed]
  • Garland AF, Hough RL, McCabe KM, Yeh M, Wood PA, Aarons GA. Prevalence of psychiatric disorders in youths across five sectors of care. Journal of the American Academy of Child and Adolescent Psychiatry. 2001;40(4):409–418. [PubMed]
  • Garland AF, Landsverk JL, Hough RL, Ellis-MacLeod E. Type of maltreatment as a predictor of mental health service use for children in foster care. Child Abuse and Neglect. 1996;20(8):675–688. [PubMed]
  • Glisson C, Green P. The effects of organizational culture and climate on the access to mental healthcare in child welfare and juvenile justice systems. Administration and Policy in Mental Health and Mental Health Services Research. 2006;33(4):433–448. [PubMed]
  • Halfon N, Berkowitz G, Klee L. Mental health service utilization by children in foster care in California. Pediatrics. 1992;89(9):1238–1244. [PubMed]
  • Harman JS, Childs GE, Kelleher KJ. Mental health care utilization and expenditures by children in foster care. Archives of Pediatrics and Adolescent Medicine. 2000;154(11):1114–1117. [PubMed]
  • Horwitz SM, Hoagwood K, Stiffman AR, Summerfeld T, Weisz JR, Costello EJ, et al. Reliability of the Services Assessment for Children and Adolescents. Psychiatric Services. 2001;52(8):1088–1094. [PubMed]
  • Hurlburt MS, Leslie LK, Landsverk J, Barth RP, Burns BJ, Gibbons RD, et al. Contextual predictors of mental health service use among children open to child welfare. Archives of General Psychiatry. 2004;61(12):1217–1224. [PMC free article] [PubMed]
  • Kaufman J. Depressive disorders in maltreated children. Journal of the American Academy of Child and Adolescent Psychiatry. 1991;30(2):257–265. [PubMed]
  • Kazdin AE, Moser J, Colbus D, Bell R. Depressive symptoms among physically abused and psychiatrically disturbed children. Journal of Abnormal Psychology. 1985;94(3):298–307. [PubMed]
  • Leslie LK, Gordon JN, Lambros K, Premji K, Peoples J, Gist K. Addressing the developmental and mental health needs of young children in foster care. Journal of Developmental and Behavioral Pediatrics. 2005;26(2):140–152. [PMC free article] [PubMed]
  • Leslie LK, Hurlburt MS, James S, Landsverk J, Slymen DJ, Zhang J. Relationship between entry into child welfare and mental health service use. Psychiatric Services. 2005;56(8):981–987. [PMC free article] [PubMed]
  • Leslie LK, Hurlburt MS, Landsverk J, Barth R, Slymen DJ. Outpatient mental health services for children in foster care: a national perspective. Child Abuse and Neglect. 2004;28(6):699–714. [PubMed]
  • Leslie LK, Landsverk JL, Ezzet-Lofstrom R, Tschann J, Slymen DJ, Garland AF. Children in foster care: factors influencing outpatient mental health service use. Child Abuse and Neglect. 2000;24(4):465–476. [PubMed]
  • McMillen JC, Fedoravicius N, Rowe J, Zima BT, Ware N. A crisis of credibility: Professionals’ concerns about the psychiatric care provided to clients of the child welfare system. Administration and Policy in Mental Health and Mental Health Services Research. 2007;34(3):203–212. [PMC free article] [PubMed]
  • Mechanic D. Correlates of physician utilization: Why do major multivariate studies of physician utilization find trivial psychosocial and organizational effects? Journal of Health and Social Behavior. 1979;20(4):387–396. [PubMed]
  • Phillips KA, Morrison KR, Andersen R, Aday LA. Understanding the context of healthcare utilization: assessing environmental and provider-related variables in the behavioral model of utilization. Health Services Research. 1998;33(3):571–596. [PMC free article] [PubMed]
  • Ryan JP, Garnier P, Zyphur M, Zhai F. Investigating the effects of caseworker characteristics in child welfare. Children and Youth Services Review. 2005;28(9):993–1006.
  • Shaffer D, Fisher P, Dulcan MK, Daview M, Piacentini J, Schwab-Stone ME, et al. The NIMH Diagnostic Interview Schedule for Children Version 2.3 (DISC-2.3): Description, acceptability, prevalence rates, and performance in the MECA study. Journal of the American Academy of Child and Adolescent Psychiatry. 1996;35(7):865–877. [PubMed]
  • Smith BD, Donovan SEF. Child welfare practice in organizational and institutional context. Social Service Review. 2003;77:541–563.
  • Stiffman AR, Chen Y-W, Elze D, Doré P, Cheng LC. Adolescents' and providers' perspectives on the need for and use of mental health services. Journal of Adolescent Health. 1997;21(5):335–342. [PubMed]
  • Stiffman AR, Foster KA, Hamburg K, Doré P. IMPROVE: A software program to improve assessments and multisector referrals. Paper presented at the Research and Training Center for Children's Mental Health; Tampa, FL: 2003.
  • Stiffman AR, Freedenthal S, Doré P, Ostmann E, Osborne V, Silmere H. The role of providers in mental health services offered to American-Indian youths. Psychiatric Services. 2006;57(8):1185–1191. [PubMed]
  • Stiffman AR, Hadley-Ives E, Doré P, Polgar M, Horvath VE, Striley C, et al. Youths' access to mental health services: The role of providers' training, resource connectivity, and assessment of need. Mental Health Services Research. 2000;2(3):141–154. [PubMed]
  • Stiffman AR, Horwitz SM, Hoagwood K, Compton WI, Cottler L, Bean DL, et al. The Service Assessment for Children and Adolescents (SACA): Adult and child reports. Journal of the American Academy of Child and Adolescent Psychiatry. 2000;39(8):1032–1039. [PubMed]
  • Stiffman AR, Pescosolido B, Cabassa LJ. Building a model to understand youth service access: The Gateway Provider Model. Mental Health Services Research. 2004;6(4):189–198. [PMC free article] [PubMed]
  • Stiffman AR, Striley C, Horvath VE, Hadley-Ives E, Polgar M, Elze D, et al. Organizational context and provider perception as determinants of mental health service use. The Journal of Behavioral Health Services and Research. 2001;28(2):188–204. [PubMed]
  • Stroul BA, Friedman RM. A system of care for severely emotionally disturbed children and youth. Washington, DC: Georgetown University Child Development Center, National Technical Assistance Center for Children’s Mental Health; 1986. [PubMed]
  • Trupin E, Tarico V, Low B, Jemelka R, McClellan J. Children on child protective service caseloads: Prevalence and nature of serious emotional disturbance. Child Abuse and Neglect. 1993;17(3):345–355. [PubMed]
  • US Department of Health and Human Services, Administration on Children, Youth and Families. Safety, permanency and well-being: Child welfare outcomes 2000: Annual report to Congress. 2003. Retrieved June 24, 2008 from http://www.acf.hhs.gov/programs/cb/pubs/cwo03/cwo03.pdf.
  • Uzzi B, Lancaster R. Relational embeddedness and learning: The case of bank loan managers and their clients. Management Science. 2003;49(4):383–400.
  • Walrath CM, Ybarra ML, Sheehan AK, Holden EW, Burns BJ. Impact of maltreatment on children served in community mental health programs. Journal of Emotional and Behavioral Disorders. 2006;14(3):143–156.