PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptNIH Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Child Abuse Negl. Author manuscript; available in PMC Jun 1, 2012.
Published in final edited form as:
PMCID: PMC3123534
NIHMSID: NIHMS300478

Changes in parental depression symptoms during family preservation services

Abstract

Objectives

Parental depression symptoms often change over the course of child welfare family preservation and parenting services. This raises the question of whether certain processes in family preservation services might be associated with depression symptom change. This study tests three correlational models of change among family preservation service participants: a) changes in depression symptoms are one facet of broad general changes in wellbeing; b) the quality of the home visitor-client relationship is associated depression symptom changes; and c) linking parents to adjunctive services is associated with symptom changes.

Methods

Participants were 2,175 parents in family preservation services, largely for child neglect, who were surveyed using standard measures at pre-treatment, post-treatment and 6 month follow-up. Change patterns were evaluated using growth models, including bivariate parallel and multivariate second-order models.

Results

Parallel growth was noted among depression symptoms and changes in social, economic, familial, and parenting domains. A second order change model positing a global change pattern fit the data well. Working alliance had a modest association with improvement, but successful linkage to outside mental health services was not associated with improvement.

Conclusions

Changes in diverse indicators of wellbeing follow a global pattern which might support use of less complex rather than more fully comprehensive service plans. Findings about lack of adjunctive usual care mental health service benefit may be related to uncontrolled factors and this is a topic in need of additional study.

Introduction

One of the interesting observations from studies of parenting programs is that parental depression symptoms tend to change over the course of services even though these types of services do not include activities designed to function as depression treatment (Ammerman et al., 2009; Bagner & Eyberg, 2003; Chaffin et al., 2004; Ho, 2005; Sanders & McFarland, 2000). Family preservation and parenting programs are among the most common services provided to parents in child welfare (NSCAW Research Group, 2005). Providers in family preservation programs are home visitors, not mental health therapists. These services are not psychotherapy, but are focused on reducing family conflicts, improving parenting problems, increasing social support, helping families meet basic concrete and child care related needs, and brokering outside services. There is little research examining which if any of these service functions may be correlated with changes in depression symptoms.

Depression symptoms among parents in child welfare

Parents in the child welfare system commonly report depression symptoms, most often at mild-moderate levels, but sometimes severe. This has been observed across both physical abuse and neglect cases (Banyard, Williams, & Sigel, 2003; Berger, 2005). In a representative population sample, depression was identified as one of the stronger prospective risk factors for the initial onset of child maltreatment. Compared with non-depressed parents, depressed parents were around three times more likely to begin maltreating their children (Chaffin, Kelleher, & Hollenberg, 1996). A history of prior mental health problems, including depression, is also associated with child welfare recidivism (Drake, Jonson-Reid, & Sapokaite, 2006).

Possible correlates of depression symptom change

The mechanism(s) by which parenting services might impact depression symptoms are unclear and perhaps conjectural. The purpose of this study is to explore correlational patterns that might be predicted by three possibilities—one involving global life circumstance change, one involving characteristics of the home visitor-client relationship, and one involving the adjunctive service linkage function of these programs. The hypotheses are not intended to be exhaustive, to describe any model of depression treatment, nor to encompass any sufficient theory about depression, but do reflect three plausible patterns that might be potentially explain change during these types of services. Background for each is discussed in the sections that follow.

Hypothesis 1: Global change hypothesis

The first hypothesis is that changes in depression could be one facet of broad or global changes—a tide that lifts or lowers all boats. Under this hypothesis, changes seen among parenting or family preservation service targets (basic needs, social support, family conflict, parenting) and changes in depression symptoms would exhibit parallel growth trajectories. This hypothesis also would predict that a single global change dimension (i.e., a second-order latent slope) would provide a strong fit with the observed data in all of these areas, including depression symptoms. The areas apart from depression symptoms that will be examined in this study include the following, each of which will be discussed in terms of why it might show a parallel growth pattern with depression symptoms. The areas that will be examined are drawn from social, familial, parental, and economic domains, and none represents another internalizing mental health condition or construct with content that significantly overlaps depression symptoms (e.g., measures of distress or anxiety), although each has some precedent in the literature for association with depression symptoms.

Concrete family resources

Perhaps the dominant demographic characteristic of families in child welfare is poverty (Drake & Zuravin, 1998). Most families in child welfare live in poverty, sometimes extreme poverty. Poverty and low socioeconomic status also exhibit a stable relationship to depression over time in prospective epidemiologic data (Murphy et al., 1991). Difficulty meeting basic family and personal needs has been found to predict levels of depression, and mediates the relationship between lower income cultural group membership and depression symptoms (Plant & Sachs-Ericsson, 2004). Family preservation services attempt to improve sufficiency of basic needs. Assistance can take a number of forms, ranging from providing direct financial support, to linking families with basic needs programs, to helping families locate employment. It is possible that changes in how well basic concrete needs are being met parallels changes in depression symptoms.

Social support

Low social support is common among parents in child welfare, and is a factor in maltreatment recurrence risk (DePanfilis & Zuravin, 1999). Social support also is related to depression, particularly the lack of a confiding relationship (Leavy, 1983). In some home based family preservation models, developing social supports is a defining service philosophy (e.g., the Family Connections Model, see Depanfilis & Dubowitz, 2005). Family preservation providers often attempt to link parents to social networks, encourage parents to more actively engage these networks, and work to increase social capital, suggesting that parallel change might be observed between these two service targets.

Conflict

Parents in child welfare may be engaged in high and chronic interpersonal conflict. This can include conflict with spouses, intimate partners, other family members, and persons outside the family. A substantial majority of all child welfare families report significant conflict in relationships, most often bilateral verbal aggression or violence (English et al., 2009). High levels of conflict and problems with others have been found to predict parental depression (Horowitz, Briggs-Gowan, Storfer-Isser, & Carter, 2007). One of the common goals of family preservation and support services is to mediate or reduce conflict using problem solving approaches. It is possible that changes in parental depression symptoms over the course of services parallels changes in the overall level of interpersonal conflict, both among family members and with others.

Parenting

Depressed parents report a higher levels of parenting stress and problems with their children. To some extent, this can reflect a depressive perceptual bias toward children, but the relationship appears more complex than this and may involve bidirectional pathways (Friedlander, Weiss & Traylor, 1986) including pathways from child behavior problems to parental depression (Pelham et al., 1997). Parenting programs focused on improving child management skills often report concomitant reductions in both parenting stress and parental depression symptoms along with improvements in child behavior (e.g., Bagner & Eyberg, 2003; Sanders & McFarland, 2000). Parenting is perhaps the single most common element in child welfare service programs (NSCAW Research Group, 2005), and changes parenting problems may parallel changes in depression.

Hypothesis 2: Provider-client relationship

The second general hypothesis is that changes in depression symptoms are correlated with aspects of the parent-home visitor relationship. This includes the extent to which a strong working alliance is established, and the extent to which the home visitor is perceived as understanding the client and his or her cultural background. This is distinct from the general social support domain described above because it is a service process factor (a quality of the service itself) rather than an intended service target. Working alliance reflects a collaborative goal-oriented affiliation between the home-visitor and the parent. It includes agreement on goals, the steps needed to reach them, and a general feeling of liking and trust. Working alliance is a non-specific service characteristic that has been found to predict improvement across a wide variety of outcomes and service types. Although working alliance should not be construed as a treatment approach per se, it has been discussed as one aspect of so-called common factors in therapy, including effective services for depression symptoms (Barber et al., 1996; Castonguay et al., 1996).

A strongly related aspect of the provider-client relationship is cultural competency. Cultural competency has been identified by federal government and professional organizations as a positive provider-client relational quality across mental health and social services (Sue, 2006). Cultural competency can exist at multiple service system levels, and have multiple facets. In this study we focus on the provider-client relationship dimension of cultural competency, including the client’s perception of the provider’s sensitivity to the clients values, respect for the family’s beliefs and values, and how well the provider communicates in a way that is understandable to the client (Sue, 2006; Hernandez et al., 2009).

Hypothesis 3: Outside service linkage

The third hypothesis is that changes in depression symptoms are correlated with the adjunctive services to which home visitors may link parents. Family preservation home visitors are encouraged to identify significant mental health and other problems, know what adjunctive usual care services are available in their community, match clients to these services, and promote service engagement. It is possible that this linkage function is strongly associated with variations in depression symptom change, particularly among parents with significant levels of depression symptoms who are linked to mental health care. In this hypothesis, our intent is not to rigorously test any defined treatment, but simply to test the hypothesis that adjunctive usual care service linkage, broadly defined, will be correlated with change, especially among those with initially higher symptoms.

Method

Participants

Participants in the study were 2,175 parents who were enrolled in a statewide network of home-based contracted family preservation and support programs operated by large non-profit community based agencies, 1 for each of the 6 child welfare administrative regions of the state. All participants were parents or caregivers referred by child welfare to the programs due to reports that they committed physical abuse and/or neglect of children in their household. Parents receiving services due to child sexual abuse were excluded from the study because these cases were felt to present distinct services issues and needs. Parents were recruited for the research in their homes by a research assistant shortly after service enrollment and were provided with a $25 gift certificate at each data collection wave as compensation. Recruitment and informed consent procedures were approved by the University IRB, and participant welfare was overseen by a Data and Safety Monitoring Board that included representatives from child welfare, the provider agencies, independent health professionals and researchers, and an independent expert on research with culturally diverse samples. Study participation involved data collection only and did not alter the services received by the family or the child welfare case disposition. Only 1 parent per household was enrolled, with first priority given to the parent identified as the primary caregiver. Three thousand one hundred sixteen prospective participants were approached, 18 did not complete the recruitment process, 23 were determined to be ineligible, and 816 declined to enroll or complete baseline data collection, yielding an overall enrollment of 2,259 (72% of all individuals approached). Eighty-four participants were withdrawn after enrollment (50 voluntarily and 34 involuntarily), yielding the analyzable sample of 2,175. Individual data on non-enrollees was unavailable. However, information on over 5,000 participants in these same programs from adjoining time periods was available, and their demographic characteristics were comparable to the study sample (88% female; 67% non-Hispanic White, 13% American Indian, 12% African American, 5% Hispanic, 2% Other; median age = 30).

In the study sample, 91% of the 2,175 participants were female with a mean age of 29 years (s.d. = 8; range = 18–75). Participants had a median of 3 children in their family, 76% had at least 1 preschool age child, and 8% of women reported being pregnant at baseline. Twenty-seven percent lived in an urban setting, 63% lived in small communities, and 10% lived rurally. Residential instability was common, with 52% having lived in their current community less than 3 years, and 54% having moved more than twice in the last 5 years. Sixty-seven percent of participants were non-Hispanic Whites, 16% were Native American, 9% were African-American, 5% were Hispanic, 0.4% were Asian, and 2.3% indicated another race/ethnicity or did not answer. Thirty-one percent were married, 15% were cohabitating, 14% were separated, 16% were divorced, 2% were widowed, and 23% were never married. Forty percent had less than a high school education, 39% had a high school diploma or equivalent, 16% completed some college and 4% had completed college. Median household income was $930/month. Applying current US federal poverty line criteria for income and family size, 83% of households fell below the federal poverty line at baseline. Twenty-seven percent indicated that they were currently unemployed, 26% were homemakers, 29% had a full-time job, 6% were students and the remainder indicated part-time or self-employment. The mean baseline score on the Beck Depression Inventory was 13 (s.d. = 12, range = 0–61), and applying a cutoff score of 19, 25% would be considered to have a clinically significant symptom level. The mean baseline score on the Child Abuse Prevention Inventory was 162 (s.d. = 106, range = 1–442), which is approximately equal to the Child Abuse Prevention Inventory signal detection cut-off score for detecting physical abusers, and is well above normative mean of 91 (Milner, 1986).

Participants had a mean of 3 and a median of 2 unduplicated prior child welfare referrals (range of 0–30; s.d. =2.7). Eighty-seven percent of all prior referrals were for child neglect. As child welfare chronicity increased, the chances of having at least 1 prior neglect report became a virtual certainty—96.5% of all cases with 2 prior referrals had at least 1 neglect report, as did 99.9% of cases with 3 or more past referrals. Thirty percent had at least 1 child placed outside their home at baseline, and for these cases services were normally part of a reunification plan.

Procedures

Data for the study were collected during 2003–2006 in participants’ homes by independent research assistants using Audio Computer Assisted Self-Interview (ACASI). An initial set of ACASI practice items was included at the beginning of the interview for the research assistant to demonstrate the system and establish that parents understood the system and the test items. Parents had the option to complete the interview with or without audio. If chosen, the audio option read each question and each response option as it was highlighted on screen. Parents gave responses by touch screen. Computer interviews were conducted while the research assistant waited or supervised the children in order to provide the parent with uninterrupted private time to respond to items. Only research assistants and not home visitors were involved in data collection. Research assistants normally did not view parents’ responses unless the parent requested assistance. A federal Certificate of Confidentiality was obtained, and no individual research data was shared with child welfare authorities or service providers. Measures were collected at baseline (i.e., around service entry), around the end of the services (median time = 205 days from baseline; s.d. = 92; n = 1279), and again at around 6 months after service exit for post-program follow-up (median = 405 days from baseline; s.d. = 87; n = 892). The dominant reason for inter-wave attrition was participants who could no longer be located despite multiple attempts to follow-up using both official and unofficial contact sources (e.g., participants who moved and left no forwarding address, who had become incarcerated, or for whom no current location could be obtained from the contact persons they identified, the home visiting service agency, or child welfare).

Measures

Beck Depression Inventory-2 (BDI-2)

The BDI-2 (Beck, Steer, & Brown, 1996) is a 21-item multiple-choice self-report questionnaire designed to measure symptoms of depression. Published internal consistency of the scale is .93, and test-retest stability is .93 (Beck, Steer, & Brown, 1996). The instrument has been found to discriminate between patients diagnosed with mood disorders and other patients or non-patients, and to correlate highly with other measures of depression (Beck et al., 1996; Steer, Ball, Ranieri, & Beck, 1997). Observed alpha in the study sample was .94. It is important to note that the BDI is a measure of depression symptoms, and does not diagnose a disorder.

Family Resources Scale-Revised

The FRS (Dunst & Leet, 1987) is a 40-item self-report scale designed to measure the adequacy of basic concrete needs in households with children. The items are ordered in terms of a hierarchy of basic needs drawn from an ecological perspective and including very basic needs (e.g., having enough food, having shelter and clothes); social needs (e.g., having enough time with family, having time for friends); needs involving transportation, medical and dental care; and finally less critical needs such has having sufficient resources for extras, entertainment, savings, and so on. Summed scores were used. The overall score reflected the average degree to which the full range of family needs was met. Observed alpha for the overall scale in the study sample was .85.

Social Provisions Scale (SPS)

The SPS is a measure of perceived social support (Cutrona & Russell, 1987). Published internal consistency estimates range from .83 to .94 (Mancini & Blieszner, 1992). Items are drawn from 6 aspects of social support (friends, family, etc.). The SPS has predicted postpartum depression among women when administered during the perinatal period (Cutrona, 1984), and has been used in studies with parents (Barnett, Kidwell, & Leung, 1998). Observed alpha in the study sample was .84.

Child Abuse Potential Inventory (CAPI)—Problems with Self and Child, Problems with Others, and Problems with Family Subscales

The CAPI (Milner, 1986) is a widely used 160-item agree/disagree format parent self-report questionnaire developed to estimate risk for child physical abuse. The CAPI has 8 subscales, 3 of which reflect constructs of interest in this study---Problems with Others (PO), Problems with Family (PF), and Problems with Self and Child (PC). Initial checks of scale attributes suggested poor loadings for some items on these 3 scales, particularly for the few parental health items included on the standard Problems with Self and Child subscale. An exploratory factor analysis for binominal response data was performed for the pool of raw items comprising these scales plus additional CAPI items with nearly identical face content. The EFA supported omitting the items previously identified as loading poorly, suggested a correlated three factor solution supporting the original PO, PF, and PC subscales, and also supported the inclusion of the additional items with very similar face content. A confirmatory factor analysis for binomial item response data was performed. Categorical CFA Omega analogue values for the modified subscales were .85, .92, and .92 for the Problems with Child, Problems with Others, and Problems with Family subscales respectively. In order to test the performance of simple additive subscale scores, factor scores from the CFA were outputted, then tested for linear and nonlinear regression fits with simple additive scores (mean value of the raw items multiplied by the number of items). Linear regression model fits ranged from r-square values of .88 to .94, so the simple scoring approach was accepted because of its better replicability and functional equivalence to factor scores. The CAP also includes an 18-item Lie Scale measuring social desirability response bias. The scale has been found to correlate significantly with other general social desirability measures (Milner, 1986). The observed alpha for the CAP Lie Scale in the study sample was 0.78. Items on the lie scale reflect denial of minor but socially undesirable faults to which most people with readily admit. Pre-testing construct validity of the Lie Scale revealed significant negative correlations will problem and symptom measures (BDI, PC, PF, PO), and significant positive correlations with strength measures (FRS, SPS), as expected, and the Lie Scale scores varied over time, suggesting changes in willingness to report.

Working Alliance Inventory (WAI)

The WAI (Tracey & Kokotovic, 1989) used in this study is the 12-item short-form measure assessing agreement on treatment goals and steps, and feelings of mutual liking, affiliation and trust. The measure was captured by client self-report at post-treatment only (i.e., the only wave with an immediately prior service interval). The WAI has generally good psychometric properties with alphas reported in the literature ranging from .68 to .87. Alpha in the study sample was .88.

Client Cultural Competency Inventory

The Client Cultural Competence Inventory (CCCI; Switzer, Scholle, Johnson, & Kelleher, 1998) is a client report instrument reflecting the client’s perception of the cultural competency of mental health services and involvement in community services. Items are answered on a 5-point ordinal scale. Item wording was adjusted slightly to reflect services from a family preservation home visitor rather than a mental health therapist. Initial examination of item properties did not support a single internally consistent factor, either empirically or based on face validity. Some items reflected client perception of provider attitudes while others reflected opinions about outside service referrals and who in the family was receiving services. An exploratory factor analysis for ordinal data using a correlated factor structure was performed for 1, 2, and 3 factor solutions. Examining the content of items loading greater than .40 on a factor in the 3 factor-solution, one factor was identified that contained items reflecting practitioner-client relationship aspects of cultural competency (respect for family beliefs and customs; use of understandable language; absence of negative judgments because of cultural difference; and accepting and respectful behavior), which is the construct of interest for this study. The main difference between the 2-factor and 3-factor solutions was a single item reflecting racial/ethnic similarity between client and provider. A confirmatory factor analysis was conducted for the relationship factor from the 3-factor solution, and the Omega value was 0.91. Adding the fifth racial/ethnic similarity item to the 4-item relationship factor, as per the 2 factor solution, reduced the Omega value, and so it was not included and the 4-item scale was accepted. Factor scores were strongly correlated but not virtually identical to simple summative scores (r-square = 0.82), and so factor scores were exported for analysis. The CCCI was captured at post-treatment only (i.e., the only wave with an immediately prior service interval).

Adjunctive mental health service linkage

Successful linkage to adjunctive usual care mental health services during the service interval was captured by self-report at Wave-2 using items drawn from the services utilization questionnaire developed by Kessler et al. and attached to the Composite International Diagnostic Interview (CITI) used in the National Comorbidity Survey (see Kessler et al., 1999). Sixteen percent of participant parents received some form of outside mental health treatment service for themselves, predominantly outpatient counseling (15% of participants). Note that this captures actual utilization of services, not simply referral for services. The instrument does not ascertain treatment specifics or level of compliance. It simply codes whether an adjunctive usual care mental health service was consumed.

Family preservation service program description

The family preservation services in the study were delivered in the home by bachelors-level home visitors (n = 229), supervised by licensed masters-level clinicians, and employed by community-based agencies under contract with child welfare. Services were normally designed for a 6–9 months duration, with some capacity for case-specific variations. Half of baseline to post-treatment time intervals were in the planned range of 6–9 months, 28% were more, and 22% were less, but fewer than 1% of those enrolled received minimal services (i.e., 3 months or less) and administrative data coded 88% of participants as having complied with the service plan The planned frequency of visits varied, but was required to be at least weekly, and was normally more frequent during early weeks of the service episode or at times of crisis. Services were designed to prevent foster care placement or to promote and stabilize reunification from foster care. Service content was specified by state contract, and included several required elements. Required or basic service elements included case management and linking to adjunctive services, direct assistance with parenting problems, direct assistance with meeting basic needs, direct assistance solving problems and conflicts, basic family violence safety planning, monitoring children’s welfare in the homes, crisis management, and providing support. Psychotherapy was specifically excluded from program content by contract. Each home visitor had access to a $500 budget to assist families in meeting basic concrete child care related needs (e.g., getting utilities turned on in the home). Because most cases involved child neglect, services focused mainly on helping parents to create a physically adequate home environment, promoting family stability, and improving basic caregiving and parenting. Home visitors were required to use a brief parental depression symptom screening tool with parents, and were provided with cut-off scores suggesting need for a referral to adjunctive usual care mental health services. The programs shared basic characteristics (contractual service goals and mandates, service duration and dose, provider credentials, target population, use of the same screening tools, home-based approach, etc.), but differed in aspects of staff training, quality control, and materials used.

Data analysis approach

Under Hypothesis 1, changes in depression symptoms during family preservation services are one facet of broad global life circumstance changes, particularly changes in the basic outcome targets for family preservation services. The data analysis approach for Hypothesis 1 involved examining a set of bivariate parallel process growth models, separately regressing BDI change slopes on change slopes for FRS, SPS, PC, PF, and PO. Prior to building any of the growth models, outcome variables were transformed in order to correct skewed distributions, and were centered. Transformation algorithms were derived from baseline data, then executed equivalently across all waves of data. As a preliminary check, correlations among raw baseline to post-treatment difference scores were examined to confirm that BDI difference scores were positively correlated with other problem area difference scores in the expected directions (positive correlations with PC, PF, and PO; negative correlations with FRS and SPS). Wave-3 coefficients in the growth models were freed in order to capture the nonlinearity that was observed in raw plotted univariate growth trajectories. Slope non-linearity typically involved greater improvement between baseline and post-treatment, followed by less improvement or partial loss of gain from post-treatment to follow-up. Additional preliminary checks were made for whether estimated slopes varied as a function of the individually varying pre-to-post and post-to-followup time intervals. The relationships between timings and estimated slopes were not significant. Clustering was managed via standard error corrections using robust maximum likelihood estimates and applying a sandwich estimator (Muthén & Asparouhov, 2002b).

The next step involved a set of bivariate parallel process growth models. These models created random intercepts and slopes for two variables (see Muthén and Muthén, 2007, p. 109 for a basic structural diagram). Separate parallel models were constructed for BDI scores paired with FRS, SPS, PC, PF, and PO. All models specified cross-variable slope-on-intercept regressions and correlated intercepts. A correlated residual structure was used in which residuals were allowed to correlate in pairwise fashion for each time point in order to account for unobserved correlated effects that might impact slope-slope correlations. All models included correction for social desirability bias by including CAP Lie Scale scores. Each observed variable was regressed on the CAPI Lie Scale score from the corresponding data collection wave. CAPI Lie Scale scores were allowed to autocorrelate across waves, but did not directly covary with slopes or intercepts. Parallel growth models were followed up by testing a single second-order model in which each individual variable’s intercept and slope were jointly modeled as reflecting a second order latent intercept and slope. Hypotheses 2 and 3 involved testing univariate growth models, allowing intercepts and slopes to be predicted by working alliance, cultural competency and adjunctive service receipt. All models were executed using M-Plus 6 software.

Approach to missing data

As an initial test, baseline values were compared across groups defined by missing data pattern to examine selection factors. Bivariate comparisons for basic demographics (age, race, household income, number of children, education) and scores on all baseline measures were tested. Missing data pattern accounted for less than 1% of the observed variance in any demographic variable or baseline score (Eta Squared from .001 to .008). Given the amount of missing longitudinal data, we opted to test models using 2 missing data approaches. Each approach makes somewhat different assumptions about the mechanism of missingness and the results are necessarily presented as a range of standardized coefficients and critical ratios. The first approach is the standard missing at random (MAR) assumption. MAR assumes that the values of missing data are covariate dependent. This is often a reasonable assumption with repeated measures data where there is normally substantial interwave correlation on the same measure, although the MAR assumption can never be fully confirmed. In this study, the observed interwave (i.e. within variable) correlations for BDI, FRS, SPS, PC, PF, and PO scores ranged from .56 to .63; .43 to .59; .53 to .57; .64 to .69; .48 to .57; and .61 to .68 respectively (all p < .001), suggesting substantial dependency in observed values over time. The MAR approach was implemented by including all available data in the growth models, using maximum likelihood estimation. The second approach explored was pattern mixture modeling (PMM; Hedeker & Gibbons, 1997). This procedure estimates how relationships among variables may be conditional on separate missing data patterns and unlike MAR is sensitive to non-ignorable missingness mechanisms given certain assumptions. With 3 waves of repeated measures data there are 7 possible missing data patterns (23–1). We observed that 4 of these 7 patterns had either 0 members or contained less than 1% of our total sample, and so patterns were collapsed to into 3 groups reflecting 1, 2, or 3 observed data points. A mixture model framework was used to implement the pattern mixture model, following the general PMM approach described in Muthen, Asparouhov, Hunter, and Leuchter (2010). The 3 missing data patterns were fixed as known class indicators in a parallel growth mixture model. Class-specific means and variances for the intercepts were allowed. Single data point cases required some assumption regarding their slopes, and so equality constraints between 1 and 2 data point cases were imposed.

Results

Parallel growth models

Findings from the five parallel process growth models are shown in the separate columns of Table 1 for both the MAR and pattern-mixture approaches to missing data. In general, BDI intercepts correlated strongly with intercepts of the other variables, and the BDI change slope tended to be correlated with the other change slopes although statistical significance was not uniform across these relationships. Although not reported in the table, models without correlated residuals were tested and yielded higher and uniformly significant slope-slope regression coefficients. We opted to retain the models with correlated residuals, because they yielded less inflated estimates of the slope-slope coefficients and better managed occasional Heywood case errors in the models. Comparing with PMM classes, some discrepancies between missing data and complete data patterns were noted, which might be interpreted as providing sensitivity boundaries. The MAR approach tended to yield results within the boundaries identified by the PMM approach.

Table 1
Bivariate Models Tested Across Missing at Random (MAR) and Pattern Mixture Model (PMM) approaches, Comparing Structural Pathways in the Models Under Differing Approaches to Managing Missing Data.

Second-order growth model

Given the generally expected pattern of correlations observed in the set of parallel growth models, we felt it was reasonable to test how well a multivariate single second-order latent growth model would fit the data as hypothesized. In order to test whether the global change hypothesis offers a good fit with the data, it is important to have a baseline alternative model against which it can be compared, and so a fully saturated model also was fitted. The fully saturated multivariate model included all latent intercepts and slopes for each of variables tested in the bivariate models (BDI, PC, PF, PO, SPS, FRS). All latent variables (intercepts and slopes) were allowed to uniquely covary. Thus, the saturated model incorporates potentially unique pathways among this set of variables, including both direct and indirect influences, rather than simply positing that there is a single general latent factor which all of the variables reflect. The second-order model is a more parsimonious model (90 free parameters in the second order model vs. 143 in the saturated model). Comparing fit statistics between the saturated and second order models favored the more parsimonious second-order model (BIC = 73,179 for the second-order model vs. 73,403 for the saturated model). A structural diagram of the second-order model with standardized effect estimates is shown in Figure 1.

Figure 1
Second-order parallel growth model1

Hypothesis 2

In the univariate growth models, a more positive working alliance was modestly but significantly related to greater reductions in depression symptoms (see the lower section of Table 1). Under the PMM approach to missing data, approximately equivalent effects and significance levels were obtained (Table 1). These analyses were repeated for the CCCI, and very similar modest effects were observed, but whereas effects for the WAI fell slightly above the customary threshold for statistical significance, effects for the CCCI fell slightly below.

Hypothesis 3

A dose relationship was found between depression symptom levels and receipt of adjunctive mental health services. Services were received by 2% of those with no depression symptoms; 11% of those with insignificant depression symptoms; 17% of those with moderate depression symptoms; 28% of those with significant depression symptoms; and 34% of those with severe depression symptoms (Gamma = 0.46, p < 0.001). The association between adjunctive usual care mental health service utilization and BDI slopes were tested for both the entire sample and for participants with significant baseline depression symptoms (defined as a raw BDI score at or above 19; n = 496). Models included slope-intercept correlations, and applied the response bias correction procedures described earlier. Findings under both the MAR and PMM methods are reported in Table 1. A significant effect was found among all participants, but in the opposite direction from prediction (i.e., receipt of outside mental health services associated with less symptom reduction). Smaller effects were noted among depressed participants, and these were not significant but remained directionally opposite from what was predicted.

Discussion

Among parents in home based child welfare services, depression symptoms were strongly correlated with problems and resources across a range of other common family preservation target areas, and changes in parental depression symptoms over the course of in-home family preservation services were found to parallel changes in these other areas. It is important to note that depression was the only mental health, internalizing or distress variable included in this intercorrelated nexus, and that the other variables spanned a range of domains—social, familial, parental, and socio-economic. A more parsimonious second-order model was ultimately accepted as best fitting the data. In effect, this model suggests that a general tide of improvement may lift (or lower) many boats including depression symptoms. The observation does not confirm that any one of these factors drives the others---it may simply be that all of these changes move in concert.

The quality of the home visitor-parent working alliance was associated with greater improvement, suggesting that attention to relationship factors may be important in family preservation services. A home visitor---client relationship characterized by goal clarity, shared goals, culturally sensitive attitudes and language, trust, and a sense of mutual liking was modestly but significantly associated with depression change. Although the hypothesis was supported, the size of the effect was small. Cultural competency and working alliance were correlated variables, with similar relationships to depression symptom change.

Contrary to prediction, linkage to adjunctive usual care mental health services was not associated with depression symptom change, controlling for baseline symptom level, and in fact the opposite finding was observed. There is a concerning precedent for this finding in the parenting and child welfare literature, including in randomized studies (Chaffin et al., 2004), and in one meta-analysis of parenting programs (Kaminski, Valle, Filene, & Boyle, 2008). There are several possible explanations for this phenomenon. The current study design cannot distinguish among these possibilities, and we believe that more definitive and focused studies are clearly needed. One possibility is that the adjunctive mental health treatment services received by the parents in this study were simply ineffective, of low quality, or had inadequate compliance. Because we measured only service utilization in the broadest terms, we do not have data on the quality of adjunctive usual care mental health services, their use of evidence-based models, or whether the service dose and client compliance were sufficient. True experimental design studies that tightly control the type and quality of depression treatment and accurately model the impact of compliance variations might help resolve this question. Another possibility is that adjunctive usual care mental health services, as currently delivered, may rely on elements that mesh poorly with the typical skill-focused and behavioral orientation of parent training and family preservation services. The other possible explanation is selection bias. There may have been any number of unobserved factors potentially biasing the selection process of who received adjunctive mental health services and who did not. For example, cases may have self-selected or been differentially encouraged into adjunctive services by home visitors if their depression was worsening and vice versa if they experienced spontaneous recovery. Any of these types of selection processes could have contributed to the findings that were observed. Studies using experimental designs to test family preservation with versus without adjunctive usual care services, and capturing finer grained data about depression symptom history and change might help address this possibility.

Consistent with the global change pattern that was observed, there also is the possibility that the current family preservation adjunctive linkage strategy is fundamentally flawed in some ways. The adjunctive service linkage strategy is founded on the assumption that services should be fully comprehensive, linking each identified problem to a corresponding and often separate service program. Given that families in child welfare commonly have a multitude of identifiable problems, this might result in service packages that are complex, that could work at cross purposes, that may create confusion, or that may add to rather than lift the burdens and demands on already thinly stretched parents. It is well known than some families in child welfare are underserved, but there also is sometimes the converse observation—that other cases are inundated by multi-component service plans that may overwhelm or prove more burdensome than helpful. Each of these potential explanations for the observed concerning findings are possible. More importantly, each would point to very different implications for child welfare service delivery. We would argue for more focused and definitive research on these possibilities before drawing conclusions about service implications.

Findings should be interpreted with some limitations in mind. Data for the study were limited to self-report. Although efforts were made to manage social desirability response bias, common method bias may still have operated. We also would note that the findings were obtained from a single statewide service network and may or may not generalize well to others. Study observations were captured on only three occasions (baseline, post-treatment, follow-up) which limited our ability to specify finer grained change trajectories. We attempted to assess the influences of missing data under certain assumptions, but these assumptions cannot be fully verified or refuted. Interpretation should respect the customary limits for any within-subjects correlational study, including caution about inferring causality.

In summary, the findings suggest that among parents in the child welfare system receiving home-based family preservation services, changes in overall wellbeing tend to involve a global improvement pattern, rather than separate or independent patterns for individual target areas, and this pattern includes parental depression symptoms. This may suggest some re-thinking of service plans based on a presumption that every identified problem requires a corresponding discrete service and that services must be fully comprehensive in order to deliver broad benefits.

Acknowledgments

This project was supported by grant number R01 MH065667 to Mark Chaffin from the National Institute for Mental Health. Additional in kind support was provided by the Violence Prevention Branch of the U.S. Centers for Disease Control and Prevention.

Footnotes

The opinions expressed are those of the authors and do not necessarily reflect those of the NIMH or the CDC.

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

References

  • Ammerman RT, Putnam FW, Altaye M, Chen L, Holleb LJ, Stevens J, Short JA, Van Ginkel JB. Changes in depressive symptoms in first time mothers in home visitation. Child Abuse & Neglect. 2009;33:127–138. [PMC free article] [PubMed]
  • Bagner DM, Eyberg SM. Father involvement in parent training: When does it matter? Journal of Clinical Child and Adolescent Psychology. 2003;32:599–605. [PubMed]
  • Baker TB, McFall RM, Shoham V. Current status and future prospects of clinical psychology. Psychological Science in the Public Interest. 2009;9:67–103. [PMC free article] [PubMed]
  • Bakermans-Kranenburg MJ, van IJzendoorn MH, Juffer F. Less is more: Meta-analysis of sensitivity and attachment interventions in early childhood. Psychological Bulletin. 2003;129:195–215. [PubMed]
  • Banyard VL, Williams LM, Siegel JA. The impact of complex trauma and depression on parenting: An exploration of mediating risk and protective factors. Child Maltreatment. 2003;8:334–349. [PubMed]
  • Barber JP, Connolly MB, Crits-Christoph P, Gladis L, Siqueland L. Alliance predicts patients' outcome beyond in-treatment change in symptoms. Journal of Consulting and Clinical Psychology. 1996;68:1027–1032. [PubMed]
  • Barnett D, Kidwell SL, Leung KH. Parenting and preschooler attachment among low-income African-American families. Child Development. 1998;69:1657–1671. [PubMed]
  • Beck AT, Steer RA, Ball R, Ranieri WF. Comparison of Beck Depression Inventories-IA and -II in psychiatric outpatients. Journal of Personality Assessment. 1996;67:588–597. [PubMed]
  • Berger LM. Income, family characteristics, and physical violence toward children. Child Abuse & Neglect. 2005;29:107–133. [PubMed]
  • Casgonguay LG, Goldfried MR, Wiser S, Raue PJ, Hayes AM. Predicting the effect of cognitive therapy for depression: A study of unique and common factors. Journal of Consulting and Clinical Psychology. 1996;64:497–504. [PubMed]
  • Chaffin M, Valle LA. Dynamic prediction characteristics of the Child Abuse Potential Inventory. Child Abuse & Neglect. 2003;27:463–481. [PubMed]
  • Chaffin M, Kelleher K, Hollenberg J. Onset of physical abuse and neglect: Psychiatric, substance abuse, and social risk factors from prospective community data. Child Abuse & Neglect. 1996;2:191–203. [PubMed]
  • Chaffin M, Silovsky JF, Funderburk B, Valle LA, Brestan EV, Balachova T, Jackson S, Lensgraf J, Bonner BL. Parent-child interaction therapy with physically abusive parents: Efficacy for reducing future abuse reports. Journal of Consulting and Clinical Psychology. 2004;72:500–510. [PubMed]
  • Cutrona CE. Social support and stress in the transition to parenthood. Journal of Abnormal Psychology. 1984;93:378–390. [PubMed]
  • Cutrona CE, Russell D. The provision of social relattionships and adaptation to stress. In: Jones WH, Perlman D, editors. Advances in personal relationships. Vol. 1. Greenwich, CT: JAI Press; 1987. pp. 37–67.
  • DePanfilis D, Dubowitz H. Family Connections: A program for preventing child neglect. Child Maltreatment. 2005;10:108–123. [PubMed]
  • DePanfilis D, Zuravin SJ. Predicting child maltreatment recurrences during treatment. Child Abuse & Neglect. 1999;23:729–743. [PubMed]
  • Drake B, Zuravin S. Bias in child maltreatment reporting: Revisiting the myth of classlessness. American Journal of Orthopsychiatry. 1998;68:295–304. [PubMed]
  • Drake B, Jonson-Reid M, Sapokaite L. Rereporting of child maltreatment: Does participation in other public sector services moderate the likelihood of a second maltreatment report? Child Abuse & Neglect. 2006;11:1201–1226. [PMC free article] [PubMed]
  • Dunst CJ, Leet HE. Measuring the adequacy of resources in households with young children. Child: Care, Health, and Development. 1987;13:11–125. [PubMed]
  • English DJ, Graham JC, Newton RR, Lewis TL, Thompson R, Kotch JB, Weisbart C. At-risk and maltreated children exposed to intimate partner aggression/violence: What the conflict looks like and its relationship to child outcomes. Child Maltreatment. 2009;14:157–171. [PubMed]
  • Friedlander S, Weiss DS, Traylor J. Assessing the influence of maternal depression on the validity of the Child Behavior Checklist. Journal of Abnormal Child Psychology. 1986;14:123–133. [PubMed]
  • Hedeker D, Gibbons RD. Application of random-effects pattern-mixture models for missing data in longitudinal studies. Psychological Methods. 1997;2:64–78.
  • Hernandez M, Nesman T, Mowery D, Acevedo-Polakovich ID, Callejas LM. Cultural competence: A literature review and conceptual model for mental health services. Psychiatric Services. 2009;60:1046–1050. [PubMed]
  • Horwitz SM, Briggs-Gowan MJ, Storfer-Isser A, Carter AS. Prevalence, correlates, and persistence of maternal depression. Journal of Women's Health. 2007;16:678–691. [PubMed]
  • Ho LKL. The effectiveness of Parent-Child Interaction Therapy with depressed mother-child dyads. Dissertations Abstracts International: Section B: The Sciences and Engineering. 2005;65:3709.
  • Kaminski JW, Valle LA, Filene JH, Boyle CL. A meta-analytic review of components associated with parent training program effectiveness. Journal of Abnormal Child Psychology. 2008;36:567–589. [PubMed]
  • Kessler RC, Zhao S, Katz SJ, Kouzis AC, Frank RG, Edmund M, Leaf P. Past-year use of outpatient services for psychiatric problems in the National Comorbidity Survey. American Journal of Psychiatry. 1999;156:115–123. [PubMed]
  • Leavy RL. Social support and psychological disorder: A review. Journal of Community Psychology. 1983;11:3–21. [PubMed]
  • Mancini JA, Blieszner R. Social provisions in adulthood: concept and measurement in close relationships. Journal of Gerontology. 1992;47(1):14–20. [PubMed]
  • Mcdonald RP, Mok MMC. Goodness-of-Fit in Item Response Models. Multivariate Behavioral Research. 1995;30:23–40.
  • Milner JS. The Child Abuse Potential Inventory Manual. 2. DeKalb, IL: Psytec; 1986.
  • Murphy JM, Olivier DC, Monson RR, Sobol AM, Federman EB, Leighton AH. Depression and anxiety in relation to social status: A prospective epidemiologic study. Archives of General Psychiatry. 1991;48:223–229. [PubMed]
  • Muthén BO, Asparouhov T, Hunter A, Leuchter A. Growth modeling with non-ignorable dropout: Alternative analyses of the STAR*D Antidepressant Trial. 2010 Downloaded 3/15/2010 from www.statmodel.com. [PMC free article] [PubMed]
  • Muthén BO, Asparouhov T. Latent variable analysis with categorical outcomes: Multiple-group and growth modeling in Mplus. 2002a Retrieved March 7, 2007, from www.statmodel.com/examples/webnote.shtml.
  • Muthén BO, Asparouhov T. Using Mplus Monte Carlo simulations in practice: A note on non-normal missing data in latent variable models. 2002b Retrieved October 5, 2009, from www.statmodel.com/examples/webnote.shtml.
  • Muthén LK, Muthén BO. Mplus user’s guide. 5. Los Angeles, CA: Muthén & Muthén; 2007.
  • NSCAW Research Group. National survey of child and adolescent well-being (NSCAW) CPS Sample Component Wave 1 data analysis report. Washington, DC: US Department of Health and Human Services, Administration for Children, Youth and Families; 2005. Apr,
  • Pelham WE, Lang AR, Atkeson B, Murphy DA, Gnagy EM, Greiner A, Vodde-Hamilton M, Greenslade KE. Effects of deviant child behavior on parental distress and alcohol consumption in laboratory interactions. Journal of Abnormal Child Psychology. 1997;25:413–424. [PubMed]
  • Plant EA, Sachs-Ericsson N. Racial and ethnic differences in depression: The roles of social support and meeting basic needs. Journal of Consulting and Clinical Psychology. 2004;72:41–52. [PubMed]
  • Sanders MR, McFarland M. Treatment of depressed mothers with disruptive children: A controlled evaluation of cognitive behavioral family intervention. Behavior Therapy. 2000;31:89–112.
  • Steer RA, Ball R, Ranieri WF, Beck AT. Further evidence for the construct validity of the Beck Depression Inventory-II with psychiatric outpatients. Psychological Reports. 1997;80:443–446. [PubMed]
  • Sue S. Cultural competency: From philosophy to research and practice. Journal of Community Psychology. 2006;34:237–245.
  • Switzer GE, Scholle SH, Johnson BA, Kelleher KJ. The Client Cultural Competence Inventory: An instrument for assessing cultural competence in behavioral managed care organizations. Journal of Child and Family Studies. 1998;7:483–491.
  • Tracey TJ, Kokotovic AM. Factor structure of the Working Alliance Inventory. Psychological Assessment. 1989;1:207–210.