NSCAW was the primary data source for this analysis, a nationally representative prospective cohort study whose primary aim was to study the health and well-being of children reported to child welfare over a 3-year follow-up period. The study included children who were recruited after a maltreatment report to child welfare from October 1999 - December 2000. Observations were collected through interviews with children, teachers, caregivers, caseworkers, and biological parents at baseline, 12 months, 18 months, and 36 months after enrollment.
From the original 5,501 children, we restricted our sample to those children residing at home during the initial investigation for maltreatment and who were subsequently placed into out-of-home care, which continually lasted for at least 18 months (see ). We also excluded subjects with missing data and the small minority of children who spent more than 9 months in a group home or residential treatment facility, as their stability in a restrictive setting might have biased our findings toward the null hypothesis. Finally, we chose to include only children continuously in out-of-home care because potential interventions with regard to placement stability would by definition impact this group the most, as opposed to those children whose family service plans carried a high probability of reunification home.
Study population derived from the National Survey of Child and Adolescent Well-Being
The primary exposure variable for this study was the child's placement stability over the first 18 months in out-of-home care. We adapted the methodology of James and colleagues in San Diego3
to identify three distinct levels of stability for children entering out-of-home care. Early stability was defined as those children who achieved a long-lasting placement within 45 days of entry into out-of-home care, which was maintained for the period of observation. Children with late stability achieved a long-lasting placement, but only after 45 days. And unstable children failed to achieve a long-lasting placement that was maintained for at least 9 months until the end of the observation period.
The primary outcome for the study was the child's behavioral well-being after 18 months in out-of-home care. We created a composite behavioral well-being variable constructed from two behavioral assessment tools: the Child Behavior Checklist (CBCL) for children aged two and older, and temperament scores for infants under two. Combining both tools allowed us to include a population of children from birth until 15 years of age.
The CBCL is an often-used measure of known reliability and validity, 18
and was administered to children two years of age and older at baseline and again after 18 months. Individual questions are rated using a 3-point Likert scale, in which the caregiver is asked about the frequency of a behavioral problem (“not or never true”, “somewhat or sometimes true”, and “very or often true”). The scores on individual items are then summed in a total behavioral problems scale, which are normed by age to identify standardized categories of normal, borderline (>83rd
percentile), and clinical range (>90th
percentile) for referral for mental health treatment. For our study, we used these normed cut-points to dichotomize our outcome variable as normal behavior vs. borderline or clinically important abnormal behavior for children over two years of age.
For children under two years of age, we included temperament scores that were developed originally for the National Longitudinal Survey of Youth (NLSY) that combined elements from Rothbart's Infant Behavior Questionnaire and Campos and Kagan's Compliance Scale.19
The temperament scores are designed to assess temperament or behavior style in infants, and although they include several domains, we restricted our analyses to the negative hedonic domain for children under one year, and the difficult/negative hedonic domain for children between 1 and 2 years of age (both asked the same questions). These domains measured possible early behavioral problems that at baseline would have rendered the child at greater risk for placement instability. Typical questions in these domains include whether the child cries around strangers, gets upset when the primary caregiver leaves, or has trouble self-soothing. Items are summed using a 5-point Likert scale for each question (Never/Almost Never to Almost Always). Individual items are totaled to report a continuous raw score, the higher of which indicates the likelihood of later behavioral problems.
The composite behavioral well-being outcome variable ultimately combined the dichotomized CBCL scores on the older children with a dichotomized variable of the infant temperament scores. The cut-point of the temperament scores was chosen as one standard deviation from the mean as determined on the sample of children included in the NLSY from 1992 (the most generalizable sample on which these scores have been applied).19
Our team felt comfortable combining the temperament and CBCL scores because prior data had suggested that temperament scores are highly correlated with behavior in older children, and preliminary analyses with our data (not shown) confirmed this to be true.19
A similar methodology was also used to encode the child's baseline behavioral well-being, likely the most important baseline attribute that might have confounded the relationship between placement stability and subsequent well-being.
Covariates included a broad array of child, birth parent and maltreatment history characteristics that are potentially important determinants of whether a child would experience placement instability and poor well-being downstream. Child-level factors included the child's age (categorized as 0−1, 2−10, and 11−15 years), race (White non-Hispanic, Black, Hispanic, Other), sex, history of chronic medical problems (yes/no), and baseline behavioral well-being (as noted above, dichotomized as normal vs. abnormal). Birth parent characteristics included a history of mental health problems, drug or alcohol use, history of domestic violence or arrests. Child maltreatment characteristics included the type of maltreatment (physical abuse, sexual abuse, or neglect/abandonment) or whether the child had a prior history of investigations, substantiated reports or out-of-home care.
Because of the unequal probabilities of selection in the stratified, clustered design elements in the NSCAW data, weighted frequencies are reported to generalize the findings to a nationally representative group of children entering out-of-home care. Due to the great variability of the design weights (range 1−6908), we trimmed the design weights above the 95th
Separate analyses (not provided) revealed that trimming the weights at the 95th
percentile had minimal effect on point estimates for unadjusted associations but reduced the variance of estimates by a factor of 2. Additional trimming did not reduce variance substantially to warrant further adjustment of the weights for analyses.
To adjust for the baseline characteristics that may have confounded the relationship between placement stability and well-being, we used a propensity score analysis in which characteristics of the child, birth parent, and maltreatment history were entered into an ordinal logistic regression model that predicted the likelihood of placement instability. Factors that were significant in bivariate chi-square analyses (p<0.2) were added to the multivariate model. Because we were not concerned with over-fitting the propensity score model, we expanded the baseline score of the CBCL and temperament scores to quartiles. Age was interacted with the child's baseline well-being score to account for the differences in interpretation of the scores and type of measure across age. After fitting the model, a post-estimation probability of placement stability was calculated for each child, and these probabilities were then divided into tertiled categories that expressed the likelihood of placement instability for each child (low, medium, or high). Further model diagnostics confirmed that the propensity score tertiles balanced the covariates in the model as intended.
The propensity score categories (or from here forward, the risk categories for placement instability) were subsequently added as a covariate to a logistic regression model predicting the likelihood of behavioral problems at 18 months as a function of a child's placement stability. After constructing the model, we used conditional standardization 21
to estimate the probability of behavioral problems (with 95% confidence intervals) for children in each level of risk for instability and by the stability they actually achieved.
All analyses were conducted using STATA 9.0 software (STATA Corporation, College Station, Texas). The study was approved by the National Data Archive for Child Abuse & Neglect, and the institutional review board at The Children's Hospital of Philadelphia.