Evaluation of an early detection tool for social-emotional and behavioral problems in toddlers: The Brief Infant Toddler Social and Emotional Assessment - A cluster randomized trial

doi:10.1186/1471-2458-11-494

BMC Public Health. 2011; 11: 494.

Published online 2011 June 24. doi: 10.1186/1471-2458-11-494

PMCID: PMC3146861

Evaluation of an early detection tool for social-emotional and behavioral problems in toddlers: The Brief Infant Toddler Social and Emotional Assessment - A cluster randomized trial

Ingrid Kruizinga,¹ Wilma Jansen,² Alice S Carter,³ and Hein Raat¹

¹Department of Public Health, Erasmus MC - University Medical Centre Rotterdam, PO BOX 2040, 3000 CA Rotterdam, the Netherlands

²Public Health Service and Environs, Rotterdam, PO BOX 70032, 3000 LP Rotterdam, The Netherlands

³Department of Psychology, University of Massachusetts Boston, 100 Morrissey Boulevard, Boston, MA 02125, USA

Corresponding author.

Ingrid Kruizinga: i.kruizinga/at/erasmusmc.nl; Wilma Jansen: w.jansen/at/ggd.rotterdam.nl; Alice S Carter: AliceS.Carter/at/umb.edu; Hein Raat: h.raat/at/erasmusmc.nl

Received May 28, 2011; Accepted June 24, 2011.

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

This article has been cited by other articles in PMC.

Abstract

Background

The prevalence of social-emotional and behavioral problems is estimated to be 8 to 9% among preschool children. Effective early detection tools are needed to promote the provision of adequate care at an early stage. The Brief Infant-Toddler Social and Emotional Assessment (BITSEA) was developed for this purpose. This study evaluates the effectiveness of the BITSEA to enhance social-emotional and behavioral health of preschool children.

Methods and Design

A cluster randomized controlled trial is set up in youth health care centers in the larger Rotterdam area in the Netherlands, to evaluate the BITSEA. The 31 youth health care centers are randomly allocated to either the control group or the intervention group. The intervention group uses the scores on the BITSEA and cut-off points to evaluate a child's social-emotional and behavioral health and to decide whether or not the child should be referred. The control group provides care as usual, which involves administering a questionnaire that structures the conversation between child health professionals and parents. At a one year follow-up measurement the social-emotional and behavioral health of all children included in the study population will be evaluated.

Discussion

It is hypothesized that better results will be found, in terms of social-emotional and behavioral health in the intervention group, compared to the control group, due to more adequate early detection, referral and more appropriate and timely care.

Trial registration

Current Controlled Trials NTR2035

Background

Psychosocial problems, such as social-emotional and behavioral problems, are prevalent among preschool children; in approximately 8-9 percent of preschool children, child health professionals identify psychosocial problems, such as anxious or depressed feelings and aggressive or disobedient behavior [1,2]. Psychosocial problems in this age group can interfere with everyday functioning [3-5]. Research has demonstrated that problems at preschool age are associated with depressive symptoms, oppositional defiant or conduct disorder, poor peer relationships and social skills, parent- and teacher reported problems with externalizing and internalizing behavior, poor academic performance and psychiatric problems later in life [6-9]. A retrospective study [10] demonstrated that adolescents with psychosocial problems already had neurocognitive, temperament and behavioral problems at age two or three years old.

It has been recommended that psychosocial problems can be detected at a very young age and followed by appropriate management [11-13]. Research has shown that detection and treatment of psychosocial problems at a young age significantly reduces problems and increases competencies [14,15]. Preventive youth health care, as part of community care or paediatrics, offers an opportunity for the early detection of psychosocial problems among preschool children. Child health professionals, such as physicians and nurses who provide preventive care, may apply early detection of psychosocial problems and if necessary adequate referral or short counselling [16]. In the Netherlands, participation of parents with their child in the preventive youth health care is free of charge and on voluntary basis; almost 95% of the parents of preschool children make use of the youth health care service [17].

Despite the potential impact of psychosocial problems at preschool age and the presence of easily accessible youth health care, only a relatively small number of children with psychosocial problems receive appropriate care [2,5]. One study showed that only 29% of the children with severe problems, based on a Child Behavior Checklist (CBCL) total problem score in the clinical range, were identified by child health professionals [2]. And, in another study, only 13% of the children who scored in the clinical range of the CBCL total problem score were referred to mental health services [5].

In current preventive youth health care in the Netherlands, child health professionals apply a structured questionnaire about psychosocial problems that parents complete before coming to the youth health care center, which helps to structure the conversation between child health professionals and parents; there are no validated norm tables or cut-off scores that indicate when the questionnaire signals a problem [18].

As an alternative to this procedure, it has been recommend to evaluate the use of the Brief Infant-Toddler Social and Emotional Assessment (BITSEA) [19] for 1-3 year olds, to detect children at-risk for psychosocial problems and to act upon detection in a coherent, effective way [20].

Objectives

The objective of this study is to evaluate the effectiveness of the BITSEA as an early detection tool for preventive youth health care on children's psychosocial health at one year follow-up, compared to 'care as usual'. Additionally, the feasibility will be evaluated. In this paper we describe the design of this study.

Methods and Design

Study design

The design of the study is a cluster randomized controlled trial in which parents of children aged about 24 months are invited to participate. Information on the study is provided to the parents and the parents are asked to provide informed consent. The parents/children are invited by preventive youth health care staff for a regular health check. We identified 31 distinct youth health care centers that were numbered to the purpose of randomization. We use a block randomization so that within each of the 4 organisations, youth health care centers were randomly allocated to either the control group or the intervention group, using random numbers. The child health professionals in the intervention group use the scores on the BITSEA and the cut-off points to assess whether children are at risk for psychosocial problems. The child health professionals in the control group offer usual care by children using a questionnaire for structuring the conversation with the parents. The effect of the intervention will be evaluated after one year of follow-up by comparing CBCL1,5-5 [21] scores between the children in the intervention group and children in the control group, taking into account the baseline measurement on the CBCL1,5-5. The course of the study is presented in Figure Figure1.1. The Medical Ethics Committee of the Erasmus Medical Centre Rotterdam approved the study protocol (reference number MEC02009-092).

Figure 1

Flow chart of the participants and allocation through the trial.

Study procedure and participants

Youth health care centers and randomisation procedure

Four youth health care organizations that consist of a total of 31 distinct youth health care centers that cover both urban and rural areas in the larger Rotterdam area, are participating in this study. Fifteen youth health care centers were randomly allocated to the intervention group; 16 youth health care centers were randomly allocated to the care-as-usual group, applying a block randomisation procedure as described above. Prior to the start of the study, the researchers arranged meetings to explain the study procedure and to instruct the child health professionals in the intervention group about the use and scoring of the BITSEA, with support of a specialized psychiatrist.

Children and their parents

Parents of 3,000 children are invited to participate in the study. The study population consists of parents or caregivers of toddlers aged 24 months old at baseline, and 36 months old at one year follow-up. Parents of children in the age range of this study have a high attendance (95%) at the regular health checks at youth health care centers [17]. Children who receive treatment of a mental health professional at baseline will be excluded from the study.

Intervention condition

BITSEA

The 42-item BITSEA is an early detection tool for emotional or behavioral problems and delays in social-emotional competence, including autism spectrum disorders, in toddlers. The BITSEA was developed and applied in the USA, and since then also studied in Turkey and Finland [19,22,23]. It is appropriate for use among children of 12-36 months old and consists of 42 items with 3 response options ('not true/rarely', somewhat true/sometimes', 'very true/often'). That are part of one of 2 multi-item scales, a Problem scale (31 items) or Competence scale (11 items). Per scale the items are summed up into a scale score. In addition to the 42 items, the BITSEA has 2 single-item questions with regard to parents' concerns. Internal consistency of the Problem scale has been reported to be 0.79 and of the Competence scale 0.65 [19]. Ten to 45 day test-retest reliability (ICC) of the Problem scale has been reported to be 0.87 and of the Competence scale 0.85 [19]. Relative to typical parent/teacher agreement [24], the parent/child-care provider correlation was relatively high for the Competence scale (ICC = 0.59) and 0.28 for the Problem scale [19]. The BITSEA was translated into Dutch according to international guidelines [25]. Psychometric properties of the BITSEA for the Dutch population will be assessed in the present study.

At the intervention centers child health professionals use the BITSEA [26] as an early detection tool during the regular health check. The child health professionals are trained to score the answers given by parents on the BITSEA and use the cut-off points provided in the literature [19] in their assessment whether children are at risk for or currently experiencing psychosocial problems. Additional information given by parents about timing, duration, intensity of problematic emotions and behavior is also considered in the assessment of the risk for problems. If, for instance, the problematic emotions/behaviors are mild or are considered to be temporarily, e.g. after a major life event, the child health professional may offer advice about how to cope with the circumstances and may choose to ask the parent back in a few weeks for a follow-up.

The child health professional may choose to refer a child and his/her parents to specialized care when the child is at risk for or currently experiencing problems based on the BITSEA scores and cut-off points, when the problematic behaviors/emotions are severe and not considered to be temporarily. A referral to specialized care is always made after consultation with the physician at the youth health care center.

Control condition

In the control condition youth health care centers at child's age 2 years provide care as usual; i.e. the child health professionals use a short questionnaire [18] that serves as a guide through the conversation between child health professionals and parents. Based on this information the child health professionals may choose to invite parents back for a follow-up visit or to refer to specialized care after consultation with the physician at the youth health care center.

Measurements

Primary outcome measures

The primary outcome of the study is the children's psychosocial health, measured with the Child Behavioral Checklist (CBCL1,5-5) [21]. Child health professionals are blind to this measurement. The 99-item CBCL1,5-5 is designed for children 18 months through 5 years and has two domains (Internalizing and Externalizing Problems and a Total Problem score). Answers are given on a 3-point scale with the following response options: 'not true', 'somewhat or sometimes true' and 'very true or often true'. We apply subclinical and clinical cutpoints for the Dutch population [27]. The primary outcome will be measured at baseline at child's age 24 months and one year after the intervention, at the child's age of 36 months. We hypothesize that children in the intervention group will have a lower Total Problem score on the CBCL1,5-5 at follow-up compared to children in the control group, due to more adequate screening, referral and more appropriate and timely care. For an overview, see table table11.

Table 1

Primary and secondary outcome measures and co-variates in the study

Secondary outcome measures

A secondary outcome is health related quality of life, i.e. General Health Perceptions subscale and the Growth and Development subscale of the Infant and Toddler Quality of Life Questionnaire (ITQOL) [28,29], measured at follow-up at child's age 36 months. For an overview, see table table11.

Co-variates

Information on parental characteristics (date of birth, ethnicity, immigration characteristics, cultural identity, socio-economic status), children's characteristics (date of birth, sex, ethnicity, day-care attendance, presence of (mental) health problems and treatment for those problems), and household composition, major life events and the perceived health of the child rated by parents are obtained from the questionnaires at baseline and at follow-up. For an overview, see table table11.

Statistical analyses

Given the cluster design of the study, multilevel analyses will be applied [30,31]. Linear multilevel analysis will be applied for continuous outcome variables and logistic multilevel analysis for dichotomous outcome variables. Interaction effects of gender and ethnic background with the outcomes will be explored.

Power of the study

Power calculations indicated that a total of 3,000 children (and their parents) are needed to detect a difference of 8 points on the CBCL1,5-5 between the control and experimental group, assuming a standard deviation of the CBCL1,5-5 of 26.5 points [32] and an intra-cluster coefficient of 0.1, with a power of 0.80 and alpha 0.05. Assuming a participation of 50% and a lost to follow-up of 30%, we will have complete data at follow-up of 2,100 children (1,050 in both the intervention and the control group).

Process evaluation

In addition to the effect evaluation, a process evaluation will be carried out, in which both the perspectives of parents and professionals will be taken into account. All parents that are included in this study, are asked to evaluate the use of the early detection tool (i.e. level of difficulty, level of understanding, consumed time and satisfaction with the early detection tool as a preparation for the regular health check at the youth health care center). All child health professionals are invited to complete a computer-based process evaluation questionnaire at 6 months after the start of the study. The process evaluation questionnaire consists of items about consumed time, adherence to work instructions, satisfaction with the early detection tool, general perception of the use of the early detection tools in the youth health care and perceived contribution of the early detection tool (a) to the quality of the conversation with parents, (b) to the assessment of the development of the child, and (c) to deciding whether or not to refer.

Furthermore, referrals and consumed care in the year after baseline measurement are assessed at 1-year follow-up, at child's age 36 months; i.e. if a referral to specialized care is made and to which professional; the extent to which parents pursue received referrals, and the diagnosis if one is made are measured at the 1 year follow-up, when children are age 36 months.

Discussion

This paper describes the design of a cluster randomized controlled trial. The trial evaluates the effectiveness of the BITSEA as an early detection tool when used by preventive child health professionals on children's psychosocial health at one year follow-up, compared to 'care as usual'. We hypothesize to find better results, in terms of psychosocial health in the intervention group at one year follow-up, compared to the control group, due to more adequate early detection, referral and more appropriate and timely care.

Strengths of the study are the cluster randomized controlled design, the power of the study, and the setting of the study, which is the daily practice of regular health checks at the youth health care centers that are highly attended by parents. The one year follow-up measurement allows evaluation of the medium term effect of the BITSEA. The study sample will include families with a non-Dutch background, which we expect will add to the generalizability of the results.

Because the study relies primarily on self-report by parents, misclassification might occur. Parents might provide socially desirable answers, e.g. by understating problems or overstating competencies.

A limitation of the study is that the questionnaires are only available in Dutch. For this reason it might be possible that parents with a relatively low level of knowledge of the Dutch language will have some difficulty with the completion of the early detection tool. However, parents have the opportunity to ask for help regarding this issue at the youth health care centers. Furthermore we assess the extent in which parents have understood the questions in the early detection tool as a process measure.

In conclusion, the study evaluates the effectiveness of the BITSEA as an early detection tool to be applied by child health professionals, with the purpose of promoting children's psychosocial health at one year follow-up, compared to 'care as usual'.

Competing interests

AC gets royalties on the sale of the BITSEA from Pearson Assessment (not in the context of this study). The other authors declare that they have no competing interests.

Authors' contributions

HR and WJ had the original idea for the study and its design and were responsible for acquiring the study grant. IK is responsible for the data collection, data analysis en reporting of the study results. AC provides expert input during the study. HR and WJ supervise the study. All authors regularly participated in discussing the design and protocols used in the study. All authors read and approved the final manuscript.

Pre-publication history

The pre-publication history for this paper can be accessed here:

http://www.biomedcentral.com/1471-2458/11/494/prepub

Acknowledgements

This study is funded by a grant from the funding body ZonMw, the Netherlands Organization for Health Research and Development (project number: 80-82435-98-8058).

We express our gratitude to J.C. Visser, S. Smeekens and J.M.A. Riksen-Walraven for the forward translation and I. Flink and Z. Op de Macks for the back translation of the BITSEA.

Study website

http://www.vestonderzoek.nl

References

Velderman M, Crone M, Wiefferink C, Reijneveld S. Identification and management of psychosocial problems among toddlers by preventive child health care professionals. European Journal of Public Health. 2010;20(3):332–338. doi: 10.1093/eurpub/ckp169. [PubMed] [Cross Ref]
Reijneveld SA, Brugman E, Verhulst FC, Verloove-Vanhorick SP. Identification and management of psychosocial problems among toddlers in Dutch preventive child health care. Archives of Pediatrics & Adolescent Medicine. 2004;158(8):811–817. doi: 10.1001/archpedi.158.8.811. [PubMed] [Cross Ref]
Brugman E, Reijneveld SA, Verhulst FC, Verloove-Vanhorick SP. Identification and management of psychosocial problems by preventive child health care. Archives of Pediatrics & Adolescent Medicine. 2001;155(4):462–469. [PubMed]
Briggs-Gowan MJ, Carter AS, Skuban EM, Horwitz SM. Prevalence of social-emotional and behavioral problems in a community sample of 1-and 2-year-old children. Journal of the American Academy of Child and Adolescent Psychiatry. 2001;40(7):811–819. doi: 10.1097/00004583-200107000-00016. [PubMed] [Cross Ref]
Verhulst FC, van der Ende J. Factors associated with child mental health service use in the community. Journal of the American Academy of Child and Adolescent Psychiatry. 1997;36(7):901–909. doi: 10.1097/00004583-199707000-00011. [PubMed] [Cross Ref]
Briggs-Gowan MJ, Carter AS. Social-emotional screening status in early childhood predicts elementary school outcomes. Pediatrics. 2008;121(5):957–962. doi: 10.1542/peds.2007-1948. [PMC free article] [PubMed] [Cross Ref]
Campbell SB, Spieker S, Burchinal M, Poe MD. Trajectories of aggression from toddlerhood to age 9 predict academic and social functioning through age 12. Journal of Child Psychology and Psychiatry. 2006;47(8):791–800. doi: 10.1111/j.1469-7610.2006.01636.x. [PubMed] [Cross Ref]
Hakman M, Sullivan M. The Effect of Task and Maternal Verbosity on Compliance in Toddlers. Infant and Child Development. 2009;18(2):195–205. doi: 10.1002/icd.599. [Cross Ref]
Mesman J, Koot HM. Early predictors of preadolescent internalizing and externalizing DSM-IV diagnoses. J Am Acad Child Adolesc Psychiatry. 2001;40:1029–1036. doi: 10.1097/00004583-200109000-00011. [PubMed] [Cross Ref]
Moffitt TE, Caspi A. Childhood predictors differentiate life-course persistent and adolescence-limited antisocial pathways among males and females. Development and Psychopathology. 2001;13(2):355–375. doi: 10.1017/S0954579401002097. [PubMed] [Cross Ref]
American Academy of Pediatrics. Committee on children with disabilities: development surveillance and screening of infants and young children. Pediatrics. 2001;108:192–196. [PubMed]
Bricker D, Schoen Davis M, Squires J. Mental health screening in young children. Infants and Young Children. 2004;17:129–144. doi: 10.1097/00001163-200404000-00005. [Cross Ref]
Horen, zien, niet zwijgen [See all, hear all, say something]. Inspectie voor de Gezondheidszorg, Inspectie van het Onderwijs, Inspectie Openbare Orde en Veiligheid, Inspectie Jeugdhulpverlening, Jeugdbescherming. 2003.
Durlak JA, Wells AM. Evaluation of indicated preventive intervention (secondary prevention) mental health programs for children and adolescents. American Journal of Community Psychology. 1998;26(5):775–802. doi: 10.1023/A:1022162015815. [PubMed] [Cross Ref]
Elliot J, Prior M, Merrigan C, Ballinger K. Evaluation of a community intervention programme for preschool behaviour problems. Journal of Paediatrics and Child Health. 2002;38(1):41–50. doi: 10.1046/j.1440-1754.2002.00713.x. [PubMed] [Cross Ref]
Basistakenpakket Jeugdgezondheidszorg 0-19 jaar [Youth Health Care Tasks 0-19 years] Ministerie van Volksgezondheid, Welzijn en Sport; 2002.
Centraal Bureau voor de Statistiek [Statistics Netherlands] http://statline.cbs.nl/StatWeb/publication/?DM=SLNL&PA=7042MC&D1=397-401&D2=0-1,5,10,15,20,25-29&VW=T
Kousemaker NPJ. Zoeken, vinden, zorgen delen: de ontwikkeling van een praktijkparadigma voor onderkenning en pedagogische preventie van psychosociale problematiek in de Jeugdgezondheidszorg [Searching, finding, sharing care; the development of a practise paradigm for the descernment and pedagogical prevention of psychosocial problems in Youth Healthcare] Dissertatie - Universiteit van Amsterdam Leiden: Rijksuniversiteit Leiden - Drukkerij Faculteit Sociale Wetenschappen; 1996.
Briggs-Gowan MJ, Carter AS, Irwin JR, Wachtel K, Cicchetti DV. The brief infanttoddler social and emotional assessment: Screening for social-emotional problems and delays in competence. Journal of Pediatric Psychology. 2004;29(2):143–155. doi: 10.1093/jpepsy/jsh017. [PubMed] [Cross Ref]
Hermans J, Ory F, Schrijvers G. Supporting development and parenting: sooner, faster and better: an advice for early detection and interventions in regard to developmental and parenting problems. Julius Centrum: Utrecht; 2005.
Achenbach TM, Edelbrock C, Howell CT. Empirically based assessment of the behavioral/emotional problems of 2- and 3-year-old children. J Abnorm Child Psychol. 1987;15(4):629–650. doi: 10.1007/BF00917246. [PubMed] [Cross Ref]
Karabekiroglu K, Rodopman-Arman A, Ay P, Ozkesen M, Akbas S, Tasdemir GN, Boke O, Peksen Y. The reliability and validity of the Turkish version of the brief infant-toddler social emotional assessment (BITSEA) Infant Behavior & Development. 2009;32(3):291–297. doi: 10.1016/j.infbeh.2009.03.003. [PubMed] [Cross Ref]
Haapsamo H, Ebeling H, Soini H, Joskitt L, Larinen K, Penninkilampi-Kerola V, Carter A, Moilanen I. Screening infants with social and emotional problems: A pilot study of the Brief Infant Toddler Social and Emotional Assessment (BITSEA) in Northern Finland. International Journal of Circumpolar Health. 2009;68(4):386–393. [PubMed]
Achenbach TM, McConaughy SH, Howell CT. Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin. 1987;101:213–232. [PubMed]
Wild D, Grove A, Martin M. et al. Principles of Good Practice for the Translation and Cultural Adaptation Process for Patient-Reported Outcomes (PRO) Measures: report of the ISPOR Task Force for Translation and Cultural Adaptation. Value Health. 2005;8:94–104. doi: 10.1111/j.1524-4733.2005.04054.x. [PubMed] [Cross Ref]
Briggs-Gowan MJ, Carter AS. Brief Infant-Toddler Social and Emotional Assessment (BITSEA) mannual, version 2.0. New Haven: CT: Yale University; 2002.
Verhulst FC, Van der Ende J, Koot HM. Handleiding voor de Teacher's Report Form (TRF). [Manual for the Theacher's Report Form (TRF)]. Rotterdam: Afdeling Kinder- en Jeugdpsychiatrie, Sophia Kinderziekenhuis/Academisch Ziekenhuis Rotterdam/Erasmus Universiteit Rotterdam. 1997.
Raat H, Landgraf JM, Oostenbrink R, Moll HA, Essink-Bot ML. Reliability and validity of the Infant and Toddler Quality of Life Questionnaire (ITQOL) in a general population and respiratory disease sample. Quality of Life Research. 2007;16(3):445–460. doi: 10.1007/s11136-006-9134-8. [PMC free article] [PubMed] [Cross Ref]
Raat H, Monhangoo AD, Grootenhuis MA. Pediatric health-related quality of life questionnairese in clinical trials. Current Opinion in Allergy and Clinical Immunology. 2006;6:180–185. doi: 10.1097/01.all.0000225157.67897.c2. [PubMed] [Cross Ref]
Campbell MK, Elbourne DR, Altman DG, Grp C. CONSORT statement: extension to cluster randomised trials. British Medical Journal. 2004;328(7441):702–708. doi: 10.1136/bmj.328.7441.702. [PMC free article] [PubMed] [Cross Ref]
Twisk JWR. Applied Multilevel Analysis: A practical guide. Cambridge: Cambridge University Press; 2006.
Achenbach TM, Rescorla LA. Manual for the ASEBA Preschool Forms & Profiles. Burlington: VT: University of Vermont, Research Center for Children, Youth, and Families; 2000. [PubMed]

Articles from BMC Public Health are provided here courtesy of
BioMed Central