|Home | About | Journals | Submit | Contact Us | Français|
Preschools provide a promising setting in which to conduct preventive interventions for childhood problems, but classroom programs can only be effective if teachers are willing and able to implement them. This study is one of the first to investigate predictors of the frequency of teacher participation in a classroom-based, randomized controlled trial of an integrated prevention program for preschoolers. The intervention was designed to promote school readiness with an integrated social and academic program, to be implemented by teachers with the support of classroom consultants. The current study is part of a larger project conducted with Head Start and community child care centers that serve primarily economically disadvantaged families; 49 teachers from 30 centers participated in this study. Overall, teachers conducted approximately 70% of the program activities. Participation decreased significantly over time from the first to the final week of the intervention, and also decreased within each week of the intervention, from the first to the final weekly activity. Teachers working at community child care centers implemented more intervention activities than did Head Start teachers. Teacher concerns about the intervention, assessed prior to training, predicted less participation. In addition, teachers' participation was positively related to their perception that their centers and directors were supportive, collegial, efficient, and fair, as well as their job satisfaction and commitment. Teacher experience, education, ethnicity, and self-efficacy were not significantly related to participation. In multi-level models that considered center as a level of analysis, substantial variance was accounted for by centers, pointing to the importance of considering center-level predictors in future research.
When children enter kindergarten, they are expected to demonstrate school readiness, which includes language, preliteracy, and numeracy skills, as well as appropriate socioemotional development. However, many children fall short of these expectations, and the consequences are significant and long-lasting. Poor emergent academic skills in young children often co-occur with early social, emotional, and behavior problems (Bramlett, Rowell, & Mandenberg, 2000; Kaiser & Hester, 1997). In turn, these problems have been linked with significant negative outcomes, including later school failure, delinquency, and peer problems (Arnold & Doctoroff, 2003; Dodge, 1993; Eccles, 1997; Neuman & Dickinson, 2001; Snow, Burns, & Griffin, 1998; White, Moffit, Earls, Robins, & Silva, 1990). These difficulties are most prevalent among poor children and children from ethnic minority backgrounds (Brooks-Gunn, Rouse, & McLanahan, 2007; Denton & West, 2002; National Science Foundation [NSF], 1996; Rech & Stevens, 1996; Rush, 1999; Snow et al., 1998).
Addressing readiness skills prior to kindergarten entry can help prevent later academic and socioemotional problems. Building children's early language and literacy skills is an important strategy for promoting school readiness and long-term academic success (Dickinson & Sprague, 2001; Vellutino & Scanlon, 2001). Fostering early mathematics skills has also become a research focus, and the National Council of Teachers of Mathematics recently called for mathematics to be included as part of preschool curricula (National Council of Teachers of Mathematics [NCTM], 2000). It has become increasingly apparent that it is similarly important to focus on socioemotional development before school entry (e.g., Smokowski, Mann, Reynolds, & Fraser, 2004; Yoshikawa & Knitzer, 1997).
Theory suggests that classroom-based programs are a promising approach to preventing academic and socioemotional difficulties. The majority of 3- and 4-year-olds attend day care, providing a means of reaching many children who might not otherwise receive services (U.S. Department of Education, National Center for Education Statistics, 2006). Children typically spend a great deal of time in preschool, providing the potential for comprehensive, multifaceted programs. Child-teacher ratios are usually smaller in preschool than in grade school, and preschool is typically less structured, allowing for social and academic functioning to be addressed in an integrative fashion. If problems can be addressed in preschool, children may be less likely to be labeled or to need special placements in kindergarten. Some empirical data support the potential of preschool-based programs to promote both social and academic development (e.g., Domitrovich, Cortes, & Greenberg, 2007; Smokowski et al., 2004; Webster-Stratton, Reid, & Hammond, 2001; Whitehurst, Epstein, et al., 1994). At the same time, better understanding regarding implementation issues is needed to fully realize the promise of preschool prevention programs.
Because most intervention researchers have focused on creating and evaluating prevention and intervention programs, their studies have typically emphasized mean differences between treatment and control groups. However, main effect comparisons mask great variability in implementation, and these differences are likely to be associated with child outcomes. One important factor is the degree of intervention implementation. Intervention implementation, or participation, can be examined in terms of the quantity of the intervention that is implemented. It stands to reason that even effective interventions will not work if teachers will not or cannot implement them at a high enough frequency to be effective. Another related but distinct factor is intervention quality, or how well teachers are implementing an intervention. If teachers fail to implement interventions at a high enough quality to be associated with treatment outcomes, then even effective interventions are unlikely to work. These effects are likely more pronounced in effectiveness trials, in which both implementation and child outcomes have been demonstrated to suffer compared to their laboratory counterparts (Hulleman & Cordray, 2009).
Given the clear importance of the amount and quality of implementation, there is an increasing call for implementation research across a wide range of areas (e.g., Mowbray, Moxley, & Brown, 1993), including how these constructs affect early childhood interventions (Pence, Justice, & Wiggins, 2008) and educational outcomes (e.g., O'Donnell, 2008). Indeed, the National Association for the Education of Young Children (NAEYC) and the Society for Research in Child Development (SRCD) recently defined the need for treatment implementation research as a priority in their position statement on early childhood education (NAEYC & SRCD, 2008). Although there is a growing theoretical framework for thinking about translating interventions from the research laboratory to real-life settings (Flay et al., 2005; Wilkinson, 2007) and encouraging educational reform at the levels of both the administration and the classroom (Evans, 1996; Fullan, 2007; Wittrock, 1986), few studies quantitatively assess teacher participation in classroom-based interventions.
Few classroom intervention studies report teacher participation rates, but those that do suggest that teacher participation is an important factor to consider in intervention research. For example, Whitehurst, Arnold, et al. (1994) conducted a 6-week literacy intervention for children who attended subsidized day care centers and reported a wide range of reading sessions per child (4.57 to 22.75 reading sessions). Whitehurst and his colleagues conducted a similar study in Head Start classrooms and reported that teacher participation ranged from 61% to 95% (Whitehurst, Epstein, et al., 1994). In a study examining implementation of a preschool intervention which utilized language-based instructional practices, Pence et al. (2008) reported that teachers never reached fidelity to their intervention because they simply did not implement these instructional practices at high enough rates. These studies suggest that children are likely receiving very different amounts of interventions based upon their teachers' participation.
Another closely related literature examines teacher participation within the context of narrow behavioral treatments, such as reinforcement schedules, typically designed for one student with a focused problem. Most of these findings have focused on school-age children rather than preschoolers. This research highlights the importance of the type of intervention, the time required to implement the intervention, the reported effectiveness of the intervention, and the severity of the target problem (Elliott, 1988; Elliott, Witt, Galvin, & Peterson, 1984; Reimers, Wacker, & Koeppl, 1987). In addition, this literature suggests that teacher participation in classroom interventions increases when project consultants provide on-site mentoring and assistance (Han, Catron, Weiss, & Marciel, 2005; Mortenson & Witt, 1998; Noell, Duhon, Gatti, & Connell, 2002; Noell, Witt, Gilbertson, Ranier, & Freeland, 1997).
Other work that investigates teacher participation in a broad, classroom-wide intervention has not yet focused on predicting differences in implementation within treatment groups. For example, Pianta and colleagues have investigated preschool teachers' participation in an intervention to improve preschoolers' transition to kindergarten, but due to small sample sizes their work has not examined predictors of within-group differences in preschool teachers' participation (La Paro, Kraft-Sayre, & Pianta, 2003; Pianta, Kraft-Sayre, Rimm-Kaufman, Gercke, & Higgins, 2001).
In sum, extant research suggests that considerable variability exists in how much and how well teachers implement interventions that target specific problems. Both constructs have been demonstrated to play important roles in the effectiveness of interventions, but the construct of amount of participation has been least studied. Teacher characteristics are one important factor that may help explain differences in participation. The current study focuses on teacher variables as one step toward understanding teacher participation.
The amount of experience a teacher has may contribute to differences in intervention implementation within a treatment group; more experienced teachers generally find treatments less acceptable (Ghaith & Yaghi, 1997; Witt, Moe, Gutkin, & Andrews, 1984). Teacher ethnicity has also been a topic of interest in relation to participation (Kazdin, French, & Sherick, 1981; Von Brock & Elliott, 1987), although to our knowledge it has only been investigated in two studies with samples large enough to detect effects. Neither found significant effects of ethnicity (Pearson & Argulewicz, 1987; Tingstrom & Silver, 1990). Finally, teachers reporting greater self-efficacy also report more positive attitudes toward implementing interventions (Ghaith & Yaghi, 1997; Kallestad & Olweus, 2003).
Other teacher characteristics that may be associated with intervention participation rates have not yet been studied. Teachers' concerns about a program may be crucial in determining whether they participate. Examples of possible teacher concerns that may limit their participation include expecting an intervention to be overwhelming, difficult to implement, or not helpful. In addition, teachers' perception of how supportive, efficient, and positive their work environments and directors are may be related to their participation. Similarly, teachers who are more satisfied with and committed to their jobs may be more likely to participate fully. In addition, teacher education and the type of center involved may be important background variables to consider when predicting teacher participation. Specifically, teachers who have more educational experience and teachers who work at Head Start may have more knowledge of and experience with classroom interventions, and as such, they may participate more in the intervention. Head Start teachers in particular may be may be more familiar with implementing pre-determined curricula and engaging with visiting staff from outside agencies than their peers at community child care centers due to the nature of Head Start programs.
In sum, researchers have only begun to examine characteristics associated with teacher participation in early childhood interventions. The present study aimed to add to this effort by being one of the first studies to investigate rates of teacher participation in a naturalistic, classroom-based, integrated preventive intervention.
Informed by the broader literature on implementation fidelity, and the smaller body of previous studies on participation quantity, we predicted that teachers with more experience would participate less in the intervention, but that teacher ethnicity would not be associated with participation rates. Further, we predicted that teachers high in self-efficacy would implement more of the intervention. Based on theory, we expected that teachers with more education would participate more, and that greater participation would be found for Head Start teachers than teachers from community child care centers. We also predicted that teachers with more concerns about the intervention would participate less, and that teachers with more positive perceptions of the work environment and job would implement more of the intervention.
Given that this is one of the first studies investigating these constructs, we sought to understand the simple relationships between participation and teacher variables. However, understanding these effects within the context of both important covariates and the effects of specific centers is also important. As such, we also utilized exploratory models that predicted teacher participation taking into account teacher-level covariates, center-level variance, and both.
This study is part of the Building Bridges (BB) project, a larger research project involving the evaluation of a kindergarten readiness program conducted in Head Start and community child care programs (Kupersmidt et al., 2007). Eligible programs were those that had one or more classrooms comprised of at least 50% 4-year-old children. Of the five Head Start programs that were identified as eligible, four agreed to participate; the fifth program was undergoing administrative changes that precluded participation. These Head Start programs were located in both urban and rural counties. Community child care centers were then identified in these counties to reflect the same geographic locations. To match the socioeconomic status of the Head Start population, community child care centers were eligible if at least 50% of their students were low-income or enrolled in subsidized slots, as identified by the center director. Eligible centers were required to have a three-star rating or higher, based on North Carolina's five-star quality rating system (North Carolina Division of Child Development, 2005).
Of the 98 eligible community programs that were invited to participate, 59 (60.2%) agreed. The most common reasons community child care programs cited for not participating included that they were busy with another intervention program, they were focusing on obtaining or renewing their license, they expected that their enrollment in the coming year would fail to meet the criteria for inclusion in the study, they were overwhelmed with staff or other structural changes, or they could not accommodate the training schedule of the program.
Seventy-five Head Start and child care centers participated in the Building Bridges program. Centers were stratified by program type (Head Start versus community child care), size, and location. Head Start and child care centers were then matched by size and location. Next, matched sets of centers were randomly assigned to one of three conditions: Workshops Plus, which included workshops, materials, and on-site classroom consultation; Workshops Only, which included workshops and materials only; or a no-treatment control group. The present study includes those centers assigned to the Workshops Plus condition. Participants were enrolled across three years in three cohorts; procedures were the same for all cohorts, with the exception that the first cohort implemented the program for 21 rather than 25 weeks.
Participants in the current study were 49 female teachers of 3- and 4-year old children from 30 centers (8 Head Start, 22 community child care). See Table 1 for teachers' demographic information.
Center directors arranged for BB project staff to meet with teachers to discuss the study and answer teachers' questions regarding participation. Once teachers consented to participate, BB project staff members individually interviewed teachers during the spring prior to the intervention year, before the teachers were trained in the BB program. All teacher-reported independent variables were gathered from this baseline interview. All questionnaires were administered in interview format, although teachers were given a copy of the interview and could opt to provide written answers once the interviewer explained the measure. This strategy was used to ensure that teachers understood the instructions associated with the measures and to prevent missing data. The interviews took 60-90 minutes to complete and were scheduled at a time that was convenient for the teachers. Teachers were compensated $20 for the interview.
Teachers attended 30 hours of group-based training during the summer prior to the intervention year, and were paid $10 per hour for the training. Makeup training was conducted in the early fall for those not able to attend the summer workshops. The training included workshops on proactive teaching strategies, relationship building, and behavior observation and management, as well as direct instruction and coaching in implementing the Building Bridges curriculum activities. Training was conducted by the teacher consultants, principal investigator, project director, and clinical supervisor. In an effort to maximize implementation support, program administrators (i.e., directors, assistant directors, education coordinators, mental health coordinators) were also invited to the training. Teachers received Second Step kits, Dialogic Reading books, BB Activity kits, and a training manual. All of these were carefully designed to be as simple to implement and user-friendly as possible. In a further effort to ensure that all teachers could utilize this multifaceted intervention, the training provided step-by-step instructions for each intervention component and provided sample lesson planning on how to incorporate BB components into their current lesson plans. Consultants were available to problem-solve, and a helpline was available to all participants. Of those teachers trained, 15% (n = 6) of lead teachers and 19% (n = 6) of assistant teachers did not return to teach in the fall.
The BB intervention, developed after seven years of preliminary research in Head Start and low-income community child care centers, was designed to promote social, emotional, academic, and behavioral school readiness in preschoolers. The BB program contains ten weekly activities subsumed under four major components: Second Step, Dialogic Reading, Pre-Literacy and Communication activities (Strengthening Speaking Skills, Things We Read, Listening to Language, Getting Ready to Write, Attending and Remembering) and Math activities (Counting and Numbers, Shapes and Sizes, Measuring and Estimating). Each activity took approximately 10-20 minutes to implement.
The Second Step Violence Prevention Curriculum, Third Edition (Committee for Children, 2002) provided the foundation for social-emotional skill building and has been empirically demonstrated to benefit preschoolers (Moore & Beland, 1992). Second Step includes Empathy Training, Emotion Management, and Problem-Solving units comprising a total of 25 lessons presented in a group setting on a weekly basis. Integrated with these lessons were Dialogic Reading activities. Dialogic Reading is an empirically-validated intervention that enhances language and emergent literacy skills in preschoolers by making shared reading more interactive (Arnold, Lonigan, Whitehurst, & Epstein, 1994; Whitehurst, Arnold, et al., 1994). Books were selected based on the social-emotional themes presented in the Second Step lessons. The BB Activity kit consists of five Pre-Literacy and Communication and three Math activities, which are integrated, sequential, developmentally appropriate lessons to build and enhance children's language and mathematics skills. The BB activities are linked to the social-emotional concepts presented in the weekly Second Step lessons and Dialogic Reading books. Individual cards containing the instructions for each activity are fully scripted and designed so that they could be conducted in a variety of settings, including circle time, transitions, outdoor time, and centers. Materials were provided to facilitate implementation (e.g., CDs, felt board, game cards).
In sum, teachers were provided a set of ten activities per week for each of the 25 weeks of lessons: one Second Step lesson, one Dialogic Reading book, five Pre-Literacy and Communication activities, and three Math activities. The intervention materials presented the activities in that order, and though teachers were trained to first implement Second Step, they were free to implement the remaining activities in any sequence. Teachers typically followed the order of presentation, so we refer to this pattern as the “within-week order” (i.e., Second Step as the first activity of the week, and the math activities as the last activities of the week). Taken together, this yielded a total of 250 activities that could be implemented across the intervention.
Following the training, teacher consultants provided on-site mentoring and assistance. The consultants were specialists in either early childhood education or mental health. They received extensive training in BB as well as weekly individual supervision and biweekly group supervision from the project's clinical supervisor. Consultants established a schedule of once-weekly classroom visits and on-site appointments across the intervention, and teachers received an average of approximately 224 (SD = 66) minutes of consultation per week. The amount of consultation varied widely across classrooms, ranging from approximately 1.5 to 6 hours per week. The content of the consultation depended on specific classroom needs as determined collaboratively by the consultant and teacher(s). During initial meetings, consultants engaged in follow-up discussions about the training, provided teachers individualized feedback from the spring baseline assessments, and worked with each teacher to set initial intervention goals. The consultation was individualized, such that classroom consultants provided teachers with the specific support they needed to manage the multifaceted BB intervention. A typical progression of goals might be to first establish a schedule for implementing the BB activities; followed by a focus on modifying the classroom environment to prevent behavior problems and enhance learning; and next, an emphasis on teachers' personal and team-building goals. Because the consultation was individualized at the classroom and teacher level, the goals varied widely. Goals were reviewed informally at each visit in order to provide focus for the visit, and formally reviewed three times during the year. At those three timepoints, teachers received $25 if they had achieved these goals, and new goals were set once earlier goals were achieved.
In cases in which teachers were not fully participating in the intervention, consultants utilized a variety of strategies to increase participation. These strategies differed depending on the reason the teacher was not implementing activities. For teachers who had difficulty fitting BB activities into their lesson plans, for example, consultants spent time reviewing individual lesson plans with the teachers to assist them in finding opportunities for implementing activities. For teachers who had difficulty remembering to implement activities, consultants placed visual cues—such as signs or posters—in the classroom to remind teachers to use BB activities at various times of the day. Teachers who lacked confidence in implementing activities were given the opportunity to observe the consultant model BB lessons in the classroom. For teachers who were resistant to the BB program or to the consultation process, consultants focused on improving their relationship with the teachers to provide a positive foundation for their participation in BB. Consultants often used tangible ‘rewards’ (e.g., hearts, stars) to acknowledge teachers' improvements in participation and implementation in the BB program.
Teachers completed weekly activity logs, the source of the outcome variable in this study. These logs were collected by consultants, and teachers were paid $10 for each completed activity log. It is important to note that teachers were paid $10 for each completed weekly log regardless of how many activities were recorded on the log. Thus, a teacher who completed only one activity in a week was paid the same as a teacher who reported completing all ten activities in a week. This procedure was put in place in order to maximize teachers' submission of the logs and encourage teachers to be as accurate as possible in reporting their activities.
Teachers provided information regarding their education, ethnicity, and experience in the field of early education.
Teacher participation was operationalized as the percentage of possible activities implemented across the course of the intervention. Implementation was measured by the Weekly Classroom Activity Log (WCAL; Kupersmidt & Voegler-Lee, 2003), designed specifically for this study. The WCAL is a highly-structured, face-valid self-report record of teachers' weekly implementation of the 10 project lessons (i.e., Second Step, Dialogic Reading, five Pre-Literacy and Communication activities, and three Math activities). This measure required teachers to record the initials of the teacher who conducted each of the ten activities and the date on which each activity was conducted. In addition, teachers rated each implemented activity on the following four-point items: “How hard or easy was it to teach this activity,” “About how many children learned the skill,” “How much did you enjoy teaching this activity,” and “How much did the children enjoy the activity.” Alpha for this scale in the current sample was .96. Activities were considered completed if teachers turned in logs and indicated that they had completed the activity. Missing WCAL data were considered incomplete activities; the missing data rate for the WCAL was 9.5%.
Teachers completed a version of the Concerns Questionnaire (CQ; Bloom, 1998; Hord, Rutherford, Huling-Austin, & Hall, 1987), modified for this study. This modified version includes seven yes-no items regarding concerns about implementing a new intervention. Example items include “I am concerned about how the children will react to a new program” and “I am concerned about how I can carry out all my responsibilities with respect to the curriculum.” Items were averaged to create a scale score, with higher scores indicating more concerns. The Kuder-Richardson 20 for this scale was .65.
Teachers completed the short form of the Early Childhood Work Environment Survey (ECWES; Jorde-Bloom, 1996), a 20-item self-report scale measuring teachers' perceptions of their work environment. Teachers responded to each item along a five-point Likert scale from “strongly disagree” to “strongly agree.” Example items include “Supervisor(s) is (are) knowledgeable and competent,” “Teachers help make decisions about things that directly affect them,” and “The program is well-planned and efficiently run.” Items were averaged to create a scale score, with higher scores indicating a more positive work environment. Adequate reliability and validity data has been established for this measure (Jorde-Bloom, 1996); in this sample, alpha was .96.
Teachers completed a five-item director's leadership style questionnaire (DLS) that assesses a teacher's perception of how much her director promotes teacher independence, teacher-teaching assistant cooperation, and professional development. Example items include “My director allows me to make my own decisions in my classroom,” and “My director supports and encourages teamwork between my teaching partner and me.” Teachers responded to each item along a five-point Likert scale (1 = not at all true, 5 = very true). Items were averaged to create a scale score, with higher scores indicating more positive director leadership style. Alpha for this sample was .77.
Teachers completed the Work Attitudes Questionnaire (WAQ; Jorde-Bloom, 1988), a five-item scale on which teachers rate how much their current work situation resembles their ideal. Teachers rated items such as “working conditions,” “pay and promotion opportunities,” and “the work itself” on a five-point scale (1 = not at all like my ideal, 3 = somewhat resembles my ideal, and 5 = is my ideal). Items were averaged to create a scale score, with higher scores indicating greater satisfaction. Adequate reliability and validity have been established for the WAQ (Jorde-Bloom, 1988); in this sample, the alpha coefficient was .76.
Teachers completed the How Committed Am I? questionnaire (OC; Jorde-Bloom, 1988), a 14-item self-report scale measuring a teacher's commitment to her center. Teachers responded to each item along a five-point Likert scale from “strongly disagree” to “strongly agree.” Example items include “It would take a big change in my present circumstances to cause me to leave this center,” and “Even if I had a slightly higher paying job offer, I wouldn't leave this job.” Items were averaged to create a scale score, with higher scores indicating greater organizational commitment. Adequate reliability and validity have been established for this measure (Jorde-Bloom, 1988); in this sample, alpha was .92.
Teachers completed a version of the Teacher Efficacy Scale (TSE; Gibson & Dembo, 1984), originally adapted by Woolfolk and Hoy (1990) and demonstrated to contain two independent dimensions of general and personal teaching efficacy. General teaching efficacy is defined as a teacher's general beliefs about the power of education, while personal teaching efficacy is defined as a teacher's beliefs that she personally can influence student learning. The short form of this scale was adapted for preschool teachers, resulting in seven self-report items (Hoy & Woolfolk, 1993). Lower scores on general and higher scores on personal efficacy subscales indicate higher feelings of self-efficacy. Teachers responded to each item along a five-point Likert scale from “strongly disagree” to “strongly agree.” Research has supported both convergent and discriminant validity of this measure (Woolfolk & Hoy, 1990). In this sample, alpha was .71 for general and .76 for personal teaching efficacy.
Teacher demographic information is presented in Table 1. Teachers at Head Start centers differed significantly in terms of both ethnicity and education from those at community child care centers (see Table 1). Means and standard deviations of the study variables, and intercorrelations between these variables, are presented in Table 2. Teachers implemented an average of 71.48% of possible intervention activities (SD = 18.84%), with participation ranging from 36% to 100% of activities. Two of the teachers implemented all of the lessons while 45% of teachers implemented at least three quarters of the activities. Approximately 14% of the teachers implemented fewer than half of the lessons.
Figure 1 presents the pattern of participation across weeks by activity type. A repeated-measures ANOVA trend analysis was conducted examining participation across the 21 weeks of the program for which all classrooms had data. This analysis indicated a linear decrease in participation as a function of time, F(1,48) = 17.41, p = .001. Teachers completed 88% of the Week 1 activities, and only 53% of the activities in Week 21. A second repeated-measures ANOVA trend analysis was conducted to evaluate whether participation decreased within weeks, by comparing participation rates across the 4 types of activities, ordered as in the activity plans (i.e., considering Second Step as the first activity, Dialogic Reading as second, Pre-Literacy and Communication activities as third, and Math activities as fourth). Within each week, participation decreased from the first lesson of the week (Second Step activities were implemented 87% of the time) to the final lesson (Math activities were implemented 62% of the time), F(1,48) = 93.93, p < .001. Although this within-week decrease in participation is striking, teachers reported similarly high ratings with regard to how much they enjoyed teaching the Second Step, Dialogic Reading, Language and Communications, and Math activities (M (SD) = 3.56 (.57), 3.75 (.47), 3.52 (.47), and 3.53 (.53), respectively).
Given the sparse previous research on the specific topic of predicting amount of implementation, we wished to examine the direct, simple relationships between various predictors and teacher participation as a first step in describing the relationships among these variables. At the same time, we recognize that more complete models of participation will need to control for background variables that might be associated with participation. We thus examined several covariates hypothesized to be important in teacher participation. In addition, because classrooms are nested within centers, models are also needed that account for this hierarchical structure, to assess the importance of center-level factors. Given the value in beginning with simple relationships, and in evaluating how these relationships change when taking into account covariates and nesting, we ran four sets of analyses for each potential predictor of participation.
First, simple relations between the study predictors and teacher participation were estimated using linear regressions. Second, these associations were evaluated adjusting for the following covariates: teacher experience, teacher education, and center type (see next section for specific rationale). Third, because these data were collected from teachers nested within centers, some of the variance accounted for by the predictors in the previous models may be accounted for by this hierarchical nature of the data structure. Therefore, in the third set of analyses, associations between predictors and teacher participation were evaluated accounting for the nesting of classrooms within centers, using a random intercepts mixed linear model. Fourth, the associations between predictors and teacher participation were evaluated, accounting for both the covariates and the hierarchical data structure. Predictors were standardized (M = 0, SD = 1) to provide some uniformity when interpreting the unstandardized regression coefficients across models. By standardizing the predictors, coefficients represent the change in the percent of the possible activities implemented associated with a one-standard deviation change in the predictor.
Table 3 presents associations between each predictor and participation under each of the four models. These patterns were also investigated separately across each of the four activity subtypes (i.e., Second Step, Dialogic Reading, Pre-Literacy and Communication, and Math) to confirm that they matched those found for overall participation. Finally, we explored whether significant relationships between predictors and teacher participation were moderated by center type (Head Start vs. community childcare).
In contrast to previous research, the number of years that teachers had worked in early childhood education was not significantly associated with teacher participation. In fact, the non-significant relationship was in the opposite direction than was expected, r(49) = .21, p = .15.
Contrary to predictions, teachers with higher levels of education were not significantly more likely to implement intervention activities. In fact, the non-significant relationship was in the opposite direction than expected, with better educated teachers participating less, r(49) = −.26, p = .07.
For the purposes of this analysis, the one Hispanic and one mixed ethnicity participant were excluded. Consistent with previous findings, no significant differences were found in the participation rates of African-American and White teachers, t(45) = 1.30, p = .20. African-American teachers (n = 31) implemented an average of 69.21% of the intervention activities (SD = 20.46%) while White teachers (n = 16) implemented an average of 76.81% (SD = 15.75%). Ethnicity was also investigated as a moderator of the simple relationships between study predictors and teacher participation; no significant interactions were found. Because no significant differences in teacher participation by ethnicity and no interactions between study variables and ethnicity in predicting teacher participation were expected or found, ethnicity was not included as a covariate in additional analyses.
In contrast to the prediction, teachers at community child care centers implemented more lessons than those at Head Start, t(47) = −3.91, p < .001. Head Start teachers (n = 21) implemented an average of 60.81% (SD = 19.50%) of the activities while community child care center teachers (n = 28) implemented an average of 79.48% (SD = 13.94%).
Consistent with our hypothesis, the unadjusted, simple association between teacher concerns and participation was significant, r(49) = −.40, p = .004 (see Table 3). Teachers who reported more concerns with implementing the intervention ahead of time in fact conducted fewer intervention activities. The regression weight of approximately −8 from this model indicates that every one standard deviation increase in teachers' CQ score was associated with the completion of about 8% (20 of the possible 250) fewer activities over the course of the intervention (see Table 3 for estimated weights for all models). This finding held even when taking into account covariates and when adjusting for the hierarchical data structure, though the magnitude of the effect was somewhat reduced in these models (b = −5.19 and −5.08 respectively). In the model that takes into account both covariates and nesting, the effect approached but did not reach significance (b = −4.27, p = .08).
Teachers who perceived their centers as promoting collegiality, professional growth, as well as providing a supportive, efficient, and fair work environment implemented significantly more intervention activities, r(49) = .54, b = 11.78, p < .001. This relationship was also significant when taking into account covariates and when adjusting for the hierarchical data structure, though the magnitude of the effect was somewhat reduced in these models (b = 7.58 and 7.23 respectively). The effect was not significant in the model that included nesting and covariates (b = 5.75).
As hypothesized, teachers who reported a more positive perception of their director participated more in the intervention, r(48) = .45 , b = 9.89, p = .001. When controlling for covariates, adjusting for the hierarchical data structure, and both, these perceptions were no longer significant predictors (b = 5.64, 3.64, and 2.58 respectively).
Teachers' job satisfaction was significantly related to participation, r(49) = .39, b = 7.82, p = .005. When controlling for covariates and adjusting for the hierarchical data structure, and both, teacher job satisfaction no longer significantly predicted participation (b = 3.88, 3.44, and 2.97 respectively).
Teacher job commitment was positively related to teacher participation, r(49) = .33, b = 6.45, p = .02. This effect was no longer significant when controlling for covariates, adjusting for the hierarchical data structure, and both (b = 1.90, .84, and .20 respectively).
In contrast to hypotheses and previous research, no significant relationships were found between self-efficacy and participation. Because self-efficacy was not significantly related to participation in any of the four models, it was not considered in any further analyses.
All analyses were also conducted using each of the four activity subtypes (i.e., Second Step, Dialogic Reading, Pre-Literacy and Communication, and Math). The pattern of results across each of the four activity subtypes was very similar to that found for overall participation. A small number of non-significant results that became significant in the sub-activity models, and the even smaller number of previously significant predictors that were not significant for sub-activities. Specifically, concerns about the intervention became significant when predicting participation in Second Step taking into account both covariates and nesting. Perceptions of the work environment became significant when predicting Dialogic Reading participation taking into account both covariates and nesting. Perceptions of the center director became a significant predictor of participation in Dialogic Reading in models taking into account covariates, nesting, and both covariates and nesting. Teacher job commitment also became a significant predictor of participation in Dialogic Reading in models taking into account covariates, nesting, and both covariates and nesting. In contrast, job commitment became a non-significant predictor of participation in Pre-Literacy and Communication Activities (p < .06) and concerns about the intervention became a non-significant predictor of participation in Math Activities in models separately taking into account covariates and nesting (all ps < .09).
A comparison of coefficients in each row of Table 3 reveals a dramatic pattern. Whereas all of the simple, unadjusted effects were significant (column 1 of Table 3), none of the effects were significant once covariates and the hierarchical data structure (teachers nested in centers) were taken into account (column 4 of Table 3). We estimated an intraclass correlation coefficient (ICC) to quantify the extent to which teachers within a center were similar with respect to their participation. This ICC was .53. This indicates a very high degree of within-center similarity regarding teacher participation in the program, and helps explain the changes in significance for any given predictor moving from the left-most two columns of Table 3 to the right two columns.
No significant interactions between center type and the predictors were found for any of the analyses (e.g., simple, covariates, nested, or covariates and nested), suggesting that the patterns of the relationships between the predictors and teacher participation did not significantly differ between Head Start and community childcare centers.
The current study is one of the first to examine patterns and predictors of teacher participation rates in a preschool preventive intervention. Overall, teachers implemented about 71% of the intervention activities, and about 14% of the teachers implemented fewer than half of the activities. Whereas complete participation in this intervention would include a child participating in 10 lessons, a typical child in this study only participated in about seven activities per week, while some children participated in as few as three or four activities per week. As a result, many children missed large portions of the program. This participation rate suggests that even though teachers were utilizing a state-of-the-art program, reported enjoying this program, were supported on a weekly basis by classroom consultants, and had access to substantial resources through the research study, there still remained barriers to full participation. These participation rates, while perhaps discouraging, are not dissimilar to previous findings (Pence et al., 2008; Whitehurst, Epstein, et al., 1994).
Two clear patterns of participation emerged from these data. Teachers gradually decreased activity implementation across the program. In addition, within a given week of the intervention teachers' implementation rates dropped on average from 85% to 55%. Several possible explanations might account for this within-week drop in participation. Teachers typically followed the within-week order of activities presented in the BB kits (i.e., Second Step as the first activity of the week and the math activities as the last activities of the week). Therefore, the drop in implementation rates within weeks could have occurred because teachers were unable to get to all of the lessons each week due to practical constraints, because they tired of program implementation as the week progressed, or because the complexity of the intervention was simply too great for them to manage on top of all of their other classroom duties. Alternatively, teachers might have been less comfortable teaching the math unit, which may be a subject area in which preschool teachers tend to be less experienced. Because these two potential explanations are confounded, we cannot conclusively determine why this within-week drop in participation rates was found. However, it appears that this effect is not due to teachers disliking any particular unit, as they rated all lessons comparably high. Unfortunately, the teacher satisfaction data suffer from a ceiling effect, making it impossible to examine teacher satisfaction as a potential moderator of teacher participation in this study.
These patterns of decreased participation over time, both across and within weeks, point to the importance of addressing this construct during program design and training. Large-scale effectiveness trials like this one allow intervention research to focus on both traditional outcomes and outcomes related to implementation. Overall, both the amount and quality of implementation have been found to be associated with better outcomes (O'Donnell, 2008), but it is likely that complete implementation of interventions is neither feasible nor necessary. In addition, interventions likely do not operate in a one-size fits all manner; from one classroom or child to the next, the optimal degree of teacher participation may differ substantially. Although the present study is unable to determine the amount of implementation necessary for optimal effectiveness, future research should focus on defining benchmarks for both participation and fidelity. We speculate that teacher participation will mediate the outcome of interventions; more generally, a comprehensive model predicting child outcomes should include both teacher participation and contextual predictors of teacher participation, including center type and teacher characteristics. We are beginning to analyze the findings of our broader research study as they relate to the efficacy of the intervention (Kupersmidt et al., 2007), and our future work will attempt to forge this link between teacher participation and child outcomes with the aim of informing efforts related to better understanding the role of teacher participation.
This study failed to replicate previously studied relationships between teacher participation and both teacher experience and teacher self-efficacy. It is possible that these variables may play a smaller or different role in classroom-based, integrated preventive interventions. This study replicated the finding that teacher ethnicity is not related to participation; as expected, no significant differences between the participation rates of African-American and White teachers were found. This study also found that teacher education, a previously uninvestigated variable, was negatively associated with participation at a level that approached significance, and that Head Start teachers had significantly lower participation rates than teachers from community child care centers.
A number of variables were strongly related to participation in terms of simple bivariate associations. Teachers who expressed more concerns about the intervention implemented fewer activities. Teachers who perceived their jobs, work environments, and directors more positively, and who reported greater job commitment implemented more activities. These effects, considered at the level of simple associations, were substantial. For example, teachers who were one standard deviation above the mean with respect to their perceptions of the workplace completed approximately 23% more of the possible activities than teachers one standard deviation below the mean on this predictor. This difference amounts to approximately 57 additional activities over the course of the program. In addition, this pattern of effects was found for each of the four activity subtypes of the intervention. Our study was among the first to examine these constructs as predictors; considering the size and consistency of these effects, and the fact that these variables are easy to measure, teacher concerns about interventions and their workplace satisfaction need to be addressed in future research. If further support is found for their importance, addressing these issues in the design of prevention programs may be critical regardless of the content or method of a program, especially given the fact that most preventive interventions do not focus on how the teacher feels about implementing a new curriculum or about her work environment and job. In addition, future research should focus on the processes and mechanisms involved in these findings; finer-grained analyses might examine, for example, what aspects of the work environment are most closely associated with participation.
Although simple bivariate relationships indicated strong effects, more complex analyses that accounted for covariates and the nesting of teachers within centers found weaker, often non-significant relationships between these teacher characteristics and participation. There are a number of possible reasons for these differing results. The finding that covariates weakened relationship estimates points to the need to understand the mechanisms involved in the simple relationships. Almost no process work has yet to be done in this research area, and such work will be critical in guiding future program development. In addition, the large center effects found in our multi-level models, and the accompanying large ICC demonstrating implementation similarity within centers, point to the importance of understanding the systemic effects of center-level variables. Although the goal of this paper was to consider teacher characteristics, center-level variables also appear likely to be important in determining teacher participation. Understanding center-level constructs would have great practical and theoretical importance, so future studies should examine constructs such as center size, resources, administrative support, structural factors, philosophical approach, and other related variables.
One center-level variable that was included in this study was the type of center. Head Start teachers implemented significantly fewer intervention activities than teachers at community child care centers. The lower participation reported within Head Start could be for several reasons. For example, there are many requirements within the Head Start program and it may be challenging for teachers to incorporate new activities within their already crowded day. Also, Head Start already endorses a preventive curriculum focused on the same aspects of early childhood development that the BB intervention targeted, and teachers may have had difficulty integrating the BB program into their weekly lesson plans. Although teacher participation differed by center type, center type did not moderate the relationship between participation and the other predictors. As such, although the results of this study suggest that considerable variance is accounted for by center type and that this difference in teacher participation by center type likely has a considerable impact on the effect of the intervention, it does not suggest that the relationships between teacher characteristics and participation differ between center types.
In addition to center-level predictors that this study failed to fully capture, additional teacher-level predictors may also play an important role in teacher participation. For example, the relationship between teachers and consultants likely plays a large role in implementation (Orrell-Valente, Pinderhughes, Valente, & Laird, 1999). In this study, the amount of consulting varied widely across classrooms, possibly due to differences in the teacher-consultant relationship. Teacher personality characteristics may also affect how they implement a classroom intervention. Although rarely studied, child effects could also be important in understanding participation, potentially including predictors like number of children in the classroom who are experiencing the problem targeted by the intervention (e.g., Kallestad & Olweus, 2003).
Approximately 10% of the activity logs were not completed. On one hand, such missing data are to some extent an inevitable part of community research. On the other hand, we cannot be sure whether these missing data represent completed or non-completed activities. For the purposes of this paper, we represented missing logs as non-completed activities. Although this method of operationalizing participation doubtless introduced measurement error, other ways of operationalizing the outcome would have been similarly concerning. For example, we could have predicted number of activities per week in which forms were completed, but this would effectively give teachers credit for activities that we had no evidence they actually completed. We believe that, for the purposes of this paper, predicting the activities that were actually reported was preferable, but not knowing what occurred during weeks with missing logs is a study limitation. In addition, some measures used in the study, including our outcome variable, lacked validity data because they were created for this project.
This study suffers from several additional limitations. The outcome variable for this study was participation frequency, which is only one narrow element of implementation. Teachers may differ widely in their approaches to executing programs, their skill in implementing activities, their ability to transfer program skills to other situations, and in many other ways. Comprehensive models for conceptualizing and measuring implementation are much needed.
In addition, the large majority of the predictors included in this study were self-reported by the teachers, a vehicle of data collection that suffers from numerous limitations. Future research should include additional reporters and observational data to reduce method variance. Teachers were compensated for their time, which could compound problems with relying on teacher-report. Resulting demand characteristics could result in over-reporting participation frequency. Alternatively, the over-justification hypothesis would suggest that an external incentive like the money that the teachers were paid to complete various aspects of the intervention may have reduced their internal motivation to participate, thus reducing their overall participation rates (Deci, Koestner, & Ryan, 1999). However, it should be noted that teachers were paid based only on completing the measures, not based on what they reported, and that although all teachers received money for their participation, teachers still varied widely in their participation rates.
Although this large sample included socioeconomically disadvantaged populations, the low-income rate in this sample was only 50%, and this study did not parse out the low-income from the high-income children, teachers, or centers to investigate whether participation differed between these groups. In particular, very low-quality centers that likely needed other supports before their staff could implement an intervention such as BB were not included in this study, so these findings may not generalize to lower quality settings. Preventive interventions for preschoolers are often aimed at those children most at risk for future problems (Orrell-Valente et al., 1999; Webster-Stratton & Hammond, 1998). As such, future implementation research should focus more closely on interventions for these at-risk children.
This study is one of the first to examine predictors of teacher participation in a classroom-based, integrated preventive intervention for preschoolers, and it raises important questions. Future research should begin to examine the interplay between contextual variables like center type and other center-level variables, teacher demographics and perceptions, child-level variables, frequency of consultation, teacher participation, and intervention outcome. Such knowledge will ultimately help in developing preschool classroom interventions that teachers will be better able to implement, which will, in turn, be more likely to result in the intended positive outcomes.
This project was supported by grant R01HD046126 (Kupersmidt), co-funded by the National Institute of Child Health and Human Development (NICHD), the Administration for Children and Families (ACF), the Office of the Assistant Secretary for Planning and Evaluation (ASPE), and the Office of Special Education and Rehabilitative Services (OSERS) in the U.S. Education Department.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.