|Home | About | Journals | Submit | Contact Us | Français|
The present study was designed to develop an innovative motivational intervention (based on Self-Determination Theory and Social Cognitive Theory) to increase physical activity (PA) in underserved adolescents. Sixty-four adolescents (35 females, 29 males; 50% minority; 65% on reduced lunch program; ages 11–13 yr) participated in either an 8-week motivational intervention after-school (n = 32) or a typical after-school program (n = 32). The conceptual framework for the intervention targeted the social environment (perceived autonomy, perceived social support, participation, fun), cognitive mediators (perceived choice, self-efficacy, and relatedness/belongingness), and motivational orientation (intrinsic motivation, commitment, positive self-concept). Formative evaluation data was collected by staff through daily forms throughout the 8-week program and through observational data completed by independent objective observers during 2 weeks of the program. The major themes that were identified addressed theoretical concepts regarding the intervention and logistical issues in delivering the intervention. The data revealed information regarding the importance of the cognitive appropriateness of the PA and motivational activities, the environmental climate for promoting nurturing relationships, developing specific strategies for increasing intrinsic rather than extrinsic reinforcement, and developing methods for preventing social “cliques” and gender conflicts to maintain an appropriate level of support in the social climate. Themes for training staff included focusing on team building, leadership, and nurturing. This formative evaluation is being used to formalize a randomized trial to test the effects of a student-centered motivational intervention on increasing PA in underserved 6th graders.
Previous studies have demonstrated that underserved adolescents, including minorities and those of low socioeconomic status (SES), are less physically active than adolescents who are non-minorities or of higher SES (Taylor & Sallis, 1997). Underserved adolescents are also more likely to engage in sedentary behaviors such as television watching than adolescents who are non-minorities or of higher SES (Sallis, Zakarian, Hovell, & Hofstetter, 1996). Thus, innovative interventions are needed for promoting PA in underserved adolescents who may be at greater risk for inactivity, obesity and other health-related complications.
The literature on school-based PA interventions has demonstrated only modest effects on increasing PA in adolescents (Baranowski, Anderson, & Carmack, 1998; Gortmaker et al., 1999; Kelder, Perry, & Klepp, 1993; McKenzie et al., 2004; Sallis et al., 2003). The Planet Health project (Gortmaker et al., 1999) evaluated a school-based Social Cognitive Theory (SCT) and behavioral choice intervention in 6th and 7th graders on increasing moderate-to-vigorous (MV) PA. There were no significant changes in PA due to the intervention, and only television viewing predicted changes in obesity prevalence. The Minnesota Heart Health Program was a quasi-experimental design where 6th graders participated either in a SCT intervention, designed to increase PA outside of school physical education classes, or no intervention (Kelder et al., 1993). After 7 years, girls were more likely to show an increase in PA than boys. In a recent review it was concluded that school-based interventions have demonstrated only modest changes in PA primarily during physical education classes (Baranowski et al., 1998).
A formative evaluation was conducted to develop a program titled “Active by Choice Today” (ACT). ACT is a large scale randomized trial (involving 24 schools) that will test a student-centered motivational intervention for increasing PA based on two theoretical models: SCT and Self-Determination Theory (SDT). Bandura's (1986) SCT conceptualizes individual-cognitive factors, environmental events, and behaviors as interacting and reciprocal determinants of each other (see Fig. 1). SDT (Ryan & Deci, 2000) proposes that behavior changes motivated by intrinsic factors such as novel, enjoyable, self-driven, and satisfying experiences, will sustain behavior more so than those behavior changes produced by extrinsic factors (external reward or coercion). The integrated conceptual framework for the ACT intervention targeted the social environment (perceived autonomy, perceived social support, participation, fun), cognitive mediators (perceived choice, self-efficacy, and relatedness/belongingness), and motivational orientation (intrinsic motivation, commitment, positive self-concept). This approach is consistent with previous research that documents the importance of choice and self-initiated behavior change on increasing intrinsic motivation for engaging in PA (Ferrer-Caja & Weiss, 2000; Thompson & Wankel, 1980; Wilson, Williams et al., 2005; Wilson, Evans et al., 2005).
In summary, school-based behavioral interventions in children and adolescents have resulted in only modest changes in PA levels in the school environment but have not had a long-term impact on PA outside of the school setting (Baranowski et al., 1998; Stone, McKenzie, Welk, & Booth, 1998). To increase the effectiveness of PA interventions, the present study integrated SCT and SDT to specifically effect change in the theoretically hypothesized mediators for increasing PA beyond the school setting. The purpose of the present study was to use formative evaluation through process evaluation strategies to finalize the intervention design for our large-scale randomized school-based trial. Specifically, the formative evaluation focused on identifying factors in the social environment and intervention curriculum that worked well and on elements of the program that needed improvement. It also allowed the intervention team the opportunity to test process evaluation measures to ensure they were accurately and completely capturing the conceptual components of the ACT intervention.
The study protocol was approved by the University of South Carolina institutional review board. All student participants and their parents provided informed consent prior to participating in the study. Using a quasi-experimental design, students enrolled in 6th grade (11–13 years) from two middle schools in a rural southeastern community were recruited with the assistance of school staff. At one school, 32 children volunteered to enroll in the student-centered motivational after-school PA program. Another middle school in this community (matched on race, gender distribution, age, and proportion on free or reduced lunch) was selected as a comparison group (n = 32) to determine the PA levels of adolescents who (participated in a health curriculum after-school program. Table 1 depicts the baseline characteristics for both the intervention and comparison participants. The participation rate at both schools was approximately 55% throughout the 8-week program.
Participants provided background and demographic information. Height and weight measurements were obtained by a trained assistant. Students at both the intervention and comparison schools participated in an 8-week program. The goal for the student-centered intervention was to increase MVPA to 60 min per day based on the guidelines for PA standards in adolescents (Sallis & Patrick, 1994). Adolescents in the student-centered program took ownership in developing the program (name, choices of activities, and ideas for promoting activities to peers), selecting a variety of physical activities that were fun and interesting, and in generating their own positive coping strategies for making effective PA behavior changes. The participants in the comparison group received an 8-week program of general health education during an after-school program.
The methodology for the student-centered motivational intervention has been previously described (Wilson, Evans et al., 2005). Briefly, the student-centered intervention was implemented on Mondays, Tuesdays, and Thursdays for 2 h after school. Four trained staff provided oversight for the program and one staff specifically trained in physical activity and injury prevention provided structure for the PA elements of the program. The program had 3 main components: homework/snack component (30 min), a PA component that included activities which the students selected each week of moderate and vigorous intensity (60 min), and a SCT and motivational component during which trained graduate students taught participants behavioral skills and motivational strategies to increase their PA with friends and at home (30 min).
The intervention was consistent with SDT and SCT Theory in that it emphasized increasing intrinsic motivation and behavioral skills for PA. Strategies to teach specific SCT behavioral skills included self-monitoring, goal setting, and developing strategies for engaging in PA with friends and family. The student-centered intervention included two elements consistent with SDT (1) providing students the opportunity to develop positive coping strategies for making lifestyle changes in overall PA using a strategic self-presentation (videotaped interview), and (2) providing students the opportunity to participate in program development by selecting a variety of the physical activities offered weekly, developing a program name and motto, and by developing ideas for promoting PA to friends and peers. Each week the students generated a list of physical activities that they wanted to engage in for the following week and the entire group voted on the top two choices.
Participants in the student-centered intervention participated in a strategic self-presentation videotape—a methodology that has been previously described in detail (Eitel & Friend, 1999; Wilson et al., 2002). Briefly, this motivational approach facilitates high levels of personal involvement through strategic self-presentation that enhances motivation and self-concept for PA by encouraging adolescents to develop their own strategies for improving PA habits. Participants were informed that their task was to advise other students on how to handle problem situations in a videotaped interview portraying their own positive coping strategies. To induce self-presentation processes students were asked to focus on the positive things they did to reach their daily PA goal and to focus on how they managed to overcome challenges in changing their PA habits. To induce commitment to their public behavior, students were (1) videotaped during an interview session, (2) viewed the videotapes during their next session and (3) were given the opportunity to revise their videotapes until they approved of the quality of their presentation.
The intervention team went through a six-step process to refine and operationalize the ACT intervention curriculum for the large-scale trial. These six steps are outlined below. Steps 1–3 were conducted iteratively (not a linear process) with each step being revisited several times. The steps are outlined below (see also Fig. 1):
A review of the final reports generated from the pilot study process evaluation yielded the following themes regarding program design and implementation.
The pilot study process evaluation produced several lessons regarding program strengths and areas in need of revision prior to full-scale implementation. These lessons are presented as themes and grouped into categories. While some of the lessons revealed completely new issues, others simply emphasized how to better implement program components with fidelity.
During the pilot study we discovered that many of our SCT/motivational skills, small group activities were too cognitively advanced for the majority of the students in the program. For example, we learned that many students struggled with some of our planning and tracking activities. As a result we have revised the curriculum to make these activities and discussion more focused and time-limited by working to develop skills for the present and very short-term future.
During the study we also learned that all of the behavioral skills and group discussions must be interactive for all of the participants. The students enjoyed playing the physical activity games, and did not want to stop playing to complete other activities such as homework. By making the SCT/motivational skill-building segment of the program more interactive through role playing and use of experiential education techniques (that often just felt like playing another game to the students) with debriefings we were better able to maintain the students' interest and participation. The curriculum was revised to include more interactive behavioral skill-building activities in small groups.
We found that students wanted some familiarity, routine and structure but also quickly became bored with the repeated activities. Students were also able to assess skill level in playing new physical activity games after several times of playing. Once this occurred all of the patterns of favoritism, exclusion, etc. that occurred with more traditional sports games would begin to appear with the new game. During the pilot study we introduced the students to 6 new games over the course of 8 weeks. However, more activities could be introduced and for the ACT trial students will be exposed to 5 new games every 4 weeks and the SCT/motivational skills will be addressed through a much wider variety of activities.
The intervention is student-centered, meaning that students have many opportunities to provide input and have choice in the activities they do each day. During the pilot study we found that when this method, if used, too often lost its meaning. Therefore, in the ACT trial we will use several different methods for students to provide feedback and have choice regarding program activities. This will begin on the very first day with the students developing their ground rules. It will continue on a daily basis with students having choice in what physical activity they want to do each day also students will have a way to choose the games they want to play in the next week. This will be done through a variety of different voting activities. In addition, staff will continually ask students about the games and what they did and/or did not like about them.
The student-centered motivational intervention emphasizes a social environment that is positive and nurturing for students. During the pilot study we discovered that because staff made such an effort to be positive and nurturing, they did not put enough emphasis on also maintaining discipline. In the beginning of the pilot study ground rules were established; however, consequences were not clear and not consistently enforced. This was counterproductive to much of the other work that the staff were doing to create a positive nurturing environment for the students. To address these findings the ACT trial will include 2 h of training devoted to constructive discipline. This training will include role-playing and simulations so that the staff will have an opportunity to experience methods of positive discipline. In addition, the curriculum has been revised to include more time dedicated to establishing ground rules and consequences at the very beginning of the program. Lastly, a step-by-step set of consequences has been established for anyone violating a group rule. The first step will be a warning, the second step will be placement in a staffed “time-out” area for 15 min, the third step will be removal from the program for the remainder of the day and the final step will be removal from the program completely.
Central to SDT is the concept of intrinsic motivation; doing an activity because you want to do it for yourself not for an external reward. During the pilot study we learned that some staff, particularly staff with prior experience in coaching or working with student recreation programs, had a tendency to reward students with small trinkets for either performance or participation. In addition, staff had a tendency to praise students verbally for performance. While in many programs this is a perfectly acceptable practice, it does not comply with the intervention emphasis on intrinsic motivation. During staff training more time will be dedicated to explaining the concept of intrinsic motivation, and helping staff to identify new methods for offering encouragement for team effort not individual performance. In addition, staff training will address the importance of emphasizing and praising students for participation, not performance, and for ensuring that all students get an equal amount of attention.
Positive student-to-student interactions have been an underlying and important characteristics of the student-centered motivational intervention since its inception. This is an essential ingredient to the program being fun for all students. In addition, many of the behavioral skills addressed in the intervention to assist students with being more active when they are not in the program build on positive coping skills for addressing environmental and social barriers to being physically active. During the pilot study we learned several new methods for how to best facilitate student interactions so that they are positive and productive.
During the pilot study staff began to notice that students would have many subtle positive student-to-student interactions. If students were praised too publicly or frequently by staff for their behavior it would reduce the frequency of positive student-to-student interactions. We addressed this delicate balance and the desire to praise a student for positive behavior by suggesting that staff give praise in more subtle ways and if possible in a setting where others would not be as aware of it.
There are cliques established among students in this age group (exclusive groups of students who do not allow other students to be in there group). We discovered that staff members need to be aware of the cliques from the very first day and have several different methods for grouping students in activities as a way to diminish the effects of the cliques and to force students to interact with other students outside their clique. During the pilot study, when staff were too explicit about trying to dismantle a clique the students got upset and resented having to be in different groups. However, when the staff grouped students using random or what appeared to be random methods then student complaints were less frequent as well as mild and short lived. Methods for recognizing cliques as well as how to subtly get students to do activities with other students outside of the clique will be addressed in staff training. There is also a section in staff training devoted to student to-student interactions and specifically to cliques, bullying, students being left out, etc. On a side note—in this pilot study we did not have any recognized incidences of bullying but the process of doing the pilot study, observing cliques and working to find ways to deal with cliques made us more keenly aware of the need to address bullying.
One-way staff members were to reinforce an emphasis on participation over mastery was by praising students for participation instead of praising students for performance. This proved to be more difficult than was originally thought. Staff often found themselves reacting to a students' performance on a shot, block, steal, hit, etc. Staff found themselves not cheering for students' actions because they were worried it might be promoting mastery. After the pilot study observations, the staff developed a series of phrases and cheers used to promote participation. While it took practice, with help from each other, the staff improved in this area.
The pilot study also identified the need to develop strategies for dealing with gender conflicts. During the pilot study we realized that having boys and girls together during homework and group time provided too many opportunities for social interactions that were counter productive to the social environment we were aiming for with the intervention. During this time if the boys and girls were in groups together much of their energy was spent on flirting and students were not as open to participating in the group activities and discussions. Students were also more reserved about sharing details of their life in mixed gender groups.
During physical activity we also found that there were some activities that one gender preferred over the other. For familiar sport-like games, such a basketball, soccer, football, we found that many of the girls were not very active in the mixed gender setting. We also found that when students were playing a new game in which they had little or no experience, they enjoyed playing in mixed gender groups and most of the students continued to be active in the mixed gender group. During the intervention phase of the project students will be grouped into single-gender small groups for the majority of the behavioral skills and group discussion activities and during all of their homework time.
During the pilot study it also became clear that staff were going to need to be supportive and able to help each other implement the program as it is designed. It is not enough for staff to have feelings of camaraderie they must also have a sense of shared responsibility. We found that when the staff worked as a team and shared responsibility, along with knowing their roles and how to perform their role, the staff's excitement about the program was infectious.
This formative work allowed the intervention team the opportunity to work out many logistical issues related to program delivery. Logistical issues experienced during the pilot study were many and varied. However, we are going to highlight the three most common logistical issues.
Even with the planning and good communication between the project staff and school there were several times when at the last minute we lost access to the space needed for our activity. It required last minute scrambling that was stressful to staff and confusing to students. From this we learned the necessity of always having viable alternative activities and equipment on hand to carry out these activities. The activities need to be flexible enough to play in large or small spaces and with many or just a few students. This will be addressed in staff training for the ACT trial.
During the pilot study we learned that homework time is better at the end of the day as opposed to right after school. The students were to wound up from being in school all day to sit still for homework as the first component of the program each day. After about 3 weeks of the pilot study we switched homework to the end of the day and it became much more productive with far fewer discipline issues.
Prior to implementing the pilot study we were cautioned that transition times between activities are important. During the pilot study we learned several methods for shepherding students from one activity to the other while keeping them moving at a fairly brisk pace and maintaining order and safety. This required at least two staff members with each small group, one staff to stay at the location of the activity awaiting the next group and one to walk with students to the next activity. In addition an extra staff member to serve as a general hall and bathroom monitor was helpful.
During the pilot study we also were able to add more clarity and depth to our list of desired characteristics for program staff. These included: (1) Flexibility—someone who is able to adjust to change and find ways to make these changes work to their advantage; (2) Loves spending time with students—someone who enjoys being around adolescents and being active with them; (3) Tolerant of chaos-ability to handle noise and chaos and see order within the chaos; (4) Leader—someone who can take charge of a situation and not just wait to be told what to do; (5) Smart—someone who understands when the students are trying to provoke them and does not allow it to happen and also addresses the provoking in a positive manner. These are the five key characteristics that we use in hiring staff or the ACT trial.
Table 4 shows the results of the process evaluation by intervention versus comparison schools. The major elements of the intervention that were implemented successfully in the intervention school included engaging in moderate to vigorous physical activity, having a choice of physical activities that the students selected, and emphasizing optimal challenge where team participation rather than winning (or competition) was the key focus of the activities for students. Clarity of rules and expectations was slightly better in the comparison school than in the intervention school, perhaps because the intervention was more complex than the programs offered in the comparison school. Interestingly, the level of relatedness and sense of belonging were quite high and similar across the two schools.
The process evaluation of the ACT pilot intervention served as formative evaluation for developing the intervention for the large-scale ACT randomized trial to test the effects of a student-center motivational intervention on increasing PA in underserved 6th graders. It allowed us the opportunity to identify with more clarity exactly how the various components of the conceptual model should be defined; how to better implement these components within the intervention; and how to more accurately ensure that we are implementing the program with fidelity to the conceptual model and underlying theories.
This formative work did not prompt the investigative team to revisit the building blocks of the conceptual framework or logic model. However, it did provide a wealth of information regarding how to better operationalize the various components of the intervention model for this project targeting 6th grade adolescents in an after-school setting. In addition, it provided the information necessary to more clearly define program fidelity so that it can be observed, measured and tracked and to continue to ensure the program is being delivered as designed. Lastly, the formative evaluation provided a wealth of information that will be used in staff training. It identified training needs and provided the investigative team a springboard to use for discussing staff structure, characteristics and training components. It allowed the investigative team an opportunity to work out many of the details that will help to enhance the success of implementing the intervention program as part of the ACT randomized school-based trial.
Other investigators have also used formative process evaluation to better understand the strengths and limitations of interventions. For example, Pate et al. (2003) tested the effects of a community-based PA intervention designed to increase PA and conducted extensive process evaluation of the intervention. The process evaluation demonstrated that the after-school and summer PA program was implemented as planned. However, the home, school, and community components of the program were not implemented as planned because of time limitations and lack of resources. An advantage in the present study was that we could learn from the pilot data how the program would need to be modified in order to increase the likelihood that implementation would be successful.
In summary, this study provides a framework for formative evaluation that identified both theoretical concepts regarding the intervention as well as logistical issues in delivering the intervention. This approach may help future investigators refine their interventions before large-scale implementation. Further research on the benefits of formative evaluation is needed to enhance our understanding of the effectiveness of theoretically based interventions to promote physical activity in youth.
This article was supported by a Grant (R01 HD 045693) funded by the National Institutes of Child Health and Human Development to Dawn K. Wilson, PhD. Send reprint requests to Dawn K. Wilson, PhD. Department of Psychology, Barnwell College, University of South Carolina, Columbia, SC 29208, USA