|Home | About | Journals | Submit | Contact Us | Français|
The purpose of this study was to evaluate ACTive: Athletic Concussion Training using Interactive Video Education, an interactive e-learning program designed to train community coaches of youth ages 10–18 in effective sports concussion prevention and management practices. Seventy-five youth sports coaches from across the country completed the study over the Internet. Results of a randomized control trial demonstrated significant differences between treatment and control participants on measures of (a) knowledge about sports concussion, management, and prevention; (b) attitudes about the importance of preventing sports concussion; and (c) intention and self-efficacy in sports concussion management and prevention. The results suggest that ACTive is an effective method of training youth sports coaches who are in an important position to reduce risks associated with sports concussion.
Approximately 1.6–3.8 million sports and recreation-related traumatic brain injuries occur in the United States each year.1 Statistics show the highest rates of emergency department visits for sports concussion occurring for youth ages 10 to14, followed by those ages 15 to19,2 although many believe the figure is much higher due to significant under-reporting by youth athletes.3 Once considered little more than a “ding” or a “bell-ringer,” concussions are now known to have potentially harmful cumulative effects resulting in long-term changes in brain function.3-5 The common symptoms of concussion include headache, confusion, disorientation, decreased information processing, loss of equilibrium, and visual or auditory disruptions.6
Young athletes appear to be particularly vulnerable to the effects of concussion.7-8 In terms of cognition, adolescents with mild brain injuries are more likely than similarly injured college athletes to experience memory and attention problems after the usual 7–10 day recovery period.9-10 There is also the relatively rare but catastrophic occurrence of second impact syndrome (SIS), which occurs when an athlete who is still recovering from a concussion sustains a subsequent brain injury. This results in rapid swelling of the brain and almost always results in death or severe long-term injury.11-12 As SIS has been primarily reported in athletes 13–18 years of age13 the importance of proper recognition and management of concussed young athletes cannot be over-emphasized.
Each year an estimated 7.5 million youth participate in high school sports14 and 38 million children participate in organized, community-supported athletic programs annually.2, 15 Coaches are in an important position to identify the typical behavioral and physical signs of concussion and to mitigate the risks associated with concussion in young athletes. Like the general public,16 coaches have misconceptions about concussion.17 However, with adequate training, they are able to recognize subtle symptoms and patterns of effects that can point to serious problems.17-18
Recently, a comprehensive public health information program was initiated by the Centers for Disease Control with their “Heads Up” concussion information kit for high school sports.19 This free packet contains information for coaches, parents, and physicians in both printed and linear video format. Initial data from focus groups and a survey suggested that use of the toolkit was associated with gains in knowledge and awareness of the severity of concussions and resulted in increased efforts to minimize the risk of concussion.2 Similar results have been reported with the ThinkFirst Canada training for hockey coaches.20 A series of studies in New Zealand have demonstrated that systematic training for rugby coaches is associated with gains in knowledge and a decrease in injury rates.21-23
The “Heads Up” program was widely distributed to youth and high school coaches across the US when first released. However, anecdotal evidence based upon the authors’ conversations with coaches and athletic administrators suggests that the material was not widely utilized. Unfortunately, such passive educational material is not always widely adopted.
The Internet has been shown to be an effective medium for providing training and promoting behavior change.24-27 Training offered over the Internet is both cost-effective and easy to deliver.28-29 On-demand learning allows users to access training on their own schedule, ideal for youth sports coaches who are often volunteers. Interactive Multimedia (IMM), which links video with computers,26, 30 is a promising approach to providing individually tailored video-based training when delivered over the Internet.31-32 The branching capabilities of IMM allow the presentation of material to be tailored and/or remediated specifically to the user (e.g., Marcus et al.33) and messages and modeling (e.g., via video) can be made more effective as they are tailored to the audience. 30, 32, 34
The purpose of this study was to evaluate an e-learning program on sports concussion for youth sports coaches. ACTive: Athletic Concussion Training using Interactive Video Education, is an interactive multimedia (IMM) program designed to train community coaches of youth age 10–18 on effective sports concussion prevention and management practices. The evaluation sought to determine whether the training program, when compared to a control group, had measurable effects in (a) knowledge about sports concussion, management, and prevention; (b) attitudes about the importance of preventing sports concussion; and (c) intention and self-efficacy in sports concussion management and prevention.
The study was conducted over the Internet with a sample of youth sports coaches throughout the country. Participants were recruited nationally via youth sports organizations’ websites (e.g. Boston Youth Sports Initiative). A total of 142 coaches completed the screening instrument. Seventy-five of those coaches (53%) qualified for (because they were currently coaching youth ages 10–14) and completed all assessments. Participants in the study included 52 males and 23 females. Nearly 75% (n = 56) were between the ages of 30–49. Reports of ethnicity indicated that the sample was 79% Caucasian, 5% African American (n = 4), 5% Asian (n = 4), 4% Hispanic (n = 3), 4% Mixed Race (n = 3), 1% American Indian or Alaska Native (n = 1), 1% Native Hawaiian or Pacific Islander (n = 1), and three participants did not report. Eighty-nine percent of the sample had attended college, had completed a bachelor's degree, or had some postgraduate education. Slightly less than half (n = 35) of the study participants reported that one or more of their players had sustained a concussion in the previous season. The study was approved through the research approval process, and participants provided online informed consent.
The ACTive e-learning program consists of 3 short modules covering information about youth sports concussion, prevention, and recognition and management. The program relies on simple graphics and video segments with easy-to-follow navigation controls. The user is guided through the modules sequentially (a “tunnel” experience), although they are free to repeat modules from previously covered material. The tunnel information architecture design is particularly appropriate for a program that guides participants through a series of steps that build upon each other in a logical manner.35
The conceptual framework underlying the training is the Health Belief Model,36-37 which suggests that individuals will be more likely to take action if they perceive greater risk, if the danger seems serious, and if they believe that there will be positive consequences and few negative aspects to engaging in the action. Consistent with the Expanded Theory of Reasoned Action38-42 messages in the ACTive program are delivered by referent others: a sports medicine physician, a former NFL quarterback who had suffered multiple concussions, a professional high school coach, and a volunteer youth sports coach. Each narrator focuses on key elements of concussion management. Critical to the training are application exercises in which the user responds to sample athletic scenarios involving a possible concussion (see Figure 1).
Users receive immediate feedback on the application exercises. The training also includes frequent interactive quizzes to reinforce key teaching points and reference materials on concussion management.
Content for the ACTive program was based on recommendations of the National Athletic Trainers’ Association and the International Conference on Concussion in Sport.43-44 Specific content and design elements were modified based on input from focus groups with youth sports coaches and interviews with a range of physicians, neuropsychologists, public health professionals and certified athletic trainers, representing different athletic and community contexts. Information from focus groups and interviews were transcribed, coded, and reviewed. Concerns from the various groups were prioritized to create the core content areas of the program. Findings from focus groups and interviews suggested that the training should: (a) stress the seriousness of concussion, especially in young people, (b) teach coaches that they can't be too conservative in removing a child from play and that the ramifications for concussion in youth are long term, (c) teach coaches to recognize, not diagnose concussion, and (d) use brief video segments and professional athletes to communicate key messages. Following the recommendations of U.S. Department of Health and Human Services45 (www.usability.gov) and Bailey,46 we tested program function, effectiveness, efficiency, and user satisfaction. The iterative development process involved usability testing with 20 individuals and multiple revisions. The final version of the program was delivered via the Internet, and users were issued unique logins and passwords.
A random control design was employed to evaluate the program. Once they completed the online screening, participants were sent an e-mail invitation that included their ID number and a link to an online Informed Consent. If they consented, they were linked to the pre-test (T1). Once the T1 was complete, they were randomly assigned to either treatment or control. Those in the treatment group received a link to the program, which included a link to the post-test (T2). Control subjects received a link to CDC materials on bicycle and pedestrian safety and were instructed to spend 15-20 minutes reading them (equivalent time to the intervention program). They were also sent a link to the T2 and told to complete the survey immediately after viewing the safety materials. When T2 was complete, the participant was sent a check for $75.
The evaluation questionnaire items were based on the constructs of social cognitive theory,41-42, 47 the health belief model,36 and the theory of reasoned action.38-40 Items assessed knowledge, attitude, self-efficacy, and behavioral intention. In addition, program usability and user satisfaction were assessed.
Knowledge items included a 16-item symptom checklist in which the participant identifies concussion symptoms (e.g., blurred vision, dizziness) by marking yes/no.
General sports concussion knowledge was measured with 23 true/false items (e.g., “an athlete who reports having a headache after a concussion will likely demonstrate other signs”).
The third set of knowledge items assessed the participant's reaction to common misperceptions about concussion. The participant rated 6 common misperceptions (e.g., “It is easy to tell if a person has brain damage from a head injury by the way the person looks and acts”) on a 4-point scale (true, probably true, probably false, false).
Items in the Guilmette et al48 measure were taken from surveys administered in three previous studies.16, 49-50 Information about the psychometric properties of these items is not available. McLeod et al17 report strong internal consistency (Cronbach's alpha = 0.83). Test-retest reliability of the knowledge measures using control participants’ pre- and post-tests was satisfactory: knowledge of symptoms (r = .82); general knowledge (r = .68); knowledge misperceptions (r = .48).
Five scenario-based questions assessed coach behavioral intention and self-efficacy to respond appropriately to each scenario using a Likert scale (1–5). A sample scenario is presented below.
Jared shows up to lacrosse practice 15 minutes late. When the coach asks why he is late, Jared tells him that he hit a rock on the bike path and fell off his bike. Jared says he can still practice but his neck hurts.
The user was first asked to rate how confident they were that could handle a situation such as the one presented (self-efficacy). Responses were rated on a 5-point Likert scale ranging from 1 (not at all confident) to 5 (very confident). A scale score for each individual was constructed by averaging the items (alpha = .89). They were next asked how likely they would be to keep a player out of the practice in a similar situation (behavioral intention). Again, a scale score for each individual was constructed by averaging the items (alpha = .67).
Participants in the intervention conditions were asked to rate their satisfaction with the program on a 7-point Likert scale. Eight satisfaction items, using agree-disagree statements with 5-point Likert scales, were adapted from Internet evaluation instruments (e.g., Chambers et al.,52 Vandelanotte et al.53). The items elicited responses about the program's functionality, credibility, and usability.
Participants in the two conditions did not differ significantly from each other on any of the pre-test characteristics or measures. An overall multivariate analysis of variance model was tested for five post-test measures, controlling for pre-test levels, followed by univariate analysis of variance (ANCOVA) models. The multivariate test was significant in that coaches in the intervention group were found to have significant and large overall gains compared to coaches in the control condition, F (5, 63) = 14.51, p < .001, partial eta-square = .54. The descriptive statistics and univariate ANCOVA results are presented in Table 1. As can be seen, the Treatment group differed significantly from the Control participants on all five of the post-test measures. The greatest gains were obtained for knowledge about concussion symptoms (eta-square = .46, large effect size); followed by general knowledge about concussions (eta-square = .37, large effect size); knowledge of misperceptions about concussion (eta-square = .12; medium effect size); self-efficacy regarding perceived self-confidence about taking appropriate action based on the five scenarios presented (eta-square = .29; large effect size); and intention to take action based on the five scenarios presented (eta-square = .17, large effect size). Thus, significant and medium-to-large effects were obtained on all five outcome measures.
Usability measures indicated a favorable response from participants. Participants in the ACTive group were asked to agree or disagree with statements by responding on a 5-point scale (1 = strongly disagree; 5 = strongly agree). Responses were very positive, with participants rating the website high on interest (M = 4.42), ease of use (M = 4.45), and navigation (M = 4.2).
On a 7-point scale (1 = not at all; 7 = extremely positive), users in the Treatment group reported the ACTive program to be helpful (M = 6.13) and enjoyable (M = 5.73). Most coaches said that they would recommend it (M = 6.08) to others.
The evaluation instrument provided users with an opportunity to offer typed-in comments on ways to improve the program. A total of 35 out of the 40 Treatment subjects offered opinions. Comments were overwhelmingly positive. Over half (n = 18) reported that they had learned a great deal. One user wrote, “I learned a lot of information I did not know. It was presented in a non-threatening personal point of view. It made me wonder if I have handled things in the past correctly, but now I will definitely do things the correct way in the future.” Another commented, “The program reinforced some things I already knew, but I also learned very much and will never feel bad about making the decision to remove a player if I suspect a concussion.”
The purpose of this evaluation was to document whether the ACTive program, when compared to a control group, had effects on coaches’ knowledge, attitudes, intention and self efficacy in sports concussion management and prevention. The results of the randomized trial indicate that the participants who viewed the ACTive program showed significantly greater improvement than those who viewed the CDC safety materials in (a) their knowledge of concussion symptoms (large effect size) and general knowledge of concussion (medium effect size), (b) their self-efficacy regarding recommended actions following concussions like those presented in sample scenarios (large effect size), (c) their intention to take appropriate action in situations like those presented in the scenarios (large effect size), and (d) their attitudes about brain injury (medium effect size). Overall, improvement on these four sets of items suggests that the training can have a measurable impact on coaches’ understanding of how to prevent and manage sports concussion.
A number of limitations exist for this initial study. The small sample size limits generalization of our results to other youth athletic coaches across the United States. External validity is also limited by a fairly homogenous sample in terms of ethnocultural diversity, making it difficult to determine differential effects of the program on individuals from other backgrounds. An additional sampling limitation is that the majority of the coaches participating in the study were well educated. While our sample size was not sufficient to analyze the effect of educational level, it is possible that coaches with less education or familiarity with computers may not derive similar benefits from participation.
Our initial findings provide much fertile ground for future research. Expanding the measures to include behavioral assessments would yield important information about skill application in real life contexts. Our data showed that coaches could identify concussion symptoms and apply the knowledge they gained to sample scenarios. Following the Theory of Reasoned Action, this would predict the application of skills in real life situations. However, we cannot determine the extent to which coaches will use the skills in game or practice situations. Future research should assess the impact of the training on injury rates and sports concussion management practices in athletics. For example, injury logs, which have been used to effectively capture injury rates in home55-56 and sports contexts,57 could be utilized to validly assess injury rates in sports setting when recorded by athletic trainers, coaches, or school nurses. In addition, future studies could examine the durability of these findings to identify whether changes in knowledge, attitude, self-efficacy and intention maintain over time.
The potential short-term and long-term ramifications of concussions in youth and high school sports are becoming increasingly evident. Given the number of children and adolescents participating in organized sports who are at-risk for these injuries, and the fact that most coaches in community sports programs are volunteers who receive no formal training,58 there is a critical need to educate athletic coaches about the seriousness of concussion. Any such educational program must also provide training in the initial management of concussion in practice and game situations.59 E-Learning is a promising approach for providing this training to coaches. Used in conjunction with education for athletes60-61 and the larger athletic and school community (e.g., parents, teachers, and school administrators),62-63 this training has the potential to minimize the risks associated with sports concussion in youth and high school athletics.
This work was funded by grant #1R43HD56610-01 from the National Institute of Child Health & Human Development. The authors thank Chris Miller, Will Doolittle, Craig Walker, and the technology team at Oregon Center for Applied Science, led by Beth Machamer.
Ann Glang, Oregon Center for Applied Science, Inc. 260 East 11th Avenue, Eugene, Oregon 97401 ; Email: aglang/at/orcasinc.com.
Michael C. Koester, Slocum Center for Orthopedics and Sports Medicine 55 Coburg Road, Eugene, Oregon 97401 ; Email: mkoester/at/slocumcenter.com.
Sherry Vondy Beaver, Oregon Center for Applied Science, Inc. 260 East 11th Avenue, Eugene, Oregon 97401 ; Email: sbeaver/at/orcasinc.com.
Janet E. Clay, Oregon Center for Applied Science, Inc. 260 East 11th Avenue, Eugene, Oregon 97401 ; Email: janeteclay/at/gmail.com.
Karen A. McLaughlin, Oregon Center for Applied Science, Inc. 260 East 11th Avenue, Eugene, Oregon 97401 ; Email: kdurany/at/uoregon.edu.