PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Crisis. Author manuscript; available in PMC 2011 January 1.
Published in final edited form as:
PMCID: PMC2913886
NIHMSID: NIHMS217918

Does a brief suicide prevention gatekeeper training program enhance observed skills?

Wendi Cross, Ph.D.,1,2,3 Monica M. Matthieu, Ph.D., LCSW,4 DeQuincy Lezine, Ph.D.,1 and Kerry L. Knox, Ph.D.1,3,5

Abstract

Background

Suicide is a significant public health problem worldwide that requires evidence-based prevention efforts. One approach to prevention is gatekeeper training. Gatekeeper training programs for community members have demonstrated positive changes in knowledge and attitudes about suicide. Changes in gatekeeper skills have not been well established.

Aims

To assess and predict the impact of a brief, gatekeeper training on community members’ observed skills.

Methods

Participants in a community gatekeeper training were employees at US universities. 50 participants were randomly selected for skills assessment and videotaped interacting with a standardized actor prior to and following training. Tapes were reliability rated for general and suicide-specific skills.

Results

Gatekeeper skills increased from pre- to posttest: 10% of participants met criteria for acceptable gatekeeper skills before training, while 54% met criteria after training. Pretraining variables did not predict increased skills.

Limitations

Results do not provide conclusions about the relationship between observed gatekeeper skills and actual use of those skills in the future.

Conclusions

Gatekeeper training enhances suicide-specific skills for the majority of participants. Other strategies, such as behavioral rehearsal, may be necessary to enhance skills in the remaining participants.

Introduction

Suicide is one of the leading causes of death worldwide. In the United States, annual rates of deaths from suicide are greater than for homicide (Goldsmith, Pellmar, Klienman, & Bunney, 2002). A variety of approaches have been identified for prevention efforts including gatekeeper training (CDC, 1992). Gatekeeper training programs teach community members to identify signs of depression and other behaviors that put individuals at heightened risk for suicide (CDC, 1992; Gould & Kramer, 2001; Mann, Apter, Bertolote, Beautrais, Currier, Haas, et al., 2005). Trainees may include police officers, teachers, coaches, co-workers, or other key stakeholders in the community who are in a natural position to carry out informal surveillance, detection, and assistance for those in need. While gatekeeper training programs vary in terms of duration and comprehensiveness, the fundamental goal is to enhance participants’ knowledge, attitudes and skills to identify individuals at risk for suicide and refer them to services.

Several uncontrolled and controlled studies have demonstrated changes in knowledge and attitudes following gatekeeper training (Cross, Matthieu, Cerel, & Knox, 2007; Goldsmith et al., 2002; Grossman & Kruesi, 2000; Lezine, Matthieu, Cross, Bajorska, Conwell, & Knox, 2009; Keller, Schut, Puddy, Williams, Stephens et al. 2009; Matthieu, Cross, Batres, Flora & Knox, 2008; Wyman, Brown, Inman, Cross, Schmeelk-Cone, Jing, & Pena, 2008). At present, however, there is limited evidence for enhanced gatekeeper behavioral skills as a result of brief community level training (Cross et al., 2007). Given the importance of building an evidence-base for prevention efforts to reduce deaths by suicide (Marušic, 2008), examination of changes in observed gatekeeper skills is necessary to understand the impact of the training. One reason for the lack of evidence for skill enhancement is the methodological and measurement challenges associated with assessing behavior. Some studies have used paper and pencil self-reports to assess skills but these survey methods fail to assess the actual behaviors that are central to demonstrating skills (Palmieri, Forghieri, Ferrari, Pingani, Coppola, et al., 2008; Stuart, Waalen, & Haelstromm, 2003). Observational methods are preferred for assessing behaviors and have been used to measure skills in a variety of interventions, including suicide prevention programs (Cross et al., 2007; Snyder, Reid, Stoolmiller, Howe, Brown, et al., 2006; Tierney, 1994). User-friendly, brief, and reliable observational measures of skills are integral to learning about behavior change associated with training. Moreover, given the proliferation of gatekeeper training for suicide prevention, research on knowledge, attitudes and observed skills is critical to ascertaining proximal as well as longitudinal training outcomes.

One function of training is to reduce the variance of knowledge acquisition and skills among trainees. Nevertheless, there is evidence that individual characteristics, such as previous educational experience or exposure to a particular subject may mitigate training effects. For example, a recent dissemination trial that trained clinicians to treat substance abuse with cognitive behavioral therapy (CBT) found that trainees who were in recovery themselves benefited more from the intensive training condition that included supervision with expert feedback, than other training conditions (Sholomskas, Syracuse-Siewert, Rounsaville, Ball, Nuro, et al., 2005). Wyman and colleagues (2008) also found that pre-training characteristics were associated with outcomes among gatekeeper-trained school personnel. Specifically, school staff who reported closer communication with students at baseline asked more students about suicide at one year follow up than those who did not. Previous experience with suicide training and exposure to suicidal individuals are two pre-training characteristics that may impact learning outcomes in the context of a brief, community-level suicide prevention training program.

Personality characteristics have also been proposed as a factor that influences training outcomes. Openness to Experience (OTE) is one of the Big Five personality factors as measured by the NEO-Five Factor Inventory (NEO-FFI; Costa & McCrae, 1992). OTE is defined as “tolerance for the unfamiliar, interest in ideas and problems, and appreciation of experiences involving actions, fantasy, values, feelings, and aesthetics (Tesch & Cameron, 1987, p. 617). Individuals who score higher on OTE items have been found to have positive attitudes toward new learning experiences, to be more likely to be active rather than passive in training, and excel in training that engages them in new approaches to learning (Gully, Payne, Koles, & Whiteman, 2002; Mount & Barrick, 1995). These studies suggest that psychologically ‘open’ individuals may demonstrate more positive learning outcomes, particularly for emotionally challenging topics such as suicide prevention and in the context of role play assessments.

In summary, examining the relationship between pre-training individual level characteristics and post training changes in the performance of newly acquired skills will contribute to knowledge about readiness of community members to benefit from training, and potentially, the selection of participants for cost-effective training.

Observational methods to assess skills

Observing behavior in ‘real world’ settings provides accurate information about the transfer of new behaviors from training programs to other contexts. For example, standardized patients have been used in medical research to assess actual provider skills in the primary care office following training (Epstein & Hundert, 2002; Humair & Cornuz, 2003; McDaniel, Beckman, Morse, Seaburn, Silberman, & Epstein, 2007). These field studies have high levels of external validity (results are generalizable) but are prohibitively costly and not feasible for brief community delivered interventions such as gatekeeper training for suicide prevention. Another observation strategy is to use analogue methods that employ simulations or standardized role plays to assess trainees’ skills. For example, analogue methods, using standardized role play scripts with a trained research assistant to interact with trainees, have been used in a variety of investigations to assess communication skills training in HIV and AIDS for dental consultants and motivational interviewing skills among therapists (Lewis, Brain, Cushing, Hall & Zakrzewska, 2000; Mounsey, Bovbjerg, White & Gazewood, 2006).

In suicide prevention programs, a few studies have used analogue methods to assess generic (e.g., empathy, warmth) as well as specific (e.g. identifying coping skills, exploring alternatives) helping skills among trainees (Neimeyer & Pfeiffer, 1994). Despite the advance over paper and pencil surveys, there are limitations to these observational methods used in suicide prevention studies including: 1) lack of inter-rater reliability for assessment of role play ratings; 2) lack of assessment of specific intervention skills; and, 3) lack of standardization of role plays (Stuart et al., 2003). In addition, few studies have rigorously assessed standardized actors’ adherence to a script. Standardized scripts that can be scored for adherence (and used to provide feedback to actors) prevent ‘drift’ in delivery across participant interactions and ensure consistent performances with the trainees (DeViva, 2006).

In one suicide prevention training program, workshop trainees participated in simulations (i.e., interacting with actors in role plays) before and after training (Tierney, 1994). The Suicide Intervention Protocol, a 17-item measure, was used to rate observations of general helping skills as well as specific skills related to suicide. There was no significant difference in general helping skills from pre- and post-training, but there was a significant difference from pre- to post- in suicide specific skills. This study has a number of strengths including inter-rater reliability for total scores on post-test observations, independent raters and the use of trained research assistants to enact the role of a suicidal individual. Limitations of this study include small sample size, lack of a standardized actor script, and thus it lacked fidelity ratings for actors’ adherence to the scripted role.

Gask and colleagues (Gask, Dixon, Morriss, Appleby & Green, 2006) also studied skills among participants of a 1 to 2 day suicide prevention program for clinicians and front line staff in mental health centers. Skills were rated at baseline and 4-month follow-up. Participants were videotaped while interacting for 15 minutes with a “suicidal” actor. There was no significant difference between baseline and 4-month follow-up on any observed skills. The authors suggest that there may have been a ceiling effect at baseline because participants were volunteer mental health professionals and may have been skillful prior to training. These findings may not be generalizable to community gatekeeper training which typically targets a lay audience with a brief intervention. In addition, because only a small subset of the sample agreed to participate in the video assessment of skills, there is high likelihood of sample bias. The study’s strengths (i.e., pre-post assessment, use of behavioral data and videotape) are offset by variable duration of intervention (participants attend from 1 to 2 days of training), lack of multiple raters and thus no measure of inter-rater reliability, and no measure of actor adherence to the role play scenario.

In another study, gatekeeper skills were assessed using analogue methodology with trained actors in a one-hour community level gatekeeper training program (Cross et al., 2007). Using an observational rating scale designed for the study, investigators rated two general abilities (empathy, active listening) and three suicide specific skills based on the training content (ability to question directly about suicide, to be persuasive about help seeking, and refer a suicidal individual for help). Brief (7–10 minute) videotaped interactions were conducted immediately post-training and independently scored. A total of fifty-five observations were rated. Inter-rater reliability was assessed and found to be within acceptable ranges. Just over half of the participants (55%) demonstrated satisfactory skills at post-training. Limitations of this study are: lack of pre-assessment of skills and lack of adherence rating for actors.

In sum, although the use of observational methods provides critical information about training outcomes, there are a variety of methodological issues essential to drawing conclusions about gatekeeper skills for suicide prevention based on analogue studies reviewed here. They include: a need for consistency in the design and use of observational rating scales including standardized scripts, attention to inter-rater reliability, and assessment of actor adherence to standardized scripts. Finally, studies of sensitive topics such as suicide require the judicious use of assessment time and effort to keep administrative and participant burden low. Observational assessment of a limited number of key items contributes to acceptability for participants and feasibility for researchers. In an effort to advance the field of suicide prevention, an important goal of the current study is to develop and to test a “user friendly” observational methodology that is scientifically rigorous to study behavior change in gatekeeper skills

Therefore, the aims of this study are to: 1) use a reliable observational measure of suicide prevention gatekeeper skills; 2) test the impact of a brief, community level gatekeeper training program by measuring pre-post changes in observable skills; and, 3) examine individual variables (e.g., personality, pre-training experience with suicide) as predictors of positive change in observed skills. Although not the central focus of the study, participants’ knowledge and attitudes (self-efficacy) were also assessed prior to and following training.

Method

Community Gatekeeper Training Program

We chose a relatively brief community-level gatekeeper training program for study. Standardized community gatekeeper suicide prevention training (QPR; Question, Persuade, Refer; Quinnett, 1995) was conducted by a certified QPR Institute, Inc. instructor (MM), to groups of 25–75 attendees. The one-hour gatekeeper training program consists of a lecture, a 10-minute introductory video, distribution of overview booklets and referral cards, and a question-and-answer discussion period (Quinnett, 1995). The lecture provides an overview on the epidemiology of suicide and current statistics, myths and misconceptions about suicide and suicide prevention, general warning signs for suicide, and three gatekeeper skills. A short video includes interviews with people who have been faced with the risk of suicide in their lives, families that have lost a loved one to suicide and others that serve in a clinical capacity (i.e., physician, school counselor). The booklet contains an overview of the didactic presentation and reviews the gatekeeper role. Referral cards are intended to function as wallet cards with prompts to recall gatekeeper skills emphasized in the training. Referral information was tailored for each site. For each specific type of referral (e.g., students, co-workers, friends/family) information regarding local resources by name and phone number was provided. Additionally, referral information to the national crisis hotline was provided. The final element of the training includes a question-and-answer period with the instructor.

Setting and Participants

This study occurred in the context of a larger suicide prevention study in a convenience sample of five universities in the northeastern part of the United States (see Lezine et al., 2009 for a full description of the larger study). We report findings on a randomly selected sub-sample of participants who consented to engage in skill assessments immediately before and after gatekeeper training, in addition to completing pre and post training surveys. Participants were employed by the universities in a variety of positions (e.g., campus /residence assistants, faculty, facilities workers, student affairs staff, and coaches). No compensation was provided for participation.

Campus groups were identified by representatives as key stakeholders to participate in a study of gatekeeper training. These representatives provided the study team with a list of all the participants who were invited to attend the training. Twenty (20) participants in four of the universities were selected using a random number program to enroll approximately 10 participants from each site. At one site, all 68 training participants were contacted about the opportunity to participate in the study because of time constraints so that 10 participants could be enrolled in a timely manner. The overall acceptance rate for participation was 33.8%. Respondents who indicated a desire to participate in the skills assessment study were scheduled to meet with the research team prior to the large group training to discuss the study protocol and participate in the informed consent process. Participants who were contacted but who did not respond to the invitation to participate were not re-contacted. A copy of the signed consent was provided to each participant. All of the participants who attended the informational meeting about the skill assessment chose to participate in the study.

Data Collection Procedures

All participants in the larger study completed self-report questionnaires before and after training (Lezine et al., 2009). The survey packet included items about socio-demographics, pre-training variables (e.g., OTE, previous experience), knowledge about suicide, and perceived efficacy for intervening with suicidal individuals.

Participants in the current study also met individually with a research team member who reviewed the study procedures and provided each participant with a brief, standardized “back story” to review which included the setting and details about the individual they were meeting (i.e., an actor portraying a co-worker, student). The participant was informed of the following: they would be accompanied to a room set up with a video camera and a waiting actor, they would begin the role play with a specific line provided in the back story, the study team member would be present throughout the interaction (i.e., off camera, away from the participant), the actor would signal the end of the role play (i.e., deliver a specific line) and the team member would escort the participant out of the room. Participants were instructed to converse naturally with the actor for 5 to 10 minutes and respond to the best of their ability within the context of their current job role. After the interaction, a brief informal assessment was conducted by a study team member who was a mental health professional to ensure the participant was not distressed and willing to continue with the training and participation in the research. This pre-training skills assessment took place up to 90 minutes prior to the large group gatekeeper training.

All participants in the observational study returned to complete the post-training skills assessment within 90 minutes of the completion of the training program. A second back story was provided to the trainee. Procedures were identical to the first skills assessment role play except that the participants met with an actor of a different gender and the study team member offered to provide brief feedback to the participant on their skills privately after the second role play. Feedback focused on the participants’ ability to directly ask about suicide and provide an appropriate, specific referral. Participants completed a post-training survey which included items specific to the acceptability of the role play experience. A research assistant debriefed the participant about the purpose of the study and answered any remaining questions. A crisis plan was in place for each school setting should a participant need services; no incidents warranted initiation of the protocol.

Standardized Actors

To increase the rigor of the role play for observational assessment, two research assistants (a male and a female) were individually trained for 6 hours to deliver scripted scenarios in an interactive manner during the role play procedure (Yardley-Matwiejczuk, 1997). A back story included a description of the actor’s role as a person at risk for suicide, the context for the interaction, and additional details on the history of risk factors prompting this suicidal crisis. The detailed back story was designed to assist the actor in developing character and to deliver the scenario interactively with appropriate affect. The scripted scenarios included a precipitating problem, multiple warning signs of suicide, and increasingly direct cues about needing intervention (e.g., feelings of hopelessness, plan to overdose). Scenarios were purposely designed to be simple due to the brevity of the interaction and at a moderate level of suicide risk (i.e., means available, suicide risk factors presented). Two scripted standardized scenarios were developed and matched for the level of suicide risk, number of warning signs of suicide, and number of cues about needing intervention. Actors met alternatively in either pre-or post-test with each research participant to reduce familiarity.

Data were collected from the five university sites from January to November 2006. Data collection procedures were the same for all participants regardless of the group or setting. Ethical approval was obtained from the Institutional Review Boards at the University of Rochester and each of the colleges that participated in the study.

Measures

Socio-demographics and pre-training variables

The baseline survey included socio-demographic items of age, gender, education, race, and job role. In addition, because we anticipated that pre-training experiences could have an effect on skill acquisition, we gathered self-report data on previous experience with suicide prevention training, exposure to suicide, and prior clinical interviewing experience.

Personality: Openness to Experience

Based on the literature that openness to new experiences and change may play a role in learning and skill development, we administered three facets (ideas, feelings, actions) of the Openness to Experience subscale of the NEO-FFI personality measure (Costa & McCrae, 1992) for a total of 24 items. Participants rate statements on a 5-point Likert scale (0 = Strongly Disagree to 4 = Strongly Agree). Internal consistency for the 24 items was high (Cronbach’s alpha = .83). Analyses used the total score for participants in this study.

Declarative Knowledge

Participants completed a 14-item assessment of declarative knowledge about suicide-related facts provided in the training before and after training (Cross et al, 2007; Wyman et al., 2008). Items included multiple choice and true/false questions. The scale score for knowledge is the sum of items answered correctly on the test.

Perceived Efficacy

As part of the larger study, all participants were asked to respond to 10 items about their perceived knowledge and perceived ability to fulfill the gatekeeper role using a 5-point Likert scale: 0 (“Poor”), 1 (“Fair”), 2 (“Good”), 3 (“Very Good”), 4 (“Excellent). For example, participants rated their knowledge of how to get help for someone and their ability to persuade someone to get help. These items were used in a previous study as two separate scales (Cross et al, 2007; Matthieu et al., 2008). An exploratory factor analysis conducted in the context of the larger study (Lezine et al., 2009), however, indicated that all 10 items load on a single factor. The 10-item scale had good internal consistency for the entire sample (Cronbach’s alpha = .94) and results are presented using an average score.

Observational Rating Scale of Gatekeeper Skills (ORS-GS)

A measure of gatekeeper skills was developed by two of the authors (WC, MM) in consultation with the gatekeeper program developer and based on the training materials. Development followed methodology from the observational coding literature (Cone, 1999; Fiese, Sameroff, Grotevant, Wamboldt, Dickstein & Fravelm, 2001; Forgatch, Patterson, & DeGarmo, 2005) and involved: operationalization of each construct with behavioral anchors, development of scoring rules and descriptions, and development of a final rating manual. An iterative process was used during the development phase with video tapes of skill assessments not included in the current analyses. The goal was to create a brief, reliable observational assessment of key general and specific gatekeeper skills.

Two general and three suicide specific items were rated using a Likert scale. The general items are: 1) Active listening (i.e., reflects feelings); and 2) Clarifying Questions (i.e., asks about thoughts, feelings, meaning of indirect communication). The suicide specific items include: 3): Question (i.e., asks directly about suicide); 4) Persuade (i.e., uses convincing phrases); 5) Refer (i.e., demonstrates knowledge of specific referral). An earlier version of this scale was refined to extend the response options to 4 points and provide greater specificity of behavioral markers (Cross et al., 2007).

Adherence to standardized script

During the 6-hour actor training preparation, actors practiced responding to a variety of scenarios that would challenge their standardization of delivery of the scripted lines. Given the transactional nature of the interaction, strict adherence to the script was not expected or desirable. Nevertheless, delivery of at least 80% of the standardized script (i.e., phrasing, timing, order of cues delivered) was expected. Actor adherence to the major prompts and lines delivered from the scripted scenario were scored dichotomously and rated independently for all 100 observations. A consensus score was calculated and resulted in a measure of percent adherence to the script for each interaction. Adherence ratings were conducted separately from scoring the observations of the newly trained gatekeepers.

Data Analysis

All data analyses were conducted using the SPSS for Windows Version 14.0 (SPSS, 2007) and included paired t-tests, binary logistic regression to examine predictors of change in gatekeeper skills from pre- to post-training, and Chi Square analysis to assess differences in categorical variables.

Results

Sample characteristics and pre-training variables

The sample participants were primarily Caucasian, campus residence assistants and student affairs staff, with post-secondary education. Participants ranged in age from young (19 years) to older adults (61 years). Almost half (44%) of the sample had some experience with clinical interviewing or other suicide prevention training, or both, and 54% knew someone who died from suicide (see Table 1).

Table 1
Sample socio-demographic and training variables (n=50).

Participants’ scores on the three facets of Openness to Experience (OTE) were within the normal range for non-clinical adults (Costa & McCrae, 1992): Feelings = 23.5 (3.84); Actions = 18.7 (3.92), Ideas = 22.6 (4.44). Total scores on the combined 24 items were used in the analyses.

Training outcomes

Declarative Knowledge and Perceived Efficacy

Participant scores showed significantly enhanced knowledge and self-efficacy scores from pre- to post-training. Participants’ mean score on knowledge after training was higher (11.30 ± 1.25 vs. 9.50 ± 1.49; t= 8.07, p <.001) than before. Participants also showed significantly higher perceived efficacy for intervening after training (2.88 ± .69 vs. 1.74 ± 1.00; t= 10.396, p <.001) than at baseline.

Observational Rating Scale of Gatekeeper Skills (ORS-GS)

Table 2 summarizes the inter-class correlation calculations for the ORS-GS. Reliability was high for all items with the exception of Persuade and pre-training Active Listening scores which were in the low end of acceptable range (.58, .55 respectively). Consensus scores were used for subsequent analyses. Table 3 summarizes t-test analyses of observational ratings of participants’ skills at pre- and post-test. Suicide specific skills were significantly improved at post-test; however, there was no significant change in general skills following training.

Table 2
Inter-rater reliability of pre-post observations (n = 50)
Table 3
Pretest and Posttest observational ratings (n = 50)

To further understand the impact of training, a combination of suicide-specific gatekeeper skills were examined at baseline and post-test. We operationalized adequate suicide prevention gatekeeper skills in terms of ORS-GS scores. A participant was classified as demonstrating adequate gatekeeper skills by asking directly about suicidality (Q score = 3), using persuasive communication for help seeking (P score ≥2), and providing a referral that would successfully connect the distressed person to appropriate assistance (R score ≥2). Any other combination of skills resulted in classification as not adequate gatekeeper skills. Prior to training, 5 participants (10%) met criteria for adequate gatekeeper skills. Following training, 27 participants (54%) met criteria for adequate gatekeeper skills. We tested the change in proportion of acceptable gatekeeper skills using a McNemar test for correlated proportions. The test assesses the significance level for differences in proportions when both proportions refer to the same sample. For this case, proportion 1 = 10% (5/50), proportion 2 = 54% (27/50), McNemar Test, p < .001.

Predicting Gatekeeper Skills

Analyses were conducted to assess the relationship between pre-training variables and adequate gatekeeper skills as defined above. We conducted a binary logistic regression predicting positive change in gatekeeper skills. All variables were entered simultaneously and included sociodemographics, prior experience related to gatekeeper training (clinical interviewing, suicide prevention training), exposure to suicide, and OTE scale score. Education (college vs. less than college education) and prior experience (clinical interview experience and/or suicide training vs. no experience/training) were entered using dummy variables with indicator coding. The results from the logistic regression were not significant (X2 (9) = 12.37, p = .19). None of the variables tested, either alone or together, significantly predicted positive change in observed gatekeeper skills. We then conducted a second analysis to include post training Knowledge and Perceived Efficacy in the model. There was an association between Gatekeeper Skills and Perceived Efficacy at post training while controlling for other factors in the logistic regression, Beta (SE) = 1.64(.83), Wald (1), = 3.97, p <.05. Independent sample t-tests showed small but significant mean differences between the Adequate Gatekeeper Skills (Mean = 3.09, SD = .57) and Not Adequate Gatekeeper Skills (Mean = 2.63, SD=.76) groups (t (48) = 2.43, p = .02).

Because 10% of participants demonstrated adequate skills prior to training, we also examined relationships among the variables associated with pre-training observed skills. Results showed that prior suicide prevention training was associated with adequate gatekeeper skills prior to training (X2 (1) = 4.85, p = .03); however, after training there was no longer a significant relationship (X2 (1) = .23, p = .63).

Fidelity: Actor adherence to script

All 100 observations (50 pre, 50 post) were reviewed for actors’ accurate delivery of the standardized scripts. Each cue and prompt was scored dichotomously for presence/absence. A total percent adherence score was assigned to each observation. Based on consensus scores, 74% of the observations met the expectation of 80% or more of the standardized script delivered as written.

Discussion

The current study was conducted to evaluate the proximate impact of a brief community level suicide prevention training program on observed gatekeeper skills. To our knowledge, this is the first study of observed gatekeeper skills in a community sample using rigorous methodology with a sample this size. The first objective was to develop methodology to assess skills that addresses a number of limitations of previous studies including: refining a brief, reliable, observational measure of general and specific gatekeeper skills; providing standardized stimuli with trained actors; assessing actor adherence to the standardized script; and assessing skills before and after training.

The second objective was to analyze the magnitude and statistical significance of changes in knowledge, attitudes and skills from pre- to post-training. The final objective was to examine the relationship between pre-training characteristics and skill enhancement after training. We expected that previous training and clinical interviewing, as well as a personality that is “open” to new learning and behaviors, would predict positive change in observable gatekeeper skills.

The current study replicated previous findings (e.g., Cross et al, 2007; Matthieu et al., 2008; Lezine et al, 2009; Wyman et al., 2008) that showed positive proximate changes in knowledge about suicide and efficacy for intervening with suicidal individuals. Wyman and colleagues (2008) and Keller et al. (2009) have demonstrated maintenance of knowledge and efficacy changes at follow-up for this gatekeeper training program.

The Observational Rating Scale of Gatekeeper Skills (ORS-GS) developed for a previous study and refined for the current one, operationalized five gatekeeper skills (two general, three specific). Inter-rater reliability was generally high with the exception of pre-Active Listening and the Persuade item. Raters found the description of the Persuasion item to be unclear and consensus meetings were used to come to agreement on all differences, and to refine the measure for future studies. Nevertheless, there was significant change in participants’ suicide specific skills. The one-hour training program enhanced participants’ ability to ask directly about suicidality, to be persuasive about getting assistance, and to provide a helpful referral.

In contrast to positive changes in suicide specific skills, and consistent with at least one other study (Tierney, 1994), we found no change in observed general skills (e.g., active listening) from pre- to post-training in this study. This is not surprising because this one-hour gatekeeper training is brief and does not specifically focus on teaching ‘soft’ skills such as empathic reflections. The training indicates that sensitive communication and clarifying questions are important to interacting with suicidal individuals, but, focuses specifically on dispelling myths about suicide and providing information about the three suicide specific skills (questioning, being persuasive, providing an appropriate referral). This finding strengthens Wyman et al.’s (2008) suggestion that identifying students at high risk for suicide in school settings requires open communication about issues of emotional distress, and that the current QPR training does not change communication styles. To have an impact on general communication skills, it is likely that a more comprehensive training that focuses specifically on communication skills (e.g., empathy, active listening) and identifies participants who are ready to engage others in emotionally charged discussions, would be necessary.

The finding from the current study that 54% of participants demonstrated adequate gatekeeper skills post-training is consistent with a previous study that used a somewhat different criterion (Cross et al., 2007). The current study defined adequate skills based on a combination of three suicide specific skills (question directly about suicide, moderate level of persuasiveness, contextualized referral). The previous study used a total score cut off for acceptable gatekeeper skills (11+ out of 15 on the scale) in a post-only design. Nevertheless, the percentage of participants is almost identical (54% and 55% respectively). The consistency of the results across two studies with different designs (post-only; pre-post) and populations, argues against practice effects alone as the mechanism underlying our results.

The finding that about half the participants in a one-hour training program demonstrate adequate gatekeeper skills following training is encouraging from a public health perspective—it appears to deliver a rather large impact in a relatively short amount of time Another way to measure cost-effectiveness from training would be to assess participants’ dissemination of knowledge, attitudes and skills to other community members. Broad diffusion of accurate information about suicide through personal contact among friends, and family could serve to reduce stigma and, over time, change social norms.

Nevertheless, there are two caveats to the finding that just over half of the participants demonstrated adequate gatekeeper behaviors. Despite a significant improvement in skills for the sample overall, 46% of participants did not show the ability to ask about suicide and make an adequate referral for help. This proportion is far from ideal for responding to a potentially life-threatening situation in a community setting. Moreover, post-training skills were assessed immediately after training and likely represent the ‘best case’ scenario for using rigorous observational methods with this type of training intervention (Snyder et al, 2006). Future research is needed to establish whether a longer time lag (e.g. days, weeks) between training and assessment would result in improved long term outcomes of participants with adequate gatekeeper skills.

Clearly, the one-hour, community level gatekeeper training did not uniformly enhance skills among participants. There is significant room for improvement if the goal of gatekeeper training extends beyond knowledge and attitude enhancement.

Finally, it is interesting to note that a small proportion of participants (10%) demonstrated adequate gatekeeper skills prior to training. We examined relationships among the variables associated with pre-training observed skills and found that suicide prevention training was associated with adequate gatekeeper skills prior to training, but the relationship disappeared after training. Thus, other participants increased their abilities at the end of the hour such that prior training was no longer associated with gatekeeper skills. The implication is that some potential participants with previous suicide prevention training may not enhance their learning or skills with this level of gatekeeper training, and may not be good candidates for training particularly if resources are limited. On the other hand, not all participants with previous training demonstrated adequate gatekeeper skills at baseline.

There are several limitations to the current study. Although participants were randomly selected from training groups, sampling bias remains. Specific groups were invited to gatekeeper training at each university site and these groups were chosen by the sites. There is a self-selection bias because participants who volunteer may be quite different from those who choose not to respond to the invitation to participate in the study. In addition, the overall sample size, while substantial for the observational methods, is small for generalizability purposes and a control group is not included. A third limitation is related to the observational measure. Low inter-rater reliability for the Persuasion item is a limitation of the scale, although we used consensus ratings in the analyses. Finally, our findings do not address the critical issue of maintenance of skills over time or the relationship between observed gatekeeper skills and actual use of those skills. Longitudinal studies are needed to establish maintenance, loss or gain in skills over time, as well as the critical question about the relationship between observed gatekeeper skills and use of skills in daily life.

Implications for Practice

Several strategies may be useful in boosting the effectiveness of gatekeeper training. One strategy focuses on selection of participants. In the current study, we found that enhanced skills was not predicted by any of the pre-training characteristics under study, including previous training and interviewing experience. The gatekeeper training intervention had a significant impact on participants who did not have experiences of suicide or suicide specific skills. Similarly, Openness to Experience, a personality characteristic shown in previous studies to be related to new learning and behaviors (Barrick & Mount, 1991), did not predict enhanced skills. Our sample size is small, however, and not representative of the population at large. Moreover, we assessed Openness to Experience using only 3 subscales. We are therefore cautious in interpreting the finding that openness is not a factor in skill development and recommend that these results be replicated in a variety of populations. Moreover, these or other variables may predict learning outcomes (e.g., skills) in comprehensive trainings programs for clinicians, where the goal is to develop very skillful interveners, or in programs that are longer and more intensive.

Another strategy is to modify the training model used in this type of gatekeeper training program. The current QPR community level training model goal is primarily knowledge transfer. Participants passively receive information about suicide and messages to increase awareness. Although the training is engaging and highly rated by participants as valuable (Cross et al., 2007; Matthieu, et al 2007; Lezine et al., 2009), from an adult learning perspective such a passive learning experience is not likely to result in transfer of that training for behavior change and use in daily life (DeNeve & Heppner, 1997; Humair, & Cornuz, 2003; Joyner & Young, 2006). The QPR training program draws an analogy with cardiopulmonary resuscitation (CPR) in that both training programs teach a skill that community members can learn and use to aid others. Studies of CPR training have shown, however, that simulations of real life scenarios where CPR may be needed, as well as behavioral assessment with instructor feedback, are necessary to achieve skill acquisition and retention (Hamilton, 2005). Although knowledge enhancement has been shown to persist for many months following CPR training, skills decline over that period, particularly if behavioral simulation is not a training factor (Hamilton, 2005). Brief gatekeeper training, such as QPR, which are primarily lecture-based would benefit from research using similar training strategies. Role play practice with supportive feedback, as well as other active learning strategies, increases the likelihood of gatekeeper skill development and use. Our finding that there was an association between participants’ efficacy after training and enhanced skills is consistent with adult learning models that emphasize the mediating role of self-efficacy in behavior change (Cross, 2009; Holton & Baldwin, 2003). Training strategies that further enhance feelings of efficacy for intervening with suicidal individuals are likely to result in a greater proportion of participants with adequate gatekeeper skills. A study that randomly assigns participants to training as usual and training plus behavioral rehearsal through simulation would be needed to assess maintenance, decline or improvement in general and specific gatekeeper skills immediately following training and at follow up.

In sum, a brief, community-level gatekeeper program designed for “novice” trainees--individuals without previous knowledge or specific traits—resulted in positive changes in knowledge, efficacy and observable skills among a variety of new learners. Importantly, about half of participants demonstrated adequate suicide specific gatekeeper skills following a program that does not provide behavioral rehearsal or focus on skill development. The duration of these skills, however, has yet to be established through longitudinal studies and the strategies needed to increase the proportion of participants who acquire gatekeeper skills have yet to be studied. An effective population-based approach to suicide prevention requires broad-based training that is necessarily cost effective. Brief gatekeeper training programs, such as the one we studied, may be enhanced by a simple, “low technology” use of active learning strategies (i.e., role play rehearsal) in the training. Transfer of training to actual use of gatekeeper skills may also be improved when training provides opportunities for practice. Studies that test the impact of different training models on knowledge, attitudes, skills and use are critical to improving community-based suicide prevention.

Acknowledgements

We acknowledge the generous contribution of our collaborators at each university site. We also thank Joey Steinhagen, Erica Steinhagen, Scott Shutts, and Lynda Chauncey for participating as standardized actors. We thank Eric Caine, MD and Yeates Conwell, MD of the University of Rochester’s Center for the Study and Prevention of Suicide; David Litts, OD of the Suicide Prevention Resource Center; the NIMH-funded Prevention Science Methodology Group, and Phil and Donna Satow of the Jed Foundation, for their support.

Support: This project was supported by the NIMH P20 Developing Center for Public Health and Population-Based Approaches to Suicide Prevention (MH071897; PI: Caine) and NIMH Institutional T32 grants supporting Dr. Cross (MH018911; PI: Caine) and Drs. Lezine and Matthieu (MH020061; PI: Conwell). Dr. Cross is currently supported by NIMH K23 (MH73615). The study took place in the context of a larger study led by Dr. Knox and supported in part by NIMH K01 (MH055317).

Biographies

• 

Wendi Cross, Ph.D. is an Associate Professor of Psychiatry (Psychology) and Pediatrics at the University of Rochester Medical Center (URMC), and is with the Center for the Study and Prevention of Suicide at URMC. Her research focuses on models, methods and measures of training and transfer of interventions, including suicide prevention programs, to community settings.

• 

Monica M. Matthieu, Ph.D., LCSW is a Research Assistant Professor at the George Warren Brown School of Social Work, Washington University in St Louis, and a Research Social Worker at the Department of Veterans Affairs Medical Center in St Louis, Missouri. Prior to this position, she was an NSRA postdoctoral fellow at the Center for the Study and Prevention of Suicide at the University of Rochester. With a joint appointment at the VA and the Brown School, she continues to focus her energies on suicide prevention among veterans and conduct research on the implementation of evidence based practices, trauma treatment, and violence and injury prevention.

• 

DeQuincy Lezine, Ph.D. was an NRSA Post-doctoral Fellow conducting research on public health and population based approaches to suicide prevention when this study was conducted. Dr. Lezine is currently an evaluation associate with Pima Prevention Partnership in Arizona.

• 

Kerry Knox, Ph.D. is an Associate Professor of Psychiatry and Community and Preventive Medicine at University of Rochester Medical Center and the Director of the VISN 2 Center of Excellence for Suicide Prevention (COE) in the Department of Veterans Affairs. Her research focus is on the relationship among psychiatric, psychosocial and physiological and stress and risk for suicide.

Footnotes

Portions of this study were presented at the 15th Annual Conference of the Society for Prevention Research, Washington, D.C. (May, 2007), and at the 36th Annual Conference of the American Association of Suicidology, Seattle, WA (April, 2006).

References

  • Barrick MR, Mount MK. The Big Five personality dimensions and job performance: A meta-analysis. Personnel Psychology. 1991;44:1–26.
  • Center for Disease Control and Prevention CDC. Youth suicide prevention programs: A resource guide. Atlanta, GA: National Center for Injury Prevention and Control; 1992.
  • Cone J. Observational assessment: Measure development and research issues. In: Kendall PC, Butcher JN, Holmbeck GN, editors. Handbook of research methods in clinical psychology. 2nd ed. New York, NY: Wiley; 1999. pp. 183–223.
  • Costa PI, Jr, McCrae RR. Revised NEO Personality Inventory and NEO Five-Factor Inventory (NEO-FFI) Manual. Lutz, FL: Psychological Assessment Resources; 1992.
  • Cross W. A model of training and transfer of training. Paper presented to the 2nd Annual Conference on the Science of Implementation and Dissemination: Building Research Capacity to Bridge the Gap from Science to Service; Bethesda, Maryland: NIMH; 2009. Jan,
  • Cross W, Matthieu MM, Cerel J, Knox KL. Proximate outcomes of gatekeeper training for suicide prevention in the workplace. Suicide and Life Threatening Behaviors. 2007;37(6):659–670. [PubMed]
  • DeViva JC. The effects of full-day and half-day workshops for health care providers in techniques for increasing resistant clients’ motivation. Professional Psychology: Research and Practice. 2006;37:83–90.
  • DeNeve K, Heppner MJ. Role play simulations: The assessment and active learning technique and comparisons with traditional lectures. Innovative Higher Education. 1997;21:231–248.
  • Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA. 2002;287:226–235. [PubMed]
  • Fiese BH, Sameroff AJ, Grotevant HD, Wamboldt FS, Dickstein S, Fravel DL. Observing families through the stories they tell: A multidimensional approach. In: Kerig PK, Lindahl KM, editors. Family observational coding systems: Resources for systemic research. Mahwah, NJ: Lawrence Erlbaum Associates; 2001. pp. 259–271.
  • Forgatch MS, Patterson GR, DeGarmo DS. Evaluating fidelity: Predictive validity for a measure of competent adherence to the Oregon model of Parent Management Training. Behavior Therapy. 2005;36:3–13. [PMC free article] [PubMed]
  • Gask L, Dixon C, Morriss R, Appleby L, Green G. Evaluating STORM skills training for managing people at risk of suicide. Nursing and Healthcare Management and Policy. 2006:739–750. [PubMed]
  • Goldsmith SK, Pellmar TC, Klienman AM, Bunney WE, editors. Reducing suicide: A national imperative. Washington, DC: The National Academies Press; 2002.
  • Gould MS, Kramer RA. Youth suicide prevention. Suicide and Life-Threatening Behavior. 2001;31 Suppl:6–31. [PubMed]
  • Grossman JA, Kruesi MJP. Innovative approaches to youth suicide prevention: An update of issues and research findings. In: Maris RW, Canetto SS, McIntosh JL, Silverman MM, editors. Review of suicidology. New York, NY: Guilford Press; 2000. pp. 70–201.
  • Gully SM, Payne SC, Koles KLK, Whiteman J-A. The impact of error training and individual differences on training outcomes: An attribute-treatment interaction perspective. Journal of Applied Psychology. 2002;87:143–155. [PubMed]
  • Hamilton R. Nurses' knowledge and skill retention following cardiopulmonary resuscitation training: A review of the literature. Journal of Advanced Nursing. 2005;51(3):288–297. [PubMed]
  • Holton EF III, Baldwin TT, editors. Improving learning transfer in organizations. San Fransisco: Jossey-Bass; 2003.
  • Humair J-Pl, Cornuz J. A new curriculum using active learning methods and standardized patients to train residents in smoking cessation. Journal of General Internal Medicine. 2003;18:1023–1027. [PMC free article] [PubMed]
  • Joyner B, Young L. Teaching medical students using role play: Twelve tips for successful role plays. Medical Teacher. 2006;28:225–229. [PubMed]
  • Keller DP, Schut LJ, Puddy RW, Williams L, Stephens RL, McKeon R, Lubell K. Tennessee lives count: Statewide gatekeeper training for youth suicide prevention. Professional Psychology: Research and Practice. 2009;(40):126–133.
  • Lewis DA, Brain G, Cushing AM, Hall A, Zakrzewska JM. Description and evaluation of an education and communication skills training course in HIV and AIDS for dental consultants. European Journal of Dental Education. 2000;4:65–70. [PubMed]
  • Lezine DA, Matthieu MM, Cross W, Bajorska A, Conwell Y, Knox KL. Gatekeeper training: characteristics that predict finding suicidal people. Unpublished manuscript. University of Rochester; 2009.
  • Mann JJ, Apter A, Bertolote J, Beautrais A, Currier D, Haas A, et al. Suicide prevention strategies: a systematic review. Journal of the American Medical Association. 2005;294(16):2064–2074. [PubMed]
  • Marušic A. Seven steps to integrating suicidology. Crisis: The Journal of Crisis Intervention and Suicide Prevention. 2008;29(3):115–117. [PubMed]
  • Mattthieu MM, Cross W, Batres AR, Flora CM, Knox KL. Evaluation of gatekeeper training for suicide prevention in veterans. Archives of Suicide Research. 2008;12:148–154. [PubMed]
  • McDaniel SH, Beckman H, Morse D, Silberman J, Seaburn DB, Epstein RM. Physician slef-disclosure in primary care visits: Enough about you, what about me? Archives of Internal Medicine. 2007;167(12):1321–1326. [PubMed]
  • Mounsey AL, Bovbjerg V, White L, Gazewood J. Do students develop better motivational interviewing skills through role play with standardised patients or college students? Medical Education. 2006;40:775–780. [PubMed]
  • Mount MK, Barrick MR. The Big Five personality dimensions: Implications for research and practice in human resources management. Research in Personnel and Human Resources Management. 1995;13:153–200.
  • Neimeyer RA, Pfeiffer AM. Evaluation of suicide intervention effectiveness. Death Studies. 1994;18:131–166. [PubMed]
  • Palmieri G, Forghieri M, Ferrari S, Pingani L, Coppola P, Colombini N, Rigatelli M, Neimeyer RA. Suicide intervention skills in health professionals: A multidisciplinary comparison. Archives of Suicide Research. 2008;12(3):232–237. [PubMed]
  • Quinnett P. QPR: Certified QPR Gatekeeper Instructors Training Manual. Spokane, WA: The QPR Institute; 1995.
  • Sholomskas D, Syracuse-Siewert G, Rounsaville BJ, Ball SA, Nuro KF, Carroll KM. We don’t train in vain: A dissemination trial of three strategies of training clinicians in cognitive-behavioral therapy. Journal of Consulting and Clinical Psychology. 2005;73:106–115. [PMC free article] [PubMed]
  • Snyder J, Reid J, Stoolmiller M, Howe G, Brown CH, Dagne G, Cross W. Measurement systems for randomized intervention trials: The role of behavioral observation in studying treatment mediators. Prevention Science. 2006;7(1):43–56. [PubMed]
  • SPSS. (Version 14.0) [Computer software] Chicago, Illinois 60606: SPSS Inc.; 2007.
  • Stuart C, Waalen JK, Haelstromm E. Many helping hearts: An evaluation of peer gatekeeper training in suicide risk assessment. Death Studies. 2003;27:321–333. [PubMed]
  • Tesch SA, Cameron KA. Openness to experience and the development of adult identity. Journal of Personality. 1987;55:615–630.
  • Tierney RJ. Suicide intervention training evaluation: A preliminary report. Crisis: Journal of Crisis Intervention & Suicide. 1994;15(2):69–76. [PubMed]
  • Wyman PA, Brown CH, Inman J, Cross W, Schmeelk-Cone K, Jing G, Pena J. Randomized trial of a gatekeeper program for suicide prevention: 1-year impact on secondary school staff. Journal of Counseling and Clinical Psychology. 2008;76:104–115. [PMC free article] [PubMed]
  • Yardley-Matwiejczuk KM. Role play: Theory and practice. Thousand Oaks, CA: Sage Publications; 1997.