|Home | About | Journals | Submit | Contact Us | Français|
Although progress has been made in developing a scientific basis for alcohol screening and brief intervention (SBI), training packages are necessary for its widespread dissemination in primary care settings. This paper evaluates a training package developed for the Cutting Back® SBI program. Three groups of medical personnel were compared before and after SBI training: physicians (n = 44), medical students (n = 88), and non-physicians (n = 41). Although the training effects were at times dependent on group membership, all changes were in a direction more conducive to implementing SBI. Physicians and medical students increased confidence in performing screening procedures, and students increased self-confidence in conducting brief interventions. Non-physicians perceived fewer obstacles to screening patients after training. Trained providers reported conducting significantly more SBIs than untrained providers, and these differences were consistent with patients’ reports of their providers’ clinical activity. Thus, when delivered in the context of a comprehensive SBI implementation program, this training is effective in changing providers’ knowledge, attitudes, and practice of SBI for at-risk drinking.
Alcohol consumption in amounts that significantly increase the chances of health problems (i.e., at-risk drinking) is common among patients presenting to primary care (1) and imposes a significant economic burden on the health care system (2,3). Screening procedures have been developed to identify at-risk drinkers (4,5), and significant reductions in drinking and related risks can be achieved by brief interventions (6–10). Screening can also identify persons likely to be alcohol dependent, and referral for diagnostic evaluation may encourage patients to seek treatments that have been shown to be effective. The Institute of Medicine (11) and the U.S. Clinical Preventive Services Task Force (12) have recommended that alcohol screening and brief intervention (SBI) be undertaken in a variety of medical settings, including primary care. This paper reports on the evaluation of a primary care training program called Cutting Back®, a comprehensive approach to alcohol screening and brief intervention. It describes the effects of the program on trainees’ knowledge, attitudes, and the subsequent practice of SBI in routine clinical practice.
Introducing new screening and prevention activities into primary care practices presents significant logistical, attitudinal, and behavioral challenges. Medical schools and residency programs devote limited faculty resources and curriculum time to alcohol and alcoholism (13–15). Many physicians feel inadequately trained when faced with patients who have alcohol-related problems (16). Similar observations have been made with regard to nursing education (17,18). Barriers to adequate coverage of alcohol-related problems in both medical schools and continuing professional education include traditional attitudes about the moral culpability of chronic alcoholics, confusion as to whether problem drinking is a medical or psychiatric concern, lack of faculty role models, lack of training materials, and role ambiguity regarding who is responsible for screening and intervention (19,20). Another factor could be the relative lack of awareness that SBI leads to significant reductions in drinking and risk.
Research on medical education has shown that training can be effective in improving students’ and physicians’ knowledge and skills in addressing alcohol issues (21–23), but changes in knowledge may be easier to produce than changes in attitudes and behavior (24). A recent review of the components and outcomes of medical education in substance-related disorders concluded that the selection of a combined didactic and interactive educational strategy may be the most cost-effective learning strategy, but there is little empirical evidence to support this approach (24).
Although some progress has been made in the development of SBI for medical practitioners, medical students, and health care organizations (25,26), a necessary step toward dissemination is the development of successful training packages that include program implementation procedures. The Cutting Back® program was developed to train medical providers to implement SBI in primary care settings. In this article we compare the effects of this program on three groups of medical personnel: physicians, medical students, and non-physicians. It was expected that the physicians would demonstrate higher initial levels of competency prior to training, and that all groups would show positive changes after training. It was also predicated that the program would produce tangible evidence of successful SBI implementation at training sites compared with sites where no training was conducted.
The study, conducted between 1997 and 2001, involved the implementation and evaluation of the Cutting Back® program in five managed care organizations (MCOs) in the West, Southwest, Midwest, and Northeast regions of the United States. Four MCOs were exclusive group model Health Maintenance Organizations (HMOs); one was a consortium of independent practice association model HMOs. Each MCO was required to have at least three comparably sized Family or Internal Medicine clinics with annual visits of at least 7,000 unduplicated adult patients, a designated MCO liaison to coordinate SBI activities, and no current alcohol screening programs. MCOs were reimbursed for some, but not all, costs of participation, including data collection. A total of 10 practices received Cutting Back® training after which they administered the program without on-site research supervision. Technical assistance was available via telephone or email and through occasional site visits. While the essential elements of SBI procedures were established in common among the five MCOs, each site had flexibility in adapting the program to its particular setting and style of operations. Another five practices, which did not receive training, served as control conditions for this and other aspects of the study. As described in related papers from this project dealing with patient outcomes, cost, and benefit findings, intervention and control clinics were randomly assigned to conditions (27,28). This paper reports on changes in three types of medical providers who received the training: primary care physicians recruited from practices associated with managed care organizations (MCOs) in five sites, non-physicians from the same settings, and, for comparison, third year medical students from the University of Connecticut (UConn) School of Medicine. The evaluation employed a before and after repeated measures group design to test short-term changes in providers’ knowledge about SBI, perceived obstacles to practicing it, expectations about its effectiveness, and self-efficacy or confidence in performing SBI. Longer-term changes in practice of SBI among practitioners were also evaluated in comparison to those who did not receive training.
The Cutting Back® training curriculum, developed in collaboration with the University of Wisconsin Department of Family Medicine, draws upon SBI literature and prior training curricula (29) and the National Institute on Alcohol Abuse and Alcoholism (NIAAA) Physicians’ Guide to Helping Patients with Alcohol Problems (30). It takes a practical, systems approach, aiming to facilitate the implementation of Cutting Back® in clinic operations rather than merely educating staff. The training curriculum contains four modules addressing practical issues deemed essential to implementing the program. Module 1 reviews the full continuum of alcohol misuse from occasional hazardous use to dependence, its importance to medical practice, and the concepts of screening and brief intervention. It also discusses research demonstrating the positive impact interventions have on patients. Module 2 presents the program’s two-stage screening protocol, its instruments, scoring procedures, and the content of the patient brochure, which was designed especially for this project. The screening procedure employed the Alcohol Use Disorders Identification Test (AUDIT), a well-validated screening instrument developed by the World Health Organization (31). Module 3 reviews the clinic’s implementation plan, which was prepared by a representative planning team using a decision-making guide that prompted decisions on the critical administrative activities required for each Cutting Back® procedure and the responsibilities of each person in the clinic. Critical administrative activities included administration and scoring of the screening instruments, assuring availability of patient brochures, sequencing of interventions with treatment of presenting health problems, the essential elements of an intervention, and the management of SBI records. Module 4 provides practice and skill development in administering and scoring screening instruments and delivering brief interventions. The training was complemented by videos that introduce the program and provide examples of screening and brief intervention procedures. Medical students trained as part of their third year clerkship in ambulatory primary care at UConn received a shorter version of the same curriculum.
The training at MCO sites was delivered in less than three hours in order to meet the busy schedules and high hourly costs of primary care practices. Sites chose a variety of time arrangements for the training. Most selected one three-hour session, but it was delivered at one practice in segments that required no closing of clinic operations and no after-hours work. In all cases a physician trainer and the project site consultant delivered the training. As many practice staff as possible were invited to the training, including physicians, physician assistants, nurse practitioners, nurses, administrators, medical assistants, medical records personnel, and reception staff. Continuing Medical Education credits were available for qualified staff.
A total of 14 trainings were conducted, including ten for primary care practices and four for medical students. The training participants consisted of 44 physicians, 41 clinical non-physicians, and 88 medical students. The mean ages of participants in the three groups were 47 for the physicians, 43 for non-physicians, and 26 for medical students. The majority of physicians were male (73%). Most of the non-physicians were female (71%). The medical students were 57% male. The non-physician group consisted of 29% physician assistants, 24% nurse practitioners, 17% registered nurses, 12% licensed practical nurses, and 17% who had a M.S.W. or other master-level counseling degree. Fifty-seven percent of the physicians practiced family medicine and 42% were internists.
The evaluation of the effects of training and program implementation were measured in two ways. A simple 9-question pre- and post-test administered by the trainers immediately before and after the training measured changes in knowledge among those who participated in training. More subtle and complex changes of attitudes, typical practice patterns, and confidence were measured before training, immediately after training (to determine the effects of training), and at the end of the project (to differentiate the effects of practicing SBI from the training itself). Approximately one week prior to the training in the MCO clinics, staff who were going to be trained were asked to complete a 123-item questionnaire that2 included questions on beliefs, practices, and self-efficacy related to preventive medicine, including at-risk drinking and alcoholism. This questionnaire was re-administered by MCO personnel a few weeks after training, after providers had begun to practice SBI with their patients, and again at the end of the Cutting Back® program implementation period, approximately nine months after training. The same questionnaire was completed by physicians at the five control clinics, who had not received training, at the end of the program to allow comparison with those who had both been trained and had practiced SBI. The protocol differed somewhat for medical students, who received the 123-item questionnaire and knowledge test immediately prior to and after the training session.
Knowledge essential to conducting SBI was measured by nine objective questions. For example, trainees were asked to define moderate drinking in terms of the USDA upper limit of number of drinks per day for men. Pre- and post-training knowledge items were scored according to the number of correct responses.
The 123-item questionnaire measured a variety of other factors that might affect a clinician’s ability to provide SBI to patients. Confidence in screening was measured separately from confidence in performing brief intervention. Five items instructed respondents to “indicate the amount of confidence you have in your ability to perform each of the activities.” For instance, respondents were asked how confident they were in asking questions to collect information about a patient’s risk for alcohol-related problems and how much confidence they had in recommending low-risk consumption limits. Response options consisted of “no confidence,” “very little confidence,” “moderate degree of confidence” and “quite a lot of confidence.” Internal consistency reliability (Cronbach’s alpha) was .85 for the screening items and .86 for the brief intervention items, which were summed to form two separate measures of confidence.
The remaining measures were based on Likert scale ratings of a series of belief statements, each with five response options ranging from strongly agree to strongly disagree. Item content was informed by the alcohol health services literature including Project NEADA (18) and the World Health Organization Study of General Practitioners (32).
Belief items were arranged into three sets of statements. Fifteen statements referred to screening activities, 16 dealt with brief interventions, and a third set of 19 statements related to working with at-risk drinkers. Principal Components Analyses were applied separately to each item set. Items with markedly skewed distributions were eliminated prior to the analyses. A scree-test was applied to the eigenvalues to determine the number of principal components.
The analysis of screening items resulted in four components with eigenvalues greater than one. Only the first component was retained based on a significant drop in eigenvalue between the first and second component, from 3.7 to 1.2. The component labeled “Perceived Obstacles to Screening” was based on the content shared by the eight items loading .40 or more (alpha = .80). A representative item from the scale is, “I don’t know what questions to ask patients to obtain information about their alcohol consumption.” Likewise the analysis of brief intervention items yielded four components with eigenvalues greater than one. Again, only the first component was retained, based on a marked drop in eigenvalues after the first component, from 4.2 to 1.6. The component was named “Perceived Obstacles to Brief Intervention” (alpha = .82). A sample item from this scale is, “There is not enough time to advise patients about drinking.” The analysis of the final set of items resulted in four components. The first and second components were retained based on the decrease in eigenvalues. One component was named “Self-efficacy for SBI” (alpha = .76) and the other “Expectations for Brief Interventions” (alpha = .48), in accordance with the content shared by items loading .40 or above. A representative item from the Self-efficacy for SBI scale is, “I feel I can appropriately advise my patients about drinking and its effects.” A sample item from the Expectations for Brief Intervention scale is, “Talking to patients about health risks related to drinking will help them alleviate their own alcohol-related problems.” Items from the retained components were reverse coded where necessary to create consistency in scoring weights and summed into four separate scales.
Although the primary focus of this evaluation is on short-term effects of the training on knowledge, beliefs, attitudes, and self-efficacy, the larger study of which this research was a part (27) also gathered information about the implementation of SBI at clinics affiliated with five managed care organizations. Screening was conducted at 15 clinics, five of which served as control sites for a related study of patient outcomes. Cutting Back® implementation and training for SBI was provided at the 10 remaining clinics. A representative sample of patients consuming alcohol at hazardous levels (N = 1,329) was contacted by telephone approximately three months after the screening to evaluate the effect of brief intervention. As part of the follow up interview, patients were asked if the “provider” had talked with them about their alcohol use. These data were used to determine whether patients treated in clinics where SBI training was conducted reported receiving brief interventions more frequently than did patients treated in clinics where SBI training was not conducted. In addition, at the end of the study implementation period, physicians and non-physicians at all sites were asked about the extent to which they practiced screening (i.e., number of times they used a standard screening test) and brief intervention (i.e., “How many patients did you manage for drinking?”) activities during the previous year.
Group means of the knowledge and confidence scales were compared across time (before and after training) and between groups (physicians, non-physicians, medical students) using a Repeated Measures Analysis of Variance (ANOVA). The Tukey B method was used for post hoc comparisons. A similar analysis was conducted with the scales measuring perceived obstacles, self-efficacy and expectations, except that the between groups factor did not include the medical students because, as non-practitioners, they were not asked these questions. In addition, physician and non-physician staff at the control sites were compared with their counterparts at the Cutting Back® sites to evaluate their use of SBI using two-way ANOVAs.
Table 1 shows the average scores for the measures of knowledge, confidence, perceived obstacles, self-efficacy and expectations before and after the training exercise. Non-physicians had the lowest mean knowledge pre-test score, whereas physicians and medical students scored significantly higher. More importantly, all groups significantly increased in knowledge after receiving the training (F(1, 159) = 320.74, p < .000).
Compared to physicians and medical students, non-physicians reported less confidence in screening patients prior to training. After training an interaction effect was found indicating that the level of confidence gain was dependent on group membership (F(2, 172) = 4.21, p = .016). Post hoc comparisons indicated that physicians and medical students reported significantly increased confidence in implementing screening procedures in comparison to non-physicians, who indicated about the same level of confidence after training as they did at baseline.
Physicians had significantly more confidence in their ability to conduct brief interventions than non-physicians and medical students prior to training. An interaction effect was observed between group membership and confidence in conducting brief interventions (F(2, 170) = 9.97, p < .000). Post hoc comparisons indicated that medical students gained significantly more confidence in their ability to perform brief interventions than did the other groups. Interestingly, physicians did not increase their confidence in conducting brief interventions after training.
The remaining measures listed in Table 1 pertain only to the physicians and non-physicians. Medical students were not given these items because they were not expected to practice SBI as the MCO personnel were. At baseline, physicians perceived significantly fewer obstacles to screening patients than non-physicians. After training, an interaction was observed involving group membership (F(1, 90) = 5.59, p = .02). Physicians remained basically the same in their perceptions of obstacles to screening, whereas non-physicians reported significantly fewer concerns than they had endorsed prior to training.
Regarding brief interventions, at baseline physicians perceived fewer obstacles than did non-physicians. After training, both physicians and non-physicians perceived significantly fewer barriers to helping patients (F(1, 90) = 20.18, p < .000).
Before training, physicians indicated higher self-efficacy in implementing SBI than non-physicians. At follow-up, both physicians and non-physicians increased in self-efficacy (F(1, 89) = 40.34, p < .000).
Prior to training, physicians reported significantly higher expectations for the benefits of brief interventions than did non-physicians. After training, both physician and non-physician providers raised their level of expectations significantly relative to their respective baseline scores (F(1, 91) = 57.19, p < .000).
Changes in knowledge, confidence, self-efficacy and perceived obstacles are important objectives of any training program, but the ultimate goal is to change provider behavior. In the ten clinics where training was conducted, and at the five comparison clinics where no training was offered, a sample of patients who screened positive for at-risk drinking on the Alcohol Use Disorders Identification Test was contacted three months after their office visit to determine whether they had received an intervention. The results indicated that only 21.9% of the comparison clinic patients reported that their provider talked to them about their drinking. In contrast, 47% of patients who screened positive in the clinics in which physicians were trained to conduct interventions reported that their doctor talked with them about alcohol use (χ21 = 65.13, p < .000). Similarly, whereas only 3% of patients in the comparison clinics reported that their doctor gave them printed materials, 42.4% of patients in the provider trained clinics remembered receiving printed materials (χ21 = 219.54, p < .000).
Table 2 summarizes the findings from provider questionnaires completed at the end of the program implementation period in both the trained intervention and untrained control clinics. Providers were asked the same questions about how often they used standard alcohol screening tests and how many patients they managed for drinking during the past year. Providers in the intervention clinics reported significantly more screening and patients managed for drinking than their counterparts in the control clinics who were not trained.
At-risk drinking and alcoholism are significant public health problems in the United States (2). Numerous studies (7–10) have shown that screening and brief intervention can be effective in decreasing alcohol intake in patients at risk. The results of this study demonstrate that, when delivered in the context of implementing a comprehensive SBI program, even relatively short training of health care providers in screening and brief intervention techniques can result in significant gains in provider knowledge, self-efficacy, and expectations about the value of SBI. For example, fewer than one third of the physicians in this study had adequate knowledge prior to training to advise patients on what constitutes moderate drinking. After training, 90% could correctly report the NIAAA Guidelines for moderate drinking. Results for non-physicians were comparable. There was also a decrease in perceived obstacles to performing brief interventions with patients. More importantly, these short-term changes in attitudes and confidence were associated with long-term changes in provider behavior.
Although the training effects were at times dependent on group membership, all changes were in a direction more conducive to implementing SBI. For instance, both physicians and medical students experienced increased confidence in performing screening procedures and students reported greater self-confidence in conducting brief interventions. In addition, non-physician clinicians perceived fewer obstacles to screening patients after receiving the training.
For practicing health care providers, the various belief changes for the physician and non-physician groups could be attributed in part to the opportunities to implement SBI in their clinical practices after the training. Although the medical students had the opportunity to role-play SBI in the training session, they did not have the benefit of direct experience with actual patients. It is therefore reassuring that the medical students’ confidence ratings increased as much as the other groups. At least among students, a minimum of practice within training is sufficient to increase self-efficacy.
The findings provide support for the Theory of Planned Behavior (33), which predicts that health care practitioners will voluntarily practice what they perceive as efficacious and easy to perform. To the extent that training increases knowledge of effective practices, enhances confidence in performing them, and decreases perceived obstacles to SBI, the training program should produce positive changes in practice behavior. The results are consistent with other studies of provider behavior (34,35) which show that physicians and other health care providers trained in a brief, provider-delivered alcohol intervention will counsel their at-risk patients when cued to do so and when supported by a primary care office system. Our results also replicate previous findings (21) that changes in provider behavior are associated with parallel changes in providers’ attitudes, knowledge and counseling skills following a group training program.
The findings are relevant not only to the design of continuing medical education programs, but also to medical education itself. Physicians in training are increasingly expected to learn an expanding array of clinical preventive services (36), including the use of screening tests and prevention counseling techniques. The findings of this study indicate that medical students respond well to a training exercise that increases their confidence to practice SBI in the future.
The data collected from records of patient interventions in clinics that received (N = 10) and did not receive (N = 5) the training program indicate that SBI was practiced with much greater frequency in the training clinics than in the comparison clinics. Although the results suggest that training in alcohol SBI results in positive changes in attitudes, knowledge, self-efficacy and clinical behavior, the findings are subject to the limitations of a quasi-experimental research design that relies primarily on self-report information from patients and providers. There was some attrition between the training and follow-up questionnaires, which may have resulted in a biased selection of medical students who were more responsive to the training. The responses of providers near the end of the program to questions about their use of screening and brief intervention could have also been affected by a perceived need to please the evaluation team. But it should be noted that all responses were anonymous and providers were encouraged to give accurate answers even if they were critical of the program. Moreover, the comparisons between intervention and control clinics produced similar results from both the patients’ and the providers’ perspectives.
Studies have shown the efficacy (7–10) and cost effectiveness (3) of SBI in primary care, but little progress has been made in disseminating and implementing this clinical preventive service (26). The results of this study indicate that the training of physicians and staff is feasible, and it is effective in preparing health care providers to implement an SBI program. There are, however, other prerequisites that must be met before SBI will find widespread application in primary care practice. Standards of practice by national accrediting and professional organizations are needed. Purchasers and payers must encourage this preventative health service and provide financial incentives to providers. Linkages to community-based organizations and behavioral health units within MCOs must be developed to support referral for treatment in an integrated service delivery system. Communications and social marketing must enlist patients in the search for better preventative health services. And finally, research will be required to develop more efficient ways to train present and future medical practitioners in the implementation of SBI in ways that assure improved performance in practice.
The authors wish to thank the Robert Wood Johnson Foundation for its support of the project upon which this article is based (Reducing Risky Drinking, Grant #029620) and Ellen M. Malaspina for help in preparing the manuscript. Mr. Jeffrey Daniels played a critical role in the overall planning and administration of the project, and Drs. Michael Fleming and James Finch contributed to the development of the training materials and provision of the training.
Thomas F. Babor, Professor and Chairman, Department of Community Medicine and Health Care, University of Connecticut Health Center, Farmington, CT.
John C. Higgins-Biddle, Assistant Professor, Department of Community Medicine and Health Care, University of Connecticut Health Center, Farmington, CT.
Pamela S. Higgins, Research Associate, Department of Community Medicine and Health Care, University of Connecticut Health Center, Farmington, CT.
Ruth A. Gassman, Assistant Scientist, Scheussler Institute of Social Research, Indiana University, Bloomington, IN.
Bruce E. Gould, Associate Dean for Primary Care, School of Medicine, and Associate Professor, Department of Medicine, University of Connecticut Health Center, Farmington, CT.