|Home | About | Journals | Submit | Contact Us | Français|
On January 20, 2004, at the Community Anti-Drug Coalitions of America (CADCA) National Leadership Forum in Washington, D.C., Science & Practice Perspectives joined with CADCA to sponsor a panel discussion on the possibilities of science-based prevention. CADCA represents some 5,000 local drug abuse prevention agencies; its National Community Anti-Drug Coalition Institute seeks to foster its members’ use of appropriate standards for evaluation and research.
Caryn Blitz, Ph.D., the National Coalition Institute’s deputy director of evaluation and research, moderated the panel. Three coalition community leaders—Deacon Dzierzawski, M.A., of Toledo, Ohio; Harry Kressler, M.A., of Tucson, Arizona; and Gwendolyn Hughes Wilson, M.A., of Akron, Ohio—and three prevention researchers—Paul Florin, Ph.D., of the University of Rhode Island; J. David Hawkins, Ph.D., of the University of Washington, Seattle; and Harold D. Holder, Ph.D., of the Pacific Institute for Research and Evaluation in Berkeley, California—exchanged insights about the state of prevention research and the art of implementing prevention programs. An audience of about 150 community leaders contributed to the discussion summarized here.
The panel kicked off on the topic of environmental strategies for drug abuse prevention and ranged widely, along the way covering risk and protective factors, homegrown programs, effectiveness evaluation, evidence versus enthusiasm, the role of community coalitions in drug abuse prevention, and claiming credit for community progress.
Harold Holder: My definition of an environmental strategy is ‘altering social, economic, and geographical community systems.’ As an example, in an environmental approach to youth smoking, the distribution of tobacco outlets is a geographical aspect; smoking by peers is social; the retail price of cigarettes is economic. In reality, each of these aspects takes in much more: economics, for example, includes what some economists call the economics of access—what you have to expend to get to a place that will sell you tobacco products. All these environmental factors interact, and the bottom line is that there are lots of environmental levels and levers to work to try to reduce youth smoking.
There is a myth that all prevention is environmental, but prevention programs that aim to change cognitive behavioral responses of individuals do not change the community environment—especially if those individuals then move away.
Deacon Dzierzawski: In the 8 years or so I’ve been working in our community, we have seen about a 38-percent reduction in kids’ substance use overall—that is, both 30-day and lifetime use of alcohol, tobacco, and marijuana. These markers are trending down parallel to the national average, always about 20 points below it. So we think we are doing something right, based on some very basic, raw, uneducated analysis.
What has gotten us there, we think, is in large part better use of environmental strategies. We scrapped all the single-day events we used to do, like ‘Don’t Drink and Drive Day,’ and instead focused on laws, policies, coordination of services, and increasing direct service. We have achieved tighter control and regulation, mobilization around substance issues, and greater exposure for a normative message of no use or responsible use.
Holder: Media advocacy is a powerful tool and absolutely critical in all environmental work. You must work with the local news, not putting in press releases, but creating real news around your issue. A school survey that provides information about what kids are using can be a good news story.
We received a lot of attention when we sent some kids out to buy alcohol; they were of legal age but looked younger. They documented that they were seldom asked for ID and came back with a documented report describing how one owner told a girl who looked about 15 that for the same price she could buy a better brand of vodka than the one she had picked out. Local news broadcast the film, and it caused a sensation.
Dzierzawski: We have an ongoing struggle convincing decisionmakers in our community that environmental strategies are science-based.
Holder: The evidence for specific environmental strategies is very good, better than anything else in the field. I will challenge anybody to match, for example, the effects of the minimum drinking age in the United States.
A short list of things we know work would include raising the retail price of substances, setting minimum ages for drinking and buying tobacco, restricting or lifting driving licenses for drinking and driving, lowering legal blood alcohol limits, and using zoning ordinances to control the density and location of alcohol outlets and the hours of sale. Other strategies have shown positive potential and call for more research: limitations on beverage service, bans and restrictions on cigarette vending machines, provision of information in primary health care settings, legal liability for substances’ harmful effects, alcohol and tobacco warning labels, and administrative license suspension.
We have had great success showing city councils a chart that links problems in the community to the factors that influence them. (See “Alcohol-Related Trauma: Environmental Influences and Interventions.”) For example, to reduce traffic accidents, you could take aim at drinking and driving. The next step would be to analyze your options and your community and customize an approach that fits.
Paul Florin: To my mind, environmental strategies are an area where communities can make their greatest mark on the prevention field. It usually takes a coalition to make an environmental strategy work. And while environmental strategies are scientifically sound, they haven’t been developed into standardized, brand-name products the way, for example, school curricula have. That gives the coalition flexibility to tailor them to local circumstances and own them in a way they can’t with standardized curricula.
Dzierzawski: Our funders want us to prove that we are making a difference right along, without waiting the 5 to 10 years or more that it often takes for community laws and norms to change. They’d like something more concrete than, ‘Well, this year we engaged a group of legislators and now they buy into our philosophy.’ How do we show that activities like that contribute to a changed environment?
Holder: Gather data on good things that are happening that you can measure. Many data are readily available with little effort. For example, my community routinely measures certain kinds of harm data, such as the number of alcohol-related car crashes. The medical community keeps track of injuries. Another type of data that I can get at low cost are tobacco retail sales. Data like these can usually convince politicians that key outcome indicators are changing in the community, and that they should continue to support you.
David Hawkins: Surveys of youths’ perceptions of the availability and of social norms surrounding substances can be very useful. We know that when youths perceive that alcohol and tobacco are less available, they report less substance use. When youths perceive that their community’s laws and norms are less favorable to alcohol, they use less of it. So if you measure trends in these factors every other year, for instance, and they go in the right directions, you can show an impact on the very factors you are trying to affect with your strategies
Dzierzawski: When I dream of pie in the sky, it’s about finding a way to measure and demonstrate what the totality of our coalition activities contributes to bettering the community. Not only the things that are easy to document, like getting laws passed to control tobacco sales, but also the day-to-day things, like when we meet with law enforcement officials or the legislature or Block Watches, or when we do community-based mobilization.
Audience Participant: Can the panelists give us any help with proving that our activities have yielded cost savings per case, anything like that? We coalitions talk about the head and heart a lot, but the wallet is often the main way to get to funders.
Holder: That’s right, the bottom line is your best argument. If you can tell community decisionmakers that even though your smoking prevention program will cost a million dollars, that’s a wise investment because it will avoid much higher costs from tobacco-related disease down the line, they will listen. I strongly recommend that you keep track of the Web site of the Washington State Institute for Public Policy, which constantly updates findings on the cost-benefit of prevention and treatment. The URL is www.wsipp.wa.gov.
Marilyn McGinnis, Audience Participant from the Oak Park Prevention Policy Board, Sacramento, California: Our coalition has been taking environmental actions around injection drug use in a very poor urban neighborhood. We have influenced local businesses to put chains across their parking lots so prostitutes cannot use those locations to get money for drugs. People from our drug-free zones have coalesced to form a neighborhood association that pickets slum lords to clean up their properties and have higher accountability to their tenants. Three years ago we did a neighborhood cleanup.
The results of these activities don’t show up straightforwardly in drug statistics for various reasons. For example, even though the number of arrests during the month of May in a single eight-block area of the city fell from 222 to 11, this decrease has been mostly in arrests for parole violations. The dealers have lookouts up and down the block, so the police have a hard time catching them with drugs and instead take them in on parole violations. As a result, our measures for success are higher property values, anecdotal reports from police, and decreased overall arrests.
Hawkins: Basic prevention consists of identifying the drug abuse risk and protective factors that are most prevalent and the drug abuse protective factors that are most depressed in a community, and addressing them with tested and effective policies, environmental strategies, prevention programs, or actions.
In my work, I use the Communities That Care youth survey to gather information on risk and protective factors.1 This instrument is in the public domain; you can use it yourself or contract with Channing Bete Company to administer it and analyze the results for you. Once you see which risk factors are elevated and which protective factors depressed, you can use Communities That Care Prevention Strategies: A Research Guide to What Works to identify policies and programs that have been effective for addressing those specific factors.
For example, suppose 65 percent of the kids in your high school say on a survey that their parents don’t know where they are or who they’re with when they’re not at home. That’s an indicator of poor monitoring or family management problems. If that is the most prevalent risk factor in the community, the community may decide that parent training is a high priority. They can choose from a number of curricula that have been shown to improve parents’ family management skills, such as Guiding Good Choices, Parenting Wisely [formerly Parenting (Adolescents) Wisely], or the Strengthening Families Program: For Parents and Youth 10–14 [formerly the Iowa Strengthening Families Program].2 If the community has limited resources, that may be the only risk factor they can address. If they have more resources, they can add another policy or program component to address whatever risk factor the survey shows is the next most prevalent.
Audience Participant: Suppose our coalition sets its sights on enhancing parental involvement. We look at a program for this, and find that we can’t do everything in it. For example, one of the elements is to recommend to parents that they attend PTA, but we know that’s not possible because our parents work evenings. How do we know what elements in these programs are absolutely necessary for them to work, and which ones can we skip and still get results?
Hawkins: This is an area in which the science has advanced, but is still advancing. Researchers have not yet done what are called disassembling studies, in which you pick apart all the pieces of a program and identify which are the active ingredients and which are expendable. At present, then, my advice to you has to be that if you want to affect the family management risk factor, your best bet is to adopt a program that has shown the desired effects in a controlled trial, and do it thoroughly. If you do anything else, your outcome is not guaranteed.
Dzierzawski: Our coalition is up against this issue. We have a new mandate to address migrant workers in a six-county area. There is very little tested material for that population, so we proposed and were granted the flexibility to take three distinct curricula and adapt them. Our population is telling us what core components they feel they need for our efforts to be effective. But we’re finding that neither our local or national evaluators can say whether we will maintain the fidelity of outcomes of the curricula if we just extract and combine those pieces.
Hawkins: Nobody can tell you that. The scientists have tested whole programs or whole environmental strategies. If you use only some parts or combine parts from different programs, you are developing something new that has to be tested all over again. So, in my judgment, the safest course is to look at the approaches that have been tested and shown effective to see if the tests included the kind of people you have in your community. If they have, take the program, rather than trying to mix something up yourself.
Harry Kressler: What if there is no tested, effective program that both fits a community and addresses the risk it has identified as most critical? Suppose the community’s needs assessment has determined that family strengthening is what will help it most. SAMHSA’s Center for Substance Abuse Prevention (CSAP) at one time had a menu of maybe 15 tested, effective parenting curricula, but none of them were readily adaptable for some populations I have worked with.
Hawkins: First I would say, really kick the tires on those 15 programs. People sometimes say, ‘Oh, that won't work for us,’ before they have fully considered how it might be adapted.
When my colleagues and I first developed the program that is now called Guiding Good Choices, we demonstrated it with African-American and white families. Subsequently a colleague, Tracy Harachi, worked with us to adapt it for Asian Americans. We found that with Cambodians, we had to open the training beyond just the families, to the whole social network. Not just the parents of the kids, but also their 23-year-old uncles.
Sometimes you find that the adaptations are relatively straightforward, and then the program can be done. Maybe now it takes 15 sessions instead of 5 to get through it, and you serve meals to help participants feel comfortable and involved. But you can do it.
Kressler: Well, we do adapt. We’re great engineers. Still, can the research community help us to loosen some of the constraints of these standardized programs that we are having a really hard time adapting? Surely there must be other approaches that work.
Gwen Wilson: Our coalition is working to reduce children’s rates of drug abuse later on, when they enter adolescence and young adulthood—a long-term outcome. However, we’ve only been given a year of funding, and the funders want proof that we are making progress.
Hawkins: The working principle for prevention is that if you have an effect on the risk factors for an outcome, you should have an impact on the outcome. Therefore, monitor the risk factors you are targeting. If they are going down, you will know you are making progress, even before you see if the substance abuse has changed.
Audience Participant: I have utmost respect for science. However, I have more respect for the people in the communities I serve. I believe success does not depend on choosing the right science-based program, but on respecting the community, acting as facilitator, and following the community’s lead.
You can do all the scientific studies you want, but give me 100 residents who are fired up in their hearts to correct something, and all those results mean nothing. Whatever the people decide to do, even if they choose the action that was weakest in scientific evaluations, they will make it work. They will change their community. It happens every time, because they feel empowered.
Now, maybe the community hasn’t seen all the prevention models that are available. Then the coalition should serve as a facilitator, to bring these models to bear, but only when asked, never before.
Hawkins: Here is how I would respond to that: The Robert Wood Johnson Foundation conducted a large-scale, long-term evaluation of a program called Fighting Back. The basic premise of Fighting Back was that communities had within themselves the answers to all their problems. So, to deal with youth drug abuse, for example, the key to being effective was to bring the community people together to find solutions. The foundation believed in Fighting Back, but when the evaluation was complete, the results were not favorable. The outcomes were not positive.
So, I agree 100 percent that we must show respect and ask people, ‘What is it that you want to achieve and how do you think it is best achieved?’ However, based on the Fighting Back experience, I think we owe it to people to offer them the tools that research has shown to work.
Kressler: The prevention field is dominated by two models: risk and resiliency, and building assets. The risk and resiliency model in particular has guided us really well. But are there other models that we might consider, perhaps one that would be a better fit for my community, which borders on Mexico and is half American Indian?
Hawkins: I don’t think you are going to see many fundamentally new approaches emerging. To my mind, the risk and protective factors and the assets approach are not so much models as they are simply the public health approach. Public health says, if you want to prevent something from happening, you have to address the factors that researchers have shown are its predictors. Both models do that. They differ mainly in how much each emphasizes building assets alone versus both reducing risks and building assets.
Florin: I sympathize with the question. We researchers have gotten very bonded to the concept of governmentally approved programs—which are often packaged as ‘brand name’ curricula. The Government’s rating system for these programs may, however, inadvertently communicate the idea that those on the list are good and everything else is bad. It obscures the fact that in some cases, inexpensive environmental strategies can be very effective, especially if tailored well to local conditions.
Rather than looking at an inventory of model programs, one sensible approach is to decide what environmental changes you want in your community, then look at what has been shown effective for that purpose in other settings. Sometimes, you may want to use a standardized curriculum. Other times, you may find that an environmental approach is best. Often, you may want to combine program and environmental approaches.
To me, this is what we need to pay attention to, not necessarily simply a program that we like or would most like to do.
Audience Participant: I think you can marry the two, community preference and evidence-based choices.
We used the Communities That Care model to get input from our community, and we asked them what interventions they wanted for preschool, school-age, and older kids. They came up with ideas. One young gentleman pointed out, for example, ‘All your recreational programs are around basketball. In case you haven’t noticed, we’re short, and that doesn’t work for us.’ Such feedback was very helpful.
People also told us they thought one reason for substance abuse was that half our kids weren’t attending kindergarten. Seeing that kindergarten wasn’t required, many parents thought it wasn’t necessary. Our response was to create a school readiness program. The people running this program don’t really see its connection to the after-school program, but it is all part of our coalition’s big picture.
Florin: Those are the key words: ‘big picture.’ Whether your coalition is going to use this tool or that tool, never forget that your job is to have the big picture.
Wilson: Our community received some grants, so we were able to add more people to our coalition. The problem was that pulling more organizations together resulted in a collaborative activity that wasn’t as powerful as we wanted. In part I think our effort was too ambitious. We worked with expectant moms on prenatal prevention, elders on prescription abuse, and everyone inbetween. Another reason our results weren’t better, we thought, was that each agency had obligations to its own funders. They had to stay within their areas of expertise.
Deacon Dzierzawski, M.A.
Deacon Dzierzawski is executive director of the Community Partnership, an anti-drug coalition in Toledo, Ohio.
Harry Kressler, M.A.
Harry Kressler is executive director of the Pima Prevention Partnership in Tucson, Arizona.
Gwendolyn Hughes Wilson, M.A.
Gwen Wilson is executive director of the Summit County Community Partnership in Akron, Ohio.
Paul Florin, Ph.D.
Paul Florin is a professor of psychology at the University of Rhode Island. His research focuses on the relationships between community conditions and the health of community populations, and on the design of systems that supply training and technical assistance for such initiatives.
J. David Hawkins, Ph.D.
David Hawkins is professor of prevention at the University of Washington, Seattle, School of Social Work. His research focuses on preventing child and adolescent health and behavior problems.
Harold D. Holder, Ph.D.
Harold Holder is a senior research scientist at the Prevention Research Center of the Pacific Institute for Research and Evaluation. His research interests include the value of environmental strategies as part of comprehensive approaches to prevention.
Hawkins: This issue of collaboration versus focus can be a real struggle. The dilemma is between trying to pull everybody in, with their various visions, and having enough focus to be able to demonstrate effects from your activities. It is very important, as you are building coalitions or collaborations, to say, ‘Where can we get the most leverage or the greatest purchase first?’ That may depend on who you can get to come to the table in the first place. It may also depend on what scientific evidence is available with regard to what is effective in prevention.
Florin: A coalition is a vehicle. The question is, ‘What do you want to do?’ As long as the coalition focuses on the outcomes it wants, it is always going to be beneficial.
I don’t personally think the best thing for a coalition to do is to run a program. It is to make sure that the best array of programs is being done for the community and to engage in environmental strategy changes.
Holder: Coalitions help when they don’t get invested in a particular program or strategy, and they hurt when they do. The biggest problem I have had with coalitions is they get invested in balloons and banners or a particular program and lose track of whether it’s really working for them. When that happens, you are dead in the water.
Caryn Blitz: CADCA endorses the principle of multiple strategies over multiple sectors, which is also the stance of the Office of National Drug Control Policy, a co-funder of the Drug Free Communities Program. We agree that coalitions should do whatever works best for the outcomes you want to get. For some that will mean mainly a strategic and coordinating role. For others, particularly rural coalitions that are the only game in their area, it will mean they must deliver direct services.
Audience Participant: This is the first time in the last 20 years or so that I have seen a real focus on coalitions. I think we have a window of opportunity, but a small one. There is less and less money. Everybody is reorganizing. Can you give us any idea of where we stand? Is there enough evidence that we can convince our legislators? Will this support continue with the funding cuts coming down?
Holder: Proving coalition effectiveness is a challenge. There is sufficient evidence that community-level organization is essential to effective programs. For example, my State would not have a minimum drinking age today if a community organization—it happened to be Mothers Against Drunk Driving—hadn’t pressed for it. That is the message I would give funders. However, research is only beginning that is designed to determine whether coalitions or other kinds of community organizations might be more or less effective than one another in general or for particular purposes.
Florin: As David [Hawkins] has commented, branded programs can give you guaranteed results when you use them appropriately with full fidelity to the whole curriculum. As Harold [Holder] has indicated, environmental strategies are more in the public domain, as far as their specific ingredients go. My recommendation to coalitions is in accord with these remarks: Decide what you want to do, then use science-based curricula whenever you can, and supplement them with environmental strategies.
Kressler: I feel there is an overemphasis on the standardized brand-name programs. Frequently they are not well suited to our communities. The communities often are not enthusiastic about them, either.
Florin: The idea is to put into practice what we know works, not to force people to do things that aren’t going to work for them. We need to work with the interventions society has collectively invested in and shown to be effective, and meanwhile researchers can engage with communities to try to continually refine and expand them. Both the science and the politics are never-ending in this thing. Where we are now in terms of choices is just a stage in this ongoing process.
Kressler: It seems to me communities get short-changed on research resources. Discoveries we make at the local level in one community might turn the tide for other communities, too.
Florin: If I were king for a day, communities would all be empowered to evaluate their own programs.
McGinnis: Our coalition is broad based. We feel strongly that to bring about a reduction in substance abuse we need a curriculum for our kids that develops all the youth competencies, not just drug and alcohol refusal skills. To this end we have been integrating several research-based curricula aimed variously at preventing substance abuse, teen pregnancy, violence, and so on. We have little funding for evaluation, and it’s hard to do pre-and post-testing because people drop in and out. How do you recommend we evaluate what we are doing?
Holder: First, be clear about the purpose of your evaluation. Is it to convince policymakers and funders that things are going in the right direction and they should continue to support your activities? If that’s your goal, you can do it inexpensively with harm data, such as numbers of single-vehicle traffic accidents and so on, as we discussed earlier. Is your goal to get on a list of recognized effective programs? That’s going to require a more stringent standard of evidence.
Florin: I would not recommend that any coalition spend much time and effort trying to build its homegrown curriculum into a model program according to SAMHSA’s National Registry of Effective Programs (NREP) criteria. It would take years and years. You would be competing against people who have had a lot more experience at that game than you have.
Holder: It’s true that it’s not easy for community agencies to get on the NREP list. However, I think it really is important to carry your program beyond being a onetime innovation to being something that could become institutionalized. Suppose you have measured your target variable and run your program and shown improvement. Maybe you’ve repeated this exercise several times, always with good results. Your issue then is, how do you get to the next step? Could you become an NREP program at some point? This is where researchers and the practice community have to come together; it is this linkage that is critical for becoming an NREP program.
Audience Participant: My question has to do with attribution. Let’s say that a community has several programs. One focuses on environmental change, others are supported by, let’s say, the Department of Education, SAMHSA, NIDA, and so forth. The community is being asked to demonstrate that these programs or these grants are working. Now, how do we disaggregate the effects? Which one of those programs will claim success? At what cost?
Florin: I say don’t try to isolate one particular effect from another. That way if anybody is doing anything good, if you are part of it and you are contributing to it, you can say, ‘I don't know what would have happened if our coalition hadn't been there.’
Holder: That’s right. Share the credit.