PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of pubhealthrepPublic Health Reports
 
Public Health Rep. 2009; 124(Suppl 1): 63–73.
PMCID: PMC2708120

Using Logic Models in a Community-Based Agricultural Injury Prevention Project

SYNOPSIS

The National Institute for Occupational Safety and Health has long promoted the logic model as a useful tool in an evaluator's portfolio. Because a logic model supports a systematic approach to designing interventions, it is equally useful for program planners. Undertaken with community stakeholders, a logic model process articulates the underlying foundations of a particular programmatic effort and enhances program design and evaluation. Most often presented as sequenced diagrams or flow charts, logic models demonstrate relationships among the following components: statement of a problem, various causal and mitigating factors related to that problem, available resources to address the problem, theoretical foundations of the selected intervention, intervention goals and planned activities, and anticipated short- and long-term outcomes. This article describes a case example of how a logic model process was used to help community stakeholders on the Navajo Nation conceive, design, implement, and evaluate agricultural injury prevention projects.

Evaluators have long been encouraged to use logic models as a foundation for understanding the linkages among program activities, objectives, and outcomes. Similarly, federal agencies often require program planners to prepare logic models to depict program strategy. However, few reports are available describing the processes that program planners and evaluators undertake together to articulate program theory and design programs and evaluations in a synergistic manner. Examples of such endeavors would be particularly useful to enlighten members of participatory research teams who seek to implement culturally appropriate, community-driven programs to address specific local problems or health disparities. This article presents a description of a logic model process that was used to engage community stakeholders from the Navajo Nation and university faculty to conceive, design, implement, and evaluate an agricultural injury prevention project.

PROBLEM OF AGRICULTURAL INJURY AMONG MINORITY POPULATIONS

While studies of agricultural injury among Hispanic and African American populations have been conducted in recent years,17 insufficient attention has been given to the occupational health problems of American Indian farmworkers, who experience significantly higher occupational mortality compared with other ethnic groups.811 Similarly, there are few descriptions of the use of tools that enhance opportunities for community members to reach a common understanding about locally responsive public health interventions and evaluations that address agricultural injury within this population.

Tools that enhance participation in planning and evaluating interventions may augment efforts to reduce agricultural injury, particularly among populations in which risk is high and few interventions have been developed and implemented.12 Nevertheless, a recent literature review of agricultural interventions minimized discussion of opportunities to increase participation.13 Real-world examples are needed to illustrate how tools that increase participation can serve as vehicles to improve the development and evaluation of agricultural injury prevention programs. Community-based approaches are increasingly viewed as essential for tailoring health-promotion strategies to local contexts.12,14,15 Participatory research is conducted through systematic inquiry in collaboration with those affected by the problem under study for the purpose of educating and effecting social change.16

In community-based participatory research, researchers and stakeholders from the intended audience as well as representatives from local agencies work in partnership to identify and define specific issues that require investigation; to design, implement, and evaluate interventions; and, finally, to disseminate research findings within community settings.17,18 The fiscal year 2009 strategic goals of the National Institute for Occupational Safety and Health (NIOSH) to reduce occupational health disparities promote the use of community-based participatory research methods, especially in intervention research.19

Logic models as planning tools for interventions and evaluation

Health promotion and disease-prevention programs often have goals that are ambitious and take a long time to achieve. Careful planning is important for goal achievement and the continued satisfaction of participants and funders.20 In addition, articulation of program theory is essential in designing effective interventions.21 Even though planning is essential, this component is often a hurried step in community-based programs, with little attention paid to its process.22 The Centers for Disease Control and Prevention (CDC) calls for evidence-based planning that incorporates the perspectives of the groups for whom the programs are intended, as well as the providers of services.23 Research on the quality of planning by community-based groups has found their plans to be poorly designed, often lacking measurable objectives and grounding in science.2426 Effective planning enhances not only intervention quality, but also the ability of the evaluation to accurately measure intended processes and outcomes.2731 Logic models can provide a productive framework for effective planning and a depiction of the theory of change of an intervention.32

A logic model is one tool in participatory research that can help stakeholders conceptualize an intervention's underlying foundation and principles. According to Renger and Titcomb, a logic model is a “visual representation of a plausible and sensible method of how a program will work under certain conditions to solve identified problems.”33,34 A logic model is intended to convey in a simple and clear-cut manner “the underlying \?\`theory,' or set of assumptions…that program proponents have about why the program will work, or about why it is a good solution to an identified problem.”27,29,35 Program-level logic models are often meta-summaries of complex processes; as a result, additional logic models may be needed to “unpack” each component in the original model so that more details can be articulated. Most often presented as sequenced diagrams or flow charts, logic models demonstrate -relationships among the statement of a problem, various causal and mitigating factors related to that problem, the purpose of the proposed program, planned activities meant to address these factors and attain program goals, and anticipated program outcomes or results. These models serve as valuable tools in the development of long-term plans of action given resource constraints, and in outlining appropriate evaluation plans, indicators, and methods. Logic models have been widely used in community health initiatives and, increasingly, in participatory evaluation efforts. The goal of using a logic model is to make explicit the theories behind the planning, implementation, and evaluation of a programmatic effort.

CDC's evaluation framework22 recommends engaging stakeholders in logic model development to increase the usefulness and validity of the resulting program. Examples of these endeavors include efforts by the CDC Prevention Research Centers, which used logic models to develop a national evaluation strategy,36 and East Carolina University's Department of Family Medicine, which used a logic model process to strengthen an existing Food Literacy Partners Program. Comparing the original program with the model allowed program partners to see “where value could be added or more efficient use of resources could be made.”37 The CDC-funded Bronx Health Racial and Ethnic Disparities in Health Status Project designed a faith-based project to increase awareness of cardiovascular disease and diabetes using a detailed logic model. This initiative successfully helped churches provide better food and health advice to citizens.38

Logic models are used in occupational health and agricultural extension program development. The Haddon Matrix, applied to injury interventions, represents the earliest application of a variation of a logic model, enabling the development of structural interventions through a systematic consideration of a set of factors about the interaction among the agent, host, and environment.39,40 Logic models have been used in the last decade at the Pennsylvania State University Cooperative Extension (http://pow.cas.psu.edu/logicModel/index.htm)41 to design programs to control workplace exposure to mining health and safety hazards, and by the University of Florida Extension Service to design a food handler training program.42 The Rand Corporation utilized logic models with the National Center for Injury Prevention and Control in strategic planning exercises.43 In 2004, the National Safety Council stated that agricultural injury prevention programs should use logic models “to increase [their] effectiveness…and more proactively identify workplace hazards and establish more challenging and innovative goals toward hazard reduction.”44 A logic model would be a useful accompaniment to an intervention mapping (IM) exercise, as it could be used to depict the outcomes of the core IM processes.45 However, few recent examples within the peer-reviewed literature illustrate the application of a logic model approach within the field.46

The W.K. Kellogg Foundation's Logic Model Development Guide47 notes that logic models often arrange the elements of a program into the following groups:

  • Resources/inputs: financial, human, and institutional resources available to implement the program
  • Activities: actions, events, and/or techniques used to reach program goals
  • Outputs: program materials or “deliverables”
  • Short-term outcomes: benefits or changes in knowledge, attitudes, or behaviors expected to occur as a result of the program
  • Long-term outcomes: a fundamental, sustained improvement in the lives of those affected by the original problem

We have augmented this traditional logic model framework to include consideration of key contextual, risk, and protective factors. These factors hinder and/or enhance the well-being of the target population, or attainment of the goal set, and may affect the program at the individual, family, programmatic/organizational, or policy level. It is vital to take into account key contextual factors and external influences that will affect the program.

The design of a logic model is a function of the thoughtfulness that goes into its construction; the logic model's apparent simplicity belies the complexity of the intervention planning and evaluation process. Indeed, poorly designed models based on a rushed job, lack of attention to local context and stakeholder views, or overly simplistic assumptions of resource needs, and those that do not clearly depict a theory of change or incorporate social science theory, can prove to be superficial and potentially less useful rather than productive.

We offer a case study about the development process and the use of a logic model that incorporated the elements found in classic logic models.21,32,47 The case study pertains to a small population of American Indian agricultural workers in the southwestern United States. In particular, we describe our experience of engaging Navajo (or Diné) farmers and ranchers in a comprehensive planning and research process that had, as its long-term goal, the reduction of agricultural injury on the Navajo Nation. This capacity-building project resulted in the implementation of several agricultural injury prevention projects, among them an integrated pest management (IPM) safety intervention. For the Navajo farmers and ranchers, the use of a logic model process represented an activity that was both occupationally and culturally appropriate, as it bore similarity to the steps involved in local agriculture: identifying environmental risk and protective factors for specific crops; using available resources; and focusing on the activities of planting, cultivating, and harvesting to reach the goal of a productive yield while minimizing risks and maximizing protective factors. This process may be applicable to other populations of agricultural workers, as well as to the general field of occupational health and safety interventions.

THE NAVAJO CONTEXT

The Navajo Nation, which stretches into Arizona, Colorado, New Mexico, and Utah, consists of five agencies representing specific geographic locations. A partnership was forged among grazing and ranching committee members in the Shiprock Agency, comprising 20 chapters. Tribal government agencies, local community agencies, staff of the Shiprock Cooperative Extension Service, and 20 community members helped to implement this partnership; faculty from two academic institutions provided technical assistance to use a science-based process to understand the agricultural injury problems facing this community, identify risk factors, identify successful agricultural safety interventions, plan the intervention, and develop evaluation designs and instruments.48 The partnership's goal was to spearhead new interventions that would be responsive to the primary causes of agricultural injury among Navajo farmers and ranchers, whose voices have not been well represented in the larger literature on occupational health.

Small family farms and ranches predominate in the Shiprock Agency. On average, each of these farms and ranches is smaller than 10 acres. Most herds have fewer than 100 cattle and include smaller animals (e.g., sheep). Tractors are used on a limited basis. Protective equipment is in short supply due to high cost (i.e., too expensive for any individual). Farmers are not formally trained, and local agricultural practices are passed down from generation to generation. The mean age of farmers and ranchers is older than that of the general population, and most engage in agricultural work on a part-time basis.49

Due to the exposure to mass-media advertisements, agricultural-related television and radio shows, commercial promotion efforts, and local agricultural extension agents, as well as increasing pressure to improve yields, IPM techniques have been promoted in the Shiprock Agency. However, until recently, there has been very little use of modern practices, such as pesticide application. In fact, there is no pesticide training available on the reservation. Those few farmers who have obtained their pesticide application licenses off reservation must also travel to nearby towns in Colorado to purchase pesticides, as pesticides are not sold on the Navajo Nation. In farm families, one member of the farm typically obtains a pesticide applicator's license on behalf of the family.

The logic model process described in this article was a component of a capacity-building project funded by NIOSH; its goal was to improve agricultural community members' ability to use science-based research processes to identify agricultural risks and to design and evaluate interventions to ameliorate those risks. During this project, stakeholders revised, translated, and implemented an epidemiologic survey50 to identify the scope of agricultural-related risks, both to farmers and ranchers on the Navajo Nation. The details of the survey content and its development and administration process have been described elsewhere.49 Some of the important areas identified as potential risks, given the limited experience with IPM, were pesticide storage, application, and mitigation of spills. As increasing crop yield was of primary importance to the farming and ranching community, the stakeholder group chose to develop an intervention to assure that in an effort to improve their financial situation, these essentially naive users of pesticide technology would not be placing themselves at risk due to lack of knowledge, incompatible attitudes, or hazardous behaviors related to pesticide use.

The overwhelming sentiment of the stakeholder group before initiating the logic model process was that the intervention should address prevailing attitudes and knowledge about IPM as depicted in the survey results, to further clarify the benefits of IPM, and to train farmers in safe handling, rather than to disseminate pesticides widely. Because pesticides themselves are not yet readily available, the project itself focused first on attitudes, knowledge, and behavior related to safe handling, in anticipation that future projects would address the use of pesticides and IPM to improve crop yield through culturally appropriate dissemination techniques. The community felt it was important to lay the groundwork for an increasing desire for IPM in this first project.

METHODS

Several steps were taken to initiate the process of planning the IPM intervention.49 First, stakeholders made a collective attempt to identify and assess local and non-local sources of data that could shed light on successful pesticide safety interventions in the United States and in other regions of the world. The scientific literature on agricultural injury was reviewed, and several experts on agricultural injury, within and outside the Navajo Nation, were consulted. The stakeholder group met several times during a one-month period to begin to articulate the different components of the logic model that would lead to development of a program theory (also called a theory of change21). In the first meeting, the group used the data from the epidemiologic survey to arrive at a set of risk and protective factors for IPM. During the second meeting, the group brainstormed about resources, including grant funds and local, state, and national agencies interested in agricultural safety issues, which could be called upon to collaborate. At this point, the stakeholder group discussed the ways in which farmers learned to farm and the ways in which information could be disseminated to the larger farming and ranching community.

Having chosen a topic to address, the stakeholder group reviewed the literature on behavior change models and chose the theory of Diffusion of Innovations (DI)51 on which to base decisions about and activities for the interventions. The components of DI provided a focus for thinking about how attitudes concerning the attributes of the innovation, or IPM, would be central to farmers' willingness to implement safety behaviors. The theory also provided opportunities to discuss communication channels and who would likely be change agents and opinion leaders in the community. Finally, the notion of early adopters and laggards helped to focus the discussion on how safe practices might catch on if the right people were carrying the message. This led the group to articulate a theory of change for the intervention. During subsequent meetings, the group again turned to the available literature for examples of promising interventions. After these preliminary steps, the group was prepared to develop a logic model for an initial IPM intervention.

The next step in the process was to identify locally appropriate activities, which they believed would address the identified risks and protective factors, while making maximal use of the identified resources. Between meetings, subgroups of stakeholders took responsibility for outlining the details of each activity and for approaching identified agencies to solicit support. During this phase, the stakeholder group discussed possible outcome objectives, while the evaluators provided examples of designs and measures to assess the success of the planned intervention.

The development of the logic model took place during many meetings, with additional fact-finding and outreach to likely collaborative partners between meetings. There were significant challenges in keeping stakeholders engaged early in the process. In the Navajo culture, talking about an injury or a death is tantamount to causing it to happen. For this reason, stakeholders were reluctant to focus too much attention on the details of risk, but they were more comfortable talking about factors that were protective. Very quickly, the group moved on to discussing available resources, which energized them and helped them to see that there were many groups whose missions would overlap with the program's goals and to whom they could turn for support.

As with most health educators, the group members were eager to develop intervention activities, and it was difficult to keep the focus on the more abstract concept of articulating change processes. Ironically, the selection of the DI theory brought the group together because there was consensus that it embodied a culturally relevant model of communication and behavior change. Once the program theory was articulated, the identification of activities easily flowed and the stakeholders marveled at the way these activities seemed to fit together. Rather than being a threatening aspect of the development work, the discussion about evaluation became an opportunity for them to demonstrate that their intervention could make a difference for their community. Subsequently, the group was engaged in selecting an evaluation design and methods; developing, piloting, and revising an instrument; and planning for its implementation.

RESULTS

Logic model development

There are many formats for logic models.21,32,45 The stakeholder group selected a format that made sense in the context of farming and ranching. The format consisted of columns stretching across a page from left to right that depicted the problem and its risk and protective factors, available resources or inputs, activities or outputs, and desired short- and long-term outcomes. This layout facilitated a focus on each component, so that the details of each could easily be elaborated in a systematic way. Each segment of the model became the focus of a separate meeting, allowing time between meetings to collect more information to prepare the subsequent step and to seek input from other resources, thus refining the work already started on previous elements. This enabled the stakeholders to implement a very collaborative process for planning, mirroring a traditional practice of consultation and joint decision-making.

Risk and protective factors.

In the first column of the Figure, risk and protective factors for pesticide safety were compiled from those identified in existing literature as well as locally available information. To focus the discussion on community-specific factors, time was spent discussing those that emerged from the epidemiologic survey, selecting risk factors amenable to intervention. In particular, pesticides and herbicides had only recently been introduced to the Navajo Nation and, as a result, there was very little knowledge or understanding of how to use them or what positive (or negative) impact one could expect from applying them. Those factors became the basis of the desired outcomes. For example, the risk factor “fear” led to a long-term outcome of “less fear to use pesticides as a result of increased knowledge.” Similarly, protective factors were identified. For example, cooperation among neighbors was characterized as a protective factor because a greater number of trained farmers working together could lead to reduction in risk.

Figure
Logic model: theory of change to improve knowledge, attitudes, and safety behaviors related to IPM

Resources or inputs.

For the second column in the Figure, the group discussed all the different resources generally available to residents on the Navajo Nation, within the state and within their chapters and smaller communities to address the problems of low knowledge, negative attitudes, and risky behaviors related to IPM. Four basic questions were posed: (1) What financial resources were available in addition to the grant funds? (2) What other kinds of resources were available? (3) Who were potential collaborators for the intervention? (4) What groups were already working on similar issues? Once a list of potential collaborators was compiled, the group discussed the potential barriers and enablers to likely collaboration.

The group recognized that agricultural extension funds already allocated for similar purposes could be leveraged if the group could demonstrate that by cooperation, the agency would be in a better position to accomplish its own goals. Examples of such potential collaborators included pesticide merchants and state injury prevention agencies. Conversely, some groups were working in similar areas but were considered not likely to collaborate because of recent loss of staff or other resources. These discussions led to follow-up meetings with additional stakeholders to determine their interest.

At this stage, the goals of other individuals and agencies were outlined, and opportunities for mutual achievement of goals were discussed. Slight adjustments were made to accommodate the concerns of collaborating agencies. For example, the Water Users Board wanted assurance that increasing the use of pesticides would not imperil the water used for irrigation throughout the Navajo Nation. This resulted in an agreement to test the water for pesticide exposure before, during, and after the intervention. After confirming the resources available to address the problem, the group began to plan the interventions.

Activities or outputs.

The activities of the intervention are also listed in the Figure. The first step in designing the activities was to consider their rationale. The stakeholder group had decided that because the purpose of the intervention was to improve the likelihood that farmers would be better informed about the benefits of proper IPM techniques, there were several sub-objectives that had to be addressed. First, farmers should be more likely to believe that IPM would easily be incorporated into their farming practices. Second, farmers should be better trained so that IPM would be applied in a safe and effective manner. An outline for activities was conceived, including knowledge-related activities, skill-related activities, and activities designed for the purpose of changing attitudes.

Next, stakeholders considered the elements of the DI,51 focusing specifically on those that would address the constructs underpinning the theory. The theory was originally developed to explain the diffusion of hybrid corn in the midwestern United States in the 1950s and, therefore, was particularly relevant to the problem facing the stakeholder group. Specifically, the DI theory has four elements: (1) the specific attributes of the innovation; (2) the social system in which the innovation is being introduced, including the innovation-decision processes, the external change agents, and the internal opinion leaders that affect adoption; (3) the time and critical mass required for the widespread diffusion and sustainability of an innovation; and (4) the ways in which information about the innovation were communicated to potential new adopters, opinion leaders, and influential change agents.

The stakeholder group understood that the activities of the intervention needed to address farmers' attitudes about several attributes so that the innovation would be accepted: compatibility, trialability, relative advantage, complexity, and observability. For example, for IPM to be observable, demonstrations of IPM use were arranged. The social system in which the innovation would be introduced was the farming and ranching community. Opinion leaders and change agents to carry the message needed to be identified. The stakeholders considered themselves to be opinion leaders, change agents, role models, and advocates for use of the new technology.51 The attitude-changing activities used such individuals to deliver messages demonstrating, for example, how pesticide use was compatible with current practices. The 20-chapter area of the Shiprock Agency includes thousands of acres of farmland. For a critical mass of farmers to implement IPM, the intervention activities would need to take advantage of natural gatherings of farmers and ranchers, such as the Shiprock agriculture days and the monthly chapter meetings held by the farming and ranching chapter leaders.

Activity design also took into account the seasonal cycle of planting, growing, and harvesting. IPM training would involve demonstrations of how pesticides affect growth. The pesticides had to be applied during the planting season so that changes could be observed in the subsequent harvest season. Some effects were determined to take longer than one annual cycle, which required long-term planning. Available materials on proper pesticide use would be used, but new locally developed materials would require before-and-after pictures of pesticide application. Activity planning not only involved consideration of the “bigger” activities, but also the steps required to develop and implement these activities. For example, “setting out demonstration plots” meant identifying local farmers who would agree to allow their land to be used for demonstration plots, choosing and procuring the correct pesticide for each type of crop, renting spraying equipment and learning the correct calibration techniques, spraying plots, and regularly observing the impact of the pesticides on crop production and pest elimination.

To bring local farmers to demonstration plots, buses had to be rented; dates set; advertisements disbursed through flyers, local newspapers, and radio; and trainers hired to conduct tours. Each step required several subactivities to be accomplished. For example, to place advertisements in local newspapers, text had to be written and negotiations for free space had to occur. These details were captured on paper by the stakeholder group in subsequent meetings, and served as the basis of budgets for the implementation process.

Outcomes.

Finally, the remaining columns (short- and long-term outcomes) were outlined, and in an iterative process, reconsidered after the program theory of change was articulated. Short-term outcomes were designed to address several components: (1) the attributes of the innovation as described in the DI theory, (2) knowledge of proper IPM, and (3) enhancing the ways in which information about the innovation were communicated to potential new adopters, opinion leaders, and influential change agents. For example, overcoming the risk that people may want to leave things the way they are was seen as an issue of relative advantage and compatibility. Thus, the short-term outcome was for farmers to believe that pesticides were innovations with positive attributes that could be incorporated into current farming practice.

Relationship between the logic model and the evaluation

The development of the activities and outcome lists led to a discussion of evaluation designs, instruments, measures, and variables. The stakeholder group selected the RE-AIM Framework for Behavior Change Interventions52 (reach, efficacy, adoption, implementation, and maintenance) as a structure for the evaluation plan. First, the stakeholders wanted to ensure that the intervention reached the target audience in large enough numbers so that a critical mass of farmers might change their behaviors. They understood that the evaluation should be designed to determine if the intervention itself was efficacious at changing knowledge, attitudes, and behaviors. They were concerned that the intervention would be adopted by local agricultural leaders and consistently implemented. Finally, the stakeholders wanted to make sure that the intervention outcomes would be maintained in the population over time.

To assess the first three elements of the RE-AIM framework—the reach of the intervention to the target audience; the efficacy of the intervention in changing knowledge, attitudes, and behaviors; and the adoption of the intervention and increase in dissemination over time—several evaluation plans were developed. Evaluation methods were designed to assess the intervention's reach (e.g., sign-up sheets at events, attendance lists at demonstration plot showings, counts of individuals at chapter meetings when IPM was discussed, and applications for new pesticide applicator licenses). The stakeholder group considered the entire Shiprock Agency as its catchment area, so the group conducted a census of the farmers in each of the 20 chapters and attempted to determine the proportion of those farmers who had been exposed to the intervention, and the ways in which they had been exposed.

To assess the intervention's efficacy, farm walk-throughs by agricultural extension agents examined demonstration plots and safe-handling practices (e.g., gloves, safe storage, and handling of spills). A pre/post survey used standardized measures to detect changes in attitudes related to attributes of IPM use (e.g., complexity) and knowledge of safe-handling practices. To assess the level of intervention adoption and to determine whether there was an increase in dissemination factors, participant observations were designed to determine if change agents and opinion leaders disbursed information flyers, whether farmers talked to one another about IPM, and whether there were discussions about IPM among farmers attending the local agriculture-related events.

Results of the logic model development process

We administered a survey to the stakeholder group to assess their perceived changes in capacity as a result of participating in the logic model development process. Stakeholders reported that they were better able to (1) understand the issues related to agricultural injury, (2) list the different things that play a role in agricultural injury, (3) choose from a list those things that the group could change to reduce or prevent agricultural injury, (4) develop a specific strategy for reducing or preventing agricultural injury, (5) understand why a specific strategy for reducing or preventing agricultural injury will work, (6) set goals and objectives for a safety program for reducing or preventing agricultural injury, (7) develop ways to determine whether the goals and objectives for a safety program have been met, (8) understand the role of research in designing and evaluating a safety program for reducing or preventing agricultural injury, (9) interpret research data, and (10) pay attention to other people's perspectives. The individuals believed that the stakeholder group as a whole was able to achieve consensus in most matters and that the overall planning process for a safety program and evaluation plan had been useful to stakeholder group members. Further details regarding change in capacity are available.49

DISCUSSION

Increasing community participation in intervention planning

Several implications for practice and policy are associated with the use of a logic model approach in occupational health interventions in general and agricultural safety projects in particular. The approach demands that the perspectives of those most intimately affected by the problem are the driving force of intervention development activities. Rather than allowing outside experts to determine the goals, objectives, and implementation steps of an intervention, community voice is considered fundamental to this process. Given this emphasis on community voice, there is greater likelihood that the intervention will be responsive to the local context at the start and for the duration of the planning process.

The inclusive nature of this process was evident because the community members were able to choose a logic model format that mimicked the thought process that they used in their agricultural planning. The focus on each step was deliberate and enabled partners to not only inventory local available resources, but also leverage them in new ways. This consultative process brought together stakeholders who otherwise might not have been in contact with one another, operating in their own silos or realizing that they could do more together than on their own. This synergy reduced the competition for scarce resources as partners worked together to address a common problem. The collaborative construction of the logic model ensured that all those involved—stakeholders, intervention planners, and evaluators—shared a common understanding of the program.

Building a visual model of how a program operates required stakeholders to state explicitly how they thought program activities and resources would lead to desired outcomes. Program planners tend to focus on activities because they are the most creative part of planning and require the most work, but the use of the logic model process forced the entire group to be more deliberate and to focus discussion on the details of how to address their specified theory of change. Articulation of how specific activities would effect change proved to be a more straightforward task when using a logic model process. Additionally, this process quickly made apparent activities lacking grounding in short- and long-term objectives and prevented wasting energy and resources on those activities that were unlikely to effect change.

Creating a unified theory of change

Developing a logic model with stakeholders serves two critical needs. First, it enables individuals with diverse expertise to create a plan of action that is understandable, doable, theoretically grounded, and likely to attain stated goals and outcomes. Although time-intensive, engaging in this initial planning work in an inclusive manner results in activities that are responsive to local contexts. A logic model developed in a participatory manner with community stakeholders makes apparent how the theory of the program comprises clearly articulated cultural components. For example, in this case, stakeholders chose the DI and described how the theory fit well with their particular way of learning about farming and how information about farming practices is diffused through their culture. This translated into an explicit and cohesive framework that guided the development of interventions.53

Evidence-based agricultural health promotion -strategies have not been well tested in American Indian settings. However, the application of these strategies can positively impact groups that experience occupational injury disparities. The challenge is to advance frameworks to adapt these strategies so that they can be appropriately applied within a variety of social contexts. Logic models offer the potential to enhance adaptation processes.

Developing a unified evaluation plan

The second critical need filled by a logic model process is that it increases receptivity to the need for and design of the evaluation. Use of an evaluation structure, such as the RE-AIM framework, helped to focus evaluation design and measurement development and provided a similarly logical process for asking questions about the intervention that could be answered by a comprehensive evaluation plan. Logic model development is helpful for gaining awareness about evaluation processes, while reducing anxiety and increasing interest in evaluation activities. Once a set of short-term objectives is articulated, the process of defining indicators is more transparent. The search for and adaptation of validated instruments or the development of new instruments becomes focused when these indicators are defined, and stakeholders become invested in obtaining objective information about their progress and program rather than threatened by an outside audit of their success or failure.

In venues where agricultural intervention effectiveness is discussed, the need to identify measurable process and outcome indicators is an imperative requiring attention. This is true if an intervention is to move beyond the research phase (efficacy) into the replication and effectiveness phase.54

CONCLUSION

Logic models can assist those working in agricultural communities to articulate inputs, outputs, and effectiveness indicators in ways that are understandable and culturally appropriate. Similarly, the likelihood of ameliorating the problems of agricultural injury and occupational health among groups with greater disparities in risk and outcomes becomes greater when the affected population participates in the design of efficacious interventions. A logic model is a tool that is useful, simple, and intuitive. As one farmer stated, “I like using the logic model because the steps we use are the same as the ones I use to plan my farm.” Using a tool that demystifies the process of intervention planning and evaluation so that community members can engage one another to address their own health problems is a good step forward to improving public health, and is a process that can be emulated in different population groups and for different public health problems.

Acknowledgments

The authors thank the stakeholders of the community collaborative, without whom this work would not have been possible. The Navajo Nation Human Research Review Board (NNHRRB) and the University of New Mexico Human Research Review Committee approved the procedures used in this project, and both institutional review boards continued to oversee and approve this work annually. This article has been approved for publication by the NNHRRB.

Footnotes

The work described in this article was supported through a grant from the National Institute for Occupational Safety and Health, Centers for Disease Control and Prevention Cooperative Agreement #U50 OH 07541-05.

REFERENCES

1. Arcury TA, Quandt SA, Russell GB. Pesticide safety among farmworkers: perceived risk and perceived control as factors reflecting environmental justice. Environ Health Perspect. 2002;110(Suppl 2):233–40. [PMC free article] [PubMed]
2. Flannery D, Portis M, Adame J. Outreach to migrant farmworkers through a pesticide safety initiative. California J Health Promot. 2003;1:13–35.
3. McCauley LA, Lasarev MR, Higgins G, Rothlein J, Muniz J, Ebbert C, et al. Work characteristics and pesticide exposures among migrant agricultural families: a community-based research approach. Environ Health Perspect. 2001;109:533–8. [PMC free article] [PubMed]
4. McCauley LA, Shapiro SE, Scherer JA, Lasarev MR. Assessing pesticide safety knowledge among Hispanic migrant farmworkers in Oregon. J Agric Saf Health. 2004;10:177–86. [PubMed]
5. Quandt SA, Arcury TA, Austin CK, Cabrera LF. Preventing occupational exposure to pesticides: using participatory research with Latino farmworkers to develop an intervention. J Immigr Health. 2001;3:85–96. [PubMed]
6. McGwin G, Jr, Enochs R, Roseman JM. Increased risk of agricultural injury among African-American farm workers from Alabama and Mississippi. Am J Epidemiol. 2000;152:640–50. [PubMed]
7. Omishakin MA. A survey of pesticide containers management among African-American agricultural workers in Mid-Delta of Mississippi, USA. J R Soc Health. 1994;114:81–2. [PubMed]
8. Crandall CS, Fullerton L, Olson L, Sklar DP, Zumwalt R. Farm-related injury mortality in New Mexico, 1980–91. Accid Anal Prev. 1997;29:257–61. [PubMed]
9. Fullerton L, Olson L, Crandall C, Sklar D, Zumwalt R. Occupational injury mortality in New Mexico. Ann Emerg Med. 1995;26:447–54. [PubMed]
10. Sugarman JR, Stout N, Layne LA. Traumatic fatalities at work: American Indians and Alaska Natives, 1980 through 1988. J Occup Med. 1993;35:1111–22. [PubMed]
11. Schnitzer PG, Bender TR. Surveillance of traumatic occupational fatalities in Alaska—implications for prevention. Public Health Rep. 1992;107:70–4. [PMC free article] [PubMed]
12. Arcury TA, Quandt SA, McCauley L. Farmworkers and pesticides: community-based research. Environ Health Perspect. 2000;108:787–92. [PMC free article] [PubMed]
13. Rautianinen R, Lehtola MM, Day LM, Schonstein E, Suutarinen J, Salminen S, et al. The Cochrane Collaboration. New York: John Wiley and Sons; 2008. Interventions for preventing injuries in the agricultural industry (review)
14. Goodman RM, Speers MA, McLeroy K, Fawcett S, Kegler M, Parker E, et al. Identifying and defining the dimensions of community capacity to provide a basis for measurement. Health Educ Behav. 1998;25:258–78. [PubMed]
15. Rosskam E. Participatory/problem-based methods and techniques for training in health safety. New Solut. 2001;11:215–27. [PubMed]
16. Green LW, George MA, Daniel M, Frankish J, Herbert CP, Bowie WR, et al. Background on participatory research. In: Murphy D, Scammell M, Sclove R, editors. Doing community-based research: a reader. Amherst (MA): Loka Institute; 1997. pp. p. 53–66.
17. Minkler M, Wallerstein N. Community-based participatory research for health. San Francisco: Jossey-Bass; 2002.
18. Israel BA, Schulz AJ, Parker EA, Becker AB. Review of community-based research: assessing partnership approaches to improve public health. Annu Rev Public Health. 1998;19:173–202. [PubMed]
19. National Institute for Occupational Safety and Health (US) [cited 2008 Feb 10]. Occupational health disparities: NIOSH strategic goals. Available from: URL: http://www.cdc.gov/niosh/programs/ohd/goals.html.
20. Hawkins JD, Catalano RR., Jr. Communities that care: action for drug abuse prevention. San Francisco: Jossey-Bass; 1992.
21. Donaldson SI. Program theory-driven evaluation science: strategies and applications. New York: Lawrence Erlbaum Associates; 2007.
22. Linney JA, Wandersman AH. Rockville (MD): Department of Health and Human Services, Office for Substance Abuse Prevention (US); 1991. Prevention plus III: assessing alcohol and other drug prevention programs at the school and community level: a four-step guide to useful program assessment.
23. Framework for program evaluation in public health. MMWR Recomm Rep. 1999;48(RR-11):1–40. [PubMed]
24. Fetterman DM, Kaftarian S, Wandersman A, editors. Empowerment evaluation: knowledge and tools for self-assessment and accountability. San Francisco: Sage Publications; 1995.
25. Butterfoss FD, Goodman RM, Wandersman A. Community coalitions for prevention and health promotion: factors predicting satisfaction, participation, and planning. Health Educ Q. 1996;23:65–79. [PubMed]
26. Kegler MC, Steckler A, McLeroy K, Malek SH. Factors that contribute to effective community health promotion coalitions: a study of 10 Project ASSIST coalitions in North Carolina. Health Educ Behav. 1998;25:338–53. [PubMed]
27. Helitzer D, Peterson AB, Thompson J, Fluder S. Development of a planning and evaluation methodology for assessing the contribution of theory to a diabetes prevention lifestyle intervention. Health Promot Pract. 2008;9:404–14. [PubMed]
28. Weiss CH. 2nd ed. Englewood Cliffs (NJ): Prentice Hall; 1998. Evaluation: methods for studying programs and policies.
29. Weiss CH. How can theory-based evaluation make greater headway? Eval Rev. 1997;21:501–24.
30. McLaughlin JA, Jordan GB. Logic models: a tool for telling your program's performance story. Eval Program Plann. 1999;22:65–72.
31. Hirose E, Taneda Y, Ishii T, Julian DA. The utilization of the logic model as a system level planning and evaluation device. Eval Program Plann. 1997;20:251–7.
32. Frechtling JA. Logic modeling methods in program evaluation. San Francisco: John Wiley and Sons; 2007.
33. Renger R, Titcomb A. A three-step approach to teaching logic models. Am J Eval. 2002;23:493–503.
34. Bickman L. The functions of program theory. In: Bickman L, editor. Using program theory in evaluation: new directions for program evaluation. San Francisco: Jossey-Bass; 1987. pp. p. 5–18.
35. Longest BB., Jr. Logic models as aids in managing health programs. J Nurs Adm. 2005;35:557–62. [PubMed]
36. Anderson LA, Gwaltney MK, Sundra DL, Brownson RC, Kane M, Cross AW, et al. Using concept mapping to develop a logic model for the Prevention Research Centers Program. Prev Chronic Dis. 2006;3:A06. [PMC free article] [PubMed]
37. Kolasa K, Lackey C. The logic model as a framework for community program evaluations: the food literacy partners program. Letter to the Editor. Fam Med. 2006;38:690–1. [PubMed]
38. Kaplan SA, Calman NS, Golub M, Ruddock C, Billings J. Fostering organizational change through a community-based initiative. Health Promot Pract. 2006;7(3) Suppl:S181–90. [PubMed]
39. Haddon W., Jr. Advances in the epidemiology of injuries as a basis for public policy. Public Health Rep. 1980;95:411–21. [PMC free article] [PubMed]
40. Baker SP, Haddon W., Jr. Reducing injuries and their results: the scientific approach. Milbank Mem Fund Q Health Soc. 1974;52:377–89. [PubMed]
41. Pennsylvania State University Cooperative Extension. [cited 2008 Feb 23];Logic models for 17 issues. Available from: URL: http://pow.cas.psu.edu/logicModel/index.htm.
42. Israel GD. Document AEC 360. Gainesville (FL): University of Florida; 2001. Using logic models for program development.
43. Greenfield VA, Williams VL, Eiseman E. Prepared for the National Center for Injury Prevention and Control. Santa Monica (CA): RAND Corp.; 2006. Using logic models for strategic planning and evaluation.
44. National Safety Council. Presented at the Control Banding Workshop: Validation and Effectiveness; 2004 Mar 2. Cincinnati: [cited 2008 Feb 10]. Safety intervention evaluation: a systematic approach. Also available from: URL: http://www.acgih.org/events/ControlBand/Thomas_SafetyIntervention.pdf.
45. Bartholomew LK, Parcel GS, Kok G, Gottlieb NH. Planning health promotion programs: an intervention mapping approach. San Francisco: Jossey-Bass; 2006.
46. Framst G. Application of program logic model to agricultural technology transfer programs. Can J Program Eval. 1995;10:123–32.
47. W.K. Kellogg Foundation. Battle Creek (MI): W.K. Kellogg Foundation; 2003. Logic model development guide.
48. Quandt SA, Arcury TA, Pell AI. Something for everyone? A community and academic partnership to address farmworker pesticide exposure in North Carolina. Environ Health Perspect. 2001;109(Suppl 3):S435–41. [PMC free article] [PubMed]
49. Helitzer D, Willging C, Hathorn G, Benally J. J Agric Saf Health. Building community capacity for agricultural injury prevention in a Native American community. [PubMed]
50. Browning SR, Truszczynska H, Reed D, McKnight RH. Agricultural injuries among older Kentucky farmers: the Farm Family Health and Hazard Surveillance Study. Am J Ind Med. 1998;33:341–53. [PubMed]
51. Rogers EM, Rogers E. 5th ed. New York: Free Press; 2003. Diffusion of innovations.
52. Dzewaltowski DA, Estabrooks PA, Klesges LM, Bull S, Glasgow RE. Behavior change intervention research in community settings: how generalizable are the results? Health Promot Int. 2004;19:235–45. [PubMed]
53. Kumanyika SK. Commentary: cultural appropriateness: working our way toward a practicable framework. Health Educ Behav. 2003;30:147–50. [PubMed]
54. Green LW, Glasgow RE. Evaluating the relevance, generalization, and applicability of research: issues in translation methodology and external validity. Eval Health Prof. 2006;29:126–53. [PubMed]

Articles from Public Health Reports are provided here courtesy of SAGE Publications