|Home | About | Journals | Submit | Contact Us | Français|
Evaluating programs targeting physical activity may help to reduce disparate rates of obesity among African Americans. We report formative process evaluation methods and implementation dose, fidelity, and reach in the Positive Action for Today’s Health trial.
We applied evaluation methods based on an ecological framework in 2 community-based police-patrolled walking programs targeting access and safety in underserved African American communities. One program also targeted social connectedness and motivation to walk using a social marketing approach. Process data were systematically collected from baseline to 12 months.
Adequate implementation dose was achieved, with fidelity achieved but less stable in both programs. Monthly walkers increased to 424 in the walking-plus-social marketing program, indicating expanding program reach, in contrast to no increase in the walking-only program. Increased reach was correlated with peer-led Pride Strides (r = .92; P < .001), a key social marketing component, and program social interaction was the primary reason for which walkers reported participating.
Formative process evaluation demonstrated that the walking programs were effectively implemented and that social marketing increased walking and perceived social connectedness in African American communities.
Intervention programs targeting neighborhood walking and physical activity (PA) have the potential to prevent obesity in underserved African American communities.1–3 African Americans experience the highest rates of obesity and the lowest rates of PA,1,4 and face increased barriers to PA, including a lack of access to safe places to be active and a lack of community support for PA.5–7 Formative process evaluation can inform program implementation and may lead to more effective health interventions for PA and obesity. In addition, social marketing (SM) approaches have been linked to increased effectiveness of health promotion interventions.8–11 The purpose of this study was to describe the application of systematic evaluation methods in the Positive Action for Today’s Health (PATH) trial, which adopted an innovative SM approach to promote walking in underserved (low-income, high-crime) African American communities,12 and to report evaluation data from the PATH walking programs. An additional aim was to present a framework for the evaluation of SM approaches that target a variety of public health issues.
Consistent with an ecological approach,13 the PATH trial developed a community-based intervention to address access and safety barriers to neighborhood walking in underserved African American communities, with one program including an additional SM component to increase social connectedness and motivation for walking. Process evaluation methods were developed to assess program implementation and reach. Social marketing strategies14,15 were developed with community input,12 and focus groups conducted in the PATH communities suggested that the strategies would meet community needs and would be culturally relevant.16,17 Additionally, the utility of incorporating evaluation methods based on ecological theory has increasingly been recognized,18–21 as has the effectiveness of SM approaches.8–10 Although ecological factors significantly influence implementation practices and program outcomes,22 fewer than half of the studies targeting minority PA have evaluated them,23 and only a handful of studies implementing SM strategies have reported summative24–29 or formative process evaluation outcomes.8,23,30 Such limitations can decrease the accuracy with which program mechanisms, efficacy, and effectiveness are understood.
As community-based interventions become more complex in design (e.g., by focusing on multiple ecological levels and components) and integrate strategies from varied disciplines (e.g., public health, marketing), evaluation becomes increasingly necessary for determining which levels and components affect outcomes, and to what extent,31 before dissemination.32 Evaluation of program implementation relies on the measurement of program dose, fidelity, and reach, which, respectively, quantify the completeness of program implementation, whether a program is delivered as intended, and the percentage of the target group engaged in the program.
A growing literature has reported evaluation methods for community-based PA programs targeting African American adults. The WATCH project promoted increased PA through participation in church-based walking groups and used process evaluation to demonstrate that intervention activities were implemented with varying degrees of dose and fidelity.33 The Health-e AME church–based, multisite intervention assessed implementation and indicated that fidelity varied by site and was positively associated with meeting PA recommendations.34 In the H.U.B. City Steps and Healthy Body, Healthy Spirit community-based trials that targeted walking and PA, formative process evaluation methods were based in part on an ecological framework, with ongoing feedback provided to counselors to address lapses in fidelity.35,36 Finally, an intervention targeting PA adopted a media-based, community-based prevention marketing approach to promote walking and PA among women.37 By contrast, the PATH trial adopted a community-based grassroots marketing approach and focused on neighborhood access, safety, and social connectedness rather than individual factors. The media-based program used process evaluation to demonstrate that campaign exposure significantly increased walking, PA, and use of a walking trail.37
In this study, we expand on previous research by reporting formative process evaluation for monitoring the implementation of a community-based intervention that integrated grassroots SM strategies to improve perceptions of access and safety, and to increase social connectedness and motivation for neighborhood walking in underserved African American communities. We present evaluation methods and dose, fidelity, and reach data from baseline to 12 months in both PATH walking programs.
We matched 3 communities by census tract on demographics such as crime, percentage of African American residents, and poverty.12 We considered communities that experienced high levels of chronic disease, poverty, and crime relative to state and national averages to be underserved. For example, 23% to 39% of households in the selected tracts experienced poverty compared with national rates of 15%.38 Communities were located in the southeastern region of the United States, with 1 or 2 census tracts targeted in each community, 5 targeted for the study, and approximately 45 to 90 miles separating each community. We considered communities to be rural on the basis of urban–rural classification criteria outlined by the US Census Bureau.39 African American adults (aged 18 years or older) residing in specified census areas were enrolled in the PATH sample if they were free of medical conditions limiting participation in moderate intensity exercise40 and did not have uncontrolled high blood pressure (<180 mm Hg systolic, <110 mm Hg diastolic) or blood sugar (< 300 mg/dL nonfasting, ≤ 250 mg/dL fasting). Participants were compensated $30 during the baseline measurement.
The PATH trial was designed to evaluate the efficacy of a 24-month environmental intervention for increasing walking and PA by targeting access, safety, and social connectedness and motivational barriers in underserved African American communities.12 Communities were randomized to a police-patrolled walking-plus-SM program (full intervention; walking-plus-SM), a police-patrolled walking-only program (i.e., no SM component; walking-only), or a health education program (health comparison program). PATH participants completed measures during health screenings occurring every 6 months. We designed PATH protocols consistent with a community-based participatory approach to ensure positive community relations and engagement, complete and valid measures, and increased participation in the walking programs. PATH community liaisons were key in building and maintaining community connections, and the health screenings were viewed as important resources for participants. These strategies resulted in high retention rates, consistent with or better than those of other community-based trials that implemented health interventions in minority populations.41–43
Residents were recruited using random-digit dialing (54%) and through volunteer mechanisms (46%). The recruited sample is not inclusive of all individuals who participated in the PATH walking programs, although sampling procedures aimed to represent individuals from the larger community. This report focuses only on baseline to 12 months of the 2 walking programs.
The intervention was based on an ecological framework,13 and program components targeted (1) access for walking in the neighborhood, (2) positive practices and perceptions related to safety, and (3) residents’ social connectedness and motivation for walking. We operationalized program components by defining essential program elements (Figure 1).
Local community centers operated as neighborhood hubs for program activities. Walking trails that began and ended at each center were identified. Walking leaders were trained in injury prevention and safety protocols and scheduled and led ongoing walks (e.g., Monday–Friday at 7:00 am). Program leaders worked with county officials to control stray dogs and address ongoing structural and aesthetic barriers on the trails, and off-duty police officers were hired to attend walks. To address seasonal and daily weather fluctuations, walking leaders used protocols directing them to lead indoor walks or aerobic activities (e.g., group exercise) during rainy or exceptionally cold (≤ 32°F) or hot (≥ 90°F) weather. Communities generally experienced hot, humid summers and mild winters, consistent with a humid, subtropical climate region.
For the walking-plus-SM program, we developed a community-based grassroots campaign in collaboration with a SM firm. Five messages framed the campaign content, conveying that walking was safe, benefited physical health, benefited mental and spiritual health, promoted social connectedness, and promoted confidence (self-efficacy) for walking. The PATH Steering Committee and community residents developed print materials, and a 12-month calendar was distributed to deliver campaign messages. The calendar featured photographs of residents walking on the trail and also provided goal-setting opportunities for walking. Door hangers were delivered to residents’ homes to invite them to walk, to reinforce campaign messages, and to highlight program incentives (e.g., hand-held fan, pedometer) that walkers could earn when walking goals were met.12 Grassroots networking was facilitated through the development of “Pride Stride” walks, which empowered community walkers to organize and lead special-event walks that promoted the campaign messages and engaged new walkers, with the support of the walking leaders.
Research staff attended program walks weekly (walking-plus-SM) and every other week (walking-only) to rate essential elements using internal process evaluation surveys. Staffing and budgetary constraints prevented weekly assessments in both programs, and we therefore paid greater evaluative attention to the walking-plus-SM program, given hypotheses that inclusion of the SM campaign would result in the greatest increase in walking.12 Because the walking-only program did not include an SM component, we did not target social connectedness and motivation for walking.
After walks, supportive, corrective feedback was given to ensure ongoing achievement of implementation dose and fidelity. For example, feedback was provided if an exercise warm-up, part of the injury prevention protocol, was not completed before beginning a walk. Intervention staff reviewed data collected through these internal evaluations immediately.
Walking leaders attended all walks and documented walkers who attended. We also calculated the correlation of walk attendance with Pride Strides as part of the grassroots SM campaign in the walking-plus-SM program to assess the relation of reach to implementation and the SM strategy. Finally, a small sample of walkers reported their perceptions of the essential elements to supplement findings from process evaluation data.
Surveys included 12 key items. Adequate dose or fidelity was achieved if (1) items with yes–no response options were endorsed in the desired directions during 75% or more of evaluations and (2) items with 3-point scale response options had mean values of 2.50 or greater. Items were linked to dose if they evaluated the degree to which essential elements were completed and linked to fidelity if they evaluated whether the program was being implemented to support improvements in walking trail access and safety, as intended.
Walking leaders recorded walker information at each walk. Although Pride Strides often occurred in tandem with regularly scheduled walks, walkers were not counted as participants for both walks but as participants for either the Pride Stride or the regular walk on the basis of their reported affiliation with one or the other. We quantified program reach by using monthly sums of the number of walkers at each walk.
Surveys included 9 key items with yes–no response options that elicited perceptions of essential program elements. Program staff administered surveys to walkers and ensured the anonymity of walker responses. We included surveys completed during months 9 through 13 because we considered program exposure and implementation to be stable by months 9 through 12, and we determined that responses collected during that time and immediately afterward would be highly relevant to evaluation outcomes and future directions for the program.
Baseline characteristics for the walking-plus-SM (n = 133) community indicated that PATH participants were predominantly female (67%), with an average age of 54.2 years (SD = 15.6) and a body mass index (defined as weight in kilograms divided by the square of height in meters) of 31.1 (SD = 7.9). In the walking-only (n = 164) community, participants were predominantly female (65%), with an average age of 48.1 years (SD = 15.6) and a body mass index of 30.3 (SD = 8.3), as described previously.12 The PATH sample aimed to represent the larger community and was not inclusive of all individuals who participated in the walking programs.
Data related to access indicated that in the walking-plus-SM program, adequate dose was achieved across all quarters and for all essential elements (Table 1). In the walking-only program, dose was achieved at rates similar to the walking-plus-SM program, with use of the neighborhood walking trail improving after the first quarter. Data related to safety indicated that the walking-plus-SM program achieved adequate dose across all quarters. In the walking-only program, adequate dose was achieved during all quarters except the fourth, during which the mean evaluation score assessing whether the trail was free from stray dogs decreased from 2.57 to 1.50. Data related to program social connectedness and motivation to walk indicated that adequate dose was achieved across all quarters in the walking-plus-SM program, with SM campaign messages reaching 81% to 87% of targeted walkers.
Data related to access indicated that in both walking programs, fidelity was initially inadequate and subsequently improved, with the exception of the presence of litter and vandalism (Table 1). During the first quarter, data indicated that the walking trails were not free from the presence of overgrowth, litter, and vandalism, and did not have adequate sidewalks, with scores of 2.00 failing to meet the 2.50 criterion indicating adequate fidelity. Scores fluctuated during subsequent quarters. In the walking-only program, access fidelity was achieved at rates slightly lower than in the walking-plus-SM program, with the presence of overgrowth, litter, and vandalism problematic in both the 3rd and the 4th quarters. Data related to safety indicated that fidelity varied by essential element, with pedestrian safety and injury prevention protocols followed inconsistently in both programs at rates ranging from 0% to 100%. However, no walkers were injured in either program, and no incidents needed to be reported to the institutional review board. Additionally, police were present at 100% of walks during most quarters, with police presence never lower than 81%. Data related to social connectedness and motivation to walk indicated difficulty reaching participants by phone in the walking-plus-SM program.
Intervention reach in the walking-plus-SM program grew over time, as intended. The total number of walkers increased steadily into the 4th quarter, with as many as 424 total walkers in 1 month (Figure 2). Monthly totals for Pride Strides and walkers were positively correlated (r = .92, P < .001). In the walking-only program, reach did not increase over time, with the total number of walkers relatively consistent over each month and averaging 41 walkers (SD = 22).
Walkers in the walking-plus-SM program reported positive perceptions of access, safety, social connectedness, and motivation components 87% to 100% of the time (Table 2). Additionally, social interaction was the number 1 reason for participating in regular walks (48%), compared with reasons such as health (25%) and feeling safe on the trail (13%). Data from the walking-only program are not reported because too few walkers were participating in the program at the time of data collection.
We presented the use of formative process evaluation for ongoing implementation monitoring of the walking programs and innovative SM campaign in the PATH trial. Based on an ecological framework, evaluation outcomes demonstrated that adequate dose, fidelity, and reach were generally achieved over time; the SM campaign led to substantial increases in walking program participation; and that walkers had positive perceptions of the program in the walking-plus-SM community. Thus, results support the effectiveness of the process used to implement the intervention rather than simply the content of the intervention. Such a framework can be applied to programs aiming to address a range of public health issues and may be tailored to meet the needs of low-income minority communities. In addition, social connectedness seemed consistently important to walkers and may have increased community capacity for promoting walking and health.
During the first 12 months of the PATH intervention, adequate dose was achieved for program components in both walking programs during a majority of quarters; implementation fidelity was achieved in a somewhat less consistent pattern. For fidelity, both communities overall were strong in access and many of the safety-related elements, although a few areas of weakness were found related to trail quality (access) and walking warm-ups (safety). However, data from walker surveys indicated that walkers’ perceptions of essential elements were positive, supporting evaluation evidence indicating that the walking-plus-SM program achieved adequate dose and fidelity.
Trail quality issues were recurrent in these underserved settings and were addressed through ongoing problem solving by community leaders, who developed an infrastructure for collaborating with local officials. Although implementation fidelity did not vary by program for most elements, the walking-only program experienced slightly increased challenges in trail quality related to the presence of overgrowth, litter, and vandalism and the quality of sidewalks (e.g., presence of cracks, uneven pavement). Both programs achieved moderate fidelity for personal safety, with injury prevention and safety protocols resulting in no walking injuries or safety threats. However, both programs struggled to consistently integrate protocols related to walking warm-ups and stretches, despite ongoing corrective feedback provided during evaluations. We observed that both walkers and walking leaders occasionally perceived some protocols as unnecessary or burdensome. Such perceptions likely gained influence over time as walkers and walking leaders began to take greater ownership of the programs. These challenges highlight the importance of incorporating flexibility into community-based intervention protocols.
In the PATH trial, walking leaders were supported in developing alternative protocols when they found existing protocols to be a poor fit for the walking group or for individual walkers. Walking leaders in the walking-plus-SM program relied less on police support over time, with police presence at walks dropping to 82% in the final quarter. This percentage did not fall below the criterion for adequate dose, and it may be a positive indicator of walkers’ increased comfort and feelings of safety on the walking trail. Efforts to decrease dependence on police support may also have been driven by goals to adopt sustainable protocols and to accommodate walkers’ varied schedules and requests to increase the number of walks.
Intervention reach differed greatly by program, with the walking-plus-SM program experiencing increased reach over time and consistently engaging more walkers than the walking-only program. When inclement weather may have prevented walking, walking leaders led indoor walks and exercises to ensure continuity of the programs. In general, the communities’ input on how to cope with inclement weather by being active indoors ensured that reach of participation decreased only slightly because of bad weather or seasonal effects.
The walking-plus-SM program demonstrated a large relationship between the number of completed Pride Strides and program reach, indicating that effective implementation of the social connectedness and motivation component may have contributed to increased program participation. The grassroots approach of the SM campaign may also have supported Pride Stride walkers in remaining engaged with the program and neighborhood, and in walking more regularly, as was often observed. Targeting social connectedness may have positively affected community resources such as social capital and collective efficacy, which have been linked to increased community capacity and positive health behaviors and outcomes.44–47 The community-based participatory approach supported participants in providing input on program components (e.g., in the PATH trial, police support, SM messages), which may have fostered program engagement, increased participant retention, and increased the likelihood of collecting valid, reliable outcomes (e.g., accelerometry-estimated PA over 7 days) over the 24-month intervention.
The barriers of access and safety that were targeted, including structural and aesthetic neighborhood conditions and the presence of crime or stray dogs, have been linked to walking and PA,48 although not all studies have supported these relations.49 Operationalizing environmental factors such as access and safety can be challenging, but these steps are a necessary precursor to evaluation and can aid in building theory-based programs. Additionally, a community-based approach that incorporates ongoing community feedback facilitates the tailoring of process methods as described, ensuring that they are applicable to a variety of public health initiatives and across culturally diverse populations.21
The evaluative findings of the PATH trial are consistent with recommendations to incorporate systematic evaluation into community-based programs32 and public health programs targeting African American communities.50 Findings are also consistent with studies suggesting that process evaluation can be used to understand outcomes for PA programs targeting African Americans34,35,36 and to assess the implementation of SM strategies to increase PA.26,27,51 Additionally, we confirmed previous evidence that exposure to a marketing campaign can increase walking and PA.37 This study is the first to our knowledge to provide evidence that using a grassroots SM approach to promote PA in African American communities is feasible and, based on these evaluation findings, effective. Additional value added from this study is the demonstration that evaluation frameworks and criteria (e.g., whether adequate dose and fidelity are achieved) can be tailored to communities to adapt to their changing needs while maintaining fidelity to the core components of an intervention. This tailoring ensures that programs remain relevant and engaging, as was achieved in the PATH trial.
This study is one of the first to report the use of formative process evaluation methods in an intervention that targeted underserved African American communities and incorporated an innovative social marketing approach. Similar evaluation methods may ensure quality implementation in future studies and contribute to the prevention of obesity in underserved communities.
This work was supported by the National Institute of Diabetes, Digestive, and Kidney Diseases (grant R01 DK067615 to D. K. W.) and in part by the National Institute on Aging (grant F31 AG039930 to S. M. C. and D.K.W.) and the National Institute of General Medical Sciences (grant T32 GM081740 for S. M. C.).
We acknowledge and thank our communities and community leaders and staff, and our participants in the programs; without their dedication and enthusiasm, this trial would not have been possible. We also thank our incredible research team and project leaders Franklin Jones, Kaya Outen, and Phil Watts, as well as Klein Buendel Inc. for their expertise and assistance in developing the social marketing intervention.
ContributorsS. M. Coulon drafted the article and participated in data collection, management, and interpretation. D. K. Wilson provided oversight on all aspects of the article, the design and implementation of the overall trial, and assisting with data interpretation, and is the principal investigator of the trial. S. Griffin assisted with drafting and editing the article, participated in development of evaluation methods as well as data management and interpretation, and is a coinvestigator of the trial. S. M. St. George participated in data collection, management, and interpretation and coordinated the walking programs. K. A. Alia assisted with data management, conducted relevant literature searches, and drafted and edited the article. N. N. Trumpeter participated in data collection and interpretation, and directed the trial. A. K. Wandersman initiated and led development of evaluation methods, participated in interpretation of data, and is a coinvestigator of the trial. M. Forthofer participated in the development of evaluation methods and interpretation of data and is a coinvestigator of the trial. S. Robinson participated in data collection, management, and interpretation. B. Gadson coordinated community- and county-level involvement with the intervention and is a coinvestigator of the trial. All authors have read and approved the final article.
Human Participant Protection
All study procedures were approved by the institutional review board of the University of South Carolina.
Sandra M. Coulon, Department of Psychology, University of South Carolina, Columbia.
Dawn K. Wilson, Department of Psychology, University of South Carolina, Columbia.
Sarah Griffin, Department of Public Health Sciences, Clemson University, Clemson, SC.
Sara M. St. George, Department of Psychology, University of South Carolina, Columbia.
Kassandra A. Alia, Department of Psychology, University of South Carolina, Columbia.
Nevelyn N. Trumpeter, Department of Psychology, University of South Carolina, Columbia.
Abraham K. Wandersman, Department of Psychology, University of South Carolina, Columbia.
Melinda Forthofer, Department of Epidemiology and Biostatistics, University of South Carolina.
Shamika Robinson, Department of Psychology, University of South Carolina, Columbia.
Barney Gadson, M. H. Newton Family Life Enrichment Center, Sumter, SC.