|Home | About | Journals | Submit | Contact Us | Français|
On June 2, 2009, Trust for America’s Health, The New York Academy of Medicine, the Robert Wood Johnson Foundation, and The California Endowment convened policymakers, researchers, and evaluators to discuss strategies for assessing the impact of the $650 million investment in community-based prevention in the American Recovery and Reinvestment Act of 2009 (ARRA). The ARRA investment was discussed as an opportunity to demonstrate the value of community-based prevention. Meeting participants generated six recommendations to address the areas of concern policymakers typically have regarding investment in community-based prevention programs.
Community-based prevention has the support of academics and practitioners in public health and preventive medicine. As a result of philanthropic investments and the growing success of local policies and programs, there has been a paradigm shift away from focusing primarily on individual behavior change to promote health, toward environmental and policy change for health improvement. However, outside the fields of public health and preventive medicine, the effectiveness of this strategy is less well known. There was consensus at the meeting that ideally, the ARRA program and evaluation of its supported interventions will be designed to help address the concerns and questions of policymakers in particular.
The following were identified as some of the areas of concern policymakers typically have regarding investment in community-based prevention programs:
Identified Concern While randomized controlled trials of clinical treatments can often show direct influence on health outcomes within a relatively short period, community interventions typically demonstrate influence on intermediate variables, and may not be able to report direct links to health outcomes within a short time frame. With only intermediate markers of success available, how can policymakers be confident they are investing in health improvement?
Further Discussion Project time frames, insufficient data, and lack of infrastructure and resources to carry out adequate evaluation are some of the reasons community-based projects are not always able to demonstrate improvements in health status. It was noted, however, that even within a limited time frame such as 2 years, projects with sufficient resources and data-collection infrastructure can capture biological or health outcomes such as physical fitness, reduced healthcare visits, and behavior changes. It was also discussed that health-related outcomes are not always essential to arguments for investment in community-based prevention. Reduced absenteeism, improved test scores, and economic benefits are examples of other outcomes that can be persuasive to policymakers. The Institute of Medicine’s Evaluation Framework for Obesity Prevention Policies and Interventions was reviewed. It illustrates that community interventions typically yield outcomes in five categories: environmental; cognitive and social; behavioral; structural/institutional/systems; and health. Projects should be framed so that results are expected in the appropriate outcome category and are not unfairly deemed unsuccessful. It was also discussed that macro-level, aggregate comparisons may be useful for explaining the value of community interventions. For example, comparison of communities who participated in the CDC REACH program versus those that did not shows that the REACH interventions are closing gaps in health disparities.
Identified Concern Community-based interventions often have multiple components and are tailored to address the needs and assets of individual communities. They are also typically undertaken by communities particularly motivated to make changes. The often idiosyncratic nature of the interventions does not serve well the strong preference among policymakers to isolate one single intervention that can be replicated in a standardized manner many times.
Further Discussion The CDC’s Guide to Community Preventive Services was discussed as a resource for identifying effective, potentially replicable program and policy interventions. It is a work-in-progress, however, and due to inadequate funding, has subject area gaps. It also lacks information about barriers to implementation. Communities need technical assistance to adapt evidence-based interventions for local conditions, and program and evaluation schemes need to provide leeway for local adaptations. It was discussed that cancer treatment may provide a useful analogy: it is well-accepted that a mix of interventions (radiation, surgery, chemotherapy, etc.) is needed to treat cancer, and that this mix varies by patient. Similarly, there are core practices that are vital for successful community interventions, but they should be applied according to local needs. School policies were discussed as a category of interventions that lend themselves well to replication. However, political will is important to their implementation. Policymakers need to understand that community “readiness for change” is an integral part of the potential success of an intervention, particularly an intervention that seeks to transform the physical or policy environment. Accordingly, efforts to cultivate leadership and coalitions should be funded and evaluated.
Identified Concern Studies of community-based efforts rarely capture cost information, and when cost information is available, it can be hard to separate program costs from research costs. This information is needed to understand adequate levels of funding and potential returns on investment.
Further Discussion It was agreed that intervention costs are not captured in most studies, and that when they are reported, the information is not usually sufficient to enable a calculation of return on investment. There was discussion of whether the economic benefits of prevention are adequately appreciated. For example, improved productivity and tax revenues generated by a healthy workforce are not generally incorporated into evaluations of the benefit of investment in health. Also, it is often argued that prevention is not an effective investment because in prolonging life, it increases opportunities for additional health care expenditures across a lifetime. It was noted that this is also true of but not raised against investment in clinical treatment. It was suggested that cost questions be reframed to ask “How can we get the most benefit per dollar invested in health?” The issue of where prevention savings are realized was discussed. Federal prevention programs are funded through discretionary funding streams. Any resulting savings in health care expenditures are realized in mandatory spending accounts (Medicare/Medicaid). Current program-scoring policies do not allow the Congressional Budget Office to acknowledge savings that occur in an account different from the account where the expenditure occurs. This is a policy decision that could be changed. It is also possible that if Medicare or Medicaid pay for community-based prevention, then savings could be accrued within the mandatory spending account and the prevention program could be more adequately scored.It was noted that the ARRA program designers have certain statutory constraints and political questions to consider. First, the program needs to demonstrate results within 2 years. It is understood that chronic disease rates can only begin to be affected within this timeframe. Therefore, the program must identify interim markers that can show progress toward eventually reducing chronic disease and its associated costs. Secondly, it is important that any measures of success be easily understood by a lay public. A baseline is needed, as well as a means to make comparisons against communities that did not receive funding. Lastly, it is not clear which populations or communities should be funded: those in most need, or those that have stronger community infrastructure and greater readiness, and so can quickly leverage the new funding.
To address the concerns regarding community-based interventions, and some of the questions raised specifically around the ARRA program, meeting participants generated the following recommendations for the ARRA program design and evaluation: