PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
AIDS Behav. Author manuscript; available in PMC 2010 October 30.
Published in final edited form as:
PMCID: PMC2966590
NIHMSID: NIHMS197215

A review of HIV/AIDS system-level interventions

Abstract

The escalating HIV/AIDS epidemic worldwide demands that on-going prevention efforts be strengthened, disseminated, and scaled-up. System-level interventions refer to programs aiming to improve the functioning of an agency as well as the delivery of its services to the community. System-level interventions are a promising approach to HIV/AIDS prevention because they focus on (a) improving the agency’s ability to adopt evidence-based HIV prevention and care programs; (b) develop and establish policies and procedures that maximize the sustainability of on-going prevention and care efforts; and (c) improve decision-making processes such as incorporating the needs of communities into their tailored services. We reviewed studies focusing on system-level interventions by searching multiple electronic abstracting indices, including PsycInfo, PubMed, and ProQuest. Twenty-three studies out of 624 peer-reviewed studies (published from January 1985 to February 2007) met study criteria. Most of the studies focused on strengthening agency infrastructure, while other studies included collaborative partnerships and technical assistance programs. Our findings suggest that system-level interventions are promising in strengthening HIV/AIDS prevention and treatment efforts. Based on our findings, we propose recommendations for future work in developing and evaluating system-level interventions.

Keywords: Systems, Structural, Social Intervention, HIV/AIDS, Review

Introduction

Researchers’ understanding of how HIV/AIDS is transmitted, prevented, and managed has increased substantially over the past twenty years. Researchers and practitioners have developed clinical and behavioral strategies that seek to decrease HIV transmission and enhance HIV-related quality of life. Multiple literature reviews have found solid evidence examplifying effective prevention strategies, including community needle exchange programs (Bastos & Strathdee, 2000), community education and advocacy (Janz, Zimmerman, Wren, Israel, Freudenberg, & Carter, 1996), linkage to antiretroviral therapy to decrease mother-to-child transmission (Volmink, Siegfried, van der Merwe, & Brocklehurt, 2007), distribution and use of the female condom (Hoffman, Exner, Leu, Ehrhardt, & Stein, 2003), and behavioral interventions to modify sex risk behaviors (Johnson, Hedges & Díaz, 2007; Lyles, Kay, Crepaz, Herbst, Passin, Kim et al., 2007). These comprehensive literature reviews have evidenced that prevention is effective, yet these efforts are sometimes labor intensive and slow. To strengthen on-going prevention efforts through dissemination and scale-up, we must ensure that the capacity and resources of the agencies carrying out these efforts are maximized.

Multiple intervention levels for HIV prevention and care

A social system refers to the network of institutional structures providing standardized and quality-assured services to large numbers of people and communities. These institutional networks may include federal agencies (e.g., CDC), local government agencies (e.g., city health departments), non-government organizations (NGOs) (e.g., AIDS service organizations [ASOs] and community-based organizations [CBOs]), and private health organizations (e.g., hospitals and healthcare providers). Delivery of HIV prevention through these institutions has been an efficient rollout mechanism because they may (a) reach and affect large numbers of people efficiently; (b) create and establish policies and procedures that maximize the diffusion of interventions; (c) increase program sustainability; and (d) incorporate the needs of specific communities into their tailored services. Nonetheless, we know very little in the HIV literature of how strengthening these social structures, individually and collectively, may ameliorate the provision of HIV prevention programs. System-level interventions are a promising approach to HIV/AIDS prevention because they focus on (a) improving the agency’s ability to adopt evidence-based HIV prevention and care programs; (b) develop and establish policies and procedures that maximize the sustainability of on-going prevention and care efforts; and (c) improve decision-making processes such as incorporating the needs of communities into their tailored services.

While most psychosocial interventions have focused on delivering individual and community programs, structural interventions have received little attention until recently (Blankenship, Bray, & Merson, 2000). We define system-level interventions as a set of strategies used to modify an agency’s culture, its internal process to preserve and maximize resources, and the efforts placed in strengthening external ties with other institutions for HIV prevention and care. An underlying assumption of this approach is that an agency’s capacity to deliver strong HIV prevention messages across different levels is tied intricately to the stability and effectiveness of its internal processes. Consequently, a system-level intervention focuses on strategies that promote institutional growth, capacity, and sustainability rather than on the mechanisms used to disseminate a program into the community.

The distinction between system-level strategies and dissemination mechanisms is vital in understanding the impact of system-level interventions. A system-level intervention may have as one of its outcomes the successful scale-up and dissemination of a program, yet it may not be the only outcomes of interest. For example, a CBO conducts a system-level intervention when it seeks to strengthen its staff skills through a technical assistance program or when it restructures its organizational hierarchy by including different community stakeholders in its Executive Board to increase diversity in its decision-making process.

A variety of theoretical frameworks and outcomes may inform system-level interventions. To this end, our goal is to review HIV/AIDS system-level interventions published in scientific journals from January 1, 1985 to February 1, 2007. In summarizing the findings by intervention strategy (i.e., technical assistance, infrastructure development, and inter-agency partnerships), we hope to illustrate different approaches and identify their strengths and weaknesses to inform future studies. We conclude with a summary discussion of intervention efforts demonstrating potential success or needing further replication and propose recommendations for future work

Theoretical Frameworks Informing System-Level Interventions

Theoretical frameworks should inform system-level interventions to facilitate program rationale and increase transferability of lessons learned when successful. Various theoretical approaches have been proposed to guide system-level intervention development, but their use has been limited in the HIV literature. We add to the basic assumption that, at a minimum, three components take place in creating and delivering services (Alter & Hage, 1993). These three components are: (1) inputs (internal and external resources that an agency can access), (2) process (an agency’s culture and its decision-making procedures), and (3) outputs (an agency’s aptitude to deliver services). We present an overview of common theoretical frameworks used to intervene in these three steps.

Stage Theory of Organizational Change (STOC)

STOC proposes a staged framework through which organizations incorporate innovations. Similar to the Transtheoretical Model (Prochaska et al., 2002), an organization’s ability to move through the stages towards adoption, implementation, and maintenance of an innovation depends on the institution’s ability to overcome external and internal barriers within each stage. Beyer and Trice (1978) propose seven stages: (1) sensing of unsatisfied demands on the system, (2) search for possible responses, (3) evaluation of alternatives, (4) decision to adopt a course of action, (5) initiation of action within the system, (6) implementation of the change, and (7) institutionalization of the change. Goodman and Steckler (1989) have argued for an eighth stage, renewal, to account for the program’s ability to evolve as demands change.

Organizational Development Theory (ODT)

Rooted in management theory and influenced by social ecology models, ODT posits that environmental influences (e.g., an organization’s norms and values) can shape organizational performance by influencing the individual and collective attitudes and behaviors of an organization’s staff (Steckler, Goodman, & Kegler, 2002). Three inter-related constructs inform interventions based on this theory: (a) organizational climate, (b) organizational culture, and (c) organizational capacity. Organizational climate can be defined as the institution’s personality and is caused and continuously shaped by the interaction of members’ collective beliefs, perceptions, and attitudes (Hoy & Miskell, 1987). Organizational culture refers to the underlying assumptions, values and norms that grow and persist as an institution evolves (Bolman & Deal, 2003). Finally, organizational capacity is characterized by an institution’s ability to create deliverables, to maintain the organization in operation, to respond to and receive support from the social and political demands in the environment, to adapt to changes within and outside of the institution, and to synchronize these competing demands appropriately (Katz & Kahn, 1978; Dill, 1994).

Interorganizational Relations Theory (IOR)

IOR focuses on how multiple organizations work together to maximize deliverables within their communities (Steckler et al., 2002). IOR posits that partnerships across organizations occur in order to tackle comprehensively the social, political, and economic demands affecting their communities. Interorganizational linkages facilitate access to new information, skills and resources, while sharing costs and responsibilities. Conversely, interorganizational linkages may delay action steps due to slow consensus development, may cause conflict if a policy or position taken by the whole is incongruent with the goals or mission of one or more of the organizations, and may divert much needed organizational resources to the collective mission of the coalition.

System-Level Intervention Strategies

Based on these frameworks, several strategies may be proposed as system-level intervention activities. We catalogue these strategies into three domains: (a) technical assistance, (b) infrastructure development, and (c) inter-agency partnerships and coalition building.

Technical assistance

System-level interventions may use technical assistance strategies to improve the delivery of services by intervening on an agency’s input component. Technical assistance may include agency staff training in the program planning process, consultation with experts in program planning and implementation, or external support in monitoring and evaluation procedures. Within HIV/AIDS, researchers and practitioners have identified evidence-based prevention strategies and programs (National Institute of Mental Health Multisite HIV Prevention Trial Group, 1998), efforts to prepare system-wide dissemination of these efforts have relied on STOC and ODT strategies. Examples of STOC and ODT strategies include the training in conducting a problem diagnosis like the one carried out in HIV Prevention Community Planning Groups (CPGs; Johnson-Masotti et al., 2000). Similarly, another example is the provision of technical assistance to agencies as they select, translate, and implement a Diffusion of Effective Behavioral Intervention (DEBI) to address community needs.

Infrastructure development

System-level interventions may seek to build an institution’s capacity by increasing an agency’s resource availability and improving its inputs. However, infrastructure development may also focus on an agency’s process component by reorganizing its internal functioning. Agency reorganization may include a shift in the agency’s mission and goals, an increase in fund-raising activities to increase personnel retention as well as resources for new and ongoing services, and/or the reorganization of an agency’s leadership and decision-making process. Within HIV/AIDS, an example of a system-level strategy that may be informed by STOC and ODT would be the development and refinement of a clinic’s goals and objectives for a high-risk group, and the monitoring of quality of care standards to ensure optimal treatment of HIV-positive patients. Another strategy might include the rearranging of an ASO’s organizational structure to diversify staff tasks, increasing volunteer manpower and ability to secure funding, or the training of staff to strengthen their ability to translate evidence-based interventions into community-relevant HIV prevention programs.

External Partnerships

System-level HIV/AIDS strategies informed by IOR rely on the development of new or existing collaborations across a social system as a way of strengthening an agency’s outputs. These strategies can include partnerships of a particular kind of institution (e.g., a coalition formed exclusively by ASOs in a defined region) or a variety of stakeholders from various kinds of institutions seeking to minimize duplicate efforts in service provision across agencies (e.g., representatives from ASOs, CBOs, religious leaders, policy-makers, hospital managers). Potential activities include the development of coalition of ASOs and CBOs to increase outreach and testing efforts in their communities, the formation of a service bundle to provide greater comprehensive care to high-risk populations such as injection drug users, and the expanded access to resources and lessons learned, locally and at a distance, through electronic networks such as listservs and newsgroups.

Markers and Outcomes of System-Level Interventions

Markers of “success” can vary according to the strategies proposed as part of a system-level intervention and can range across different levels of analysis (e.g., individual, agency, and city/region) and units of analysis (e.g., patient, care provider, geospatial unit). We present briefly an overview of potential outcome indicators for system-level interventions and categorize them as (a) institutional-based outcomes and (b) HIV-related markers.

Institutional-based outcomes

System-level evaluation plans include an agency’s internal (“inputs”) markers, external (“outputs”) markers, or both, to monitor change in organizational efficiency and productivity. A capacity building initiative may focus, for example, on improving access to care by optimizing the organizational efficiency of a hospital. Internal agency markers include indicators such as changes in team composition and responsibilities within a medical team; creation and buy-in of group rules and values among staff; ensuring transparency in agency decision-making procedures by recording the number of staff meetings and the quality of feedback from clients and personnel; developing systems to monitor quality of care; and increasing agency funding and revenue.

Similarly, system-level programs may focus on improving organizational productivity. DEBIs, for example, seek to maximize the reach and impact of prevention programs across ASOs, CBOs, and health departments by identifying evidence-based behavioral strategies that are transferable to different communities. External agency outputs may include indicators such as the number of people reached within a catchment area; number of cross-agency collaborations; the quality of services, workshops and trainings offered; the proportion of evidence-based programs and services successfully meeting the needs of the community; and, the number of counseling and outreach services sponsored by the agency, among others.

As part of a system-level intervention, changes can occur at the individual level of analysis. System-level evaluation plans may include provider-specific measures to assess changes in provider attitudes and behaviors. A technical assistance program to facilitate the scale-up of voluntary counseling and testing (VCT) in a health clinic, for example, may include intervention components seeking to increase providers’ knowledge and attitudes regarding HIV counseling and testing, maximize their perceived job satisfaction, their perceived comfort towards service delivery, and enhance their skills to deliver counseling to clients and provide social support, among others. Similarly, individual-level changes can be evaluated among clients receiving services before and after a system-level intervention. Increases in the number of services offered in health clinic, for example, may increase clients’ intention to seek medical care, facilitate centralization of multiple health services within a single setting to optimize adherence, offer a greater number of opportunities for health education, and improve the quality of life of the patient and their social support networks.

HIV-related markers

System-level interventions can modify the epidemiologic characteristics of a disease at the regional and population levels. An agency strengthened by a systems-level initiative such as its capacity building to disseminate a DEBI may implement a primary prevention intervention that may decrease the incidence of HIV infection across a region. Similarly, restructuring a hospital’s initiative to increase the time spent in quality patient-provider interactions may increase the access and adherence to medical care that may result in the decrease of a county’s AIDS-related mortality rate and improve the quality of life of people living with HIV/AIDS.

Measurement of population health markers such as HIV/AIDS incidence and prevalence may be important to test the reach of a system-level intervention at a population-level. Nonetheless, these estimates may change slowly, additional epidemiologic markers such as quality of life indicators (QALYs/DALYs) and adherence rates should be included into system-level evaluations.

Method

Literature Search

We identified an inclusive list of search terms for studies that would be added to an electronic database for cataloguing: community, network, social, system, structural, intervention, program(me), and HIV/AIDS. To maximize the search, each term was assessed individually (i.e., structural & HIV) or in conjunction with other terms (i.e., structural intervention & AIDS) using Boolean expressions.

We approached the review process from two directions: (1) an inductive systematic collection of all potentially relevant articles found through computerized searches and (2) a deductive approach by reviewing the references of identified publications. The literature search included iterative searches of MEDLINE, Pub Med, PsycInfo, and The Cochrane Library. In addition, retrieved articles’ bibliographies were examined for any additional relevant intervention studies. To maximize the search, the research team placed no initial restrictions on publication year, population, or explicit mention of psychosocial factors associated with HIV prevention. Any publication written in English and related to HIV prevention, related HIV-risk behaviors, or potential predictors of these behaviors in the social sciences, public health, medicine, social work and human organization fields were eligible for inclusion.

Eligibility Criteria for Literature Search

We had a three-step filtering of the identified citations. First, we identified 4,129 citations published in over 100 journals between January 1, 1985 and February 1, 2007 using the generated list of keywords. From the citation abstracts, we excluded 3,339 studies because they were not written in English, solely detailed medical procedures, focused solely on HIV/AIDS prevalence or incidence surveillance statistics, and/or presented the results of predictor studies. An additional 166 entries were excluded because they were duplicates.

Second, we refined the search by reading each citation’s abstract and kept citations that offered any description of a potential system-level effect (n= 624 citations). Where relevant, we also included studies describing the implementation of an intervention. To be conservative, we included a citation if the abstract did not include sufficient information to determine if there a systems-level effect (i.e., did not include an agency-level or provider-level outcome). After filtering all non-relevant citations, we identified 104 potential citations for review. We were unable to find the full-text of sixteen of these studies.

Finally, we developed additional criteria for inclusion in the review. We included in this review interventions focusing on reducing HIV risk through technical assistance, scale-up and roll-out programs, or through optimizing agency capacity of hospitals, community-based organizations (CBOs) and AIDS service organizations (ASOs), and government institutions. To be considered for review, studies had to have been published in the scientific literature between January 1985 and February 2007. Only studies between 1995 and 2007 met review criteria. This period also reflects the onset of ARV scale-up and rollout. Following Exner et al. (1997), we focused on studies designed specifically to address HIV/AIDS and with available process and outcome evaluation data. We did not restrict our review to a subset of intervention outcomes to capture the scope and breadth of eligible systems-level intervention outcomes.

We acknowledge that many systems-level interventions may be missing from our review because they are implemented by agencies (i.e., UNAIDS, FHI, WHO, World Bank, Global Fund) that disseminate their program findings through non-peer-reviewed journals (for a recent review of these initiatives, see Druce et al., 2006). Furthermore, while the reach and importance of health policies as system-level interventions should not be undermined, we chose to exclude policy interventions from our review because they require different analytic considerations (Cason et al., 2002; Lazzarini & Klitzman, 2002; Weimer & Vining, 1999). Policy analysis requires an analytic approach that takes into account the political process, the social action movements lobbying for changes in government regulations. While influencing system-level initiatives, policy interventions are beyond (more “upstream”) the scope of system-level interventions.

Results

Out of 624 citations, we reviewed 26 reports meeting study criteria (i.e., interventions focusing on reducing HIV risk through technical assistance , infrastructure development, scale-up and roll-out programs, or through optimizing partnerships between hospitals, community-based organizations (CBOs) and AIDS service organizations (ASOs), and government institutions). While most studies reported more than one intervention strategy, the most common strategies were infrastructure development (17 studies; 74%), followed by collaborative partnerships across agencies to provide bundled services (12 studies; 53%), and technical assistance for agencies trying to develop and implement new interventions (9 studies; 43%). We found, however, that the choice of a particular strategy varied by location (see Table 1). Among studies based in the United States, infrastructure development was the strategy cited most often, followed by technical assistance programs and external partnerships. Among international studies, infrastructure development was the strategy cited most often, followed by external partnerships and technical assistance programs.

Table 1
Overview of strategies used across the reviewed studies by institutional context and location

Four research designs repeatedly emerged in the studies reviewed: a pretest-posttest design with one group (11 studies; 48%) or two groups (2 studies; 9%), a one-shot case study (5 studies; 22%), a randomized control trial (4 studies; 17%), and a one-group time series design (1 study; 4%). Below, we summarize our findings by cataloguing studies across intervention strategies: (a) technical assistance, (b) infrastructure development, or (c) external partnerships. Within each strategy, we identify common intervention components and comment on selection of outcomes to test intervention effects.

Technical assistance

Nine studies reported technical assistance as a component of their intervention (see Table 2). Over 75% of all technical assistance strategies included a component aiming to improve skills in program planning (7 studies) and intervention implementation (7 studies), respectively. Two studies included technical assistance in community outreach and service delivery. Technical assistance components were found more often in the U.S. than in international settings, particularly within NGO and government institutions.

Table 2
Summary Review Table of System-Level Interventions

Interventions using technical assistance components presented internal (e.g., improved in agency performance) and external (e.g., proportion of agencies adopting a new program) agency indicators among their primary outcomes. Only one study using technical assistance reported a positive increase in agency staff behavior change. Three studies reported improved outcomes among community members; two studies reported increased HIV prevention behavior change among clients while another reported decreased HIV/STI prevalence. All studies reported positive effects from baseline to follow-up in their primary outcomes.

Taken together, these studies suggest that technical assistance is a promising system-level strategy. Nonetheless, the impact of these interventions was variable. In particular, technical assistance components were given to a large number of agencies yet a smaller proportion adopted these interventions. Harshbarger et al. (2006), for example, reported that staff members from 229 agencies were training in the VOICES/VOCES intervention. At the three-month follow-up, more than a third of the agencies had decided to forgo implementation. Another third decided to implement the intervention with modifications, with no consideration as to how it would affect the intervention’s success. On the other hand, the studies by Kelly et al. (2000, 2006) deserve particular mention because they identify critical intervention components when delivering technical assistance to NGOs. In their randomized control trial comparing different teaching components within a national technical assistance program in the United States, Kelly et al. (2000) found that technical assistance efforts may have the strongest effect when multiple teaching components (i.e., intervention manuals, staff training, and follow-up consultation calls) are delivered as a bundle. In a multi-site international study, Kelly et al. (2004, 2006) found NGOs in the treatment condition (e.g., receiving distance education and technology transfer through the telephone, emails, or instant messaging) were more likely to implement a program than control NGOs. These findings are promising and highlight that system-level interventions including technical assistance components should carefully consider the inclusion of different types of learning materials as well as the integration of multi-communication modalities.

Infrastructure Development

Seventeen studies included infrastructure development components (see Table 2). The most common intervention strategies included skill building activities (8 studies; 47%) such as increasing patient-provider competency and efficient HIV prevention outreach strategies, followed by organizational restructuring (7 studies; 41%) such as the creation of institutional decision-making groups, and access to material resources (6 studies; 35%) such as computers and condoms. Only one study by Takahashi and colleagues (2007) discussed the access to seed-money for trainings and workshops. Infrastructure development components were found more often in the U.S. than in international settings (see Table 1).

Many of the studies including infrastructure development reported agency-level outcomes. Five studies reported internal agency-level outcomes. Four of these studies reported increased internal agency functioning; one study reported decreases in internal agency functioning due to a decrease in number of volunteers as greater number of paid positions were created in the agency. Three studies reported increased external agency outcomes such as an increase in number of programs implemented by the agency. Infrastructure development programs also included positive increases in agency staff’s skills and knowledge (n=2). Five studies found increases in client health behavior (e.g., integrating HIV care to antenatal care). Two studies found decreases in STI/HIV prevalence and incidence markers.

Most studies reported positive effects across their primary outcome measures. Overall, these studies suggest that system-level initiatives including infrastructure development components can help increase agency functioning and HIV prevention outcomes. The study by Morrill et al. (2005) deserves particular mention as an interesting case study on how the Massachusetts Prevention Planning Group restructured their internal processes to improve its decision-making process. Equally important were the findings on provider-communication skill building activities as an infrastructure development initiative to improve client outputs. Bluespruce et al. (2001), for example, found that training health care providers increased their self-efficacy to counsel patients on risky sexual practices as well as their ability to integrate sexual health questions into routine check-ups. Furthermore, Grosskurth et al. (1995) found that training health care providers in clinical examinations for STD-related symptoms, combined with community mobilization, decreased the incidence of HIV among clinic attendees. Taken together, these studies suggest that system-level interventions using infrastructure development components may benefit by activities that aid refine an institution’s internal processes (e.g., organization stability and community-input in the decision-making process) while also paying attention to the training needs of agency staff.

External Partnerships

Twelve studies included external partnerships (see Table 2). The most common intervention strategies included coalition building across community stakeholders (5 studies; 42%), followed by community mobilization (4 studies; 33%) and service bundling (4 studies; 33%). External partnership programs were based in the United States (see Table 1).

Studies including external partnerships reported increases in agency-level internal (n=4) and external (n=2) outcomes, respectively. Six studies measured changes in clients’ behaviors. Five of these studies found positive increases in clients’ behaviors. One study with an external partnership component informed decreases in HIV/STI prevalence and incidence markers.

The bundling of services, per se, is variable and may not always lead to clients’ use of services. Blank et al. (2005), for example, found that only 27% of all MSM attending an event in a non-traditional health venue accessed one or more services. This suggests that the availability of bundled health services may result in poor client uptake if the program setting is not considered carefully. On the other hand, given the increasing needs in bundling services and maximizing HIV prevention and care efforts, it is encouraging to find most system-level interventions with external partnership components reported positive effects.

Discussion

Overall, our findings suggest that system-level initiatives are a promising modality if we are to maintain or increase institutional capacity to provide HIV prevention services and rollout evidence-based interventions. From our review, we found three overarching themes that may strengthen system-level strategies. Within technical assistance, Kelly et al.’s (2000, 2004, 2006) work suggests that multiple highly-active education components in technical stategy are most effective. Particularly in a global world, using new technologies for distance learning and consultation seems promising and increases iterative consultations. Within infrastructure development, restructuring agency procedures to enhance internal staff communication and feedback seems vital. Furthermore, staff-training initiatives in interpersonal communication and public health skills seem promising as a mechanism to strengthen agency productivity and stability. External partnership strategies linking existing resources into bundles were found to be effective. In an era of scare funds for new services, the impact of service linkage is particularly heartening.

Reported intervention efforts varied by geographic locale, highlighting different epidemiologic priorities in the HIV epidemic. Interventions in the U.S., for example, commonly focused on strengthening the capacity of agencies working with subgroups at greatest risk for HIV infection (e.g., men who have sex with men and racial/ethnic minority women) in the U.S. System-level efforts in the U.S. focused on infrastructure development (e.g., organizational restructuring of community planning groups, or the creation of a database to monitor quality of care in a hospital) and external partnerships (e.g., coalition building among community stakeholders interested in a particular population group). These strategies reflect the need to strengthen the individual and collective resources of U.S.-based agencies in order to deliver, implement and tailor evidence-based HIV prevention education and treatment.

International locations with high HIV prevalence and incidence, on the other hand, sought to deliver HIV prevention and treatment services to the population in a defined catchment area. Most system-level interventions focused on infrastructure development (e.g., access to material resources for day-to-day functioning such as computers and funding) and external partnerships (e.g., the bundling of reproductive and HIV prevention services within a clinic). It is important to note that the selection of infrastructure development and external partnerships may reflect the need for prompt HIV prevention and treatment rollout in regions with high infection rates and mitigating social and economic vulnerabilities. Strikingly, very few of the studies meeting eligibility criteria focused on international settings.

Effective programs, both U.S.-based and international, commonly incorporated multiple intervention strategies. The presence of multiple strategies in an intervention limits our ability to understand the efficacy of the intervention components independently. While this is a common critique of public health programs (commonly referred to as an ‘intervention black box’), our findings suggest that, similar to individual and community prevention programs, system-level interventions require multiple strategies in order to support an agency’s ‘behavior change’ during service delivery, optimization and maintenance. Below, we propose several recommendations for future work in HIV/AIDS system-level programs.

Efforts to curtail the spread of HIV/AIDS, locally and globally, have highlighted the importance of identifying contexts and populations and tailoring theoretical constructs and strategies to improve program effectiveness. Unfortunately, although perhaps implicit, none of the reports reviewed mentioned explicitly the theoretical framework informing their system-level program. Moreover, the ecological nature of system-level interventions requires that these be matched with evidence-based community-level and individual-level health education programs. Consequently, peer-reviewed journals should allocate publication space so authors may include richer descriptions of the system-level constructs informing their program. These descriptions will facilitate program replication, translation, and adaptation, and, avoid wasting material and human resources, especially in communities where these resources are severely strained. Similarly, formative work is required prior to the implementation of a system-level intervention and the importance of this should not be trivialized (Jenkins & Carey, 2005). Researchers and practitioners’ experiences in developing and implementing system-level interventions are needed in the scientific literature. Without an understanding of the needs and resources required during the formative phase, replication of a system-level intervention may be difficult.

Stronger evaluation components are required in system-level intervention programs. The acknowledgement that system-level programs are context-specific highlights the importance of developing strong causal linkages between the intervention components and the desired outcomes (internal validity) and undermines the usefulness of randomization for generalizability (external validity). While some would argue that a randomized control trial design is the “gold standard” within system-level interventions, these designs may not be feasible because of pragmatic considerations such as increased costs, ethical concerns in randomizing systems, identifying and selecting characteristics for randomization and comparison of trials arms, among others (Rapkin & Trickett, 2005). Borrowing from lessons learned from community-level interventions (Shinn, 1990), system-level interventions should include elements that, when possible, strengthen the system-level intervention’s internal validity (i.e., having multiple groups, having different operationalizations of the outcome, alternating or reverting treatment conditions across groups, offering delayed treatment to a comparison group, and including multiple observations before and after treatment).

Quantitative and qualititative system-level process and outcome indicators deserve greater attention in the literature (Lipsey & Cordray, 2000). Careful programmatic and data monitoring procedures should be in place to detect potential biases such as measurement corruptibility (Rossi, 1997). When possible, success indicators should encompass multiple levels of analysis (i.e., agency outcomes, staff outcomes, and HIV markers).

Data linking system, community, and individual-level outcomes should be explored further in intervention evaluation. Nested data allow the testing of cross-level effects that, until recently, were disheartening because they provided biased estimates and violated the assumption of independence in procedures such as ANCOVA and linear regression (Raudenbush & Bryk, 2002). New statistical advances in this area, however, have overcome these issues and allowed for computation of nested models through packages such as Mplus and HLM. While a detailed review of this analytic technique is beyond the scope of this paper (see Bingenheimer & Raudenbush (2004) for an introduction to multilevel modeling in health), we provide several examples to illustrate the potential power of this technique. Examples of multi-level models within system-level programs may include: (1) exploring the effects of an agency capacity-building program (i.e., increases in funding) on the characteristics of the communities (i.e., community capacity) these intend to serve; (2) assessing the effects of a training for physicians (i.e., provision of social support toward HIV+ clients) on patients’ behavior (i.e., antiretroviral adherence); or (3) appraising how external partnerships increase the diffusion of an evidence-based intervention across multiple agencies and impact a region’s HIV/AIDS prevalence and incidence.

We acknowledge that our review has several limitations. First, our findings may be subject to publication bias given that unpublished studies offering weak or null effects are not reported as often in the scientific literature. Second, our criteria for peer-reviewed journal publications excluded system-level interventions implemented by international agencies (i.e., UNAIDS, FHI, WHO, World Bank) or health policies that are system-level interventions (i.e., needle exchange programs). While the bulk of system-level interventions tend to be carried out by these international agencies, we chose to exclude these interventions from our analysis as a recent paper by Druce et al. (2006) addressed these iniatives. Nonetheless, our review does include multiple system-level interventions within non-government agency settings. Furthermore, as the demand for peer-reviewed evidence-based findings continues to increase, we present the evidence in peer-reviewed scholarly journals in hopes that their evidence informs on-going programmatic efforts. Nonetheless, future studies reviewing system-level programs implemented by international agencies is useful. Finally, the variety in outcome operationalization and intervention strategies did not allow us to perform a meta-analysis. As more HIV system-level interventions are published with similar outcomes, however, we hope that measuring effect sizes across different system-level intervention strategies will be possible.

These limitations notwithstanding, our literature review of the peer-reviewed evidence on system-level interventions suggests that they are effective and varied. Future research focusing on expanding/refining system-level theoretical frameworks and sharing lessons learned from implementing these initiatives are needed. On-going prevention and treatment efforts can incorporate system-level interventions to strengthen their efforts from a top-down approach that may lead to strengthening evidence-based community, behavioral, and clinical HIV prevention and care efforts.

Acknowledgment

Special thanks to Drs. Alex Carballo-Diéguez, Jenny Higgins, Shari Dworkin, Robert H. Remien, Theresa Exner, and Ray Smith for their suggestions and contributions to earlier versions of this manuscript. This research was supported by a center grant from the National Institute of Mental Health to the HIV Center for Clinical and Behavioral Studies at New York State Psychiatric Institute and Columbia University (P30-MH43520; Principal Investigator: Anke A. Ehrhardt, Ph.D.) and a training grant from the National Institute of Mental Health (T32 MH19139 Behavioral Sciences Research in HIV Infection; Principal Investigator: Anke A. Ehrhardt, Ph.D.).

References

Alter C, Hage J. Organizations Working Together. Thousand Oaks, California: Sage Publishers; 1993.
Ball AM. Making the connections: AIDS and water. Canadian Journal of Public Health. 2006;97(1):56–59. [PubMed]
Bastos FI, Strathdee SA. Evaluating effectiveness of syringe exchange programmes: current issues and future prospects. Social Science & Medicine. 2000;51:1771–1782. [PubMed]
Beyer J, Trice HM. Implementing Change: Alcoholism policies in work organizations. New York: Free Press; 1978.
Bingenheimer JB, Raudenbush SW. Statistical and substantive inferences in Public Health: Issues in the application of multilevel models. Annual Review of Public Health. 2004;25:53–77. [PubMed]
Blank S, Gallagher K, Washburn K, Rogers M. Reaching out to boys at bars: Utilizing community partnerships to employ a wellness strategy for syphilis control among men who have sex with men in New York City. Sexually Transmitted Diseases. 2005;32(10):S65–S72. [PubMed]
Blankenship KM, Bray SJ, Merson MH. Structural interventions in public health. AIDS. 2000;14 (suppl. 1):S11–S21. [PubMed]
Bluespruce J, Dodge WT, Grothaus L, Wheeler K, Rebolledo V, Carey JW, McAfee TA, Thompson RS. HIV prevention in primary care: Impact of a clinical intervention. AIDS Patient Care and STDs. 2001;15(5):243–253. [PubMed]
Bolman LG, Deal TE. Reframing organizations: Artistry, choices, and leadership. (Third Ed.) San Francisco: Jossey-Bass; 2000. Introduction: The Power of Reframing; pp. 3–17.
Cason C, Orrock N, Schmitt K, Tesoriero J, Lazzarini Z, Sumartojo E. The impact of laws on HIV and STD prevention. The Journal of Law, Medicine & Ethics. 2002;30(3):139–145. [PubMed]
11. Dill A. Institutional environments and organizational responses to AIDS. Journal of Health and Social Behavior. 1994;35(4):349–369. [PubMed]
Druce N, Dickinson C, Attawell K, White AC, Standing H. Strengthening linkages for sexual and reproductive health, HIV and AIDS: progress, barriers and opportunities for scaling up: Final Report. London: Health Resource Center, UK Department for International Development; 2006. [Accessed on April 9, 2007]. via the World Wide Web at http://www.dfidhealthrc.org/publications/srh.html.
Exner TM, Seal DW, Ehrhardt AA. A review of HIV interventions for at-risk women. AIDS & Behavior. 1997;1(2):93–124.
Goodman RM, Steckler A. A framework for assessing program institutionalization. Knowledge in Society: The International Journal of Knowledge Transfer. 1989;2(1):52–66.
15. Harshbarger C, Simmons G, Coelho H, Sloop K, Collins C. An empirical assessment of implementation, adaptation, and tailoring: The evaluation of CDC’s national diffusion of VOICES/VOCES. AIDS Education and Prevention. 2006;18 (Suppl. A):184–197. [PubMed]
Hayes R, Mosha F, Nicoll A, Grosskurth H, Newell J, Todd J, et al. A community trial of the impact of improved sexually transmitted disease treatment on the HIV epidemic in rural Tazania: Design. AIDS. 1995;9:919–926. [PubMed]
Hoffman S, Exner TM, Leu CS, Ehrhardt AA, Stein Z. Female condom use in a gender-specific family planning clinic trial. American Journal of Public Health. 2003;93:1897–1903. [PubMed]
Hoy WK, Miskel CG. Educational Administration: Theory, Research, and Practice. (Third Ed.) New York: Random House; 1987.
Janz NK, Zimmerman MA, Wren PA, Israel BA, Freudenberg N, Carter RJ. Evaluation of 37 AIDS prevention projects: Successful approaches and barriers to program effectiveness. Health Education & Behavior. 1996;23(1):80–97. [PubMed]
Jenkins RA, Carey JW. Decision making for HIV prevention planning: Organizational considerations and influencing factors. AIDS & Behavior. 2005;9(2):S1–S8. [PubMed]
Johnson WD, Hedges LV, Díaz RM. Interventions to modify sexual risk behaviors for preventing HIV infection in men who have sex with men. The Cochrane Database of Systematic Reviews. 2007;1 [PubMed]
Johnson-Masotti AP, Pinkerton SD, Holtgrave DR, Valdiserri RO, Willingham M. Decision-making in HIV prevention community planning: An integrative review. Journal of Community Health. 2000;25(2):95–112. [PubMed]
Katz D, Kahn R. The Social Psychology of Organizations. (Second Ed.) New York: Wiley; 1978.
Kelly JA, Somlai AM, Benotsch EG, Amirkhanian YA, Fernandez MI, Stevenson LY, et al. Programmes, resources, and needs of HIV-prevention nongovernmental organizations (NGOs) in Africa, Central/Eastern Europe and Central Asia, Latin America and the Caribbean. AIDS Care. 2006;18(1):12–21. [PMC free article] [PubMed]
Kelly JA, Somlai AM, Benotsch EG, McAuliffe TL, Amirkhanian YA, Brown KD, et al. Distance Communication Transfer of HIV Prevention Interventions to Service Providers. Science. 2004;305:1953–1955. [PubMed]
Kelly JA, Somlai AM, DiFranceisco WJ, Otoo-Salaj LL, McAuliffe TL, Hackl KL, et al. Bridging the gap between the science and service of HIV prevention: Transferring effective research-based HIV prevention interventions to community AIDS service providers. American Journal of Public Health. 2000;90(7):1082–1088. [PubMed]
Klein SJ, O’Connell DA, Devore BS, Wright LN, Birkhead GS. Building an HIV continuum for inmates: New York State’s criminal justice initiative. AIDS Education and Prevention. 2002;14 (Suppl. B):114–123. [PubMed]
Koenig SP, Léandre F, Farmer PE. Scaling-up HIV treatment programmes in resource-limited settings: the rural Haiti experience. AIDS. 2004;18 (Suppl. 3):S21–S25. [PubMed]
Lazzarini Z, Klitzman R. HIV and the law: Integrating law, policy, and social epidemiology. Journal of Law, Medicine & Ethics. 2002;30:533–547. [PubMed]
Lipsey MW, Cordray DS. Evaluation methods for social intervention. Annual Review of Psychology. 2000;51:345–375. [PubMed]
Lyles CM, Kay LS, Crepaz N, Herbst JH, Passin WF, Kim AS, et al. Best-evidence interventions: Findings from a systematic review of HIV behavioral interventions for US populations at high risk, 2000–2004. American Journal of Public Health. 2007;97(1):133–143. [PubMed]
Mayaud P, Mosha F, Todd J, Balira R, Mgara J, West B, et al. Improved treatment services significantly reduce the prevalence of sexually transmitted diseases in rural Tanzania: Results of a randomised controlled trial. AIDS. 1997;11:1873–1880. [PubMed]
Mitchell CG, Perloff J, McVicker J, Ebbert S, Petersen L, Oltean A. Integrating prevention in residential and community care settings: A multidimensional program evaluation. AIDS Education and Prevention. 2005;17 (Suppl. A):89–101. [PubMed]
Molitor F, Kuenneth C, Waltermeyer J, Mendoza M, Aguirre A, Brockmann K, Crump C. Linking HIV-infected persons of color and injection drug users to HIV medical and other services: The California Bridge Project. AIDS Patient Care. 2005;19(6):406–412. [PubMed]
Morrill AC, Amaro H, Blake SM, Cranston K, Averbach AR, Logan JA, Dai J, Krech L, Dunn S. HIV prevention community planning: Enhancing data-informed decision-making. AIDS & Behavior. 2005;9(2):S55–S70. [PubMed]
National Institute of Mental Health (NIMH) Multisite HIV Prevention Trial Group. The NIMH Multisite HIV Prevention Trial: Reducing HIV Sexual Risk Behavior. Science. 1998;280:1889–1894. [PubMed]
Parry MF, Stewart J, Wright P, McLeod GX. Collaborative management of HIV infection in the community: an effort to improve the quality of HIV care. AIDS Care. 2004;16(6):690–699. [PubMed]
Peltzer K, Mngqundaniso N, Petros G. A controlled study of an HIV/AIDS/STI/TB intervention with traditional healers in KwaZulu-Natal, South Africa. AIDS & Behavior. 2006;10:683–690. [PubMed]
Perez F, Orne-Gliemann J, Mukotekwa T, Miller A, Glenshaw M, Mahomva A, Dabis F. Prevention of mother to child transmission of HIV: evaluation of a pilot programme in a district hospital in rural Zimbabwe. BMJ. 2004;329:1147–1150. [PMC free article] [PubMed]
Peterson AS, Randall LM. Utilizing multilevel partnerships to build the capacity of community-based organizations to implement effect HIV prevention interventions in Michigan. AIDS Education and Prevention. 2006;18 (Suppl. A):83–95. [PubMed]
Poindexter CC, Lane TS, Boyer NC. Teaching and learning by example: Empowerment principles applied to development, delivery, and evaluation of community-based training for HIV service providers and supervisors. AIDS Education and Prevention. 2002;14(5):391–400. [PubMed]
Prochaska JO, Redding CA, Evers KE. The Transtheoretical Model and Stages of Change. In: Glanz K, Rimer BK, Lewis FM, editors. Health Behavior and Health Education: Theory, Research, and Practice. San Francisco: Jossey-Bass; 2002. pp. 99–120.
Ramos RL, Ferreira-Pinto JB. A model for capacity-building in AIDS prevention programs. AIDS Education and Prevention. 2002;14(3):196–206. [PubMed]
Rapkin B, Trickett E. Comprehensive dynamic trial designs for behavioral prevention research with communities: Overcoming inadequacies of the randomized controlled trial paradigm. In: Trickett E, Pequenaut W, editors. Increasing the community impact of HIV prevention interventions. New York: Oxford University Press; 2005. pp. 249–277.
Raudenbush SW, Bryk A. Hierarchical Linear Models: Applications and Data Analysis Methods. (Second Ed.) Thousand Oaks, California: Sage Publications; 2002.
Richie BE, Freudenberg N, Page J. Reintegrating women leaving jail into urban communities: A description of a model program. Journal of Urban Health: Buelleting of the New York Academy of Medicine. 2001;78(2):290–303. [PMC free article] [PubMed]
Richter DL, Potts LH, Prince MS, Dauner KN, Reininger BM, Thompson-Robinson M, et al. Development of a curriculum to enhance community-based organizations' capacity for effective HIV prevention programming and management. AIDS Education and Prevention. 2006;18(4):362–374. [PubMed]
Rossi PH. Program outcomes: conceptual and measurement issues. In: Mullen EJ, Magnabosco J, editors. Outcome and Measurement in the Human Services: Cross-Cutting Issues and Methods. Washington, DC: Natl. Assoc. Social Workers; 1997.
Shea MA, Callis BP, Cassidy-Stewart H, Cranston K, Tomoyasu N. Diffusion of effective HIV prevention interventions – Lessons learned from Maryland and Massachusetts. AIDS Education and Prevention. 2006;18 (Suppl. A):96–107. [PubMed]
Shinn M. Mixing and matching: Levels of Conceptualization, Measurement, and Statistical Analysis in Community Research. In: Tolan P, Keys C, Chertok F, Jason L, editors. Researching community psychology: Issues of theory and methods. Washington, D.C.: American Psychological Association; 1990. pp. 111–126.
Steckler A, Goodman RA, Kegler MC. Mobilizing organizations for health enhancement. In: Glanz K, Rimer BK, Lewis FM, editors. Health Behavior and Health Education: Theory, Research, and Practice. San Francisco: Jossey-Bass; 2002. pp. 335–360.
Takahasi LM, Candelario J, Young T, Mediano E. Building capacity for HIV/AIDS prevention among Asian Pacific Islander organizations: The experience of a culturally appropriate capacity-building program in southern California. Journal of Public Health Management and Practice. 2007;1 (Suppl.):S55–S63. [PubMed]
Volmink J, Siegfried NL, van der Merwe L, Brocklehurt P. Antiretrovirals for reducing the risk of mother-to-child transmission of HIV infection. The Cochrane Database of Systematic Reviews. 2007;1 [PubMed]
Weimer DL, Vining AR. Policy Analysis: Concepts and Practice. (Third Ed.) New Jersey, Prentice Hall: Englewood Cliffs; 1999.
Xiaoming S, Yong W, Choi KH, Lurie P, Mandel P. Integrating HIV prevention education into existing family planning services: Results of a controlled trial of a community-level intervention for young adults in rural China. AIDS & Behavior. 2000;4(1):103–110.