|Home | About | Journals | Submit | Contact Us | Français|
Since 2003, the President’s Emergency Plan for AIDS Relief (PEPFAR) has been the most ambitious initiative to address the global HIV epidemic. However, PEPFAR’s effect on HIV-related outcomes is unknown.
To assess PEPFAR’s impact on HIV-related deaths, the number of people living with HIV, and prevalence in Africa.
Comparison of trends before and after the initiation of PEPFAR’s activities in the African focus countries with other African countries.
Twelve African focus countries and 29 controls with a generalized HIV epidemic from 1997–2007.
451 country-year observations.
A 5-year, $15 billion program for HIV treatment, prevention, and care that started in late 2003.
HIV-related deaths, the number of people living with HIV, and prevalence.
Between 2004 and 2007, the annual change in the number of HIV-related deaths was lower by 10.5% in the focus countries compared to controls (p=0.001). The difference in trends between the group before 2003 was not significant. The annual growth in the number of people living with HIV was 3.7% slower in the focus countries compared to controls from 1997–2002 (p=0.05), but during PEPFAR’s activities the difference was no longer significant. The difference in the change in HIV prevalence was not significantly different throughout the study period. These estimates were stable after a sensitivity analysis.
The selection of the focus countries was not random, and limits the generalizability of the results.
We find evidence that, after four years of activity, HIV-related deaths declined in PEPFAR’s focus countries relative to sub-Saharan African controls, but trends in adult prevalence were not different. Assessing epidemiologic effectiveness should be part of PEPFAR’s evaluation programs.
Alleviating the burden of HIV in sub-Saharan Africa is one of the great challenges of our time (1). HIV is the leading cause of death in Africa, and is responsible for a reduction in life expectancy in many countries (2). Available resources for HIV in less developed countries expanded more than five-fold since the Declaration of Commitment on HIV/AIDS, adopted by the UN General Assembly in 2001 (3, 4), but the expansion has not been evenly distributed. Since 2003, the US President’s Emergency Plan for AIDS Relief (PEPFAR) provided the majority of its $18.8 billion budget to 15 focus countries – 12 of them in Africa – for HIV/AIDS prevention, treatment, and care (5, 6). While the criteria for selecting focus countries were not explicit, they were related the burden of disease, the focus countries’ governmental commitment to fighting HIV, administrative capacity, and a willingness to partner with the US government. Nearly half of PEPFAR’s resources were spent on antiretroviral drugs and treatment infrastructure, and about a fifth was spent on prevention programs, of which one third was earmarked for abstinence-only programs. Through its various activities, PEPFAR aimed to support the provision of life-saving antiretrovirals to 2 million people and prevent 7 million HIV infections in the focus countries. Among the three major funders of HIV/AIDS programs – the others are The Global Fund to Fight AIDS, Tuberculosis, and Malaria and the World Bank – PEPFAR is unique in its distinctive approaches and disproportionate funding of a few countries (7–10).
In July, 2008, the US Senate reauthorized PEPFAR with a $48 billion budget for the next five years, including a broader emphasis on strengthening health systems (11). The US leadership in the battle against HIV has been one of its most important legacies in Africa (12, 13). However, despite the substantial financial commitment and the important role PEPFAR plays abroad, no quantitative evaluation of the program’s outcomes has been completed. The original legislation mandated a short-term evaluation that was completed by the Institute of Medicine in 2007 (14). Their report scrutinized PEPFAR’s ability to meet its targets for delivery of prevention, treatment, and care services in the focus countries, and found that within 2 years, PEPFAR supported expansion of HIV/AIDS activities in the focus countries; however, it did not evaluate health-related outcomes such as HIV mortality, incidence, or prevalence.
The number of human lives affected and the financial stakes make it essential to assess the impact of PEPFAR's investment in Africa. Although the full impact of PEPFAR may not be felt for years, an ongoing evaluation of programmatic outcomes is central to the direction of future policies. We undertake the first quantitative evaluation of HIV-related outcomes in the focus countries compared with other countries in sub-Saharan Africa with a generalized HIV epidemic.
All countries of central, east, west, and south Africa and the island nations of Cape Verde, Comoros, Madagascar, Mauritius, São Tomé and Príncipe, and Seychelles were eligible for this analysis. We excluded countries if they did not have epidemiologic data or if their HIV epidemic was not generalized. Generalized epidemic was defined as HIV prevalence of over 1% in antenatal clinics and a predominantly heterosexual mode of transmission (9, 15). The countries which PEPFAR selected as focus countries were examined as an intervention group, while all other African countries with a generalized HIV epidemic were designated controls.
The joint United Nations Programme on HIV/AIDS (UNAIDS) has monitored the Declaration of Commitment on HIV/AIDS by working with individual countries to measure the global epidemiology of HIV/AIDS (16). We used country and year epidemiologic data obtained through UNAIDS from 1997 to 2007 as the outcome variables for this study. UNAIDS determines the prevalence trends using sentinel and population based surveys. The prevalence estimates, vital registry data, and model-based calculations are then used to determine mortality and the number of people living with HIV. All the estimates are given uncertainty bounds which depend on the quality of the primary data and the strength of the assumptions used in the estimation process (17).
We obtained data on Global Fund allocation of resources, population structure, governance indicators, life expectancy, and per-capita gross domestic product from publicly available databases (10, 18–20).
We chose two study periods for this analysis: an early period, from 1997–2002, prior to the onset of PEPFAR; and a late period, from 2004–2007, during PEPFAR’s activities. We excluded 2003, the watershed year when PEPFAR’s operations were getting organized. Three outcomes of interest were examined: HIV prevalence among adults age 15–49, number of deaths due to HIV/AIDS, and the number of adults living with HIV. We used these indicators as outcomes because they reflect the most consistent measures for cross-country comparisons available over time. The indicators are publicly available (21).
We compared trends of the epidemic outcomes between the focus and control countries during the two time periods noted above. The early period represented the trends of the epidemic absent PEPFAR activity, while the analyses of the late study period represented the trends during PEPFAR’s activity. We tested the hypothesis that the epidemic in the focus countries showed greater improvement (or lesser worsening) than in the control countries after the onset of PEPFAR’s activities. We included the earlier time period in order to separate the effects of PEPFAR from the natural course of the epidemic and any between-country differences; that is, we wanted to compare not only the relative trends after PEPFAR, but also any changes in the relative trends after the onset of PEPFAR.
We analyzed the trends using a fixed-effects model for longitudinal data (22). We fixed the time and country effects and clustered by country to account for within-country correlation of observations over time (23). The year, PEPFAR designation as a focus country, and an interaction term between them were included as predictors in the unadjusted model. Except for prevalence rates, the outcomes we examined depend on the size of the population. To account for that, we examined the percentage change in the outcomes of interest by using log transformations of the outcomes (24). For prevalence rates, we report the difference between the focus and control countries of the change in prevalence rather than the difference in percentage change. We report the interaction term coefficient, which estimates the difference in the trends between the focus and control countries. For example, a coefficient of −2% in the number of deaths from 2004 to 2007 suggests that, during that period, the percentage change in the number of deaths was 2% less in the focus countries than in the controls (which means that deaths were rising more slowly or descending more rapidly, depending on the overall trend). We used Stata 9.2 (StataCorp, College Station, Texas) for all analyses.
In adjusted analyses, we accounted for baseline HIV prevalence (1997), population size, life expectancy, per-capita gross domestic product, the amount of funding from the Global Fund for HIV per capita, and four measures of governance from the World Bank’s Worldwide Governance Indicators: control of corruption, political stability, rule of law, and government effectiveness (20). The Indicators measure the quality of governance in over 200 countries based on 35 data sources from 32 organizations. They have been available since 1996, and are widely used in development research (25, 26). We also changed the model structure to allow for random year effects. Since none of these additional analyses changed the direction or significance of the relative effect of PEPFAR, we report the results of the unadjusted analyses.
To account for the assumptions and imprecision in the UNAIDS data, we did two different sensitivity analyses. First, we performed a Monte Carlo process where the outcome variable was drawn randomly within the uncertainty bounds for each country-year, and the analysis was repeated 1,000 times. We also performed a sensitivity analysis where we randomly selected groups of “focus” countries and ran the analysis 1,000 times to verify our analyses were not biased in favor of finding significant differences (23).
This work was supported in part by a training grant from the Agency for Healthcare Research and Quality. The funding agency had no part in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.
We examined twelve PEPFAR focus countries and 29 control countries from sub-Saharan Africa for the analysis. Tables 1 and and22 show the selected countries and their baseline characteristics, respectively. The focus countries had a significantly higher average population (37 vs. 11.7 million people, p<0.01), but there was no difference in the life expectancy of men or women. The per-capita disbursements from the Global Fund were slightly more generous in the control countries, but the annual per-capita gross domestic product was significantly lower in those countries ($1,935 vs. $4,094). The focus countries ranked higher than the controls on all governance indicators – significantly higher in government effectiveness – at the time of PEPFAR’s start. The adult HIV prevalence was higher in the focus countries throughout both study periods (p<=0.05).
The number of deaths from HIV/AIDS rose in the first period, but declined in the second period in the focus countries and leveled off in the controls (Figure 1). Prior to 2003, the annual percent increase in the number of deaths due to HIV was 3.5% higher in the control countries, but that difference was not significant (p=0.22). During the second study period, however, the death toll from HIV decreased much more rapidly in the focus countries compared to the controls. During this period, the difference in the percentage change was 10.5% lower in the focus countries (p=0.001, Table 3). Figure 2 shows that the difference in death rates between the groups of countries started after 2003, and was most pronounced between 2005 and 2006, after nearly 3 years of PEPFAR activities.
The number of people living with HIV/AIDS rose in the focus and control countries throughout both study periods (Figure 1). The annual percent change in the number was lower by 3.7% in the focus countries during the early period (p=0.05, Table 3). In the late period, during PEPFAR’s activities, the annual percent rise in the number of people living with HIV/AIDS was slower in both groups of countries. However, the difference (1.7% lower in the focus countries) was no longer significant (p=0.12). That is, between the early and late periods the focus countries experienced a relative acceleration in the number of people living with HIV/AIDS compared with the control countries. Figure 2 shows the difference in the annual percent change diminishing (getting less negative) after 2003, while the difference in the number of people living with HIV/AIDS was negative (rise was slower in the focus countries) before 2003.
The HIV prevalence in both groups of countries peaked by 2002 (Figure 1). Before 2003, the prevalence rose faster in the focus countries compared to the controls (0.05% faster annually), an insignificant difference. During PEPFAR’s activities, the difference in prevalence change between the groups was nearly 0% (p=0.95), as both groups of countries experienced a gradual decline of about 0.1% annually in the HIV prevalence in the adult population. Figure 2 shows that the difference in the adult prevalence diminished throughout the early period, and remained stable around 0% difference after 2003. Additional details on the regression models are provided in Supplementary Tables 1–6.
To assess the stability of our analyses, we used the uncertainty bounds provided by UNAIDS to select values for the outcome measures. We used a uniform distribution to draw a value for the outcome variables, avoiding assumptions about the true value. We then used the randomly picked value for each country-year as the value for the regression. This process was repeated 1,000 times, and we collected the mean of the coefficients and p-values. Using this procedure, the difference in the annual percent change in the deaths due to HIV/AIDS was −2.0% during the early period (p=0.29), and −9.7% in the late period (p=0.02) in the focus countries compared to controls. We obtained a significant difference in deaths from HIV in 90.7% of the iterations in the late period, but only 3.7% of the iterations in the early period. The full results of the sensitivity analysis is provided in Supplementary Table 7.
Our second procedure for verifying the stability of our results, where we randomly selected groups of countries to serve as “focus” countries for the analysis, showed that any significant differences, especially in the number of deaths, disappeared.
This study examined the impact of PEPFAR by comparing changes in three outcome measures between the focus countries and other countries with a generalized HIV epidemic in sub-Saharan Africa, before and after the onset of PEPFAR. Our results show that after four years of activity, there is evidence to suggest that PEPFAR was associated with a decrease in the death toll due to HIV/AIDS, and may be linked to a relative increase in the number of people living with HIV/AIDS. We also see no evidence that PEPFAR was associated with changes in adult HIV prevalence in the focus countries compared to other sub-Saharan African countries.
The reduction in HIV-related deaths is likely the result of improved treatment and care of HIV-infected individuals in the focus countries, especially the scale-of highly active antiretroviral therapy. Increases in antiretroviral therapy coverage occurred disproportionately in the focus countries during the past 5 years (9), and nearly half of PEPFAR’s expenditures are dedicated to purchasing antiretrovirals, constructing treatment infrastructure, and providing antiretroviral services (28). Around the time of PEPFAR’s launch, other governmental and multinational organizations, most notably the Global Fund, also scaled up their activities to combat HIV, and may have contributed to the decline in HIV deaths seen across the continent. However, the added infusion of funding for antiretrovirals in the focus countries made an appreciable impact on the deaths from HIV and speaks to the power of antiretrovirals to improve survival in a relatively short period of time.
We observe a relative acceleration in the number of people living with HIV/AIDS in the focus countries relative to controls during PEPFAR’s activities. While long-term goals may target reductions in the size of infected populations, this increase probably reflects the decreasing death toll and may have several public health spillover benefits. For example, infected adults who live longer may be able to support their children and dependent elderly, reducing the burden of orphans and elderly care.
Changes in prevalence are complex and depend on the rate of new infections, deaths due to HIV/AIDS, and changes in the size of the population. We see no evidence that prevalence trends in the focus countries were different from those in the control countries during PEPFAR’s activities. To affect a reduction in HIV prevalence, the combined effect of reduced HIV incidence and increased population size has to offset the reduction in deaths from HIV/AIDS. While it may be too early to observe these changes, it is important to follow this trend for several reasons: measurement of prevalence is standardized in many countries, longitudinal trends through sentinel sites are widely available, and it is a key determinant of infection risk (29). Reduction in prevalence that may be attributable to PEPFAR would be a consequential accomplishment for the next five years of PEPFAR.
As the number of people on antiretroviral therapy and the deaths averted in the focus countries continue to increase, the cost of providing treatment is expected to increase as well. Projections of financial resources needed to sustain the treatment scale-up suggest that even with PEPFAR’s increased commitment, the gap between the available funds and those needed will continue to increase unless the incidence of HIV in Africa is substantially reduced (3). Striking the right balance between treatment and prevention with insufficient resources for the burden of the epidemic is a major challenge for comprehensive care programs such as PEPFAR.
Monitoring impact will have important implications for PEPFAR’s future as well as that of other organizations that are operating with poor information about the effectiveness and efficiency of their programs. In a recent report, the Institute of Medicine drafted general considerations for evaluating PEPFAR’s impact, including HIV prevalence, mortality, and incidence, as well as broader metrics such as system capacity, economic development, and health status (30). Incidence, in particular, has not been directly estimated in Africa, and the measurement techniques are increasingly available (31). Impact evaluations are difficult, but rigorous methods adopted specifically to resource-limited regions, including randomization into program entry, are commonly used in other disciplines (32).
We evaluated the contribution of PEPFAR to the abatement of the epidemic in the focus countries, which has implications for the program’s economic efficiency. By the end of 2007, PEPFAR spent more than $6 billion on HIV care, prevention, and treatment in the 12 focus countries examined in this study. In those countries, a reduction in the death rate of 9.5% implies about 1.1 million deaths were averted due to PEPFAR’s activities. This large benefit cost about $2,700 per death averted, assuming PEPFAR directed half of its budget towards treatment. This is a rough estimate, and it may change as the treatment infrastructure and supply chains become more established, but it could allow an evaluation of the program’s efficiency. A formal cost-effectiveness analysis will also allow a comparison with other interventions for HIV in Africa (33).
Our study has several methodological limitations. First, we use UNAIDS data that is partly derived with the help of mathematical models. We deal with this limitation by doing a sensitivity analysis using the uncertainty bounds, which take into account the imprecision in both the primary data and the model estimates. We use a resampling procedure that is agnostic about the exact values of outcomes within the uncertainty bounds, and strengthens the results of the primary analysis. It is also possible that the data in the focus countries is more reliable than in the controls. This is unlikely to change the results, as PEPFAR’s support for monitoring and evaluation programs was minor, and this effect is likely to be small.
Second, we perform cross-country comparisons between groups of countries which were not picked at random and with significant baseline differences. We address this partly by controlling for observable potential sources of bias such as population, gross domestic product, aid for HIV from other sources, and governance indicators. Because the focus countries were not selected randomly, and we cannot fully observe the differences between the groups, our measured effects may be specific to the countries and years of the study. That is, we cannot fully generalize the results to other countries or other time periods. If PEPFAR had chosen a different set of focus countries or operated at a different time, we may have observed a different impact.
Third, it is possible that the observed difference in deaths after 2004 results from a difference in the phase of the epidemics in the study countries. That is, if the epidemic in the focus countries was more mature than the controls, then the observed relative reduction in deaths could occur without any intervention or program. However, the lack of difference in HIV deaths prior to 2003 argues against this possibility as an explanation of the results.
While the Institute of Medicine scrutinized PEPFAR’s operations, it failed to evaluate outcomes that are central to the epidemic’s prevention and treatment efforts. The commitment of funds by the US government is commendable, but it is crucial to ascertain that the Plan is effective, and that the investment is cost-effective. As PEPFAR enters its next funding period, evaluating outcomes will highlight the areas which are successful and those which are not making an appreciable impact. This analysis shows the success of PEPFAR to avert HIV-related deaths in a relatively short period of time. It also underscores the importance of a continued outcome-based evaluation of this essential and expensive intervention.
The authors thank Karen Stanecki of UNAIDS for help with providing and clarifying the data used in this study and Grant Miller, PhD and Kanaka Shetty, MD MS of the Center for Health Policy at Stanford for methodologic contributions.
Grant Support: Dr. Bendavid is supported by a training grant from the Agency for Healthcare Research and Quality. Dr. Bhattacharya is supported by the National Institute on Aging.
Potential Financial Conflicts of Interest: None disclosed.
Author Contributions: Conception and design: E. Bendavid
Analysis and interpretation of the data: E. Bendavid, J. Bhattacharya.
Drafting of the article: E. Bendavid.
Critical revision of the article for important intellectual content: E. Bendavid, J. Bhattacharya.
Final approval of the article: E. Bendavid, J. Bhattacharya.
Statistical expertise: E. Bendavid, J. Bhattacharya.
Collection and assembly of data: E. Bendavid.
Author Contribution and Conflicts of Interests
Eran Bendavid: I declare that I participated in originating the concept, in conducting the data collection, and data analysis. I did most of the writing of the paper, including the final revision. I had full access to all the data in the study and had final responsibility for the decision to submit for publication. I have no conflict of interests.
Jayanta Bhattacharya: I declare that I participated in elucidating the original concept, in the data analysis, critical review of the results, and revising the manuscript. I have seen and approved the final version and I have no conflict of interests.