|Home | About | Journals | Submit | Contact Us | Français|
Tests for recent infection (TRIs), such as the BED assay, provide a convenient way to estimate HIV incidence rates from cross-sectional survey data. Controversy has arisen over how the imperfect performance of a TRI should be characterised and taken into account. Recent theoretical work is providing a unified framework within which to work with a variety of TRI- and epidemic-specific assumptions in order to estimate incidence using imperfect TRIs, but suggests that larger survey sample sizes will be required than previously thought. This paper reviews the framework qualitatively and provides examples of estimator performance, identifying the characteristics required by a TRI to estimate incidence reliably that should guide the future development of TRIs.
When monitoring HIV epidemics it is vital to estimate incidence in order to plan and evaluate HIV programmes . Prospective cohort studies are the most direct way to achieve this. They are, however, expensive, prone to recruitment and retention bias, and potentially rendered unrepresentative by ethical obligations. The use of prevalence data in conjunction with mathematical modelling is an alternative approach [2,3], but is indirect and requires accurate knowledge of mortality and migration. The disadvantages of these methods have focused attention on estimating incidence from cross-sectional surveys [4–8], with the result that a number of assays and algorithms that test for recent infection have been developed [9,10]. In the context of HIV, such an assay or algorithm has sometimes been termed a STARHS (Serological Testing Algorithm for Recent HIV Seroconversion) [9,10], but we prefer to use the generic term ‘test for recent infection’ (TRI), because it does not specify a particular disease and method of testing. Recently, the World Health Organization (WHO) Technical Working Group on Statistical Approaches for Development, Validation and Use of HIV Incidence Assays has proposed using the term ‘recent infection testing algorithm’ (RITA). The term has not, however, gained universal acceptance.
TRIs identify HIV-positive individuals who have been infected recently. By using a TRI in a serosurvey, incidence (I) can be estimated by applying the epidemiological relationship (based on ‘Prevalence = Incidence × Duration’):
where R and S are the counts of ‘recently infected’ and ‘susceptible’ (HIV-uninfected) individuals observed in the cross-sectional survey and D is the mean duration spent in the ‘recently infected’ state, often called the (mean) window period. This incidence estimate is an average of the instantaneous incidence over a period of approximately D prior to the survey. The problem of incidence estimation then reduces to measuring the prevalence of ‘recent infection’, given knowledge of its duration.
TRIs usually discriminate recent from established infections by measuring specific aspects of the immune system which evolve during the course of initial infection. For HIV, this is typically the antibody response, with the titre, proportion of HIV-specific IgG, or antibody avidity (or a combination of these) providing quantitative output . Laboratory-defined thresholds are chosen to convert these outputs into categorical results. These results may be augmented with other clinical information, such as CD4 lymphocyte counts and antiretroviral therapy (ART) status, to classify individuals as either TRI-positive (P i.e. recent) or TRI-negative (N i.e. non-recent). Positive and negative in this context should not be confused with HIV-positive and HIV-negative.
The interaction between the virus and the immune system is complex, and individuals vary in their response to infection as assessed by a particular TRI. Modest variation is not intrinsically problematic, but serious complications arise if, in some individuals, the immune response is such that they remain indefinitely classified as TRI-positive or if individuals revert back to a TRI-positive classification as a result of advanced disease or in the presence of antiretroviral therapy. Unfortunately, both these complications arise for TRIs currently in use. This not only limits the applicability of the simple incidence estimator above, but also makes it difficult to define and estimate the mean duration spent in the recently infected state (i.e. to evaluate D). Methods for ‘adjusting’ estimates of incidence have been proposed [7,8] and adopted by the United States Centers for Disease Control and Prevention  but are currently under debate [12–15]. Recently, a formally rigorous framework has been developed [16,17]. We provide a summary of the framework and explore its implications for the analysis of surveys and development of new TRIs.
We now briefly describe the theoretical framework and how it can be generalised. The key results that emerge from the analysis are:
Previous work by McDougal et al.  used terminology usually employed to characterise the performance of diagnostic tests, such as sensitivity and specificity, to characterise TRI performance. ‘Recent infection’ was defined as being infected for less than a particular time (chosen to be the mean window period). A sensitivity and two specificity parameters were introduced to characterise imperfect classification. No procedure incorporating the effect of parameter uncertainty has thus far been proposed to estimate statistical error or power for the McDougal approach. It has recently been shown that use of sensitivity and specificity parameters is a redundant description of the TRI characteristics [17,19]. In contrast, the new framework defines the condition of being ‘recently infected’ directly in terms of the TRI result. This approach is applicable under less restrictive assumptions, is less prone to bias, and admits an equally informative description of TRI performance using only ω and Pnp .
In deriving the results outlined above, two assumptions were made. Firstly, it was assumed that individuals who do not progress on the TRI have the same survival outcomes as TRI progressors. There is, however, evidence for some TRIs that individuals that fail to progress on the test have a survival advantage. For example, in Baltimore, USA, 60% of elite suppressors (individuals with naturally suppressed virus below 50 copies per ml) failed to progress on the BED assay , and elite suppressors have been observed to survive for longer than others . Secondly, it was assumed that TRI progressors never regress back to the TRI-positive state, but there are indications that this is not true for some TRIs. For example, the rate of misclassification by the BED assay is observed to be higher in individuals with advanced infection  and individuals on ART [22–24]. When these assumptions are true, Pnp is always equal to the proportion of non-recently infected individuals who are classified TRI-positive. When the assumptions are violated, this proportion, or false-recent rate, denoted by ε, varies according to the historic trajectory of the epidemic [17,25]. This would be consistent with the apparently higher BED assay false-recent rate in Uganda  (an older, declining epidemic) than in South Africa  (a younger, growing epidemic) . It is, however, still possible to estimate the number of recent infections by replacing Pnp, in the expression (1) above, with an estimate of ε applicable to the time and place of an incidence survey  (see eAppendix http://www0.sun.ac.za/sacema/publications/eAppendix.pdf for justification). The incidence estimator can then be written as:
The inputs to this estimator are of two types: survey counts (P, N and S), which need to be estimated in every incidence survey, and parameters that describe the characteristics of the TRI (ω and ε), which ideally are estimated in a smaller number of parameter estimation studies.
When ε and ω are known with sufficient accuracy, there are no theoretical reasons why an imperfect TRI should not allow the accurate estimation of incidence. However, two distinct types of practical problems arise – counting error and TRI parameter error. An important component of recent developments is the first consistent analysis of incidence uncertainty accounting for both counting and parameter error (see eAppendix http://www0.sun.ac.za/sacema/publications/eAppendix.pdf for a description of the uncertainty expression). We now illustrate this uncertainty with a somewhat idealised model of the BED assay, which has received much attention and application .
Even in the largest HIV epidemics, infection events are relatively rare (about 2% of the population per year) and ‘recent’ infections (infections in the last 155 days or so, for the BED assay [7,29]) are even less common (about 0.85% in a cross-section of the population). Thus, estimates of incidence are associated with substantial uncertainty since there are few recent infections to be counted. Figure 1 shows the coefficient of variation (CV, which is the ratio of the standard deviation to the estimate) for the estimator (2) calculated under various survey sample sizes and steady-state HIV incidence rates (see eAppendix http://www0.sun.ac.za/sacema/publications/eAppendix.pdf for a description of the uncertainty and steady-state calculations). The TRI parameters (ω and ε) are assumed to be known with absolute certainty. Low values of CV are desirable and indicate that estimates of incidence have small confidence bounds, while high values indicate that incidence estimates will be less certain. For example, in a cross-sectional survey of 5,000 individuals from a population with a steady-state incidence of 2.0 per 100 person years at risk (pyar) the CV is 25.8% – i.e. the 95% likelihood interval for an incidence estimate is 1.0 to 3.0 per 100 pyar.
To explore the ability to detect a change in incidence, a substantial reduction (halving) in incidence is simulated (initially in a steady-state epidemic, with prevalence remaining constant between the two surveys), and a two-tailed test of the null hypothesis that incidence is the same in the two surveys is performed. The possible outcomes are: sustaining the null hypothesis, or concluding that incidence has either increased or decreased. Figure 2 shows the probability of correctly inferring a reduction in incidence, when testing the null hypothesis at a significance level of α=5%. A probability close to 100% indicates that reductions in incidence will be reliably detected, with a probability of less than 90% indicating that results will be unreliable. The South African National Strategic plan for HIV AIDS  has ambitiously set a target of halving incidence between 2007 and 2012. Our calculations suggest that the sample size of each of two surveys (in 2007 and 2012) required to reliably conclude that incidence has decreased, at the 5% significance level, is approximately 25,000.
In the previous section, it was assumed that the correct TRI parameters were known with certainty. The incidence estimates are very sensitive to changes in the values of ω and ε, however, and small differences between the values used in the calculation and the true values can lead to large errors. These parameters have to be estimated in separate studies, usually using cohorts of individuals whose infection time is known approximately. Such cohorts are rare, however, and the numbers of individuals in them are typically small, resulting in substantial uncertainty for the values of ω and ε. In Figure 3 we explore the uncertainty of the estimator (expressed as a CV), as a function of the uncertainty in the TRI parameters. For example, when the BED-like parameters are known with a CV of only 15.0% (reference to a CV for ω relates to the uncertainty of the estimate of ω, not the variation associated with progression times), at a sample size of 5,000 and a steady-state incidence of 2.0 per 100 pyar the CV, as a result of both counting error and parameter uncertainty, is 35.7% – i.e. the 95% likelihood interval for an incidence estimate is 0.6 to 3.4 per 100 pyar.
Since the TRI parameter estimation study may be conducted in a separate population, it is possible to introduce systematic bias if the true values of the TRI parameters vary between populations or over time. The few estimates of ε that have been published vary widely. For example, the false-recent rate is estimated at 1.7% in a South African survey  and 26.7% in Rwanda and Zambia  presumably due to population differences in the historic courses of the epidemics, viral subtypes, host immune profiles, and uptake of antiretroviral therapy. This undermines confidence in the ability to use an estimate for ε obtained in a different population to the one in which incidence is to be estimated, and could contribute to the apparently inflated estimates of incidence reported recently [34,35]. There is also currently no general theoretically unbiased procedure for estimating ε – work on this problem is in progress . In Figure 4 we explore the systematic error in the incidence estimate, expressed as a percentage of the correct value, introduced by systematic errors in the TRI parameters, also expressed as percentages. There is a region in which bias may be small due to cancellation of systematic errors (see the zero error contour).
In the short term, reports from early studies using BED should be interpreted with caution , given the substantial uncertainties identified above. Analysis of TRI data should be performed within a more general theoretical framework [16,17], rather than using earlier methods. Most importantly, incidence surveillance should not currently rely on any single methodology, but make use of multiple methods for estimating incidence , such as interpretation of prevalence trends and epidemiological and demographic modelling [3,38].
The search for robust means of estimating incidence from cross-sectional surveys is at a crucial juncture. Although an imperfect TRI can be used to estimate HIV incidence reliably, the reliance on having accurate and precise values of two key aspects of TRI performance (ω and ε) can undermine the use of this technology. The effect of ω and ε on statistical power is shown in Figure 5. While larger values of ω provide sufficient numbers of TRI-positive individuals to ensure statistical power, ω should not be so large that the estimated incidence is not representative of the recent past. On this basis, a value of approximately six months to a year is desirable. It is also essential that ε be small (progress in this regard is being made, for instance using TRIs consisting of an assay in combination with clinical information ). Ideally, to ensure that the fraction of misclassifications is independent of time and epidemic state, inter-individual variability in TRI progression should be unrelated to survival outcomes, and there should be no regression to the TRI-positive state. These form the core requirements for the development of new TRI assays and algorithms used to estimate incidence.
In the next phase of TRI development, it will be essential to be guided by these insights into the key determinants of test performance, and to focus on characterising the performance of the test within a systematic framework.
A Welte and TA McWalter thank the Canadian International Development Agency (CIDA) for funding support. TB Hallett thanks The Wellcome Trust for funding support. This research was partly supported by the Division of Intramural Research, National Institute of Allergy and Infectious Diseases (NIAID), United States National Institutes for Health (NIH). In addition this research was sponsored in part by The HIV Prevention Trials Network (HPTN) sponsored by NIAID, National Institute on Drug Abuse (NIDA), National Institute of Mental Health (NIMH), and Office of AIDS Research, of the NIH, United States Department of Health and Human Services (U01-AI-068613).