|Home | About | Journals | Submit | Contact Us | Français|
Disease Intervention Specialists (DIS) in North Carolina (NC) have less time to conduct partner notification due to competing responsibilities while simultaneously facing increased case loads due to increased HIV testing. We developed a model to predict undiagnosed HIV infection in sexual partners to prioritize DIS interviews.
We abstracted demographic, behavioral, and partnership data from DIS records of HIV-infected persons reported in two NC surveillance regions between January 1, 2003 and December 31, 2007. Multiple logistic regression with generalized estimating equations was used to develop a predictive model and risk scores among newly diagnosed persons and their partners. Sensitivities and specificities of the risk scores at different cutoffs were used to examine algorithm performance.
Five factors predicted a partnership between a person with newly diagnosed HIV infection and an undiagnosed partner—four weeks or fewer between HIV diagnosis and DIS interview, no history of crack use, no anonymous sex, fewer total sexual partners reported to DIS, and sexual partnerships between an older index case and younger partner. Using this model, DIS could choose an appropriate cutoff for locating a particular partner by determining the weight of false negatives relative to false positives.
While the overall predictive power of the model is low, it is possible to reduce the number of partners that need to be located and interviewed while maintaining high sensitivity. If DIS continue to pursue all partners, the model would be useful in identifying partners in which to invest more resources for locating.
Partner notification is an established public health effort to control the transmission of sexually transmitted infections (STI), including human immunodeficiency virus (HIV) infection. When used in a population with a high prevalence of HIV, partner notification leads to identification of HIV-infected persons that might otherwise not have been tested.1
However, the effectiveness of partner notification programs is limited by the cost and labor associated with locating and interviewing partners.2
In North Carolina (NC), disease intervention specialists (DIS) conduct partner notification for both HIV and syphilis. Currently, 48 DIS are available to locate ~2,000 newly identified HIV cases and ~600 early syphilis cases in the state per year as well as their named sexual and drug sharing partners. HIV testing efforts in the state have increased recently in an attempt to identify undiagnosed infection, thus increasing the demand for partner notification.3
DIS are also used for assignments outside their standard scope of work, such as community awareness campaigns and public health research, leaving less time for their traditional partner notification duties.4 In the current economic environment, public health departments are facing budget cuts and hiring freezes, making it unlikely that more DIS will be hired to fulfill these responsibilities.5 If DIS are unable to trace all named partners in the future, identifying those partners most likely to be HIV-infected would be a potentially effective strategy.
In 2008, the Centers for Disease Control and Prevention (CDC) released updated partner notification guidelines that emphasize the need for setting-specific, evidence-based partner services programs.6 A predictive model to prioritize partner follow-up of named partners might improve DIS efficiency. While predictive models have not been utilized by partner services programs specifically, they have been shown to successfully increase efficiency and cost-effectiveness of STD case finding in the past.7–14 Using characteristics of index cases and named partners from DIS records, we developed and evaluated risk scores to predict undiagnosed HIV infection in named sexual partners of newly diagnosed HIV-infected persons in NC.
NC is divided into seven HIV and STI surveillance regions. We reviewed the Sexually Transmitted Disease Management Information System (STD*MIS) database from two of these regions (Winston-Salem and Raleigh regional offices) to identify persons in whom HIV was diagnosed between January 1, 2003 and December 31, 2007. These two regions include 27 of NC’s 100 counties and encompass approximately 40% of the state’s incident HIV cases. DIS maintain a chart for each index case at the regional office that contains the STD*MIS entry and their notes on the interviews with the index and partners. Demographic, sexual behavior, and sexual partner data were abstracted using a standard form and entered into an Access database. Cases were not abstracted if they were aged 10 years or younger, attributable to mother-to-child transmission, or reported no sexual history. Cases were excluded from analysis if they were unable to be located or refused the DIS interview. Sexual partners named by the index cases were excluded from analysis if they were previously diagnosed with HIV. The unit of analysis was an index-partner pair.
The outcome was newly diagnosed HIV infection in a sexual partner. DIS check NC surveillance databases for previous HIV diagnoses. If an HIV-infected partner is not found in a NC surveillance database, they are considered a new HIV diagnosis in NC. All DIS are trained to identify new HIV diagnoses in this manner. The set of possible predictors included demographic characteristics and risk behaviors of the index case, demographic characteristics of the named partner reported by the index case, and characteristics of the partnership reported by the index case.
DIS in NC have a special protocol for follow-up of index cases with acute HIV infection (HIV antibody-negative, RNA-positive cases), giving these cases the highest priority for interviewing and follow-up of partners. Partners of acute index cases were considered to be definite notifications and were removed from the model building process. The algorithm for prioritizing partner interviews is therefore a two-step process (Figure 1). If the index case of the partnership is acutely infected, all partners would be notified of their potential HIV exposure. If not, the risk score cutoff chosen for the model would determine if a particular partner should be notified. In order to evaluate the performance of the algorithm, not just the model, we included the partnerships with acute index cases as definite notifications in the assessment of algorithm performance.
Generalized estimating equations were used to address the lack of independence between index case-partner pairs for persons with multiple partners. We examined the association between each predictor and the outcome using unadjusted prevalence odds ratios with associated 95% confidence intervals. We assessed the association between each pair of candidate predictors to avoid collinearity. For dichotomous variables, two variables were considered collinear if the odds ratio was 3 or greater. If one variable was continuous and the other was categorical, we examined the magnitude of the difference in means in standardized units. A difference of more than 1.5 standard deviations was considered a strong association. Collinear variables were either recoded or one of the variables was selected based on the relationship to other variables. Variables for which P<.25 in the bivariable analyses were selected for inclusion in the multiple unconditional logistic regression model.15 We assessed interaction terms between all candidate predictors included in the model and retained interaction terms with P values <.25. This model was considered the full model.
We examined reduced models to see if they had adequate model fit without loss of predictive power. Modeling proceeded in a backward elimination process to eliminate predictors with weak predictive power, starting with interaction terms and then proceeding with the variable with the highest P value. If the Wald P value comparing two models was less than .10, the variable was retained in the model. Changes in the area under the receiver operating characteristic (ROC) curve were used to assess variations in model performance due to collapsing across categories or removing variables. Changes in the area under the ROC curve less than 0.01 were considered acceptable. Model fit was evaluated using the Hosmer-Lemeshow test. Modeling procedures were limited to those persons with complete data for all variables in the model.
We created risk scores using the β-coefficients corresponding to each predictor in the final model. β-coefficients were summed to create an overall risk score for each patient. We used 1,000 bootstrap samples with replacement to validate our model and risk score performance. Consistent performance was defined as tight confidence intervals around the sensitivity and specificity associated with each cutpoint.
To identify an ‘optimal’ strategy for prioritizing interviews, we examined the number of misclassification errors that would be made depending on the cutpoint used for additive risk score totals (i.e., over a certain cutpoint, an index case would be located and interviewed). A false positive (FP) was defined as interviewing a partner who turns out to be HIV-uninfected, whereas a false negative (FN) was defined as failing to interview a partner with undiagnosed HIV.
A FN was weighted more than a FP since it would be worse to miss an undiagnosed HIV-infected partner than to locate and test a partner that was HIV-uninfected. The following calculations were made to determine the number of errors associated with the sensitivity and specificity of the model at different risk score cutpoints:
where weight reflects the relative value of a FN compared to a FP and N is the population size. We used a hypothetical population of 1,000 to calculate the number of FNs and FPs. The weights provided in the analysis are arbitrary and are provided as examples of how an ideal cutpoint for the risk score is chosen. Currently, because DIS pursue all partners, a FN is weighted infinitely more than a FP, so we chose high weights as examples. All analyses were conducted using SAS version 9.2 (Cary, NC). The University of North Carolina Institutional Review Board approved all study procedures.
A total of 3,880 index cases from the two surveillance regions were diagnosed with HIV infection and recorded in STD*MIS between January 1, 2003 and December 31, 2007 (Figure 2). DIS interviewed 81.3% of eligible cases. Cases not interviewed were either unable to be located or refused DIS interview. Over half of these cases (61%) reported one or more partners to DIS for follow-up. Almost one-third of the partnerships (31.1%) involved a previously known HIV-infected partner, leaving 2,232 index-partner pairs for analysis. Approximately 42% of these pairs involved a partner that was unable to be located or refused testing.
Overall, 171 index-partner pairs (7.7%) had a partner that was newly diagnosed with HIV. DIS interviewed 18.3 index cases to identify one partner newly diagnosed with HIV. After restricting to complete cases, there were 164 newly diagnosed partners among 2,100 total partners pursued. Most of the index cases in the index-partner pairs were male (68.3%; 21.4% MSW and 46.9% MSM and MSM/W) and black (66.0%) (Table 1). They were also in the chronic stage of HIV infection (78.3%), with only 6.1% of cases acutely infected with HIV and 15.6% identified as AIDS cases (CD4 count or percent < 200 cells/μL or 14%, respectively, or diagnosis with an AIDS-defining illness). The median age of the index cases in the pairs was 33 years (range: 15–68 years). The partner was younger than the index case in 41.0% of the index-partner pairs, and 45.1% were same gender partnerships.
Reporting only one partner total in the past year to DIS compared to reporting four or more partners was the predictor most strongly associated with a newly diagnosed HIV-infected partner (odds ratio (OR) 2.7, 95% confidence interval (CI): 1.6, 4.4) (Table 1). Although not statistically significant, index cases with acute HIV infection were less likely to have a newly diagnosed HIV-infected partner compared to those with chronic HIV infection (OR 0.4, 95% CI: 0.1, 1.1). Other potentially important predictors of a newly diagnosed HIV-infected partner (P < .05 in bivariate analyses) were no history of crack use ever, no anonymous sex ever, exchanging sex for drugs or money ever, fewer than 4 weeks between time of HIV diagnosis and DIS interview, and having a younger partner. Hispanic ethnicity, having immigrated to the US, no incarceration history ever, HIV diagnosis at a community health center or health department, having a bisexual sex partner ever, heterosexual partnerships, and same race partnerships were also candidate predictors (.05< P<.25 in bivariate analyses) for the reference model.
Stage of infection was not a candidate predictor in the reference model since all acutely infected index cases are prioritized in the partner notification algorithm (Figure 1). Non-Hispanic ethnicity, being a native of the US, history of incarceration, and exchanging sex for drugs or money were highly correlated with crack use. Exchanging sex for drugs or money was also highly correlated with anonymous sex, as was same gender partnership. Therefore, these variables were excluded.
The reference model included time between HIV diagnosis and DIS interview, diagnosis location, history of crack use, history of anonymous sex, bisexual sex partner, number of partners reported to DIS, age difference between partners, and same race partnership. The relationship between crack use and undiagnosed HIV infection varied by the age difference between the index case and partner, so an interaction term between these variables was included (Table 2). The area under the ROC curve was 0.666 (95% CI: 0.619, 0.712).
After model simplification, the final model included six terms—time between HIV diagnosis and DIS interview, crack use, anonymous sex, number of sex partners pursued, age difference between partners, and the interaction between crack use and age difference (Table 2). The area under the ROC curve was 0.662 (95% CI: 0.619, 0.704).
The risk score for a partnership is equal to the sum of the predictors’ β-coefficients. Risk scores ranged from zero to 3.46 for an index case that was interviewed within four weeks of diagnosis (+0.55), had no history of crack use and a younger partner (1.49), had no history of anonymous sex (+0.56), and reported one partnership to DIS (0.86) in the past year (Table 2).
The overall predictive power of the model was low, as indicated by the low value for the area under the ROC curve. In order to maintain a high sensitivity, only relatively small reductions in partners pursued can be made. Using a lower risk score cutpoint (e.g., 1.00 or 1.50) entails interviewing a larger proportion of partners. Consequently, more partners who actually have undiagnosed HIV infection would be interviewed and tested, resulting in fewer false negatives. Interviewing all partners, as currently practiced, corresponds to a cutpoint of 0, with sensitivity = 100% and specificity = 0%. Using a cutpoint of 1.50 for this model, DIS would identify 95.7% of undiagnosed HIV infection in partners while reducing the number of partners pursued by 15% (Table 3).
If FNs are weighted 15 times worse than FPs, the ideal cutpoint in terms of minimizing total number of errors for the model with partnership data is a risk score of 2.00 (Table 3). Interviewing all partners at or above 2.00 has a sensitivity of 90.2% and reduces the number of partners DIS would need to locate and interview by 26%. Increasing the tradeoff weight to 30 decreases the ideal cutpoint to 1.50. Validation of the model demonstrated consistent performance over 1,000 replications.
Using demographic and behavioral data collected from DIS interviews of HIV index cases, we developed a risk score algorithm to predict undiagnosed HIV infection in named sexual partners. We identified five factors that predict a partnership with an undiagnosed partner—four weeks or fewer between HIV diagnosis and DIS interview, no history of crack use, no report of anonymous sex, fewer sexual partners reported to DIS, and sexual partnerships between an older index case and younger partner. While overall performance of the model is low with poor specificity, it is possible to reduce the number of partners that need to be located and interviewed by up to 25% while maintaining sensitivity above 90%.
In deciding to use this algorithm to reduce DIS workloads, authorities would need to decide the relative value of a FN compared to a FP. Currently, in pursuing all partners, a FN is considered infinitely worse than a FP. In order to reduce the number of partners pursued, the tradeoff between FNs and FPs must be quantified by weighing the potential public health and monetary costs of failing to diagnose an HIV infection with the monetary costs of hiring more DIS. DIS currently follow up on all partners regardless of whether or not the partner was first notified of their potential exposure by the index case or DIS. This affects DIS time spent on the partner and will therefore also need to be considered in determining the weight.
Alternatively, if DIS continue to pursue all partners, the model could be helpful in prioritizing partners in which to invest more time for locating. Currently DIS must complete an extensive checklist of locating tactics (e.g., searching the Department of Corrections database or checking for a social networking account) before declaring that a person is unable to be located. If the algorithm indicated that a partner should not be prioritized, the locating checklist could be modified so that not all tactics are attempted on this person, particularly those that are the most time consuming (e.g., driving to the person’s listed address and asking neighbors for additional locating information). If implemented, DIS could be given an Excel spreadsheet where they would enter 1’s and 0’s corresponding to the characteristics of the partnership to calculate the risk score. The spreadsheet would also include instructions as to how to interpret the risk score.
We were unfortunately unable to collect data on partner notification costs in these two regions and are therefore unable to calculate the actual weight of a FN compared to a FP or to demonstrate the cost effectiveness of using a predictive model in this capacity. The weights of a FN compared to a FP presented in this analysis are used as examples of how an ideal risk score cutpoint is chosen. However, use of this model to prioritize partner interviews could ensure that the most undiagnosed HIV infections are identified in a timely manner given the available level of resources available. Many health departments in the US currently provide inconsistent partner notification for HIV due to limited resources16 and may benefit from prioritizing particular cases.
Our analysis uses data from two regions of NC and may not be generalizable to the other field service regions in the state or to other states due to varying prevalence of risk factors in different regions. The model also assumes a fixed epidemic and would need to be re-evaluated if partner positivity according to risk factor changed over time. However, the age and racial distributions of newly diagnosed persons in these two regions are similar to those for NC as a whole. The NC HIV epidemic has also remained fairly stable over the past eight years with respect to number of new cases and risk profile of newly infected cases.17
Several factors may contribute to the relatively poor performance of the model and the limited reduction in number of partners interviewed. The strongest predictors for having an undiagnosed HIV-infected partner, such as type of sex and condom use, were undocumented. The odds of HIV transmission during receptive anal intercourse are much higher than the odds of transmission during insertive anal sex or vaginal sex.18, 19 Therefore, the inclusion of type of sex would likely improve the model’s predictive power. Additionally, when DIS identify a newly diagnosed partner, the potential transmission dynamics are difficult to determine. The partner may have infected the index case, the index case may have infected the partner, or both may have been infected through other exposures. Because the timing and directionally of infection is unknown, the partnerships reflect a mixture of transmission events. Transmission events to the index case could have different predictors that are diluting the potential predictors of transmission events to the partner.
Finally, our algorithm only addresses one objective of partner notification—identifying undiagnosed HIV infection in sexual partners. In order to reduce HIV transmission in a population, it is important to first identify persons with HIV infection and then to make sure they receive care, including possible antiretroviral treatment, and education to maintain safe sexual behavior. We unfortunately do not have data on linkage to care, maintenance in care, or antiretroviral use to evaluate the overall utility of partner notification in this population.
While it may seem counterintuitive that several of our model predictors are considered lower HIV risk behaviors, this may be explained by the amount of locating information those persons with lower risk profiles were able to provide DIS. Index cases that reported anonymous sex or crack use and named more sex partners were more likely to report partners that could not be located or refused testing compared to those of a lower risk profile (data not shown). Although we do not have the data to show this, persons reporting only one partner to DIS may also have been in partnerships of longer duration that resulted in more unprotected sexual acts and therefore increased transmission probability compared to persons who reported multiple partners. An alternative modeling approach is to count the index only once, then seek all partners or none, depending on the characteristics of the index. We did a sensitivity analysis looking at the model this way, and found that an index case-only model had reduced sensitivity at all cutpoints compared to the partnership model presented.
Our other model predictors are consistent with predictors of HIV infection identified in other studies. Persons reporting sex with an older partner were more likely to be HIV-infected compared to persons with partners their same age or younger in previous studies.20–22 Our finding that partnerships with index cases interviewed four weeks or fewer after their HIV diagnosis predict undiagnosed HIV infections in partners is also consistent with previous data.23, 24 Decreased time between diagnosis and patient interview increases the number of interviews yielding locatable contacts and therefore the number of partners notified and tested.
As resources available for partner notification decrease and HIV testing and case detection increase, public health departments are in need of novel strategies to maximize the efficiency of partner notification. Using data available from DIS interviews in two surveillance regions of NC, we demonstrate that it is possible to develop a model to predict undiagnosed HIV infection in partners, albeit with less accuracy than desired. Implementation of the model would allow DIS to prioritize partner interviews when all partners cannot be pursued and would allow DIS to reduce the number of partner interviews with high sensitivity for identifying undiagnosed HIV infection. Predictive models with additional partnership data including types and number of sex acts could potentially improve performance and should be explored as evidence-based approaches to improving partner notification.
Funding Support: Brooke Hoots’ work was supported by National Institute of Allergy and Infectious Diseases (NIAID) grant 5T32AI070114-05.
We would like to thank the North Carolina Communicable Disease Branch (Evelyn Foust, Michael Hilton, and Del Williams) and the North Carolina HIV/STD Field Services Unit (Rhonda Ashby, Maxi Mackalo, Kristen Patterson, Todd Vanhoy, and the disease intervention specialists of Regions 3 and 4) for collecting and giving us access to the study data.
Potential Conflicts of Interest: None declared.