We evaluated four key attributes of the NYC HIV incidence surveillance system—simplicity, data quality, timeliness, and acceptability—using CDC evaluation guidelines. The evaluation revealed that the incidence surveillance system can potentially provide an HIV incidence estimate stratified by borough and major demographic groups at approximately nine months after the period of interest. The strengths of the system include its relative simplicity and integration with routine HIV/AIDS surveillance. The primary weakness of the system is the incomplete reporting of TTH, a critical component of incidence estimation. Improvements in data completeness have been made during the time of this evaluation, and we expect that further improvements in data completeness and timeliness will improve the incidence information available to public health professionals and policy makers in NYC and nationally.
With respect to personnel required at NYC DOHMH, incidence surveillance is a relatively simple system, integrated within routine HIV surveillance and requiring only six additional staff for major tasks (e.g., salvaging remnant serum, shipping samples for BED testing, and calculating incidence). Obtaining TTH requires additional routine HIV/AIDS case surveillance staffing, which is provided for in the HIV incidence grant. Specimen salvage is the least simple component of incidence surveillance, requiring substantial time and effort on the part of incidence surveillance staff to coordinate with commercial laboratories. This issue might be attributable in part to the absence of a predetermined schedule for specimen shipment at most commercial laboratories, despite longstanding state requirements for submission of positive specimens for other communicable diseases. Although the process only requires about eight staff hours/month, clinical laboratories still assign two to three staff for the task. Some laboratory managers indicated lack of familiarity with packaging and shipping processes, despite being provided written instructions by incidence surveillance regarding these tasks.
Nevertheless, the NYC incidence surveillance unit achieved a high completion rate for BED testing—77%—among the highest of all the participating jurisdictions throughout the U.S. Since the beginning of this evaluation, substantial improvements in salvage completeness in NYC have been achieved by sending formal letters of noncompliance to laboratory personnel at the management level, bringing high-level attention to the issue, and improving salvage rates immediately. Incidence staff continue to evaluate whether these improvements have increased completion rates to the 85% criteria recommended by CDC (Unpublished data. Technical guidance for HIV/AIDS surveillance programs, volume 1. CDC, Division of HIV/AIDS Prevention, 2005).
Completion rates for TTH were lower compared with BED testing. Only 39% of new diagnoses have this information, which does not achieve CDC's recommended 85%. Local decreased rates of TTH are attributable to decreased rates of HIV provider case-report form completion. Noncompliance with the HIV report form likely reflects underreporting, a frequent problem for public health surveillance systems,24–27
rather than a specific problem related to routine HIV or incidence surveillance. Routine surveillance personnel regularly conduct active surveillance for these forms, including routine visits to providers and requesting report forms of newly reported cases, yet compliance remains limited.
According to our finding that 90% of BED results on salvaged specimens are received within nine months of diagnosis and 87% of TTH on provider report forms containing this information arrive within six months of diagnosis, incidence surveillance personnel can expect to calculate incidence approximately one year after the last calendar month of the year for which an estimate is sought. Calculating incidence earlier with fewer data is possible, but reduced precision and potentially biased estimates are expected. Proportionately, the majority of this time is related to establishing STARHS eligibility by routine HIV surveillance.
No substantial problems with acceptability were indicated by key personnel at CDC, NYC DOHMH, or NYSDOH. However, commercial clinical laboratorians indicated that packaging and shipping processes were not fully satisfactory. This issue might be an internal problem to individual laboratories, but one that is worth addressing more closely by DOHMH and CDC. Underreporting of testing and TTH information indicates that provider report form acceptability varies among providers.
In its current form, incidence surveillance in NYC has been useful for contributing to the national estimate of HIV incidence21
and for providing NYC with an incidence estimate that can potentially be stratified by sex, racial/ethnic categories (e.g., black, Hispanic, and white), certain age categories, certain risk factors, and four of the five NYC boroughs.28,29
Detailed examination of infection rates (e.g., comparisons of demographic groups by borough or such smaller geographic areas as zip codes) is not yet possible, and rates cannot be calculated for populations that are not enumerated (e.g., men who have sex with men and IDUs). We do not know how the inadequate provider report form completion rates that translated into missing TTH might affect accuracy and precision of the incidence estimate. Until a more thorough evaluation of the impact of this data issue is conducted, incidence estimates should be interpreted with caution, especially when making future comparisons between years or jurisdictions.
Recent studies have shown that pooled-ribonucleic acid testing for acute HIV infection can be used for incidence surveillance.30
NYC DOHMH recently began pooled nucleic acid amplification test (p-NAAT) screening to routine HIV testing in city-run sexually transmitted disease clinics.31
Currently, pooled p-NAAT screening is only routinely conducted in this setting, making citywide incidence estimates with these results not possible. Further, one important challenge of this approach is that all HIV-negative specimens must be tested, rather than retesting HIV-positive specimens as in BED testing. In many settings currently, because of the widespread implementation of oral fluid and blood fingerstick-based rapid testing, no blood sample is taken on people with a negative rapid HIV test. This is one operational issue that makes the widespread implementation of this approach for incidence estimation more challenging than the current BED-based approach.
This evaluation had several limitations. It provided no information on sensitivity or positive predictive value of incidence surveillance, primarily because we did not have an external measure of the true frequency of recent infections in the population. We did not survey providers to assess barriers to reporting TTH; a survey might have provided useful information on ways to improve reporting. We also did not survey nonmanagerial clinical personnel at commercial laboratories, which might have provided additional insights into improving specimen salvage as well as overall acceptability of the system.
An approach to improving disease reporting by providers might be to make them more aware of both the public health benefit and their legal obligation to report, which in a previous study has been shown to improve reporting rates.25
Both, however, are stated on the report form. Another approach that has been successful in NYC is asking patients directly for this information during patient interviews. NYC has been expanding the number of newly diagnosed patients interviewed for partner notification purposes, and 92.7% of the completed interviews result in obtainment of TTH. Further expansion of such interviews from the 20% of newly diagnosed cases will likely improve overall TTH ascertainment.
In the interim, incidence surveillance should carefully characterize the demographic characteristics of diagnoses with complete information because imputation of TTH and BED values among the missing or untested population are based on these characteristics. Incidence surveillance should also ensure that time trends in incidence estimates and differences between local estimates and national or other jurisdictions do not result from missing testing and treatment information.
Specimen salvage, BED testing completion, and timeliness.
As a result of this evaluation, DOHMH personnel made clinical laboratorians more aware of their legal obligation to submit remnant samples, which has improved salvage rates. We also recommend that DOHMH and CDC work more closely with clinical laboratorians to improve their standardized protocol for remnant specimen packaging and shipping. Changes to routine surveillance are necessary to improve timeliness because the majority of time between diagnosis and receipt of BED test results is related to investigating new diagnoses and establishing eligibility for STARHS. Evaluation of routine surveillance should be conducted and is planned, but was beyond the scope of this evaluation.
Another possibility is to allow clinical laboratories to perform the BED test themselves, perhaps simultaneously with WB testing at the clinical laboratory, thereby eliminating the need to ship specimens and improving timeliness. This will require FDA approval of BED testing for clinical use, which reportedly has not occurred because of concerns regarding the validity of results on an individual level.5,21