Few published studies evaluating surveillance systems presented timeliness measures. When timeliness was evaluated, standard methods were not used. Information collected by public health surveillance systems should support the quantitative assessment of timeliness by various steps in the pubic health surveillance process. Public health programs should periodically assess timeliness of specific steps in the surveillance system process to ensure that the objectives of the surveillance system are being met. A more structured approach to describing timeliness studies should be considered.
Published papers describing local or state surveillance system reporting timeliness generally do not explicitly describe the surveillance system processes contributing to the timeliness measure, such as processing and analyzing the data or implementing a public health action before data are reported from a state to CDC. To facilitate future comparisons of reporting timeliness across jurisdictions, studies should include an explicit description of the public health surveillance reporting process and the surveillance process interval being measured. Additionally, surveillance information systems must support the collection of appropriate reference dates to allow the assessment of the timeliness of specific surveillance processes.
A more structured approach to describing timeliness studies could include a description of the following characteristics: 1) the level of the public health system being assessed (e.g., local, state, or national), 2) the purpose of the surveillance evaluation, 3) goals of the surveillance system, 4) the surveillance interval being measured and a description of the reference dates that define the upper and lower boundaries of the surveillance interval, 5) the surveillance steps (processes or activities) that contribute to the surveillance interval being measured, 6) whether the measured timeliness met the needs of the surveillance step being evaluated, and 7) whether the timeliness met the goals of the surveillance system. No single timeliness measure will achieve the purpose of all evaluations or meet all the goals of the surveillance system. In addition, if the goal of the surveillance evaluation is to identify ways to improve timeliness, the analysis should identify factors associated with delayed reporting, such as the role of specific case ascertainment sources.
The 1999–2001 national notifiable diseases data were timely enough to support the following surveillance objectives: monitoring trends over time, informing allocation of public health resources, monitoring the effectiveness of disease control, identifying high risk populations, and testing hypotheses. If NNDSS data are to be used to support timely identification of and response to multistate outbreaks at the national level, the timeliness of reporting needs to be enhanced for all diseases, but especially for diseases with the shortest incubation periods (e.g., cryptosporidiosis, E. coli
O157:H7, meningococcal disease, salmonellosis, and shigellosis). Until reporting timeliness is enhanced, the application of aberration detection analytic methods to NNDSS data to aid in the identification of changes in disease reporting that may indicate a multistate outbreak in time to alert states for the purposes of disease control and prevention may be of limited use. Future work to improve reporting timeliness will need to address the substantial variation across states. As states enhance their reporting mechanisms with the use of automated electronic laboratory reporting systems [18
], there may be less variation in state-specific reporting timeliness, but this should be assessed.
NNDSS timeliness improved compared to timeliness of notifiable infectious diseases measured in previous reports [11
]. However, the methods or variables used in these analyses were different. A few factors may have contributed to improvements in timeliness seen in this study. Since 1992, states have been routinely transmitting electronic case-specific records intended to improve reporting procedures and protocols. In addition, the use of automated electronic laboratory reporting to enhance infectious disease case reporting may have contributed to increased timeliness.
Our study findings are subject to several limitations. The variables available for assessing NNDSS reporting timeliness are based on the MMWR week numbers that are assigned by states and the earliest known date reported in association with the case. While these variables might provide an estimate of national reporting timeliness, NNDSS data do not include a fixed date defining when a case report was initially transmitted to CDC or received at CDC, which would provide a more precise measure of national reporting timeliness. NNDSS data management protocols should be modified to permit direct calculation of national reporting timeliness. If the ability to support outbreak detection at the national level using NNDSS data is generally viewed as an important and sustainable enhancement for the NNDSS, states and CDC programs should facilitate reporting that more closely approximates real-time and define reporting protocols and data requirements to ensure that reporting timeliness can be improved and accurately monitored. The current NNDSS practice of weekly reporting and data processing limits reporting timeliness to CDC. Lastly, 72,293 (26.4%) cases were excluded from our analysis because the information contained in the database would not permit calculation of timeliness and this exclusion may have resulted in our study results either falsely overestimating or underestimating the magnitude of NNDSS reporting lags.
The reporting timeliness variations across states may result from different reporting protocols in the states (e.g., centralized versus distributed reporting within the state's public health system) or from variations in how states assign MMWR week numbers. Other factors that might have contributed to reporting delay in our study included: the patient's recognition of symptoms; the patient's acquisition of medical care; the use of confirmatory laboratory testing; reporting by the health care provider or the laboratory to the local, county, or state public health authority; the volume of cases identified in the state; case follow-up investigations to verify the case report or to collect additional case information; periods of decreased surveillance system activity due to variable staffing levels; computer system down-time for maintenance, upgrades, or new application development; and data processing routines, such as data validation or error checking. Following a structured approach to evaluation of timeliness by specifying the surveillance objectives and the process(es) being measured may allow better definition of the factors that contribute to reporting delay. It was beyond the scope of this study to assess how these factors contribute to NNDSS reporting timeliness.
In addition to reporting timeliness, other surveillance system attributes are important to assess (e.g., completeness of reporting). Completeness of notifiable infectious diseases reporting in the United States varies from 9% to 99% [7
]. Six of the eight papers reviewed for this study assessed completeness of reporting [12
]. One paper [14
] noted that although the timeliness of the AIDS passive and active surveillance systems were comparable, the completeness of the active AIDS reporting system far exceeded the reporting completeness for the passive system. This highlights the importance of evaluating completeness and timeliness and other surveillance system attributes concurrently, before contemplating any changes to a surveillance system based on the assessment of a single attribute.
To improve public health surveillance infrastructure and performance in the United States, CDC and local and state health agencies are integrating a number of public health surveillance systems monitoring infectious diseases in the United States, including the NNDSS, into the National Electronic Disease Surveillance System (NEDSS) [19
]. NEDSS outlines a standards-based approach to disease surveillance and intends to connect public health surveillance to the clinical information systems infrastructure. As a result, NEDSS promises to improve the accuracy, completeness, and timeliness of disease reporting to state and local health departments and CDC.