In this study, prospective surveillance of hospitalization data was simulated using retrospective data, to evaluate whether syndromic surveillance can effectively detect local outbreaks of lower-respiratory infections (LRIs). Over 1999–2006 (400 weeks), 35 space-time LRI-clusters were detected by weekly analysis, with a total of 221 generated cluster-signals. This represents an average rate of approximately 5 new clusters per year, or 3 per year using a threshold recurrence interval ≥5 years. The number of clusters detected per year differed over the study period, reflecting substantial annual variation in influenza epidemics.
Two clusters were related to the Legionnaires' disease “test-case” outbreaks and would have been detected around the same time as the outbreaks were actually detected. This indicates that syndromic surveillance will pick up similar outbreaks of severe respiratory disease in a timely manner. Note that the Legionnaires' disease outbreaks are used here as “positive controls” (or Gold Standard) for realistic severe respiratory outbreaks by uncommon pathogens that may not be (timely) detected by traditional surveillance, such as the Dutch Q-fever outbreak in 2007, for which the initial diagnoses were delayed by several weeks 
. As 17 out of the total 35 LRI clusters probably reflected local RSV and/or influenza activity, many signal investigations could be limited to checking their concurrence with local RSV and/or influenza activity. The 3 clusters with “unknown cause”, that concur with local ILI-elevations outside the influenza season, possibly represent very early local influenza activity or local activity of another respiratory pathogen reflected in both GP-ILI-data and hospital LRI-data. For these 3 clusters and the other 8 clusters for which no likely cause was defined, it would have been interesting to investigate possible causes in a truly prospective setting (e.g., by additional diagnostics). Some of these clusters possibly represent unreported and/or undetected local LRI-outbreaks.
As a threshold value for the significance of cluster signals, we used a threshold of recurrence intervals ≥1 year, and only evaluated the LRI clusters that were above this threshold. To illustrate the impact of changing the threshold we repeated the analyses for recurrence intervals ≥5 years. At both threshold levels, two LRI clusters showed a higher proportion of Legionnaires' disease cases (p:<0.0001, see also results
section) overlapping with the known outbreak areas, which made us conclude that these LRI clusters indeed detected the Legionnaires' disease outbreaks.
The results of the sensitivity analysis show that the test outbreaks are still detected with the restricted time and spatial windows (at both threshold levels), without loss of timeliness and with less signals generated in time. To limit the computation time we only performed a modest sensitivity analysis. In this study, the restrictions on the time window almost halved the number of signals (42% decline), whereas the clusters in time to investigate declined much less (14% decline). The spatial restrictions resulted in less decline in generated signals (25% decline in signals and 6% decline in clusters). This indicates that with little loss of sensitivity, the restricted time window would be most appropriate to limit the number of generated signals.
To our knowledge this is the first study that evaluates the performance of syndromic surveillance with nationwide high coverage data (80–99% of hospitalizations) over a longer period (8 years) with all detected clusters analyzed and (if possible) explained in a systematic way. Feasibility of localized outbreak detection is demonstrated without swamping true signals by excessive false alarms. Some other studies evaluating the performance of space-time syndromic surveillance have concluded differently, but these studies were based on shorter periods, had lower coverage or lacked comparable outbreaks which could be tested 
. Cooper et al. tracked the spatial diffusion of influenza and norovirus, using space-time analysis on syndromic data from a telephone help line system in the UK, but did not test space-time detection for more localized outbreaks 
. Using syndromic surveillance for detection of local gastro-intestinal outbreaks in New York City, Balter et al. found numerous cluster-signals in time, but these could not be used for effective surveillance because of insufficient comparable diagnostic data 
. Respiratory disease outbreaks could not be evaluated in the NYC study, because no local respiratory outbreaks had been reported in the study period. Nordin et al. used simulated anthrax attack data injected in true physician's visit data to confirm that a respiratory outbreak initiated by bioterrorism will be detected in a timely manner by syndromic surveillance 
. However, no results on the number of possibly false alarms were presented. These studies present space-time cluster detection analyses over relatively few years and are therefore prone to miss the effects of annual variation. Furthermore, sensitivity for local outbreaks is reduced by using data with relatively low coverage levels. For such data sources with low coverage, methods other than space-time scan statistics seem more appropriate to generate useful information for public health practice (like aberration detection in time).
We performed weekly analyses (instead of daily) over the whole study period, because these analyses consume considerable computation time. Daily analyses in 1999 and 2006 detected fewer clusters than weekly analyses because the threshold level for recurrence intervals (≥1 year) is more strict (see Appendix S2
). Daily analyses would therefore probably not detect more epidemiological events but would yield more timely signals.
Hospital based syndromic surveillance could be a helpful tool in detecting local LRI-outbreaks, complementing outbreak detection by laboratory surveillance or astute clinicians. Syndromic surveillance might be most valuable for outbreaks due to uncommon or novel pathogens (like the SARS outbreak), as these seem more likely to be missed by the laboratory and clinicians. Furthermore, outbreaks due to more common pathogens could also be missed, as for community acquired pneumonia often no causative pathogen is detected 
. Apart from that, under-notification can complicate outbreak detection through laboratories and clinicians 
A prerequisite for prospective syndrome surveillance is the real-time availability of hospitalization data, including clinical diagnoses and symptoms by date of hospitalization. Although at present not available in the Netherlands, such real-time syndromic data collection may become feasible after the nationwide implementation of electronic health-care information exchange. In this light, the results of our study justify further development of these methods, including retrospective evaluation of other types of documented health events than the ones presented in our study.
Besides that, further research should focus on prospective application of these methods. In a prospective setting, sustaining reliable data with high coverage and few data artifacts might be more challenging, thus possibly leading to higher numbers of false alarms. In addition, it should be evaluated to what extent 3 to 5 new syndromic clusters per year would indeed be manageable in a prospective setting. Responding to such clusters is complicated, because the cause and thus possible threat will initially often be unknown. For each new cluster, it should first be verified whether plausible explanations can be found in epidemiological or laboratory data. For example, LRI clusters need to be interpreted in relation to local influenza or RSV activity similar as we did in our study, and provided the age distribution of cases reflects the usual pattern, further investigation would seem unnecessary. Internet-based ILI-monitoring 
combined with virological self -sampling (at home) 
could increase the microbiological base for interpreting syndromic surveillance data. Age stratified syndromic surveillance with a multivariate space-time scan statistic 
may further facilitate quick interpretation of clusters by revealing the affected age groups.
This retrospective study shows that space-time syndromic surveillance on hospitalizations can timely detect local LRI-outbreaks independent of detection of the causative pathogen. The frequency of cluster detection, when interpreted in the light of available epidemiological and microbiological data, does not give rise to excessive levels of further investigations.
Consequently, we recommend real-time syndromic surveillance as an additional tool for detection of local LRI outbreaks, but only if syndromic data with sufficient quality and coverage can be collected, coupled with epidemiological and microbiological data. Public health responses can be based on a combination of syndromic surveillance data, reports by astute clinicians and early diagnostic test results, which all could generate the first alarm for different kinds of disease events. Future research on prospective syndromic surveillance should therefore focus on practical methods for integrating syndromic surveillance alarms with clinical reports and laboratory information for effective public-health responses.