Review of syndromic surveillance: implications for waterborne disease detection

doi:10.1136/jech.2005.038539

J Epidemiol Community Health. 2006 June; 60(6): 543–550.

doi: 10.1136/jech.2005.038539

PMCID: PMC2563943

Review of syndromic surveillance: implications for waterborne disease detection

Magdalena Berger, Rita Shiau, and June M Weintraub

M Berger, J M Weintraub, San Francisco Department of Public Health, Environmental Health Section, USA

R Shiau, San Francisco Department of Public Health, Communicable Disease Control and Prevention Section

Correspondence to: MsM Berger
San Francisco Department of Public Health, Environmental Health Section, 1390 Market Street, Suite 910, San Francisco, CA 94102, USA; mberger@health.nyc.gov

Accepted December 19, 2005.

This article has been cited by other articles in PMC.

Abstract

Syndromic surveillance is the gathering of data for public health purposes before laboratory or clinically confirmed information is available. Interest in syndromic surveillance has increased because of concerns about bioterrorism. In addition to bioterrorism detection, syndromic surveillance may be suited to detecting waterborne disease outbreaks. Theoretical benefits of syndromic surveillance include potential timeliness, increased response capacity, ability to establish baseline disease burdens, and ability to delineate the geographical reach of an outbreak. This review summarises the evidence gathered from retrospective, prospective, and simulation studies to assess the efficacy of syndromic surveillance for waterborne disease detection. There is little evidence that syndromic surveillance mitigates the effects of disease outbreaks through earlier detection and response. Syndromic surveillance should not be implemented at the expense of traditional disease surveillance, and should not be relied upon as a principal outbreak detection tool. The utility of syndromic surveillance is dependent on alarm thresholds that can be evaluated in practice. Syndromic data sources such as over the counter drug sales for detection of waterborne outbreaks should be further evaluated.

Keywords: syndromic surveillance, waterborne disease, surveillance

Syndromic surveillance is a tool for outbreak detection that has been used by public health departments since the mid‐1990s.¹ The CDC has defined syndromic surveillance as “…surveillance using health‐related data that precede diagnosis and signal a sufficient probability of a case or an outbreak to warrant further public health response.”² In the USA, interest in syndromic surveillance increased after September 11, 2001 because of concern about the possibility of bioterrorist attacks.

Theoretically, syndromic surveillance systems have the potential to supplement traditional infectious disease surveillance systems by providing information about the extent of an outbreak or seasonal increases in disease incidence. They may also provide reassurance that an outbreak is not happening. As syndromic data are gathered before diagnostic or laboratory information is available, health departments may be able to recognise and respond to increases in disease incidence before formal diagnoses are made, and to respond to outbreaks early in their course. In this way, syndromic surveillance has the potential to effectively mitigate the extent of morbidity, mortality, and social and financial unrest resulting from natural or manmade outbreaks.

The effectiveness of syndromic surveillance in terms of informing timely and successful public health interventions has not been demonstrated in public health practice. Although the impetus for its development has been its potential use as a bioterrorism preparedness tool, it may be well suited to monitoring naturally occurring infectious and chronic disease. The utility of syndromic surveillance for monitoring waterborne or environmentally mediated disease incidence has not been assessed to date. However, any decision about the implementation of a syndromic surveillance system cannot focus on only one aspect of disease surveillance and response. This decision has to be made within the context of the existing public health system and its ability to respond to public health threats of all types. Additionally the design of an effective syndromic surveillance system must take into account the data needs and response capabilities of many different public health responders.

This review summarises the evidence gathered from retrospective, prospective, and simulation studies to assess the efficacy of syndromic surveillance for waterborne diseases detection. The original aim of this review was to determine the utility of syndromic surveillance systems for waterborne disease detection in the San Francisco Bay Area. All articles were selected from peer reviewed articles found on Medline under the search term “syndromic surveillance”.

Data sources

Three levels of data may be used for disease surveillance: pre‐clinical data, clinical pre‐diagnostic data, and diagnostic data.³ Syndromic surveillance usually uses two types of data sources: pre‐clinical and clinical pre‐diagnostic data. Traditional surveillance generally focuses on diagnostic data.

Pre‐clinical data sources

Public behaviour may be an early indicator of an increase in disease incidence in the population. Information about school and work absenteeism,¹^,4 calls to nurse help lines,⁵ poison control centres,⁶ water utility complaint lines,⁷ and sales information of over the counter and prescription drugs¹^,4^,7^,8^,9^,10^,11^,12^,13^,14 may give information about the level of disease in the population. Other environmental data, such as drinking water turbidity levels, may also be useful indicators of disease incidence. While pre‐clinical data sources tend to be more timely than clinical data sources, they are much less specific and therefore provide a less solid basis for public health decision making.

Clinical pre‐diagnostic data sources

Most clinical data sources currently used by syndromic surveillance systems consist of electronic data gathered for an independent purpose and adapted to the needs of the specific surveillance system. Sources of clinical data used for syndromic surveillance include health care utilisation records such as registers of emergency department chief complaints,¹^,4^,15^,16^,17^,18^,19^,20^,21^,22^,23^,24 discharge data,²⁰ ambulance dispatch data,²⁵ and ICD‐9 or other procedure codes used in inpatient and outpatient settings.⁴^,20^,26 Electronically captured orders for diagnostic tests may also be used as a data source.

Different groupings of symptoms, ICD‐9 codes, or drugs, as well as the most effective free text search algorithms are being researched to determine how these data are best analysed to accurately reflect disease activity in the population being studied.¹⁷^,21^,27^,28

Data timeliness and accuracy

Usually, data sources that are chronologically distant from diagnostic information are timely but not as specific as data sources that are chronologically closer to diagnosis.³ Potential syndromic surveillance data sources along with each source's capacity for being timely and specific are described in figure 11.. Comparing two data sources from this figure illustrates the trade off between data timeliness (the amount of lag time from an event to data about the event being available for interpretation by a public health department) and specificity/accuracy (the degree to which information with relevant patient characteristics is available). Over the counter (OTC) drug data have the benefit of timeliness; for example, people who are beginning to have symptoms of an intestinal illness may purchase OTC drugs a day or two before presenting at the emergency department (ED) or other health care facility. However, OTC sales data are not very specific; while a rise in sales of OTC antidiarrhoeal remedies may be the result of an increase in incidence of diarrhoeal illness, it may be unrelated to disease incidence. An increase in OTC sales may be explained by store specials, hoarding behaviours, or duplicate data transfers. It would be difficult to make a public health decision based solely on increased OTC sales because it does not contain any patient specific data. Involving pharmacists or pharmacy managers in initial models and categorisation of OTC data may increase the correlation of OTC sales data and possible human disease. Timeliness of data acquisition is only valuable if the signal is accurate and specific enough to inform public health decision making.

Figure 1

Surveillance data sources and health seeking behaviour.

The timeliness and specificity of OTC data may be compared with ED chief complaint data. An increase in chief complaints of diarrhoea is an unambiguous signal that the incidence of intestinal illness has increased. It is less timely than the OTC data but yields much more detailed information. None the less this information does not distinguish between a natural and expected increase, a natural outbreak, a result of a bioterrorism event, or a coincidental, simultaneous increase in intestinal illnesses of varying aetiologies. It would still be difficult to make a public health decision based on these data alone without further investigation.

An increase in sales of OTC antidiarrhoeal drugs occurring concurrently with spikes in other data such as increases in worker absenteeism, and increases in ED chief complaints of diarrhoea would be a more reliable indicator of a true increase in disease incidence than any of these signals on their own. Developing a mechanism for evaluating data from disparate sources and of varying relevance to public health response is one of the challenges in finding practical applications for syndromic surveillance information.³

The accuracy of aberration detection signals generated by a syndromic surveillance system is also greatly affected by the amount of baseline data that are available. Studies reporting the use of short term, or “drop‐in” syndromic surveillance systems set up specifically for outbreak detection during a high profile event consistently cite the lack of baseline data as a factor that hampered the determination of an appropriate signal threshold for these systems, often leading to systems that were overly sensitive and did not account for seasonal or other temporal factors.¹⁵^,29^,30 Before a syndromic surveillance system is instituted for use as a regular surveillance tool, individual jurisdictions should ensure that at least one year of historical data are available for use as a reference for signal threshold determination and signal investigation.

Syndromic surveillance evaluations

Retrospective evaluations

Retrospective analyses show that syndromic surveillance systems may provide information that would allow public health departments to predict outbreaks earlier than by using traditional surveillance. For example, in the Milwaukee, Wisconsin cryptosporidiosis outbreak of 1993,³¹ newspaper reports of over the counter antidiarrhoeal drugs being sold out at pharmacies were one of the first clues that an unusual event had occurred.⁷ Calls to nurse hotlines showed a fourfold increase in the standard deviation of diarrhoea rates one day before observation of unusual activity by local pharmacists and five days before the local public health department was notified of the possible outbreak.⁵ Similarly, a retrospective review of pharmacy records showed that sales of over the counter drugs increased fivefold during an outbreak of cryptosporidiosis in North Battleford, Saskatchewan.⁷^,32 During a 1983 outbreak of waterborne campylobacteriosis in Florida, pharmacy sales of antidiarrhoeals increased eightfold when compared with the same period in the previous year.¹⁶ Retrospective analysis of Canadian outbreaks of Cryptosporidium, E coli O157:H7, and Campylobacter also all showed increases in OTC sales that corresponded to increases in disease incidence.³³ In New York City, retrospective analyses of ambulance dispatch data and ED data showed that they were effective in predicting city wide respiratory, gastrointestinal, and influenza outbreaks.²⁵^,34

Simulations

Simulated outbreaks can examine the ability of syndromic surveillance systems to accurately predict outbreaks while minimising false alarms.³⁴^,35 According to most simulations, detection algorithms used in syndromic surveillance can best identify large, geographically widespread increases in disease. Syndromic surveillance detection algorithms are less successful at identifying small counts or small increases in disease, showing that syndromic surveillance may augment traditional disease surveillance but may not improve timeliness or sensitivity. Simulations of syndromic surveillance suggest that it is best suited for detecting diseases that have a narrow incubation period distribution, a steep epidemic curve, a long prodromal phase, are not included on routine diagnostic tests, and do not have a specific disease identifying clinical feature.³⁶

To date no simulations of waterborne disease outbreaks have been conducted. Stoto et al tested a hypothetical syndromic surveillance system by simulating a “fast” outbreak (a disease that would have a steep epidemic curve) and a “slow” outbreak (a disease with a more gradual epidemic curve) of influenza‐like illness (ILI) using three years of data from an ED.³⁷ Results of this exercise show that an episode of either “fast” or “slow” ILI during the annual influenza season was undetectable by syndromic surveillance when set to a 1% false positive rate. During the non‐flu (RSI) season syndromic surveillance algorithms were successful at detecting a “fast” outbreak by day three. However, the “slow” outbreaks were much harder to detect. Only one algorithm had a 50% probability of detecting this outbreak by day nine. By day nine, such outbreaks would usually be detected through traditional surveillance systems. As Stoto et al point out, “The sudden appearance of large number of ILI cases…five times the daily average…especially in the summer‐ is clearly exceptional and ER physicians do not require any sophisticated statistical algorithm to tell them so”.³⁷

Prospective reports

Several prospective evaluations of long term surveillance systems¹⁸^,23^,34^,38^,39^,40^,41^,42^,43 have been published. Syndromic surveillance has been in use in New York City since 1995 to detect outbreaks of waterborne diarrhoeal disease. The waterborne disease surveillance system uses data from OTC antidiarrhoeal sales, ED chief complaint logs, reports from sentinel nursing homes, and reports of positive stool samples from clinical laboratories. Prospective studies evaluating this system between 2001 and 2004³⁴^,40 reviewed citywide signals and smaller spatial clusters to find out if signals represented real outbreaks, and whether outbreaks detected by traditional surveillance were also detected by the syndromic system. These studies found that, 75% of the citywide outbreak signals correlated with true citywide viral gastroenteritis outbreaks. However, there was no correlation between smaller spatial clusters detected by the syndromic surveillance system and gastrointestinal (GI) outbreaks detected by traditional surveillance. Additionally, there were 36 GI outbreaks detected by traditional surveillance, none of which were detected by the syndromic surveillance system. Despite the extensive use of staff resources, the availability of fully electronic and timely data, and the ability to quickly initiate responses to various alerts, the syndromic system did not appreciably improve timeliness or sensitivity to overall disease surveillance.⁴⁰ A system in Connecticut that tracks hospital admissions shows similar discrepancies between GI outbreaks detected by traditional and syndromic surveillance.⁴² This system generated 35 GI illness alarms in the course of one year, only one of which turned out to be a true GI outbreak. Additionally, none of 15 GI illness clusters detected by laboratory surveillance were detected by the syndromic surveillance system.

The University of Maryland implemented a syndromic surveillance system in its university hospital and evaluated its system's ability to detect actual clusters of patients with GI or respiratory symptoms presenting to the hospital from 2001 to 2002.³⁹ This system incorporated admission, discharge and transfer data as well as laboratory information system data. The evaluation showed that a peak in ED visits and an increase in stool test orders corresponded to a cluster of patients who later tested positive for infections with Shigellaspp. The authors found the system to be timely and sensitive for their needs. A similar system is operated by the Department of Defense (DoD).²³ The DoD system detected three outbreaks of diarrhoeal diseases in three different locations, one of which was laboratory confirmed to be caused by rotavirus. A system based on ED chief complaint data administered by the Westchester County Department of Health was less specific. Over the course of 277 days, 59 signals were detected, none of which corresponded to an outbreak of a communicable disease.⁴³ A multi‐jurisdictional system implemented for the 2002 Olympic games in Utah tracked encounters at urgent care centres and EDs using the real‐time outbreak and disease surveillance system.¹⁸ During the two month Olympic event the system's detection algorithms were exceeded twice; neither of these alarms corresponded to a real outbreak. The system was successfully maintained after the end of the Olympic event.

A study by Henry et al quantified the sensitivity, specificity, and positive predictive value (PPV) of the Kaiser Mid‐Atlantic Region nurse hotline syndromic system by comparing the level of concordance between syndrome assignments based on the nurse hotline algorithm and the diagnosis made based on a subsequent Kaiser office visit. The authors found that the nurse hotline system achieved the highest sensitivity and PPV for respiratory and GI illness syndromes.³⁸

Staff and resources used

Performance evaluations often do not provide information about time and resources that are used in maintaining the system and investigating the alarms. In New York City, staff maintain the system, spending several hours a day, seven days per week to download and analyse data.¹ Investigation of false signals is a significant burden on staff resources.³⁷ An often mentioned solution is to investigate only those alarms that are maintained for two or more days and in two or more geographical areas. However, if the main goal of syndromic surveillance is to increase timeliness of outbreak detection and response then waiting two or more days to start an investigation would negate the timeliness benefit of the system.⁴⁴ Similarly, if, because of the low specificity of syndromic signals, local health departments require that syndromic signals are substantiated by specimen collection and laboratory confirmation before public health action may be taken, any potential advantages of timeliness are lost.⁴⁵ Decision makers must take into account the ability of the local public health system to respond to signals when considering the implementation of any syndromic surveillance system.

Syndromic surveillance uses for waterborne disease detection

Criteria derived from simulations can be used to assess which waterborne diseases may be appropriate for identification by syndromic surveillance.³⁶

(1) The disease should have a narrow incubation period distribution.

Incubation periods of waterborne disease agents vary widely depending on dose, host susceptibility, and other factors.

(2) The disease should have a steep epidemic curve and a long prodromal phase.

Most waterborne diseases do not have a prodromal phase and an outbreak will not necessarily have a steep epidemic curve.

(3) The disease should not have a specific disease identifying clinical feature.

Initial symptoms of most potentially waterborne diseases are non‐specific and generally include diarrhoea and other GI distress. Most of these illnesses do not have a disease identifying clinical or historical feature that allows clinicians to pinpoint the cause before performing laboratory tests.⁴⁶

(4) The disease should not be included in routine diagnostic tests.

Laboratory tests are not routinely ordered for many waterborne diseases,⁴⁶ and acute GI illnesses are generally under diagnosed and underreported.⁴⁷^,48^,49

What this paper adds

To date, little attention has been paid to the possibility of using syndromic surveillance for monitoring waterborne disease incidence. This review of the benefits and shortcomings of syndromic surveillance may prove useful to public health practitioners and planners who are considering approaches to improve traditional surveillance systems.

Table 11 lists characteristics of potentially waterborne pathogens. Because many waterborne diseases lack clinically identifying features and are not part of routine testing, they may be good candidates for detection by syndromic surveillance, based on the four criteria listed above. However, many of the diseases that fit these criteria, while good candidates for syndromic surveillance, are often commonly occurring and nor necessarily of high public health importance.

Table 1

Characteristics of potentially waterborne disease agents

Waterborne disease surveillance in the San Francisco Bay Area

In the San Francisco Bay Area four counties receive water from a common water utility. With the exception of a multi‐county cryptosporidiosis surveillance project, surveillance for potentially waterborne disease in the San Francisco Bay Area is conducted by each county separately. There is no formal, timely coordination of waterborne disease surveillance across county lines. A system with multi‐jurisdictional disease monitoring capabilities could provide public health benefit by permitting early detection of a multi‐county waterborne outbreak. Surveillance data captured from multiple jurisdictions and interpreted centrally may lead to earlier outbreak detection than data gathered and interpreted by staff in separate counties who may not be aware of disease incidence in neighbouring jurisdictions.

Syndromic surveillance data sources may potentially provide cross‐jurisdictional data, information about the geographical scope of an outbreak once one was identified, and additional reassurance that an outbreak was not occurring. However, the practical utility of syndromic data sources is uncertain; their outbreak detection benefits are currently theoretical and will remain so until accurate electronic capture of data and signal detection algorithms are refined or data sensitivity increases. The benefits of a regional waterborne disease surveillance system must be weighed against the resources required to set up such a system and the true risk of a waterborne disease outbreak in the San Francisco Bay Area. Based on the absence of any known waterborne outbreaks in the history of the water utility, extensive watershed protection measures, and a protected water source located in a national park it would seem that the risk of a waterborne disease outbreak occurring in the San Francisco Bay Area is quite small.

Policy implications

This review will help policy makers weigh the costs and benefits of implementing a syndromic surveillance system and will clarify the drawbacks and advantages of potential data sources.

Potential waterborne disease syndromic surveillance data sources

Table 22 lists the potential waterborne disease syndromic surveillance data sources along with indicators of data quality and utility. Potential data sources in the table are divided into data that are currently available and accessible in electronic format in the San Francisco Bay Area and data that are not currently automated or electronically available but could be useful once they became automated and electronic.

Table 2

Potential syndromic surveillance data sources

Compared with other options, OTC surveillance for waterborne disease is currently the most feasible source of syndromic surveillance data available in the San Francisco metropolitan area because of the relative ease of implementing an existing, nationally funded system. Nursing home surveillance entails large inputs in terms of health department and on‐site staff and fiscal resources for specimen testing, data monitoring, and signal investigation. Water utility complaint, nurse call line, and school and worker absenteeism logs are currently not compiled electronically in a central location; setting up a system to electronically capture these data would entail considerable commitments of will and resources from the department of public health, other city agencies, and private and public partnerships with water utilities, insurance providers, hospitals and clinics, and large employers in the San Francisco Bay Area. Mechanisms for data storage, sharing, and retrieval would need to be established for each partnership. Finally and most importantly, dedicated public health staff who would compile, manage, and analyse syndromic data on a regular basis, and respond to syndromic surveillance signals would be needed. Signal verification and response activities may include: (1) determining data import and aberration detection algorithm problems that may lead to erroneous signals (for example, duplicate data, batch transfers from certain institutions, miscoding of information at the point of entry, text‐string search algorithms that are too specific or not specific enough, etc); (2) verifying the validity of the signal by looking for the presence of signals in other data sources; (3) if the signal is deemed to be composed of possible true cases, hospital logs and charts may need to be manually reviewed by hospital or public health department staff and a line list compiled for clinical and/or laboratory based case verification; (4) traditional outbreak investigation activities and application of interventions. Timeliness provided by a syndromic surveillance system can only be useful if all of the above functions are supported by and integrated into the activities of the local health department on a sustained basis.

It is possible that outbreaks of cryptosporidiosis, cyclosporiasis, legionellosis, hepatitis A, and others may be detected through monitoring of OTC sales or that a suspected outbreak may be confirmed or better characterised through the use of these data. If OTC surveillance is to be used primarily as a back up data source to traditional surveillance, staff time for checking the web based interface would not exceed 15 minutes per day. Signal investigation would require additional resources. While the usefulness of OTC monitoring for waterborne diseases is only theoretical, given the availability of multi‐county data and the low amount of staff time and effort needed to monitor the data, utilisation and prospective evaluation of these data may be recommended for two purposes: (1) reassurance of the absence of a waterborne disease outbreak and (2) establishing baseline familiarity that may prove helpful in the event of a waterborne disease outbreak. OTC data need to be correlated with known outbreaks in the geographical area where surveillance is occurring to clarify the validity and representativeness of these data before they can be used for prospective outbreak detection.

Conclusion

Because the efficacy of syndromic surveillance is not proved, it remains unclear whether it is worth the investment of personnel and financial resources to implement. In addition to the issues discussed in this review, it is challenging to develop sensible response protocols for syndromic surveillance systems because the likelihood of false alarms is so high, and because information is currently not specific enough to enable more timely outbreak detection or disease control activities. Although most syndromic surveillance systems do not use personally identifiable information, there are also considerations of personal privacy and public comfort with health information storage and analysis. The data needs of a syndromic surveillance system must be weighed against the needs and public demands for patient privacy.

There are many theoretical benefits of syndromic surveillance, including potential timeliness, increased response capacity, ability to establish baseline disease burdens, and ability to delineate the geographical reach of an outbreak. However, in the absence of empirical evidence of its efficacy to mitigate the effects of natural or intentional waterborne disease outbreaks through earlier detection and response, syndromic surveillance should not be implemented at the expense of traditional public health activities, and it should not be relied upon as a principal waterborne disease outbreak detection tool at this time. For waterborne diseases, syndromic surveillance systems should continue to be assessed and implemented as a supplemental system to help develop better statistical methods and sensible alarm thresholds that can be applied and evaluated in practice.

Footnotes

Funding: none.

Competing interests: none.

References

1. Heffernan R, Mostashari F, Das D. et al New York City syndromic surveillance system. Syndromic surveillance: reports from a national conference, 2003, MMWR 2004. 53(suppl)25–27.27. [PubMed]

2. Centers for Disease Control http://www.cdc.gov/epo/dphsi/syndromic.htm (accessed on 3 Oct 2005)

3. Buckeridge D, Graham J, O'Connor M. et al Knowledge‐based bioterrorism surveillance. Proc AMIA Symp 2002. 76–80.80. [PMC free article] [PubMed]

4. Lombardo J, Burkom H, Elbert E. et al Systems overview of the electronic surveillance system for the early notification of community based epidemics (ESSENCE II). J Urban Health 2003. 80(suppl)i32–i41.i41. [PMC free article] [PubMed]

5. Rodman J, Frost F, Jakubowski W. Using nurse hotline calls for disease surveillance. Emerg Infect Dis 1998. 4329–332.332. [PMC free article] [PubMed]

6. Derby M, McNally J, Ranger‐Moore J. et al A comparison of foodborne illness cases reported to a poison control center and public health department. Syndromic surveillance: reports from a national conference, 2004, MMWR 2005. 54(suppl)35–40.40. [PubMed]

7. Proctor M E, Blair K A, Davis J P. Surveillance data for waterborne illness detection: an assessment following a massive waterborne outbreak of cryptosporidium infection. Epidemiol Infect 1998. 12043–54.54. [PubMed]

8. Magruder S, Lewis S, Najmi A. et al Progress in understanding and using over‐the‐counter pharmaceuticals for syndromic surveillance. Syndromic surveillance: reports from a national conference, 2003, MMWR 2004. 53(suppl)117–122.122. [PubMed]

9. Foldy S. Linking better surveillance to better outcomes. Syndromic surveillance: reports from a national conference, 2003, MMWR 2004. 53(suppl)12–17.17. [PubMed]

10. Beaudeau P, Payment P, Bourderont D. et al A time series study of anti‐diarrheal drug sales and tap water quality. Int J Environ Health Res 1999. 9293–311.311.

11. Hogan W, Tsui F, Ivanov O. et al Detection of pediatric respiratory and diarrheal outbreaks from sales of over‐the‐counter electrolyte products. J Am Med Inform Assoc 2003. 10555–562.562. [PMC free article] [PubMed]

12. Davies G, Finch R. Sales of over‐the‐counter remedies as an early warning system for winter bed crises. Clin Microbiol Infect 2003. 9858–863.863. [PubMed]

13. Sacks J, Lieb S, Baldy L. et al Epidemic campylobacteriosis associated with community water supply. Am J Public Health 1986. 76424–428.428. [PubMed]

14. Wagner M, Tsui F, Espino J. et al National retail data monitor for public health surveillance. syndromic surveillance: reports from a national conference, 2003, MMWR 2004. 53(suppl)40–42.42. [PubMed]

15. Foldy S, Biedrzycki P A, Barthell E N. et al Syndromic surveillance using regional emergency medicine internet. Ann Emerg Med 2004. 44242–246.246. [PubMed]

16. Beitel A J, Olson K L, Reis B Y. et al Use of emergency department chief complaint and diagnostic codes for identifying respiratory illness in a pediatric population. Pediatr Emerg Care 2004. 20355–360.360. [PubMed]

17. Begier E M, Sockwell D, Branch L M. et al The National Capitol Region's emergency department syndromic surveillance system: Do chief complaint and discharge diagnosis yield different results? Emerg Infect Dis 2003. 9393–396.396. [PMC free article] [PubMed]

18. Gesteland P H, Gardner R M, Tsui F C. et al Automated syndromic surveillance for the 2002 winter Olympics. J Am Med Inform Assoc 2003. 10547–554.554. [PMC free article] [PubMed]

19. Silka P A, Geiderman J M, Goldberg J B. et al Demand on ED resources during periods of widespread influenza activity. Am J Emerg Med 2003. 21534–539.539. [PubMed]

20. Lober W B, Trigg L J, Karras B T. et al Syndromic surveillance using automated collection of computerized discharge diagnoses. J Urban Health 2003. 80(suppl 1)i97–106.106. [PMC free article] [PubMed]

21. Flaherty J, Gillam M. Correlation of West Nile virus infection with emergency department chief complaints by using a passive syndromic surveillance model. (Abstract). MMWR 2004. 53(suppl)237.

22. Paladini M. Daily emergency department surveillance system‐Bergen County, New Jersey. Syndromic surveillance: reports from a national conference, 2003, MMWR 2004. 53(suppl)47–49.49. [PubMed]

23. Lewis M, Pavlin J, Mansfield J. et al Disease outbreak detection system using syndromic data in the greater Washington DC area. Am J Prev Med 2002. 23180–186.186. [PubMed]

24. Irvin C, Nouhan P, Rice K. Syndromic analysis of computerized emergency department patients' chief complaints: an opportunity for bioterrorism and influenza surveillance. Ann Emerg Med 2003. 41447–452.452. [PubMed]

25. Mostashari F, Fine A, Das D. et al Use of ambulance dispatch data as an early warning system for communitywide influenzalike illness, New York City. J Urban Health 2003. 80(suppl)i43–i49.i49. [PMC free article] [PubMed]

26. Lazarus R, Kleinman K, Dashevsky I. et al Use of automated ambulatory‐care encounter records for detection of acute illness clusters, including potential bioterrorism events. Emerg Infect Dis 2002. 8753–760.760. [PMC free article] [PubMed]

27. Reis B Y, Mandl K D. Syndromic surveillance: the effects of syndrome grouping on model accuracy and outbreak detection. Ann Emerg Med 2004. 44235–241.241. [PubMed]

28. Day F C, Schriger D L, La M. Automated linking of free‐text complaints to reason‐for‐visit categories and International Classification of Diseases diagnoses in emergency department patient record databases. Ann Emerg Med 2004. 43401–409.409. [PubMed]

29. Osaka K, Takahashi H, Ohyama T. Testing a symptom‐based surveillance system at high‐profile gatherings as a preparatory measure for bioterrorism. Epidemiol Infect 2002. 129429–434.434. [PubMed]

30. Cho J, Kim J, Yoo I. et al Syndromic surveillance based on the emergency department in Korea. J Urban Health 2003. 80i124.

31. Mac Kenzie W R, Hoxie N J, Proctor M E. et al A massive outbreak in Milwaukee of cryptosporidium infection transmitted through the public water supply. N Engl J Med 1994. 331161–167.167. [PubMed]

32. Stirling R, Aramini J, Ellis A. et al Waterborne cryptosporidiosis outbreak, North Battleford, Saskatchewan, Spring 2001. Can Commun Dis Rep 2001. 27185–192.192. [PubMed]

33. Edge V, Pollari F, Lim G. et al Syndromic surveillance of gastrointestinal illness using pharmacy over‐the‐counter sales. A retrospective study of waterborne outbreaks in Saskatchewan and Ontario. Can J Public Health 2004. 95446–450.450. [PubMed]

34. Heffernan R, Mostashari F, Das D. et al Syndromic surveillance in public health practice, New York City. Emerg Infect Dis 2004. 10858–864.864. [PubMed]

35. Kulldorff M, Zhang Z, Hartman J. et al Benchmark data and power calculations for evaluating disease outbreak detection methods. Syndromic surveillance: reports from a national conference, 2003, MMWR 2004. 53(suppl)144–151.151. [PubMed]

36. Buehler J W, Berkelman R L, Hartley D M. et al Syndromic surveillance and bioterrorism‐related epidemics. Emerg Infect Dis 2003. 91197–1204.1204. [PMC free article] [PubMed]

37. Stoto M, Schonlau M, Mariano L. Syndromic surveillance: Is it worth the effort? Chance 2004. 1720–25.25.

38. Henry J, Magruder S, Snyder M. Comparison of office visit and nurse advice hotline data for syndromic surveillance—Baltimore‐Washington, DC, Metropolitan Area, 2002. Syndromic surveillance: reports from a national conference, 2003, MMWR 2004. 53(suppl)112–116.116. [PubMed]

39. Peterson D, Perencevich E, Harris A. et al Using existing electronic hospital data for syndromic surveillance. J Urban Health 2003. 80i122–i123.i123.

40. Balter S, Weiss D, Hanson H. et al Three years of emergency department gastrointestinal syndromic surveillance in New York City: What have we found? Syndromic surveillance: reports from a national conference, 2004, MMWR 2004. 54(suppl)175–180.180. [PubMed]

41. Cooper D, Smith G, Baker M. et al National symptom surveillance using calls to a telephone health advice service—United Kingdom, December 2001–February 2003. Syndromic surveillance: reports from a national conference, 2003, MMWR 2004. 53(suppl)179–183.183. [PubMed]

42. Dembek Z, Carley K, Siniscalchi A. et al Hospital admissions syndromic surveillance‐ Connecticut, Septemeber 2001–November 2003. Syndromic surveillance: reports from a national conference, 2003, MMWR 2004. 53(suppl)50–52.52. [PubMed]

43. Terry W, Ostrowsky B, Huang A. Should we be worried? Investigation of signals generated by an electronic surveillance system‐ Westchester County, New York. Syndromic surveillance: reports from a national conference, 2003, MMWR 2004. 53(suppl)190–195.195. [PubMed]

44. Reingold A. If syndromic surveillance is the answer, what is the question? Biosecur Bioterror 2003. 177–81.81. [PubMed]

45. Steiner‐Sichel L, Greenko J, Heffernan R. et al Field investigations of emergency department syndromic surveillance signals‐ New York City. Syndromic surveillance: reports from a national conference, 2003, MMWR 2004. 53(suppl)184–189.189. [PubMed]

46. Centers For Disease Control and Prevention Diagnosis and management of food borne illness: a primer for physicians and other health care professionals MMWR 2004. 53RR–NaN4.NaN4. [PubMed]

47. Hellard M, Sinclair M, Fairley C. et al An outbreak of cryptosporidiosis in an urban swimming pool: why are such outbreaks difficult to detect? Aust N Z J Public Health 2000. 24272–275.275. [PubMed]

48. Kasowski E, Gackstetter G, Sharp T. Foodborne illness: new developments concerning an old problem. Curr Gastroenterol Rep 2002. 4308–318.318. [PubMed]

49. Townes J. Acute infectious gastroenteritis in adults: seven steps to management and prevention. Postgrad Med 2004. 11511–18.18. [PubMed]

50. Chin J. Control of communicable diseases manual. 17th ed. Washington, DC: American Public Health Association, 2000.

51. Mulazimoglu L, Yu V. Can Legionnaires disease be diagnosed by clinical criteria? Chest 2001. 1201049–1053.1053. [PubMed]

52. Fiore A, Butler J. Detecting nosocomial Legionnaires' disease. Infect Med 1998. 15625–635.635.

Articles from Journal of Epidemiology and Community Health are provided here courtesy of
BMJ Group