PMCC PMCC

Search tips
Search criteria

Advanced
Results 1-25 (1166813)

Clipboard (0)
None

Related Articles

1.  SAGES Update: Electronic Disease Surveillance in Resource-Limited Settings 
Objective
The Suite for Automated Global Electronic bioSurveillance (SAGES) is a collection of modular, flexible, open-source software tools for electronic disease surveillance in resource-limited settings. This demonstration will illustrate several new innovations and update attendees on new users in Africa and Asia.
Introduction
The new 2005 International Health Regulations (IHR), a legally binding instrument for all 194 WHO member countries, significantly expanded the scope of reportable conditions and are intended to help prevent and respond to global public health threats. SAGES aims to improve local public health surveillance and IHR compliance with particular emphasis on resource-limited settings. More than a decade ago, in collaboration with the US Department of Defense (DoD), the Johns Hopkins University Applied Physics Laboratory (JHU/APL) developed the Electronic Surveillance System for the Early Notification of Community-based Epidemics (ESSENCE). ESSENCE collects, processes, and analyzes non-traditional data sources (i.e. chief complaints from hospital emergency departments, school absentee data, poison control center calls, over-the-counter pharmaceutical sales, etc.) to identify anomalous disease activity in a community. The data can be queried, analyzed, and visualized both temporally and spatially by the end user. The current SAGES initiative leverages the experience gained in the development of ESSENCE, and the analysis and visualization components of SAGES are built with the same features in mind.
Methods
SAGES tools are organized into four categories: 1) data collection, 2) analysis & visualization, 3) communications, and 4) modeling/simulation/evaluation. Within each category, SAGES offers a variety of tools compatible with surveillance needs and different types or levels of information technology infrastructure. SAGES tools are built in a modular nature, which allows for the user to select one or more tools to enhance an existing surveillance system or use the tools en masse for an end-to-end electronic disease surveillance capability. Thus, each locality can select tools from SAGES based upon their needs, capabilities, and existing systems to create a customized electronic disease surveillance system. New OpenESSENCE developments include improved data query ability, improved mapping functionality, and enhanced training materials. New cellular phone developments include the ability to concatenate single SMS messages sent by simple or Smart Android cell phones. This ‘multiple-SMS’ message ability allows use of SMS technology to send and receive health information exceeding normal SMS message length in a manner transparent to the users.
Conclusions
The SAGES project is intended to enhance electronic disease surveillance capacity in resource-limited settings around the world. We have combined electronic disease surveillance tools developed at JHU/APL with other freely-available, interoperable software tools to create SAGES. We believe this suite of tools will facilitate local and regional electronic disease surveillance, regional public health collaborations, and international disease reporting. SAGES development, funded by the US Armed Forces Health Surveillance Center, continues as we add new international collaborators. SAGES tools are currently deployed in locations in Africa, Asia and South America, and are offered to other interested countries around the world.
PMCID: PMC3692858
software; surveillance; electronic; open-source
2.  The Biosurveillance Resource Directory - A One-Stop Shop for Systems, Sources, and Tools 
Objective
The goal of this project is to identify systems and data streams relevant for infectious disease biosurveillance. This effort is part of a larger project evaluating existing and potential data streams for use in local, national, and international infectious disease surveillance systems with the intent of developing tools to provide decision-makers with timely information to predict, prepare for, and mitigate the spread of disease.
Introduction
Local, national, and global infectious disease surveillance systems have been implemented to meet the demands of monitoring, detecting, and reporting disease outbreaks and prevalence. Varying surveillance goals and geographic reach have led to multiple and disparate systems, each using unique combinations of data streams to meet surveillance criteria. In order to assess the utility and effectiveness of different data streams for global disease surveillance, a comprehensive survey of current human, animal, plant, and marine surveillance systems and data streams was undertaken. Information regarding surveillance systems and data streams has been (and continues to be) systematically culled from websites, peer-reviewed literature, government documents, and subject-matter expert consultations.
Methods
A relational database has been developed and refined to allow for detailed analyses of data streams and surveillance systems. To maximize the utility of the database and facilitate one-stop-shopping for biosurveillance system information, we have expanded our scope to include not only biosurveillance systems, but also data sources, tools, and biosurveillance collectives. Captured in the information collected about the resource (if available) is the name and acronym of the resource, the date the resource became available, the accessibility of the resource (is it open to all, or are there limitations to access), the primary sponsors, if the resource is associated with GIS functionality, and if the focus is health. Also collected is contact information, information regarding the scope and domain of the resource, the pertinent diseases or disease categories, and the geographic and population coverage of the resource. Websites associated with the resource are directly accessible from the database. Data stream information is also captured based on our developed data stream framework. If the resource uses other specified systems/sources/tools for data gathering or analysis, then that is also captured and directly linked within the database.
Results
The Biosurveillance Resource Directory (BRD) is in the process of being tested by multiple potential end users in the public health, biosecurity, and biosurveillance communities. Feedback from these testers is being used to refine the database to maximize functionality and utility. Additionally, methods for dynamically updating and maintaining the database are being evaluated. Automated and semi-automated queriable reports have been developed and are integral to demonstrating specific use-case scenarios in which the BRD would be beneficial for end-users.
Conclusions
A need for a biosurveillance one-stop shop has been increasingly called for to help in evaluating what data streams and systems are available and relevant for many different biosurveillance needs and goals. The prototype Biosurveillance Resource Directory is a search-able, dynamic database for biosurveillance systems, sources, and tools information.
PMCID: PMC3692785
infectious disease; biosurveillance; database
3.  SAGES: A Suite of Freely-Available Software Tools for Electronic Disease Surveillance in Resource-Limited Settings 
PLoS ONE  2011;6(5):e19750.
Public health surveillance is undergoing a revolution driven by advances in the field of information technology. Many countries have experienced vast improvements in the collection, ingestion, analysis, visualization, and dissemination of public health data. Resource-limited countries have lagged behind due to challenges in information technology infrastructure, public health resources, and the costs of proprietary software. The Suite for Automated Global Electronic bioSurveillance (SAGES) is a collection of modular, flexible, freely-available software tools for electronic disease surveillance in resource-limited settings. One or more SAGES tools may be used in concert with existing surveillance applications or the SAGES tools may be used en masse for an end-to-end biosurveillance capability. This flexibility allows for the development of an inexpensive, customized, and sustainable disease surveillance system. The ability to rapidly assess anomalous disease activity may lead to more efficient use of limited resources and better compliance with World Health Organization International Health Regulations.
doi:10.1371/journal.pone.0019750
PMCID: PMC3091876  PMID: 21572957
4.  Operational Experience: Integration of ASPR Data into ESSENCE-FL during the RNC 
Objective
The Florida Department of Health (FDOH), Bureau of Epidemiology, partnered with the U.S. Department of Health and Human Services (HHS) Office of the Assistant Secretary for Preparedness and Response (ASPR) to improve surveillance methods in post disaster or response events. A new process was implemented for conducting surveillance to monitor injury and illness for those presenting for care to ASPR assets such as Disaster Medical Assistance Team (DMAT) sites when they are operational in the state. The purpose of the current work was to field test and document the operational experience of the newly implemented ASPR data module in ESSENCE-FL (syndromic surveillance system) to receive near real-time automated data feeds when ASPR federal assets were deployed in Florida during the 2012 Republican National Convention (RNC).
Introduction
Florida has implemented various surveillance methods to augment existing sources of surveillance data and enhance decision making with timely evidence based assessments to guide response efforts post-hurricanes. Historically, data collected from deployed federal assets have been an integral part of this effort. However, a number of factors have made this type of surveillance challenging: logistical issues of field work in a post-disaster environment, the resource intensive manual data collection process from DMAT sites, and delayed analysis and interpretation of these data to inform decision makers. The ESSENCE-FL system is an automated and secure web-based application accessed by FDOH epidemiologists and staff at participating hospitals.
Methods
ESSENCE-FL was configured by the Johns Hopkins University Applied Physics Laboratory (JHU/APL) to receive ASPR electronic medical record (EMR) data. A scheduled program to generate data files for FDOH was created using SAS Enterprise Business Intelligence (EBI) software and a script was set up on the ASPR server to send an updated file via secure file transfer protocol (sftp) every 15 minutes. A case definition was created by ASPR field teams to identify which encounter visits would be entered into the electronic medical record (EMR) and received in ESSENCE-FL. To assess completeness of data elements and total patient encounters received in ESSENCE-FL, DMAT field teams maintained Excel line lists of patient encounters and emailed them to FDOH three times daily during the RNC. ASPR data were reviewed and analyzed by FDOH staff multiple times a day in near real time utilizing the existing ESSENCE-FL robust analysis tools.
Results
Three separate ASPR missions were deployed to Florida to support the RNC. ASPR EMR data files were received at 15-minute intervals by ESSENCE-FL from the ASPR central server during each day of the 2012 RNC (August 26–31). Reduced patient counts within ESSENCE-FL as compared with DMAT-maintained Excel line lists indicated an incomplete input, upload, or transfer of patient data from one of two ASPR sites to the central ASPR servers. Although only 11 of 34 total patient encounters were received by ESSENCE-FL during the event, the system design enabled users to run specific queries and display the results of their queries in time series graphs, pie and bar charts, GIS maps, dashboards, and statistical tables.
Conclusions
There is a great need to have timely access to data sources to enhance disease surveillance efforts and help guide decision makers’ situational awareness and disease control efforts during a response. The FDOH, Bureau of Epidemiology’s collaboration with JHU/APL and ASPR takes advantage of ASPR’s EMR-S to make data sharing and analysis efficient as evidenced during the RNC. Automated data feeds to ESSENCE-FL removed resource intensive manual data collection by public health, improved standardization of syndrome and demographic categorizations, increased access to these data by local, state, and federal epidemiologists in a timely manner, and expedited analysis and interpretation for situational awareness. Future recommendations include pre-event testing of the entire data flow process, establishing an on-site specialist to immediately assist with any issues, greater understanding of the field team use of the EMR-S, and ensuring field staff is aware of data quality needs for effective public health surveillance.
PMCID: PMC3692919
surveillance; response; disaster
5.  Using Cloud Technology to Support Monitoring During High Profile Events 
Objective
In May 2012, thousands of protesters, descended on Chicago during the NATO Summit to voice their concern about social and economic inequality. Given the increased numbers of international and domestic visitors to the Windy City and the tension surrounding protesting during the summit, increased monitoring for health events within the city and Chicago metropolitan region was advised. This project represents the first use of cloud technology to support monitoring for a high profile event.
Introduction
Hospital emergency departments in Cook and surrounding counties currently send data to the Cook County Department of Public Health (CCDPH) instance of ESSENCE on CCDPH servers. The cloud instance of ESSENCE has been enhanced to receive and export all meaningful use data elements in the meaningful use format. The NATO summit provided the opportunity for a demonstration project to assess the ability of an Amazon GovCloud instance of ESSENCE to ingest and process meaningful use data, and to export meaningful use surveillance data to the Cook County Locker in BioSense 2.0.
Methods
In the three weeks leading up to the NATO Summit, HL7 data extracts were sent to BioSense 2.0 and a data feed was established to the Amazon GovCloud instance of ESSENCE. Queries specific to anticipated health events associated with the summit such as injuries, tear gas exposure, and general exposure, were developed. Several features of the cloud instance of ESSENCE enhanced the ability of CCDPH staff epidemiologists to conduct analyses, including the sharing capabilities of queries and the myESSENCE dashboard feature. The sharing capabilities within the cloud instance of ESSENCE allowed queries to be easily shared with multiple staff epidemiologists and across health jurisdictions. The myESSENCE dashboard feature was used to create dashboards of surveillance results, including time series graphs, maps, and records of interest for relevant queries, that were shared with public health staff monitoring population health during the summit. This information was used to provide situational awareness on a daily basis in the Chicago Metropolitan region.
Results
Data feeds to BioSense 2.0 and the Amazon GovCloud instance of ESSENCE were successful. The NATO Summit did not produce any remarkable public health concerns in suburban Cook County. The use of the cloud instance of ESSENCE enhanced the timeliness of generating situational awareness reports for distribution to public health partners in the Chicago Metropolitan region.
Conclusions
While further evaluation of cloud resources to conduct syndromic surveillance is warranted, use of the cloud instance of ESSENCE during the NATO Summit demonstrated the ability of the cloud to support surveillance for both routine and high profile events.
PMCID: PMC3692921
Cloud; Meaningful Use; Surveillance; Public Health
6.  Enhanced health event detection and influenza surveillance using a joint Veterans Affairs and Department of Defense biosurveillance application 
Background
The establishment of robust biosurveillance capabilities is an important component of the U.S. strategy for identifying disease outbreaks, environmental exposures and bioterrorism events. Currently, U.S. Departments of Defense (DoD) and Veterans Affairs (VA) perform biosurveillance independently. This article describes a joint VA/DoD biosurveillance project at North Chicago-VA Medical Center (NC-VAMC). The Naval Health Clinics-Great Lakes facility physically merged with NC-VAMC beginning in 2006 with the full merger completed in October 2010 at which time all DoD care and medical personnel had relocated to the expanded and remodeled NC-VAMC campus and the combined facility was renamed the Lovell Federal Health Care Center (FHCC). The goal of this study was to evaluate disease surveillance using a biosurveillance application which combined data from both populations.
Methods
A retrospective analysis of NC-VAMC/Lovell FHCC and other Chicago-area VAMC data was performed using the ESSENCE biosurveillance system, including one infectious disease outbreak (Salmonella/Taste of Chicago-July 2007) and one weather event (Heat Wave-July 2006). Influenza-like-illness (ILI) data from these same facilities was compared with CDC/Illinois Sentinel Provider and Cook County ESSENCE data for 2007-2008.
Results
Following consolidation of VA and DoD facilities in North Chicago, median number of visits more than doubled, median patient age dropped and proportion of females rose significantly in comparison with the pre-merger NC-VAMC facility. A high-level gastrointestinal alert was detected in July 2007, but only low-level alerts at other Chicago-area VAMCs. Heat-injury alerts were triggered for the merged facility in June 2006, but not at the other facilities. There was also limited evidence in these events that surveillance of the combined population provided utility above and beyond the VA-only and DoD-only components. Recorded ILI activity for NC-VAMC/Lovell FHCC was more pronounced in the DoD component, likely due to pediatric data in this population. NC-VAMC/Lovell FHCC had two weeks of ILI activity exceeding both the Illinois State and East North Central Regional baselines, whereas Hines VAMC had one and Jesse Brown VAMC had zero.
Conclusions
Biosurveillance in a joint VA/DoD facility showed potential utility as a tool to improve surveillance and situational awareness in an area with Veteran, active duty and beneficiary populations. Based in part on the results of this pilot demonstration, both agencies have agreed to support the creation of a combined VA/DoD ESSENCE biosurveillance system which is now under development.
doi:10.1186/1472-6947-11-56
PMCID: PMC3188469  PMID: 21929813
7.  Enhanced Influenza Surveillance using Telephone Triage Data in the VA ESSENCE Biosurveillance System 
Objective
To evaluate the utility and timeliness of telephone triage (TT) for influenza surveillance in the Department of Veterans Affairs (VA).
Introduction
Telephone triage is a relatively new data source available to biosurveillance systems.1–2 Because early detection and warning is a high priority, many biosurveillance systems have begun to collect and analyze data from non-traditional sources [absenteeism records, over-the-counter drug sales, electronic laboratory reporting, internet searches (e.g. Google Flu Trends) and TT]. These sources may provide disease activity alerts earlier than conventional sources. Little is known about whether VA telephone program influenza data correlates with established influenza biosurveillance.
Methods
Veterans phoning VA’s TT system, and those admitted or seen at a VA facility with influenza or influenza-like-illness (ILI) diagnosis were included in this analysis. Influenza-specific ICD-9-CM coded emergency department (ED) and urgent care (UC) visits, hospitalizations, TT calls, and ILI outpatient visits were analyzed covering 2010–2011 and 2011–2012 influenza seasons (July 11, 2010–April 14, 2012). Data came from 80 VA Medical Centers and over 500 outpatient clinics with complete reporting data for the time period of interest. We calculated Spearman rank-order coefficients, 95% confidence intervals and p-values using Fisher’s z transformation to describe correlation between TT data and other influenza healthcare measures. For comparison of time trends, we plotted data for hospitalizations, ED/UC visits and outpatient ILI syndrome visits against TT encounters. We applied ESSENCE detection algorithms to identify high-level alerts for influenza activity. ESSENCE aberration detection was restricted to the 2011–2012 season because limited historical TT and outpatient data from 2009–2010 was available to accurately predict aberrancy in the 2010–2011 season. We then calculated the peak measure of healthcare utilization during both influenza seasons (2010–2011 and 2011–2012) for each data source and compared timing of peaks and alerts between TT and other healthcare encounters to assess maximum healthcare system usage and timeliness of surveillance.
Results
There were 7,044 influenza-coded calls, 564 hospitalizations, 1,849 emergency/urgent visits, and 416,613 ILI-coded outpatient visits. Spearman rank correlation coefficients were calculated for influenza-coded calls with hospitalizations (0.77); ED/UC visits (0.85); and ILI-outpatient visits (0.88), respectively (P< 0.0001 for all correlations). Peak influenza activity occurred on the same week or within 1 week across all settings for both seasons. For the 2011–2012 season, TT alerted with increased influenza activity before all other settings.
Conclusions
Data from VA telephone care correlates well with other VA data sources for influenza activity. TT may serve to augment these existing clinical data sources and provide earlier alerts of influenza activity. As a national health care system with a large patient population, VA could provide a robust early-warning system for influenza if ongoing biosurveillance activities are combined with TT data. Additional analyses are needed to understand and correlate TT with healthcare utilization and severity of illness.
PMCID: PMC3692747
Surveillance; Influenza; Telephone triage; Veterans
8.  Enabling ESSENCE to Process and Export Meaningful Use Syndromic Surveillance Data 
Objective
The objective of this project is to enable the ESSENCE system to read in, utilize, and export out meaningful use syndromic surveillance data using the Health Level 7 (HL7) v2.5 standard. This presentation will detail the technical hurdles with reading a meaningful use syndromic surveillance data feed containing multiple sources, deriving a common meaning from the varying uses of the standard and writing data out to a meaningful use HL7 2.5 format that can be exported to other tools, such as BioSense 2.0 (2). The presentation will also describe the technologies employed for facilitating this, such as Mirth, and will discuss how other systems could utilize these tools to also support processing meaningful use syndromic surveillance data.
Introduction
In order to utilize the new meaningful use syndromic surveillance data sets (3) that many public health departments are now receiving, modifications to their systems must be made. Typically this involves enabling the storage and processing of the extra fields the new standard contains. Open source software exists, such as Mirth Connect, to help with reading and interpreting the standard. However, issues with reliably reading data from one source to another arise when the standard itself is misunderstood. Systems that process this data must understand that while the data they receive is in the HL7 v2.5 standard format, the meaning of the data fields might be different from provider to provider. Additional work is necessary to sift through the similar yet disjoint fields to achieve a consistent meaning.
Methods
This project utilized 3 separate instances of ESSENCE and BioSense 2.0. For both importing and exporting HL7 v2.x standard files, the project used the open source tool Mirth Connect. For importing data the project adapted versions of Tarrant County and Cook County ESSENCE systems in the Amazon GovCloud to receive meaningful use syndromic surveillance data files sent from BioSense 2.0. For exporting data to BioSense 2.0, the project used Mirth Connect to poll the local version of Cook County’s ESSENCE database and export the data into an HL7 v2.5 file. The resultant file was sent over secure file transfer protocol (SFTP) to BioSense 2.0. The team then evaluated the process by comparing the data in the local instances of ESSENCE and the corresponding instances hosted on the Internet cloud.
Results
Many issues were encountered during the reading of the HL7. While the standard suggests that hospitals and hospital systems would all send data in the same fields for the same data, the reality was far different. Although HL7 v2.5 is a standard and there is a defined use for each field, it can be interpreted in many ways. A large portion of time was spent communicating with the local health department to determine exactly what each field meant for a particular hospital. Comparing the Internet cloud and local versions did have some difficulties due to local filtration rules that eliminated non-ER related records from the local Tarrant County system. The project was able to utilize new query features in ESSENCE to filter down to only ER related records on the Internet cloud version to support the comparisons. The project was able to re-use much of the configuration that was created when moving from one jurisdiction to the other. This will help when describing how others may use the same technology in their own systems.
Conclusions
Reading and interpreting the data consistently from a data feed containing multiple sources can be challenging. Confusion with the HL7 v2.3 or 2.5 standards causes many health organizations to transmit data in inconsistent ways that betrays the notion of a messaging standard. However, with the tools this project have created and the lessons we have learned, the pain of implementing meaningful use syndromic surveillance data into a system can be reduced.
PMCID: PMC3692901
Analytics; Electronic Medical Records for Public Health; Interoperability; Meaningful Use; Internet Cloud
9.  National Collaborative for Bio-Preparedness 
Objective
Demonstrate the functionality of the National Collaborative for Bio-Preparedness system.
Introduction
The National Collaborative for Bio-Preparedness (NCB-Prepared) was established in 2010 to create a biosurveillance resource to enhance situational awareness and emergency preparedness. This joint-institutional effort has drawn on expertise from the University of North Carolina- Chapel Hill, North Carolina State University, and SAS Institute, leveraging North Carolina’s role as a leader in syndromic surveillance, technology development and health data standards. As an unprecedented public/private alliance, they bring the flexibility of the private sector to support the public sector. The project has developed a functioning prototype system for multiple states that will be scaled and made more robust for national adoption.
Methods
NCB-Prepared recognizes that the capability of any biosurveillance system is a function of the data is analyzes. NCB-Prepared is designed to provide information services that analyze and integrate national data across a variety of domains, such as human clinical, veterinary and physical data. In addition to this one-health approach to surveillance, a primary objective of NCB-Prepared is to gather data that is closer in time to the event of interest. NCB-Prepared has validated the usefulness of North Carolina emergency medical services data for the purposes of biosurveillance of both acute outbreaks and seasonal epidemics (1).
A unique model of user-driven valuing of data-providing value back to the provider in their terms-motivates collaboration from potential data providers, along with timely and complete data. NCB-Prepared approaches potential data providers, partners and users with the proposition that enhanced data quality and analysis is valuable to them individually and that an integrated information system can be valuable to all. With the onboarding of new data sources, NCB-Prepared implements a formal process of data discovery and integration. The goal of this process is three-fold: 1) to develop recommendations to enhance data quality going forward, 2) to integrate information across data sources, and 3) to develop novel analytic techniques for detecting health threats. NCB-Prepared is committed to both utilizing standard methods for event detection and to developing innovative analytics for biosurveillance such as the Text Analytics and Proportional charts method (TAP). The sophisticated analytic functionality of the system, including improved time to detection, query reporting, alert detection, forecasting and predictive modeling, can be attributed to collaboration between analysts from private industry, academia and public health.
NCB-Prepared followed the formal software development process known as agile development to create the user interface of the system. This method is based on iterative cycles wherein requirements evolve from regular sessions between user groups and developers. The result of agile development and collaborative relationships is a system which visualizes signals and diverse data sources across time and place while providing information services across all levels of users and stakeholders.
Conclusions
Lessons Learned: Understand the functionality of new biosurveillance system, NCB-PreparedIdentify the benefit of creating collaborative relationships with data providers and usersAppreciate the value of a public/private partnership for biosurveillance and bio-preparedness
PMCID: PMC3692851
Biosurveillance; Analytics; Preparedness; Emergency
10.  Sharing Public Health Information with Non-Public Health Partners 
Objective
The objective of this project is to provide a technical mechanism for information to be easily and securely shared between public health ESSENCE users and non-public health partners; specifically, emergency management, law enforcement, and the first responder community. This capability allows public health officials to analyze incoming data and create interpreted information to be shared with others. These interpretations are stored securely and can be viewed by approved users and captured by authorized software systems. This project provides tools that can enhance emergency management situational awareness of public health events. It also allows external partners a mechanism for providing feedback to support public health investigations.
Introduction
Automated Electronic Disease Surveillance has become a common tool for most public health practitioners. Users of these systems can analyze and visualize data coming from hospitals, schools, and a variety of sources to determine the health of their communities. The insights that users gain from these systems would be valuable information for emergency managers, law enforcement, and other non-public health officials. Disseminating this information, however, can be difficult due to lack of secure tools and guidance policies. This abstract describes the development of tools necessary to support information sharing between public health and partner organizations.
Methods
The project initially brought together public health and emergency management officials to determine current gaps in technology and policy that prevent sharing of information on a consistent basis. Officials from across the National Capital Region (NCR) in Maryland, Virginia, and the District of Columbia determined that a web portal in which public health information could be securely posted on and captured by non-public health users (humans and computer systems) would be best. The development team then found open source tools, such as the Pebble blogging system, that would allow information to be posted, tagged, and searched in an easily navigable site. The system also provided RSS feeds both on the site as whole and specific tags to support notification. The team made modifications to the system to incorporate spring security features to allow the site to be securely hosted requiring usernames and passwords for access. Once the Pebble system was completed and deployed, the NCR’s aggregated ESSENCE system was adapted to allow users to submit daily reports and post time series images to the new site. An additional feature was created to post visualizations every evening to the site summarizing the day’s reports.
Results
The system has been in testing since March of 2012 and users of the system have provided valuable feedback. Based on the success of the tests, public health users in the NCR have begun working on the policy component of the project to determine when and how it should be used. Modifications to the system since deployment have included a single sign on capability for ESSENCE users and the desire to allow other features of ESSENCE to be posted beyond time series graphs, such as GIS maps and statistical reports.
Conclusions
Having tools that can promote exchange of information between public health and non-public health partners such as emergency management, law enforcement, and first responders can greatly enhance the situational awareness and impact overall preparedness and response. By having tools embedded in ESSENCE, users are able to integrate the information sharing aspects into their daily routines with a small amount of effort. With the use of open source tools, the same type of capability can be easily replicated in other jurisdictions. This presentation will describe the lessons learned and potential improvements the project will incorporate in the future.
PMCID: PMC3692892
Open Source; Emergency Management; Information Sharing
11.  Advancing a Framework to Enable Characterization and Evaluation of Data Streams Useful for Biosurveillance 
PLoS ONE  2014;9(1):e83730.
In recent years, biosurveillance has become the buzzword under which a diverse set of ideas and activities regarding detecting and mitigating biological threats are incorporated depending on context and perspective. Increasingly, biosurveillance practice has become global and interdisciplinary, requiring information and resources across public health, One Health, and biothreat domains. Even within the scope of infectious disease surveillance, multiple systems, data sources, and tools are used with varying and often unknown effectiveness. Evaluating the impact and utility of state-of-the-art biosurveillance is, in part, confounded by the complexity of the systems and the information derived from them. We present a novel approach conceptualizing biosurveillance from the perspective of the fundamental data streams that have been or could be used for biosurveillance and to systematically structure a framework that can be universally applicable for use in evaluating and understanding a wide range of biosurveillance activities. Moreover, the Biosurveillance Data Stream Framework and associated definitions are proposed as a starting point to facilitate the development of a standardized lexicon for biosurveillance and characterization of currently used and newly emerging data streams. Criteria for building the data stream framework were developed from an examination of the literature, analysis of information on operational infectious disease biosurveillance systems, and consultation with experts in the area of biosurveillance. To demonstrate utility, the framework and definitions were used as the basis for a schema of a relational database for biosurveillance resources and in the development and use of a decision support tool for data stream evaluation.
doi:10.1371/journal.pone.0083730
PMCID: PMC3879288  PMID: 24392093
12.  Coverage and Timeliness of Combined Military and Veteran Surveillance Systems 
Objective
We determined the utility and effective methodology for combining patient record information from the Departments of Veterans Affairs (VA) and Defense (DoD) health surveillance systems.
Introduction
An objective of the Joint VA/DoD BioSurveillance System for Emerging Biological Threats project is to improve situational awareness of the health of combined VA and DoD populations. DoD and VA both use versions of the Electronic Surveillance System for the Early Notification of Community-Based Epidemics (ESSENCE). With a retrospective outpatient data collection available, we analyzed relative coverage and timeliness of the two systems to understand potential benefits of a joint system.
Methods
We used the US Office of Management and Budget’s core-based statistical area (CBSA) to group data from the respective systems by megapolitan (>1 million), metropolitan (50,000-1 million) and micropolitan (10,000–50,000) areas. We performed frequency analyses and mapped coverage of the VA and DoD medical systems in these CBSAs. To determine comparability, we compared International Classification of Diseases, 9th Revision (ICD-9) code usage from 2007–2010 by age group in the respective systems and then formulated a working definition of influenza-like illness (ILI). We then compared CBSA-level temporal detection timeliness in the two systems for the H3N2 epidemic of 2007–9 and the H1N1 pandemic in 2009.
Results
We identified a total of 939 CBSAs, with generally diffuse geographic coverage by VA facilities and higher concentration in larger metro and mega areas for DoD facilities. Of the 51 mega CBSAs, all have at least one VA facility and 63% have a DoD facility. Coverage is sparser for the metro CBSAs and lighter still for the micro CBSAs (Table 1). Although the VA coverage is greater, in many CBSAs with dual coverage, the DoD visit volume is comparable or greater. Patient age distribution differs sharply, with >85% of the VA patients over 45 years of age compared to 22% of DoD patients. For all CBSAs, the overall VA/DoD visit ratio is 1.92, but the ratios for 0–17 years is 0.004, 18–44 years 0.33, 45–64 years 5.20 and >65 years 11.63.
Based on an analysis of ICD-9 codes used in the two systems, the DoD uses symptom-based ILI codes far more frequently than the VA, especially codes for diseases often seen in children (e.g., otitis media). Analysis of ILI-related codes assigned in both systems led to a common code set for comparative analysis. From applying alerting algorithms to visit counts based on this code set, detection was better in DoD data for 57% and 77% of CBSAs for seasonal and pandemic influenza, respectively, and better in VA data for 37% and 14% of CBSAs (Table 2). The VA system performed better during the typical H3N2 seasonal flu compared to the H1N1 outbreak. The DoD system performed better during the H1N1 pandemic, although outperformed the VA for both.
Conclusions
The coverage analysis demonstrates two complementary surveillance systems with evident benefits to a fused national health picture. The VA system patient volume roughly doubles the DoD system, and provides better geographic coverage in smaller CBSAs; however, the DoD includes younger populations, better coverage in strategic metro areas, and more pre-diagnostic ILI coding. From analysis of both outbreaks, relative timeliness could be improved in 92% of CBSAs with access to both systems, with more information provided in CBSAs where only one type of facility exists.
PMCID: PMC3692883
Syndromic surveillance; Merged systems; Government
13.  Biosurveillance Ecosystem (BSVE) Workflow Analysis 
Introduction
The Defense Threat Reduction Agency Chemical and Biological Technologies Directorate (DTRA CB) has initiated the Biosurveillance Ecosystem (BSVE) research and development program. Operational biosurveillance capability gaps were analyzed and the required characteristics of new technology were outlined, the results of which will be described in this contribution.
Methods
Work process flow diagrams, with associated explanations and historical examples, were developed based on in-person, structured interviews with public health and preventative medicine analysts from a variety of Department of Defense (DoD) organizations, and with one organization in the Department of Health and Human Services (DHHS) and with a major U.S. city health department. The particular nuanced job characteristics of each organization were documented and subsequently validated with the individual analysts. Additionally, the commonalities across different organizations were described in meta-workflow diagrams and descriptions.
Results
Two meta-workflows were evident from the analysis. In the first type, epidemiologists identify and characterize health-impacting events, determine their cause, and determine community-level responses to the event. Analysts of this type monitor information (primarily statistical case information) from syndromic or disease reporting system or other sources to determine whether there are any unusual diseases or clusters of disease outbreaks in the jurisdiction. This workflow involved three consecutive processes: triage, analysis and reporting. Investigation and response processes to disease outbreaks are both parallel and overlapping in many circumstances. In the second meta-workflow type, analysts monitor for a potential health event through text-based sources and data reports within their particular area of responsibility. This surveillance activity is often interspersed with other activities required of their job. They may generate a daily/weekly/monthly report or only report when an event is detected that requires notification/response. There are similar triage, analysis and reporting workflow stages to the first meta-workflow type, but in contrast these analysts are focused on informing leadership and response in the form of policy modification. They are also subject to answering leadership-driven biosurveillance queries.
Conclusions
In these interviews, analysts described the shortcomings of various technologies that they use, or technology features that they wish were available. These can be grouped into the following feature categories:
Data: Analysts want rapid access to all relevant data sources, advisories for data that may be relevant to their interests, and availability of information at the appropriate level for their analysis (e.g., output of interpretations from expert analysts instead of raw data).
Enhanced search: Analysts would like customization of information based on relevance, selective filtering of sources, prioritization of search topics, and the ability to view other analysts searches.
Verification: Analysts want indications of information that has been verified or discarded by other analysts, a trail of information history and uses, and automatic verification (e.g., data quality editing) if possible.
Analytics: Analysts want access to forecasting models, services to suggest analysis methods, pointers to other analysts’ expertise, methods, and reports, and tools for “big data” exploitation.
Collaboration and communication: Analysts want assistance identifying people who may have needed information, real-time chat, the ability to compare analyses with colleagues, and the ability to shield data, results, or collaborations from selected others.
Archival: Analysts want automation to provide lessons learned, methods and outcomes for related events, the ability to automatically improve baselines with analyzed data, and assistance with reporting on interim analytic decisions.
The current understanding of the biosurveillance analyst’s functions and processes, based on the results of these interviews, will continue to evolve as further dialog with analysts are combined with results of evaluations during subsequent phases of the new BSVE program.
PMCID: PMC3692935
biosurveillance; workflow; collaboration; operations
14.  An ISDS-Based Initiative for Conventions for Biosurveillance Data Analysis Methods 
Objective
The panel will present the problem of standardizing analytic methods for public health disease surveillance, enumerate goals and constraints of various stakeholders, and present a straw-man framework for a conventions group.
Introduction
Twelve years into the 21st century, after publication of hundreds of articles and establishment of numerous biosurveillance systems worldwide, there is no agreement among the disease surveillance community on most effective technical methods for public health data monitoring. Potential utility of such methods includes timely anomaly detection, threat corroboration and characterization, follow-up analysis such as case linkage and contact tracing, and alternative uses such as providing supplementary information to clinicians and policy makers.
Several factors have impeded establishment of analytical conventions. As immediate owners of the surveillance problem, public health practitioners are overwhelmed and understaffed. Goals and resources differ widely among monitoring institutions, and they do not speak with a single voice. Limited funding opportunities have not been sufficient for cross-disciplinary collaboration driven by these practitioners. Most academics with the expertise and luxury of method development cannot access surveillance data. Lack of data access is a formidable obstacle to developers and has caused talented statisticians, data miners, and other analysts to abandon the field. The result is that older research is neglected and repeated, literature is flooded with papers of varying utility, and the decision-maker seeking realistic solutions without detailed technical knowledge faces a difficult task.
Regarding conventions, the disease surveillance community can learn from older, more established disciplines, but it also poses some unique challenges. The general problem is that disease surveillance lies on the fringe of disparate fields (biostatistics, statistical process control, data mining, and others), and poses problems that do not adequately fit conventional approaches in these disciplines.
In its eighth year, the International Society of Disease Surveillance is well positioned to address the standardization problem because its membership represents the involved stakeholders including progressive programs worldwide as well as resource-limited settings, and also because best practices in disease surveillance is fundamental to its mission. The proposed panel is intended to discuss how an effective, sustainable technical conventions group might be maintained and how it could support stakeholder institutions.
Methods
Members of a Technical Conventions Group would have experience and dedication to advancing the science of disease surveillance. Primary functions would include: Specify and communicate technical problems facing professionals involved in routine monitoring of population health. Alternative use applications would also be considered, such as the use of epidemiological findings to inform clinical diagnoses.Independently evaluate the utility of proposed analytical solutions to well-defined problems in public health surveillance and confer approval or certification, perhaps on several levels, such as whether results can be replicated with shareable data. Approved solutions might be restricted to commonly available software such as the R language or Microsoft EXCEL.Facilitate sharing of tools and methodologies to evaluate methods and to visualize their results
The framework to be discussed in the proposed panel would be a means of keeping open lines of collaboration and idea-sharing. Overcoming obstacles toward this goal is worthy of a conference panel discussion whether or not it concludes that a conventions group is a viable approach.
Results
Three 15-minute panelist talks are proposed: Background: in-depth description of the dimensions of the problem aboveConstraints facing public health practitioners and requirements for practical analytic toolsStrawman conventions group: role, logistics, inclusiveness, methods of communicating with stakeholders and related organizations and producing/disseminating output.
For the 45 minutes of discussion, the first 15–20 will invite reactions to the first two talks to sharpen the scope of the effort. The remainder of the session will cover the advisability, feasibility, and logistics of an ISDS-based conventions group, and modify the straw-man group concept.
PMCID: PMC3692949
Standards; Data Analysis; Statistical Algorithms; Certification
15.  Evaluation of ESSENCE in the Cloud Using Meaningful Use Syndromic Surveillance Data 
Objective
This project represents collaboration among CDC’s BioSense Program, Tarrant County Public Health and the ESSENCE Team at the Johns Hopkins University APL. For over six months the Tarrant County Public Health Department has been sending data through the BioSense 2.0 application to a pilot version of ESSENCE on the Amazon GovCloud. This project has demonstrated the ability for local hospitals to send meaningful use syndromic surveillance data to the Internet cloud and provide public health officials tools to analyze the data both using BioSense 2.0 and ESSENCE. The presentation will describe the tools and techniques used to accomplish this, an evaluation of how the system has performed, and lessons learned for future health departments attempting similar projects.
Introduction
In November of 2011 BioSense 2.0 went live to provide tools for public health departments to process, store, and analyze meaningful use syndromic surveillance data. In February of 2012 ESSENCE was adapted to support meaningful use syndromic surveillance data and was installed on the Amazon GovCloud. Tarrant County Public Health Department agreed to pilot the ESSENCE system and evaluate its performance compared to a local version ESSENCE they currently used. The project determined the technical feasibility of utilizing the Internet cloud to perform detailed public health analysis, necessary changes needed to support meaningful use syndromic surveillance data, and any public health benefits that could be gained from the technology or data.
Methods
This project investigated database and visualization changes necessary to support meaningful use syndromic surveillance data in ESSENCE. It evaluated the Internet cloud environment and determined the benefits and disadvantages to using this technology as a platform for ESSENCE. This included scalability, performance, and cost analysis of the Internet cloud platform. After using the system for a period of time, the Tarrant County users evaluated the Internet cloud version of the system.
Results
Many technical adaptations to the ESSENCE system were made to support the new meaningful use syndromic surveillance elements. Several optimizations, including a new database schema and cube table structures, were developed to improve performance of ESSENCE in the Internet cloud and incorporating the meaningful use requirements. The Internet cloud platform offered many levels of performance that could alter the ESSENCE user experience. Smaller configurations allowed for 100 concurrent users to experience 16 second response times, whereas larger configurations supported experiences of 2 second response times.
Conclusions
Public health departments are dealing with new meaningful use syndromic surveillance data elements and the cost of maintaining local systems. This collaborative team have researched and evaluated tools, technologies, and solutions that can be used throughout the country.
PMCID: PMC3692900
Electronic Medical Records for Public Health; Interoperability; Meaningful Use; Syndromic Surveillance; Internet Cloud
16.  Localized Cluster Detection Applied to Joint and Separate Military and Veteran Subpopulations 
Objective
We examined the utility of combining surveillance data from the Departments of Defense (DoD) and Veterans Affairs (VA) for spatial cluster detection.
Introduction
The Joint VA/DoD BioSurveillance System for Emerging Biological Threats project seeks to improve situational awareness of the health of VA/DoD populations by combining their respective data. Each system uses a version of the Electronic Surveillance System for Early Notification of Community-Based Epidemics (ESSENCE); a combined version is being tested.
The current effort investigated combining the datasets for disease cluster detection. We compared results of retrospective cluster detection studies using both separate and joined data. — Does combining datasets worsen the rate of background cluster determination?
— Does combining mask clusters detected on the separate datasets?
— Does combining find clusters that the separate datasets alone would miss?
Methods
Cluster determination runs were done with a spatial scan statistics implementation previously verified [1] by comparison with SaTScan software [2] using DoD data from the Biosense system.
Input data files were extracted from a repository of outpatient records from both DoD and VA facilities covering 4 years beginning Jan. 1, 2007. This repository includes over 37 million DoD records and over 86 million VA records. Input files were matrices of daily Influenza-like-Illness (ILI) or gastrointestinal (GI) visit counts. Matrix rows were consecutive days, columns were patient residence zip codes, and entry (i, j) was the number of visits on day i from with zip code j. These files were made for DoD data, VA data, and combined data.
For assessing the alerting burden from combining datasets, three sets of runs were executed using data from three regions, Baltimore/Washington D.C. (dominated by DoD data), Los Angeles (mainly VA data), and Tampa (representation of both). For each region, sets of 1672 daily runs were executed for ILI and GI syndrome data. Lastly, focused runs were done to investigate known outbreaks in New York (GI, Jan–Mar 2010), San Diego (ILI, Dec 2007–Apr 2008 and Fall 2009), and New Jersey (GI, Jan–Mar 2010).
Results
Combining the data sources increased the rate of significant cluster alerting by a manageable 1–10% across run sets. Some clusters found only when the data were combined persisted over several days and may have indicated small events not reported in either system; however, we were unable to validate minor events that may have occurred in past years.
Retrospective looks at known outbreaks were successful in that clustering evidence found in separate DoD and VA runs persisted when data sets were combined. For the New York run, a West Point outbreak was seen in repeated clusters of combined data, beginning days before the event report. However, clustering did not consistently produce alerts before outbreak report dates. In the New Jersey DoD runs, repeated clusters indicated a 10-week GI outbreak at Fort Dix; adding VA data that dominated the record counts gave the same clusters with no added cases, so the DoD event was probably self-contained. The San Diego runs were aimed at detecting unusually severe influenza epidemics in February 2008 and in the fall of 2009, and numerous clusters were found but did not enhance regional disease tracking.
Conclusions
From the analysis, combining DoD and VA data enhances cluster detection capability without loss of sensitivity to events isolated in either population and with manageable effect on the customary alert rate. For cluster detection, there may be many geographic regions where a health monitor in one of the systems would benefit from combined data. More detailed outbreak information is needed to quantify the timeliness/sensitivity advantages of combining datasets. In events examined, clustering itself yielded an occasional but not consistent timeliness advantage.
PMCID: PMC3692743
ESSENCE; Department of Defense; scan statistics; cluster detection; Veterans Administration
17.  SMART Platforms: Building the App Store for Biosurveillance 
Objective
To enable public health departments to develop “apps” to run on electronic health records (EHRs) for (1) biosurveillance and case reporting and (2) delivering alerts to the point of care. We describe a novel health information technology platform with substitutable apps constructed around core services enabling EHRs to function as iPhone-like platforms.
Introduction
Health care information is a fundamental source of data for biosurveillance, yet configuring EHRs to report relevant data to health departments is technically challenging, labor intensive, and often requires custom solutions for each installation. Public health agencies wishing to deliver alerts to clinicians also must engage in an endless array of one-off systems integrations.
Despite a $48B investment in HIT, and meaningful use criteria requiring reporting to biosurveillance systems, most vendor electronic health records are architected monolithically, making modification difficult for hospitals and physician practices. An alternative approach is to reimagine EHRs as iPhone-like platforms supporting substitutable apps-based functionality. Substitutability is the capability inherent in a system of replacing one application with another of similar functionality.
Methods
Substitutability requires that the purchaser of an app can replace one application with another without being technically expert, without requiring re-engineering other applications that they are using, and without having to consult or require assistance of any of the vendors of previously installed or currently installed applications. Apps necessarily compete with each other promoting progress and adaptability.
The Substitutable Medical Applications, Reusable Technologies (SMART) Platforms project is funded by a $15M grant from Office of the National Coordinator of Health Information Technology’s Strategic Health IT Advanced Research Projects (SHARP) Program. All SMART standards are open and the core software is open source.
The SMART project promotes substitutability through an application programming interface (API) that can be adopted as part of a “container” built around by a wide variety of HIT, providing readonly access to the underlying data model and a software development toolkit to readily create apps. SMART containers are HIT systems, that have implemented the SMART API or a portion of it. Containers marshal data sources and present them consistently across the SMART API. SMART applications consume the API and are substitutable.
Results
SMART provides a common platform supporting an “app store for biosurveillance” as an approach to enabling one stop shopping for public health departments—to create an app once, and distribute it everywhere.
Further, such apps can be readily updated or created—for example, in the case of an emerging infection, an app may be designed to collect additional data at emergency department triage. Or a public health department may widely distribute an app, interoperable with any SMART-enabled EMR, that delivers contextualized alerts when patient electronic records are opened, or through background processes.
SMART has sparked an ecosystem of apps developers and attracted existing health information technology platforms to adopt the SMART API—including, traditional, open source, and next generation EHRs, patient-facing platforms and health information exchanges. SMART-enabled platforms to date include the Cerner EMR, the WorldVista EHR, the OpenMRS EHR, the i2b2 analytic platform, and the Indivo X personal health record. The SMART team is working with the Mirth Corporation, to SMART-enable the HealthBridge and Redwood MedNet Health Information Exchanges. We have demonstrated that a single SMART app can run, unmodified, in all of these environments, as long as the underlying platform collects the required data types. Major EHR vendors are currently adapting the SMART API for their products.
Conclusions
The SMART system enables nimble customization of any electronic health record system to create either a reporting function (outgoing communication) or an alerting function (incoming communication) establishing a technology for a robust linkage between public health and clinical environments.
PMCID: PMC3692876
Electronic health records; Biosurveillance; Informatics; Application Programming Interfaces
18.  A Systematic Evaluation of Data Streams for Global Disease Surveillance 
Objective
The overall objective of this project is to provide a robust evaluation of data streams that can be leveraged from existing and developing national and international disease surveillance systems, to create a global disease monitoring system and provide decision makers with timely information to prepare for and mitigate the spread of disease.
Introduction
Living in a closely connected and highly mobile world presents many new mechanisms for rapid disease spread and in recent years, global disease surveillance has become a high priority. In addition, much like the contribution of non-traditional medicine to curing diseases, non-traditional data streams are being considered of value in disease surveillance. Los Alamos National Laboratory (LANL) has been funded by the Defense Threat Reduction Agency to determine the relevance of data streams for an integrated global biosurveillance system through the use of defined metrics and methodologies. Specifically, this project entails the evaluation of data streams either currently in use in surveillance systems or new data streams having the potential to enable early disease detection. An overview of this project will be presented, together with results of data stream evaluation. This project will help gain an understanding of data streams relevant to early warning/monitoring of disease outbreaks.
Methods
Three specific aims were identified to address the overall goal of determining the relevance of data streams for global disease surveillance. First, identify data streams as well as define metrics for the evaluation. Second, evaluate data streams using two different methodologies, decision analysis modeling using a support tool called Logical Decisions® that assigns utility scores to data streams based on weighted metrics and assigned values specific to data stream categories; and a Surveillance Window concept developed at LANL that assigns a window or windows of time specific to a disease within which information coming from various data streams can be determined to have utility. This would obtain a ranked list of useful data streams. Additionally, evaluate data integration algorithms useful for a global disease surveillance system through a review of scientific literature. Finally, validate the top-ranked data streams by application of specific historical outbreaks to determine whether the data streams are capable of providing early warning or detection of the particular disease before it became a large outbreak.
Results
Seventeen categories of data streams were identified that ranged from traditional ones such as clinic/healthcare provider and laboratory records to newly emerging sources of information such as social media and internet search queries. The Logical Decisions® based evaluation of data streams identified 5 data streams that consistently showed utility regardless of the goal of biosurveillance. However, different data streams varied in rank, given different biosurveillance goals, and there is no one top ranked data stream. Surveillance window based evaluation of data streams during disease outbreaks identified data streams that had high utility for early detection and early warning regardless of disease, while others were more disease and operations specific. Additionally, we have built a searchable biosurveillance resource directory that houses information on global disease surveillance systems.
Conclusions
LANL has developed a robust evaluation framework to determine the relevance of various traditional and non-traditional data streams in integrated global disease surveillance. Through the use of defined surveillance goals, metrics and data stream categories, not only have we identified data streams currently in use that have high utility, but also new data streams that could be exploited for the early warning/monitoring of disease outbreaks. Our robust evaluation framework facilitates the identification of a defensible set of options for decision makers to use to prepare for and mitigate the spread of disease.
PMCID: PMC3692853
evaluation; disease surveillance; data streams
19.  Establishing a Federal and State Data Exchange Pilot for Public Health Situational Awareness 
Objective
U.S. Department of Health and Human Services (HHS) Office of the Assistant Secretary for Preparedness and Response (ASPR) partnered with the Florida Department of Health (FDOH), Bureau of Epidemiology, to implement a new process for the unidirectional exchange of electronic medical record (EMR) data when ASPR clinical assets are operational in the state following a disaster or other response event. The purpose of the current work was to automate the exchange of data from the ASPR electronic medical record system EMR-S into the FDOH Electronic Surveillance System for the Early Notification of Community-based Epidemics (ESSENCE-FL) system during the 2012 Republican National Convention (RNC).
Introduction
ASPR deploys clinical assets, including an EMR system, to the ground per state requests during planned and no-notice events. The analysis of patient data collected by deployed federal personnel is an integral part of ASPR and FDOH’s surveillance efforts. However, this surveillance can be hampered by the logistical issues of field work in a post-disaster environment leading to delayed analysis and interpretation of these data to inform decision makers at the federal, state, and local levels. FDOH operates ESSENCE-FL, a multi-tiered, automated, and secure web-based application for analysis and visualization of clinical data. The system is accessible statewide by FDOH staff as well as by hospitals that participate in the system. To improve surveillance ASPR and FDOH engaged in a pilot project whereby EMR data from ASPR would be sent to FDOH in near real-time during the 2012 hurricane season and the 2012 RNC. This project is in direct support of Healthcare Preparedness Capability 6, Information Sharing, and Public Health Preparedness Capability 13, Public Health Surveillance and Epidemiological Investigation.
Methods
In 2011, FDOH approached ASPR about securely transmitting raw EMR data that could be ingested by ESSENCE-FL during ASPR deployments in the state. Upon conclusion of an agreement for a date exchange pilot, data elements of interest from the ASPR EMR were identified. Due to the modular design ESSENCE-FL Microsoft SQL databases were easily adapted by the Johns Hopkins University Applied Physics Laboratory (JHU/APL) to add a new module to handle receipt of ASPR EMR data including code to process the files, remove duplicates and create associations with existing reference information, such as system-defined geographic regions and age groups. Scripts were developed to run on the ASPR server to create and send updated files via secure file transfer protocol (SFTP) every 15 minutes to ESSENCE-FL. Prior ASPR event deployment data was scrubbed and sent to ESSENCE-FL as a test dataset to ensure appropriate receipt and ingestion of the new data source.
Results
EMR data was transmitted through a central server at ASPR to ESSENCE-FL every 15 minutes during each day of the 2012 RNC (August 26–31). In ESSENCE-FL, configuration allowed the data to be queried, analyzed, and visualized similar to existing ESSENCE-FL data sources. In all, data from 11 patient encounters were successfully exchanged between the partners. The data were used by ASPR and FDOH to simultaneously monitor in near real-time onsite medical response activities during the convention.
Conclusions
Timely access to patient data can enhance situational awareness and disease surveillance efforts and provide decision makers with key information in an expedient manner during disaster response and mass gatherings such as the RNC. However, data are siloed within organizations. The collaboration between FDOH, ASPR and JHU/APL made EMR data sharing and analysis more expeditious and efficient and increased timely access to these data by local, state, and federal epidemiologists. The integration of these data into the ESSENCE-FL system created one location where users could go to access data and create epidemiologic reports for a given region in Florida, including the RNC. To achieve these successes with partners in the future, it will be necessary to develop partnerships well in advance of intended data exchange. Future recommendations include robust pre-event testing of the data exchange process and planning for a greater amount of lead-time between enacting the official agreement and beginning data exchange.
PMCID: PMC3692893
Syndromic surveillance; Public health informatics; Data exchange; Federal and state collaboration
20.  The Epidemiologic Vocabulary of the West and the Former Soviet Union: Different Sides of the Same Science 
Objective
The purpose of this project was to develop an English-Russian Epidemiology Dictionary, which is needed for improved international collaboration in public health surveillance.
Introduction
As part of the US Department of Defense strategy to counter biological threats, the Defense Threat Reduction Agency’s Cooperative Biological Engagement Program is enhancing the capabilities of countries in the former Soviet Union (FSU) to detect, diagnose, and report endemic and epidemic, man-made or natural cases of especially dangerous pathogens. During these engagements, it was noted that Western-trained and Soviet-trained epidemiologists have difficulty, beyond that of simple translation, in exchanging ideas.
The Soviet public health system and epidemiology developed independently of that of other nations. Whereas epidemiology in the West is thought of in terms of disease determinants in populations and relies on statistics to make inferences, classical Soviet epidemiology is founded on a more ecological view with the main focus on infectious diseases’ spread theory. Consequently many fundamental Soviet terms and concepts lack simple correlates in English and other languages outside the Soviet sphere; the same is true when attempting to translate from English to Russian and other languages of the FSU. Systematic review of the differences in FSU and Western epidemiologic concepts and terminology is therefore needed for strengthening understanding and collaboration in disease surveillance, pandemic preparedness, response to biological terrorism, etc.
Methods
Following an extensive search of the Russian and English literature by a working group of Western and FSU epidemiologists, we created a matrix containing English and Russian definitions of key epidemiologic terms found in FSU and Western epidemiology manuals and dictionaries, such as A Dictionary of Epidemiology (1), Epidemiology Manual (2) and many other sources. Particular emphasis was placed on terms relating to infectious disease surveillance, analysis of surveillance data, and outbreak investigation. In order to compare the definitions of each term and to elucidate differences in usage and existing gaps, all definitions were translated into English and Russian so that the definitions could be compared side by side and discussed by the working group.
Results
Six hundred and thirty one terms from 27 English and 51 Russian sources were chosen for inclusion based on their importance in applied epidemiology in either the West or the FSU. Review of the definitions showed that many terms within biosurveillance and infectious disease public health practice are used differently, and some concepts are lacking altogether in the Russian or English literature. Significant gaps in FSU epidemiology are in the areas of biostatistics and epidemiologic study designs. There are distinctive differences in FSU and Western epidemiology in the conceptualization and classification of disease transmission, surveillance practices, and control measures.
Conclusions
Epidemiologic concepts and definitions significantly differed in the FSU and Western literature. To improve biosurveillance and international collaboration, recognition of these differences must occur. Detailed analysis of epidemiology terminology differences will be discussed in the presentation and paper. Major limitations of the work were scarcity of prior research on the subject and lack of bilingual epidemiologists with the good understanding of FSU and Western approaches. A bilingual reference in the form of a dictionary will greatly improve mutual comprehension and collaboration in the areas of biosurveillance and public health practice.
PMCID: PMC3692890
Surveillance; Dictionary; Collaboration
21.  Evaluating Biosurveillance System Components using Multi-Criteria Decision Analysis 
Objective
The use of Multi-Criteria Decision Analysis (MCDA) has traditionally been limited to the field of operations research, however many of the tools and methods developed for MCDA can also be applied to biosurveillance. Our project demonstrates the utility of MCDA for this purpose by applying it to the evaluation of data streams for use in an integrated, global biosurveillance system.
Introduction
The evaluation of biosurveillance system components is a complex, multi-objective decision that requires consideration of a variety of factors. Multi-Criteria Decision Analysis provides a methodology to assist in the objective analysis of these types of evaluation by creating a mathematical model that can simulate decisions. This model can utilize many types of data, both quantitative and qualitative, that can accurately describe components. The decision-maker can use this model to determine which of the system components best accomplish the goals being evaluated. Before MCDA can be utilized effectively, an evaluation framework needs to be developed. We built a robust framework that identified unique metrics, surveillance goals, and priorities for metrics. Using this framework, we were able to use MCDA to assist in the evaluation of data streams and to determine which types would be of most use within a global biosurveillance system.
Methods
MCDA was implemented using the Logical Decisions® software. The construction of the evaluation framework was carried out in several steps: identification and definition of data streams, metrics and surveillance goals, and the determination of the relative importance of each metric to the respective surveillance goal being evaluated. Sixteen data streams types were defined and identified for evaluation from a survey we conducted that collected over 200 surveillance products. A subject matter expert (SME) panel was assembled to help identify the biosurveillance goals and metrics in which to evaluate the data streams. To assign values for the metrics, we referenced properties of data streams used in currently operational systems.
Results
Our survey identified sixteen different classes of data streams: Ambulance Records, Clinic/Health Care Provider Records, ED/Hospital Records, Employment/School Records, Established Databases, Financial Records, Help Lines, Internet Search Queries, Laboraotry Records, News Aggregators, Official Reports, Police/Fire Department Records, Personal Communication, Prediction Markets, Sales, and Social Media.
Four biosurveillance goals were identified: Early Warning of Health Threats, Early Detection of Health Events, Situational Awareness, and Consequence Management.
Eleven metrics were identified: Accessibility, Cost, Credibility, Flexibility, Integrability, Geographic/Population Coverage, Granularity, Specificity of Detection, Sustainability, Time to Indication, and Timeliness.
Using the framework, it was possible to use MCDA to rank the utility of each data stream for each goal.
Conclusions
The results suggest that a “one size fits all” approach does not work and that there is no ideal data stream that is most useful for each goal. Data streams that scored more highly for speed tended to rank more highly when the biosurveillance goal is early warning or early detection, whereas data streams that scored more highly for data credibility and geographic/population coverage ranked highly when the goal was situational awareness or consequence management. However, there are several data streams that rank consistently within the top 5 for each goal: Internet Search Queries, News Aggregators, Clinic/Health Care Provider records, ED/Hospital Records, and Laboratory Records and may be considered useful for integrated, global biosurveillance for infectious disease.
PMCID: PMC3692806
evaluation; biosurveillance; multi-criteria decision analysis; data stream; evaluation framework
22.  Data Quality: A Systematic Review of the Biosurveillance Literature 
Objective
To highlight how data quality has been discussed in the biosurveillance literature in order to identify current gaps in knowledge and areas for future research.
Introduction
Data quality monitoring is necessary for accurate disease surveillance. However it can be challenging, especially when “real-time” data are required. Data quality has been broadly defined as the degree to which data are suitable for use by data consumers [1]. When compromised at any point in a health information system, data of low quality can impair the detection of data anomalies, delay the response to emerging health threats [2], and result in inefficient use of staff and financial resources. While the impacts of poor data quality on biosurveillance are largely unknown, and vary depending on field and business processes, the information management literature includes estimates for increased costs amounting to 8–12% of organizational revenue and, in general, poorer decisions that take longer to make [3].
Methods
To fill an unmet need, a literature review was conducted using a structured matrix based on the following predetermined questions: -How has data quality been defined and/or discussed?-What measurements of data quality have been utilized?-What methods for monitoring data quality have been utilized?-What methods have been used to mitigate data quality issues?-What steps have been taken to improve data quality?
The search included PubMed, ISDS and AMIA Conference Proceedings, and reference lists. PubMed was searched using the terms “data quality,” “biosurveillance,” “information visualization,” “quality control,” “health data,” and “missing data.” The titles and abstracts of all search results were assessed for relevance and relevant articles were reviewed using the structured matrix.
Results
The completeness of data capture is the most commonly measured dimension of data quality discussed in the literature (other variables include timeliness and accuracy). The methods for detecting data quality issues fall into two broad categories: (1) methods for regular monitoring to identify data quality issues and (2) methods that are utilized for ad hoc assessments of data quality. Methods for regular monitoring of data quality are more likely to be automated and focused on visualization, compared with the methods described as part of special evaluations or studies, which tend to include more manual validation.
Improving data quality involves the identification and correction of data errors that already exist in the system using either manual or automated data cleansing techniques [4]. Several methods of improving data quality were discussed in the public health surveillance literature, including development of an address verification algorithm that identifies an alternative, valid address [5], and manual correction of the contents of databases [6].
Communication with the data entry personnel or data providers, either on a regular basis (e.g., annual report) or when systematic data entry errors are identified, was mentioned in the literature as the most common step to prevent data quality issues.
Conclusions
In reviewing the biosurveillance literature in the context of the data quality field, the largest gap appears to be that the data quality methods discussed in literature are often ad hoc and not consistently implemented. Developing a data quality program to identify the causes of lower quality health data, address data quality problems, and prevent issues would allow public health departments to more efficiently and effectively conduct biosurveillance and to apply results to improving public health practice.
PMCID: PMC3692854
Biosurveillance; Data quality; Literature review
23.  Incorporation of School Absenteeism Data into the Maryland Electronic Surveillance System for the Early Notification of Community-based Epidemics (ESSENCE) 
Objective
The state of Maryland has incorporated 100% of its public school systems into a statewide disease surveillance system. This session will discuss the process, challenges, and best practices for expanding the ESSENCE system to include school absenteeism data as part of disease surveillance. It will also discuss the plans that Maryland has for using this new data source, as well as the potential for further expansion.
Introduction
Syndromic surveillance offers the potential for earlier detection of bioterrorism, outbreaks, and other public health emergencies than traditional disease surveillance. The Maryland Department of Health and Mental Hygiene (DHMH) Office of Preparedness and Response (OP&R) conducts syndromic surveillance using the Electronic Surveillance System for the Early Notification of Community-based Epidemics (ESSENCE). Since its inception, ESSENCE has been a vital tool for DHMH, providing continuous situational awareness for public health policy decision makers. It has been established in the public health community that syndromic surveillance data, including school absenteeism data, has efficacy in monitoring disease, and specifically, influenza activity. Schools have the potential to play a major role in the spread of disease during an epidemic. Therefore, having school absenteeism data in ESSENCE would provide the opportunity to monitor schools throughout the school year and take appropriate actions to mitigate infections and the spread of disease.
Methods
DHMH partnered with the Maryland State Department of Education (MSDE), local health departments, and local school systems to incorporate school absenteeism data into the syndromic surveillance program. There are 24 local public school systems and 24 local health departments in the state of Maryland. OP&R contacted each local school superintendent and each local health officer to arrange a joint meeting to discuss the expansion of the ESSENCE program to include school absenteeism data. Once the meetings were arranged, OP&R epidemiologists traveled to each local jurisdiction and presented their plan for the ESSENCE expansion. At each meeting were representatives from the local health department, as well as school health, school attendance, and school IT staff. This allowed all questions and concerns to be addressed from both sides. In addition to the targeted meetings and presentations, the Secretary of Health issued an executive order which required all local school systems to sign a memorandum of understanding (MOU) with DHMH. This MOU detailed the data elements to be shared with the ESSENCE program and the process by which this would be shared. While this order made data contribution mandatory, the site visits by the OP&R staff created a working relationship and partnership with the local jurisdictions. Data was collected from all public schools in the state including elementary, middle, and high schools.
Results
As of June 30, 2012, Maryland became the first state in the United States to incorporate 100% of its public school systems (1,424 schools) into ESSENCE. Each school system reports absenteeism data daily via an automated secure FTP (sFTP) transfer to DHMH. Due to its unique properties, Johns Hopkins Applied Physics Laboratory (JHUAPL) designed a new detection algorithm in ESSENCE specifically for this data source. OP&R epidemiologist review and analyze this data for disease surveillance purposes in conjunction with other data sources in ESSENCE (emergency department chief complaints, poison control center data, thermometer sales data, and over-the-counter medication sales data). Integrating school absenteeism data will provide a more complete analysis of potential public health threats. The process by which Maryland incorporated their public school systems’ data could potentially be used as a best practice for other jurisdictions. Not only was DHMH able to obtain data from all public schools in the state, but the process also enhanced collaboration between local health departments and public school systems.
PMCID: PMC3692827
ESSENCE; Surveillance; Absenteeism
24.  Early Detection of Influenza Activity Using Syndromic Surveillance in Missouri 
Objective
To assess how weekly percent of influenza-like illness (ILI) reported via Early Notification of Community-based Epidemics (ESSENCE) tracked weekly counts of laboratory confirmed influenza cases in five influenza seasons in order to evaluate the early warning potential of ILI in ESSENCE and improve ongoing influenza surveillance efforts in Missouri.
Introduction
Syndromic surveillance is used routinely to detect outbreaks of disease earlier than traditional methods due to its ability to automatically acquire data in near real-time. Missouri has used emergency department (ED) visits to monitor and track seasonal influenza activity since 2006.
Methods
The Missouri ESSENCE system utilizes data from 84 hospitals, which represents up to 90 percent of all ED visits occurring in Missouri statewide each day. The influenza season is defined as starting during Centers for Disease Control and Prevention (CDC) week number 40 (around the first of October) and ending on CDC week 20 of the following year, which is usually at the end of May.
A confirmed influenza case is laboratory confirmed by viral culture, rapid diagnostic tests, or a four-fold rise in antibody titer between acute and convalescent serum samples. Laboratory results are reported on a weekly basis. To assess the severity of influenza activity, all flu seasons were compared with the 2008–09 season, which experienced the lowest influenza activity based on laboratory data. Analysis of variance (ANOVA) was applied for this analysis using Statistical Analysis Software (SAS) (version 9.2).
The standard ESSENCE ILI subsyndrome includes ED chief complaints that contain keywords such as “flu”, “flulike”, “influenza” or “fever plus cough” or “fever plus sore throat”. The ESSENCE ILI weekly percent is the number of ILI visits divided by total ED visits.
Time series of weekly percent of ILI in ESSENCE were compared to weekly counts of laboratory confirmed influenza cases. Spearman correlation coefficients were calculated using SAS. The baseline refers to the mean of three flu seasons with low influenza activity (2006–07, 2008–09 and 2010–11 seasons). The threshold was calculated as this baseline plus three standard deviations.
The early warning potential of the ESSENCE weekly ILI percent was evaluated for five consecutive influenza seasons, beginning in 2006. This was accomplished by calculating the time lag between the first ESSENCE ILI warning versus the first lab confirmed influenza warning. A warning was identified if either lab confirmed case counts or weekly percent of ILI crossed over their respective baselines.
Results
For each influenza season evaluated, weekly ILI rates reported via ESSENCE were significantly correlated with weekly counts of laboratory-confirmed influenza cases (Table 1). The baseline of ILI activity in ESSENCE was 1.8 ILI /100 ED visits/week and the threshold was set at 4.1 ILI visits per 100 ED visits/week. The ESSENCE ILI baseline provided, on average, two weeks of advanced warning for seasonal influenza activity. Figure 1 shows that two influenza seasons (2007–08 and 2009–10) were more severe than others examined based on the ESSENCE percent ILI threshold analysis, this result is consistent with the examination of severity of influenza activity based on lab confirmed influenza data (p<0.05).
Conclusions
The significant correlation between ILI surveillance in ESSENCE and laboratory confirmed influenza cases justifies the use of weekly ILI percent in ESSENCE to describe seasonal influenza activity. The ESSENCE ILI baseline and threshold provided advanced warning of influenza and allowed for the classification of influenza severity in the community.
PMCID: PMC3692881
ESSENCE; syndromic surveillance; influenza-like illness (ILI); baseline; threshold
25.  Enhanced Disease Surveillance during the 2012 Republican National Convention, Tampa, FL 
Objective
To describe disease and illness surveillance utilized during the 2012 Republican National Convention (RNC) held August 26–30, 2012 in Tampa, FL.
Introduction
While the Tampa Bay Area has previously hosted other high profile events that required heightened disease surveillance (e.g., two Super Bowls), the 2012 RNC marked the first national special security event (NSSE) held in Florida. The Hillsborough County Health Department (HCHD), in conjunction with the Pinellas County Health Department (PinCHD) coordinated disease surveillance activities during this time frame. This presentation will focus of the disease surveillance efforts of the Hillsborough County Health Department during the 2012 RNC.
In addition to the surveillance systems that are used routinely, the HCHD Epidemiology Program implemented additional systems designed to rapidly detect individual cases and outbreaks of public health importance. The short duration of RNC, coupled with the large number of visitors to our area, provided additional surveillance challenges.
Tropical Storm Isaac, which threatened Tampa in the days leading up to RNC, and an overwhelming law enforcement presence likely dissuaded many protestors from coming to Tampa. As a result, a tiny fraction of the number of protestors that were expected actually showed up.
Methods
Our normal daily analysis of the emergency department (ED) data using the Electronic Surveillance System for the Early Notification of Community-based Epidemics (ESSENCE) was expanded to look in detail at ED volumes and chief complaints of those patients who live outside of a 5-county Tampa Bay area. This analysis used patient zip code to determine place of residence. Additionally, ESSENCE queries were utilized to look for heat, tear gas, and RNC related exposures. The ESSENCE system also receives Poison Control data every 15 minutes. Expanded analyses of the Poison Control data were conducted as well. Two Disaster Medical Assistance Teams (DMATs) were deployed in Tampa during the RNC. Data was collected electronically and transmitted through ESSENCE as well.
The HCHD also asked infection preventionists, health care providers, hotels, labs, and Mosquito Control to lower their reporting threshold to us during the RNC period. We provided updates to all our partners with respect to diseases and outbreaks of public health importance occurring in our county.
Results
No epidemiologic events linked to the RNC were detected through the HCHD’s enhanced surveillance that was conducted. Decreased patient volumes were seen during the RNC at our EDs closest to the convention site. No significant increases in ED visits from outside of our 5-county area were noted during the RNC. Urgent care centers reported seeing patients associated with the RNC for a variety of reasons including respiratory and gastrointestinal illness. DMAT surveillance showed mainly routine visits but four secret service agents did seek care for respiratory illness during the convention.
Conclusions
Substantial time and resources were devoted to disease surveillance in the 6 months leading up to the RNC and during the event. While no epidemiologic events were detected, the public health surveillance infrastructure has clearly been strengthened in our county. We are receiving our ED syndromic data, from many of our hospitals, every two hours as opposed to every day. We have established relationships with our urgent case centers and hope to begin receiving urgent care center data on a daily basis in the near future. Receiving DMAT data through ESSENCE could prove very useful in the future, especially in Florida where hurricanes are always a threat. Lastly the improved relationships with our health care providers should be beneficial as we move forward.
PMCID: PMC3692915
mass gathering; national special security event; convention

Results 1-25 (1166813)