Politics and Regulatory Issues
Two of the biggest challenges in the planning process were accommodating the security and confidentiality mandates of regulatory agencies and obtaining institutional approvals. Our first contact was with our own system's chief information officer, who required that we obtain secondary approval from the chief operating officer at each of the three hospitals and from each of the clinical department administrators ().
Barriers and Responses (Countermeasures) to the Development of eID Clinical Data Warehouse
Keepers of data—often self-viewed as “owners” of the data—have a fiduciary responsibility to maintain data in a confidential and secure manner and to ensure that data are interpreted accurately. Allowing a third party direct access to computer servers may be perceived as compromising data-keeper responsibilities. Our request to create a copy of patient level data for CARP projects challenged the paradigm of data “ownership.” The introduction of the Health Insurance Portability and Accountability Act of 1996 (HIPAA),15,16
which addresses confidentiality of patient level data and the safeguards required for security of electronic data transfer, further complicated our task by introducing another level of scrutiny. HIPAA does not specify how patient data should be protected but does note that institutions and individuals will be held accountable if confidentiality is breached.
To overcome the barriers associated with ownership and confidentiality concerns, we obtained senior administrative-level endorsement of the project. In addition, we informed administrators and data keepers that use of the information by the infectious diseases and infection control divisions of our system's hospitals would help meet regulatory mandates for infection control and improve patient care. Administrative acceptance included approval and endorsement from the chief of the Cook County Bureau of Health Services, hospital attorneys, and department directors. We implemented a security program that included policies, procedures, and technological safeguards to protect patient-level data and to assure compliance with our institutional review board (IRB) requirements. Establishment of the data warehouse was determined to be exempt from IRB review.
The administration of the hospitals' information system data repository is a contracted service through an extramural vendor. This operational practice limited the development of technical expertise in database management within the hospitals' information services staff and restricted their ability to assist us in establishing a connection to the application servers. We learned that staff from each clinical department (e.g., pharmacy, laboratory, medical records, radiology) assisted with the design and management of data in its own server. Understanding the design of the information system software architecture and determining data flow of the two-tiered server configuration were critical to successfully build our clinical data warehouse.
To create links to local servers, we found it essential to use established, or to develop new, relationships with the clinical department directors and staff who managed each local server. In exchange for the time provided by these staffers, we helped them access data. For example, we abstracted pharmacy data about unit-level drug dispensing practices, information the pharmacy needed to plan implementation of automated drug dispensing machines.
Database Content Knowledge
Selecting data elements from each database required knowledge of the database model and the data dictionary and clinical content expertise. Ideally, we would have had the documentation supporting each application, but often this was not available. Once we could view the database contents, clinically trained staff helped interpret the data variables and relational table design. The pharmacy database was relatively simple in design and required abstracting drug data from only one table. In addition, one of the pharmacy directors was able to provide important assistance by directing us to tables containing census information that we needed to report the antimicrobial use per patient days. In contrast, the laboratory database was complicated, and we required assistance from the software vendor's application specialist. The microbiology laboratory database had over 600 tables, only a fraction of which contained data elements relevant to the CARP study. We created one microbiology table by linking ten different tables; some of these tables contained only two variables (e.g., tables that contained encoded data descriptors). After data aggregation and validation, we learned that some laboratory results were archived to other data tables. Much effort was required to review the large number of tables, to determine the meaning of encoded variables, to locate stored variables, and to learn how to link tables and to abstract complete data sets. In addition to the complicated table structure, the microbiology data were partitioned into time-dependent preliminary and final results, and each of the final pathogens had a panel of 1 to ≥10 antimicrobial susceptibility results.
Usability of Data
Few published studies report on the accuracy or usability of clinical warehouse data. Because data accuracy is highly variable,14
methods should be applied for continuous monitoring. In addition, data may be accurate but not in a usable format for computation. For example, in our system's pharmacy tables, the field containing the dose of a medication could be expressed in three different ways: the dose field could be a number representing the actual dosage (e.g., 250 mg of medication); the dose field could represent the volume of fluid to be infused to administer one dose (e.g., 10 mL of medication); or the dose field could indicate the number of units to be administered (e.g., two pills). To address this problem, we had to create a new dose field by having clinical staff manually transform the free text into uniformly comparable data. Once the initial mapping of the 3,247 unique antibiotic dosing combinations in our pharmacies' formularies was completed (approximately 80 person-hours of labor), the ongoing update of new lines of data was minimal but repeated every three months. Another usability problem occurred when the clinical data warehouse encountered nonstandardized data formats. For example, in our pharmacy tables, at times, the drug name also included the preparation strength, which made simple queries by medication name nearly impossible. A third set of usability issues resulted from storage of data as text. The microbiology table contained the results of infrequently performed tests in a free-text comment field that was not abstracted easily. Also, the radiology text-based report files were compressed, and access to them required proprietary extraction software and natural language processing software to interpret these reports.17
Changes in application software and vendors are inevitable, and this requires clinical data warehouse developers to be continually in “development and refinement” mode. provides an overview of our hospitals' information systems, the data sets that we have been able to obtain, and the time frame for new systems. In 2002, the Cook County Bureau of Health Services contracted with a new clinical information system vendor. This evolution in the information systems will require that CARP repeat the data acquisition process ( and ) or access these data from a data repository function planned by the Cook County Bureau of Health Services Information Systems Department.
Having information stored in a database and analyzing these data to answer hypotheses are two separate domains. Although programming simple queries to provide descriptive statistics was performed readily, the desire for applying business intelligence models or analyzing and displaying patient level data in a complex arrangement of variables required more advanced programming and analytic skills. We considered purchasing an Application Service Provider, i.e., a software program developed, marketed, and sold for specific applications, but we found none to meet our specific objectives. We developed business specifications outlining our requirements and solicited bids from four consulting firms. The four responses provided plans to use our existing clinical data warehouse and to program the data into information about the care received by patients with infectious diseases. We assessed these proposals and determined that purchasing analytical services was expensive, that intellectual property would be jointly owned, that the final product (programming code) would be proprietary, and that any product modifications or enhancements would require an ongoing contractual relationship. Entering into long-term contractual agreements was deemed too expansive for the scope of the project and assumed that the consultant could ensure data integrity and provide ongoing product support. In addition, the consultant required the study investigators with content expertise to work intensely with contractors in the design of the application. As an alternate approach, we chose to develop internal resources by employing a statistician and recruiting a master's-level graduate student.