Crossing the Chasm: Information Technology to Biomedical Informatics

doi:10.231/JIM.0b013e31821452bf

J Investig Med. Author manuscript; available in PMC 2012 June 1.

Published in final edited form as:

J Investig Med. 2011 June; 59(5): 768–779.

doi: 10.231/JIM.0b013e31821452bf

PMCID: PMC3137749

NIHMSID: NIHMS279293

Crossing the Chasm: Information Technology to Biomedical Informatics

Brenda G. Fahy, M.D., F.C.C.M., C. William Balke, MD, Gloria H. Umberger, PhD, MSPH, RPh, Jeffery Talbert, PhD, Denise Niles Canales, MA, Carol L. Steltenkamp, MD, MBA, and Joseph Conigliaro, MD, MPH, FACP

Brenda G. Fahy, Professor, Departments of Anesthesiology and Neurosurgery, College of Medicine, University of Kentucky, 800 Rose Street, N-263, Lexington, KY 40536-0293, Phone: (859) 323-5956, extension 80365, Fax: (859 323-1080;

Contributor Information.

Brenda G. Fahy: bgfahy2/at/email.uky.edu; C. William Balke: bill.balke/at/ucsf.edu; Gloria H. Umberger: geanie.umberger/at/uky.edu; Jeffery Talbert: jtalb1/at/uky.edu; Denise Niles Canales: denisecanales/at/uky.edu; Carol L. Steltenkamp: steltenkamp/at/uky.edu; Joseph Conigliaro: jconi2/at/email.uky.edu

The publisher's final edited version of this article is available at J Investig Med

Abstract

Accelerating the translation of new scientific discoveries to improve human health and disease management is the overall goal of a series of initiatives integrated in the National Institutes of Health (NIH) “Roadmap for Medical Research.” The Clinical and Translational Research Award (CTSA) program is, arguably, the most visible component of the NIH Roadmap providing resources to institutions to transform their clinical and translational research enterprises along the goals of the Roadmap. The CTSA program emphasizes biomedical informatics as a critical component for the accomplishment of the NIH’s translational objectives. To be optimally effective, emerging biomedical informatics programs must link with the information technology (IT) platforms of the enterprise clinical operations within academic health centers.

This report details one academic health center’s transdisciplinary initiative to create an integrated academic discipline of biomedical informatics through the development of its infrastructure for clinical and translational science infrastructure and response to the CTSA mechanism. This approach required a detailed informatics strategy to accomplish these goals. This transdisciplinary initiative was the impetus for creation of a specialized biomedical informatics core, the Center for Biomedical Informatics (CBI). Development of the CBI codified the need to incorporate medical informatics including quality and safety informatics and enterprise clinical information systems within the CBI. This paper describes the steps taken to develop the biomedical informatics infrastructure, its integration with clinical systems at one academic health center, successes achieved, and barriers encountered during these efforts.

Keywords: CTSA, information technology clinical and translational research, biomedical informatics

INTRODUCTION

In 2003, the National Institutes of Health (NIH) (Table 1, abbreviations/acronyms used in this manuscript) launched a series of initiatives collectively known as the NIH Roadmap for Medical Research with the goal of accelerating the translation of basic biomedical discoveries into advances in clinical practice.^1,2 The Clinical and Translational Science Award (CTSA) program is one of the largest and most visible of these NIH Roadmap initiatives.³ The CTSA program identifies biomedical informatics as a fundamental component for the achievement of the NIH Roadmap objectives.^4–6 Several studies^4,6 have examined the current status of IT within the research infrastructure of academic medical centers and the potential opportunities afforded by utilizing a full array of informatics tools and services. These studies recommend the development of an integrated informatics strategy consisting of the following key components: a coherent vision, implementation strategy, governance structure, and budget. Linking an emerging informatics program with enterprise clinical operations was identified as being crucial to the ultimate effectiveness of both the research and clinical operations of an institution.⁶ This approach has the potential to leverage constrained resources and align the optimization of clinical operations with the inevitable shift to the priorities of clinical and translational science. Despite the promulgation of these recommendations, a recent survey revealed that academic medical centers had yet to achieve an informatics strategy based on aforementioned components.⁵

Table 1

Manuscript Abbreviations

This report details a transdisciplinary initiative to create an integrated academic discipline of clinical and translational science with a comprehensive informatics strategy. Created in 2005, the University of Kentucky (UK) Center for Clinical and Translational Science (CCTS) emerged from an in-depth campus-wide transdisciplinary restructuring of the institution’s entire clinical and translational research enterprise. As such, it became the impetus for the creation of a specialized biomedical informatics core, the Center for Biomedical Informatics (CBI). The CBI promoted UK’s evolution via improved communication and integration of disparate data sources (clinical, laboratory, regulatory, and administrative). Development of the CBI provoked discussions with senior leadership and other stakeholders over the recognized need to incorporate the academic discipline of biomedical informatics (bioinformatics, imaging informatics, clinical informatics, public health informatics) with the clinical systems and operations of the university’s health care enterprise, UK HealthCare. In this paper, we discuss the strategies employed to develop a comprehensive institution-wide research informatics program built on a shared vision, transparent and inclusive governance structure, and a composite budget integrating contributions from multiple stakeholders. We also review the barriers encountered in UK’s evolution from an IT-focused institution to one offering a bona fide informatics program. Aspects of this strategy can be generalized to other universities who desire to create a transdisciplinary biomedical informatics approach.

MATERIALS AND METHODS

In 1997, UK advanced a strategic and business plan to reach the rank of one of the top 20 public research universities by the year 2020.⁷ The transformation of the institution’s clinical and translational research enterprise along the lines of the NIH CTSA program became an important element in UK’s top 20 quest. While CTSA programs include multiple components deemed necessary for the establishment of the discipline of clinical and translational science, a comprehensive and functional bioinformatics program (Table 2) was a key element. The required development of biomedical informatics infrastructure stimulated a series of dialogues between senior university and medical center leadership and key faculty involved with research, clinical care, and information systems. This dialogue identified university-wide needs, which stimulated the creation of a university-wide biomedical informatics roadmap. A consultant, IBM (Armonk, NY), was engaged to assist in the development of the roadmap that defined the personnel, strategic architecture, implementation strategies, and timeline necessary to achieve UK’s informatics objectives.

Table 2

Clinical and Translation Science Award (CTSA) Application

An early objective of the informatics roadmap was to establish the mission, implementation strategy, and guiding principles with key informatics stakeholders. The CBI’s mission centers on providing UK investigators and community health care researchers and providers with the informatics tools and support necessary to promote the translation of scientific breakthroughs to advance the health and well being of the people of the Commonwealth of Kentucky and beyond. This ambitious goal required UK to evolve beyond locally implemented IT-based activities to an institutionally integrated and comprehensive informatics infrastructure and processes. Consequently, the CBI was developed using the guiding principle of providing an accessible, high-quality biomedical informatics infrastructure to the entire University community. One potential impediment to this goal was the barriers between the academic departments of UK’s colleges and the business operations groups of UK HealthCare. The academic units possessed intellectual property focused on theoretical informatics, while the business and operations staff possessed the applied knowledge and infrastructure to operate informatics tools. The business operations group consisted of UK HealthCare IT resources, while the research resources consisted of faculty and staff from academic departments in computer science; the Colleges of Business, Medicine, and Pharmacy; and the Office of the Provost. The vision for the CBI was to pool these resources together and create an environment that encouraged collaborative projects that focused on both informatics research and informatics tool development. Remarkably, the resulting CBI supported the development of resources to support clinical care, education, and research within and outside of the NIH CTSA. (Figure 1 and Table 2).

Figure 1

Bioinformatics Core Governance

UK assessed the demand for high-level informatics infrastructure using a multi-faceted approach. In 2006–2007, IBM conducted an institutional assessment of current UK informatics capacity and unmet needs and developed the UK biomedical informatics roadmap. The resulting recommendations spanned six categories of activity: 1) operational planning, 2) governance, 3) change management, 4) reporting, analytics, and data mining requirements; 5) data analytics environment; and 6) technical infrastructure. Based on the IBM recommendations, formation of the CBI coalesced faculty from across UK in support of biomedical informatics development. Using SWOT (strengths, weaknesses, opportunities, and threats) and gap analyses at CBI leadership meetings, the CBI faculty formulated project plans based on institutional needs and the needs of their specific research.

Ten principles were formulated that articulate ideas about UK’s informatics vision, roles, governance, standardization, access, priorities, and funding:

Facilitating clinical and translational research by collaborating around a shared biomedical informatics platform,
Establishing a UK-wide, multi-tiered governance structure,
Using informatics to accelerate excellence in research, patient care and education,
Driving toward University-wide standards of data, and infrastructure, without impairing collaboration,
Developing guidelines that protect research integrity and recognizing data as a university asset,
Providing funds and resources to the functions determined by biomedical informatics governance,
Balancing organizational priorities when deploying the roadmap and addressing early value for all constituents,
Implementing the roadmap through a series of progressive leaps,
Defining a multi-channel, stakeholder-specific communication plan,
Adopting a comprehensive and systematic change management program.

The CBI, working with UK academic, clinical, external partners, and members of the National CTSA Consortium, is in its third year of implementing the UK biomedical informatics roadmap. The CBI supports translational research by allowing CCTS investigators to access existing data technologies to readily store, mine, and retrieve data from a variety of sources. This allows for linkage of individual research data sources from biomedical studies, health services, and epidemiologic research, and genomic, proteomic, and sequencing data integrated with clinical information. Specifically, CBI faculty and staff collaborate and provide technical assistance across three critical areas: (1) research and service: support innovative storage, retrieval, and dissemination technologies to apply and advance biomedical knowledge to support clinical care, translational research, education, and administration; (2) education: provide education and training to undergraduate, graduate, professional, and postgraduate students, as well as practitioners in the theory and practice of biomedical informatics as it pertains to translational science; (3) professional and ethical standards: develop and maintain collaborative ties with local, national, and international colleagues to develop and disseminate biomedical informatics research ethical and professional standards.

In addition to technical infrastructure recommendations, the roadmap outlines a governance structure, a comprehensive change management program, and a detailed communication plan. We focused on several highly effective, nationally recognized informatics programs as models of an integrated and expanded CBI infrastructure.

Based on roadmap recommendations, a CBI governance structure linking separate resources and expertise was established. This governance structure includes a senior leadership team; oversight and governance committees; an operations group; and an external advisory board. The senior leadership team meets at least quarterly and includes the Provost, the executive vice president for health affairs, the vice president for research, and selected Deans from the Colleges of Medicine, Public Health, Nursing, and Pharmacy. This team approves high-level informatics recommendations including capital expenditures and funding. It has approved and funded the proposed governance structure, granted authority to the CBI and recruited its new director. The Biomedical Informatics Oversight Committee has responsibility for policy and budget allocation as they pertain to biomedical informatics in the context of overall IT decision-making and prioritization. During its monthly meetings, it establishes policy and allocates budget with the UK HealthCare IT Governance Committee. The UK HealthCare IT Governance Committee focuses on enterprise wide IT decisions and investment prioritization; it advises and provides recommendations to the executive vice president for health affairs on enterprise-wide IT strategy and resource deployment. The Biomedical Informatics Operations Group, led by the CBI director, meets weekly and sets, organizes, establishes standards and priorities, and assigns specific tasks to work groups. This group works with the Regulatory Knowledge and Research Integrity Core, a key function of a CTSA (Table 2), to establish guidelines and policies addressing the Health Insurance Portability and Accountability Act (HIPAA), privacy, and Institutional Review Board issues related to establishing the data warehouse. Work groups are formed on an ad hoc basis to assess and advise specific proposed initiatives (e.g., data integrity, security standards). One example of a created BIC work group is the Center for Enterprise Quality and Safety (CEQS) Data Oversight Committee, which meets monthly. Current members represent clinical practice, quality and safety, finance, administration, IT, and computer science. The group has inventoried UK’s databases and data resources throughout the university and UKHealthCare for inclusion into the data warehouse and has developed a standard procedure for requesting existing clinical and administrative data for quality improvement, research, administration, and health care delivery.

The External Advisory Board, which reports to the CBI director, includes a group of interdisciplinary advisors external to the CBI from well-established informatics programs and CTSA programs. The Board meets biannually to discuss CBI initiatives, grants and strategies and to address both issues raised by UK informatics thought leaders and issues deemed of significance by the Advisory Board members themselves. To date, this group has proved critical in discussions related to: location of biomedical Informatics within the institution, director appointment, shaping informatics aims, and curriculum development planning. In 2010, the External Advisory Board reviewed a CBI self-assessment report and copies of grant proposals submitted by CBI faculty as part of the biomedical informatics development and participated in workshops with CBI leadership, informatics investigators, and UK and UK HealthCare senior leaders. The resulting report findings provided significant impetus for moving forward.

The CBI provides CCTS researchers with the resources to fulfill the objectives of the NIH CTSA program by (1) providing opportunities and informatics resources to study methods and approaches to clinical and translational science; (2) collaborating between departments and schools at UK and with corporate partners [Eclipsys (Eclipsys Corporation, Atlanta, GA), IBM]; (3) providing portals for partnerships with community physicians, industry, and foundations; and (4) providing research education, training and career development in biomedical informatics for clinical and translational researchers (including physicians, nurses, dentists, pharmacists, and other health professionals). The CBI also participates in activities with other established CTSA programs at Vanderbilt University, the University of Pittsburgh, and Harvard University.

To leverage resources, an inventory of relevant clinical, administrative, and health-related databases located at or affiliated with UK was performed. To assess the needs of community based health professionals qualitative methods were used (focus groups, surveys). UK’s four practice-based research networks in ambulatory care, oral health, rehabilitative medicine, and public health, which link academic clinicians with their counterparts in community settings and which form a vital link in the health communication network were evaluated. Biomedical informatics infrastructure and sharing were identified in both workshop sessions and a conference evaluation as priorities.

Several existing databases were integrated and federated and a university-wide data warehouse was developed to leverage resources. The current data warehouse infrastructure consists of an Oracle 11g database using SAN Storage for both the Development and Production Environment along with our Extract, Transform and Load Environment. Informatics is our chosen Extract, Transform and Load tool from which the central database is loaded and operated.

The data warehouse is designed to support two general user systems within the University community, research and operations/business. The research user environment uses a trusted agent model for de-identification and access via the CCTS e-Workbench (see below). Operationally, the data warehouse allows researchers to easily and quickly find subjects across disparate systems, including complex multi-criteria selection and “refocus” the database to concentrate on key areas such as patient, disease state, provider, diagnosis, procedure, protocol, biomarker, tissue sample, clinical trial or grant, intervention, or other user-defined area. Design features include mapping to recognized standards and extension to multiple disease states in an integrated manner. A high priority is compliance with governmental and organizational privacy requirements.

An operations/business user environment provides with tools for dashboards, multi-dimensional analysis, data mining, and ad-hoc queries. UK users and authorized UK collaborators can extract data from internal and external sources with tools and applications available via an innovative web portal, eWorkBench. Accessed through the CCTS web site, this portal allows UK users including CCTS faculty and staff as well as collaborators flexible, efficient and effective means of collaboration and interaction with data. Large patient or disease state datasets can be analyzed using a concise, flexible, easy-to-use unit focused on reporting, analysis, and data mining. This knowledge portal provides research data mining, public health-related applications access to various public databases, presentation tools to interact with data, and a UK research portal to facilitate research and allow collaboration with UK and other researchers via the web. The eWorkBench has access to other operational systems, which include the electronic medical record (EMR) and financial data. This approach provides an enhanced collaborative working environment reducing dependency on other communication systems (electronic or postal services mail, fax) while enhancing other means of communication including online discussion forums, interactive multimedia-rich online meetings, and continuing medical educational learning.

The creation of this data model has proceeded through four primary activities: (1) data discovery and analysis to identify data sources and analyze content and source standard common to multiple sources; (2) establishing a common data model to ensure individual fields within the same domain are consistent and data structures compatible; (3) data integration to either load data into the data warehouse or associate data from larger information sources; and (4) establish a presentation format through which users interact with data.

Access to the data warehouse (Figure 2) is granted to UK CCTS staff, researchers and collaborators and/or UK faculty or staff with appropriate certification and Institutional Review Board approval for any research use. CCTS researchers are granted access to data through a process requiring certification and completion of a data use agreement consistent with requirements of their work or research. As a condition of CCTS membership, a researcher must sign a commitment, which is reviewed and approved by the CCTS to maintain the security and confidentiality of all CBI information systems. Detailed user guides for accessing the inpatient and outpatient datasets, data security and other relevant topics are maintained by the CBI. Maintaining data in a central location that is specifically designed for research purposes facilitates compliance with HIPAA privacy and security rule requirements. The CBI and the appropriate CCTS key functions (Table 2) ensure that appropriate members of the workforce with access to protected health information follow HIPAA rules regarding use or disclosure of information. Our approach to HIPAA compliance in research includes use of subjective de-identification methodologies, limited data sets and data use agreements, and certifications for reviews conducted preparatory to research. The public portion of data in the central warehouse is made anonymous. Any links to non-anonymized sources are under strict HIPAA and Institutional Review Board control. To ensure patient confidentiality, we implemented a Trusted Agent, both automated and manual, that fully meets HIPAA requirements and use-reporting tools that enforce rules returning only properly aggregated data.

Figure 2

Functional Relationships of University of Kentucky High Performance Analytical Data Warehouse (HPADW)

RESULTS

CBI Outcomes

As a consequence of the centralization of resources and personnel, the CBI produced several important returns on investment. The CBI and its integration of previously disparate informatics tools effectively aligned the informatics goals of the CCTS and UK with the overall national CTSA efforts. In addition, the CBI drove (1) a marked enhancement of informatics infrastructure (e.g., federation of databases, creation and implementation of a high powered analytical data warehouse, and a centralized portal for data access), (2) formation of collaborative and matrixed relationships within UK, regionally and nationally, and (3) improved educational offerings.

Informatics Integration

CBI activities encompass but are not limited to bioinformatics (genomics and proteomics), medical informatics, public health informatics, and quality informatics (outcomes and performance measures). The CBI also provides a wide range of resources and opportunities for collaboration between individual faculty members and other entities including UK centers (e.g., CEQS), the clinical enterprise (e.g., UK HealthCare), and the local and national research communities.

Through the CBI work group and UK IT, the CEQS continues to develop systems for reporting and assessing quality and safety. Outcomes and other research databases including the National Surgical Quality Improvement Program, Acute Physiology & Chronic Health Evaluation, and National Trauma and Tumor Registries provide significant resources and outcome data for translational and operational research.

The CBI creation was an important factor in heightening awareness of the need for IT to support the mission of the CBI and CCTS. This resulted in expanded job descriptions and hiring of two new individuals to serve in these newly defined and larger interactive roles, the UK HealthCare enterprise chief information officer with a total staff of 127 and university chief information officer with a direct reporting staff of 6 individuals. In addition, to date, four external faculty recruitments with expertise in several areas in data mining, database architect, computational modeling, and an informaticist. Recruitment of internal faculty included 8 individuals to date with expertise including genomics, analysis of single nucleotide polymorphism data, classification of Internet information-based resources, quality, safety and public health informatics, data mining, and knowledge management. Other important recruitments included a data architect with expertise in designing and implementing data warehouses.

The CBI was stimulated by the CTSA initiative however the CBI represents a large integrated array of activities with the CCTS serving as one of its customers. The CBI is thus a multidisciplinary center composed of faculty and staff from across the university and UK HealthCare fostering integration of resources and fostering broader collaborations and serves as one example of the benefits to the larger university system of the CBI creation simulated by the CTSA initiative. CBI faculty members have academic appointments in various UK colleges encompassing a multitude of interests, disciplines, and activities. The CBI supports UK’s goal for the expansion of its clinical and translational research enterprise as well as provides support for the CBI and other CCTS key functions (Table 2) including community outreach and educational initiatives.

Enhanced Infrastructure

Federation of existing databases

As described in the methods section, existing relevant databases were inventoried and then integrated and federated into a data warehouse. Early integration included a partnership with the Institute for Pharmaceutical Outcomes and Policy (IPOP). The IPOP is a research center located in the College of Pharmacy that conducts health outcomes research and evaluations, develops business analytic tools, and implements decision support systems for health care organizations and providers. The IPOP maintains several separate databases (relational when warranted) designed to efficiently support a wide range of projects. Essential IPOP activities for CBI include monitoring data flow, maintaining and managing records/databases, tracking databases usage, data audits, and developing procedures for data collection training. It also provides expertise in design, maintenance and management of data and systems, performs analyses, and participates in web development. For example, the data management component can assist investigators’ sharing and transferring data amongst projects.

State and national data sets needed to assist investigators (e.g., Medicare and Medicaid data, death certificates, voter registration lists, etc.) are acquired, housed, and maintained by IPOP. IPOP’s wider scope involves development and maintenance of a mental health hospitals’ performance measure reporting system, and a web site/portal for real time reports and outcome measures for other entities including the Kentucky Department for Behavioral Health, Development and Intellectual Disabilities and the Kentucky Department for Medicaid Services. For example, national hospital data is collected using a custom-designed Performance Measuring System for behavioral healthcare for over 200 hospitals. Data collected and accessible to researchers include Medicaid data; patient/client demographic and service data from community mental health clients; demographic, admission, discharge, and treatment data on patients in state mental hospitals; data from disease management and medication therapy management programs; survey results (e.g., Brief Psychiatric Rating Scale); and the Kentucky Medicaid Health Plan Employer Data and Information Set Patient and Provider Survey.

Utilizing IPOP as one resource, the CBI supports translational research by allowing UK investigators to access existing and emerging computer and data technologies to readily store, mine, and retrieve data from a variety of sources. This allows linkage of individual research data sources from biomedical studies, health services, and epidemiologic research, and genomic, proteomic, and sequencing data integrated with clinical information.

Other CBI databases for federation and integration involve pathology bioinformatics with the UK Biospecimens Core and Metadata Framework for Digitized Pathology Images. The UK Biospecimens Core acquires, processes, stores, dispenses, and annotates biological specimens with pertinent demographical, epidemiological, and clinical-pathological information. This core system to study biomarkers has broad-based support from the Kentucky Lung Cancer Research Program, the National Center for Research Resources under the KY-Institutional Development Award Networks of Biomedical Research Excellence, and the Markey Cancer Center’s Cancer Bioinformatics CBI Division. The latter maintains a statewide population-based cancer registry as part of the National Cancer Institute’s Surveillance, Epidemiology, and End Results Program.

The UK Biospecimens Core developed and implemented an open-source relational database specimen tracking system,⁸ caTISSUE. This system was transferred to the National Cancer Institute Cancer Biomedical Informatics Grid⁹ with further funding to develop a novel metadata framework for pathologic imaging data. The CBI is creating a standardized metadata framework for pathologic images in collaboration with the UK Markey Cancer Center Tissue Procurement Service and Massachusetts General Hospital. ⁹ This system will allow well-described, integrated biomedical imaging information to be efficiently stored, managed, retrieved, and shared in compliance with federal regulations. Standardization of the data set information is required. Domain experts from pathology, imaging, ontology and library science determine relevant test image descriptions and construct standards that effectively represent the imagery information. Although biomedical images and visual findings are increasingly captured and stored in digitized formats, pathologists’ access is limited because digitized images are rarely associated with meaningful imaging information.¹⁰ This novel metadata framework will support biomedical imaging at currently unprecedented levels translating core concepts in information. Current development includes cataloging, classifying, authority and access control, subject analysis, arrangement and display, and vocabulary control specifically for non-traditional “library” collection, pathologic images.¹¹

High Performance Analytical Data Warehouse (HPADW)

The Provost and UK HealthCare are jointly supporting the development and implementation of a high performance analytical data warehouse (HPADW) to form the basis of UK’s data analytics environment.¹² The IPOP and CBI have developed systems to extract clinical, financial and research data from existing source systems to form the HPADW (Figure 2).

The HPADW supports existing core measures and quality indicators and permits future development of automated data collection and formatting for reporting. The CEQS identifies process improvement opportunities while the CBI can develop measures to permit factors analyses and impact assessment. For example, CBI, CEQS, and College of Pharmacy investigators use HPADW to perform aggregate analysis of improvement measures and automate processes of identifying incidents and potential incidents for manual review.¹³

Computer Science and College of Engineering research strengths in information visualization enhance the utility of the HPADW infrastructure. An effective information visualization approach will provide immediate data comprehension and allow researchers and clinicians to interactively explore and link data across multiple databases. The scale and complexity of the system are challenging for data management, analysis, and information presentation.

A high priority for HPADW analytical environment is compliance with governmental and organizational privacy requirements. The measures described ensure access to the data warehouse is granted to appropriate individuals and is compliant with HIPAA privacy and security rules. To aid in the development of data access policies and facilitate technical and workflow approaches to access control, UK HealthCare IT, the CCTS and the CBI have convened a data governance board with representation from users and data owners; both operational and research. To date this group has established data use agreements with data owners and the CBI and is currently working with the UK Institutional Review Board in the development of policies for research access including use for research in preparation and for the purpose of identifying and following research subjects over time.

Collaborative Infrastructure and Relationships

The CBI has played a pivotal role in the development of a collaborative infrastructure and relationships at the University, state and national levels.

University

The CBI provides crucial support to a number of university-wide groups. The CBI provides integration of CCTS informatics with every CCTS key function, and provides the CCTS a centralized data-coordinating center providing financial support for its director, data manager, and web designer to integrate personnel. Data management activities are implemented by appropriate center staff including the data manager, Statistical Analysis Software (Cary, NC) programmer, and web designer. This data management group develops standard operating procedures. Data entry systems available include web entry, research electronic data capture, computer aided telephone interview, manual double data entry, and hand-held tablets/personal digital assistants. Newer data collection modalities are implemented as appropriate. Databases are set up in Oracle (Redwood Shores, CA), Microsoft Structured Query Language Server, MySQL, or Statistical Analysis Software. Statistical Analysis Software/IntrNet allows reports and data to investigators using the web.

University collaboration has also resulted in NIH-funded Alzheimer’s Disease Research Center Biostatistics and Data Management Core. This core, the first of its type in the country, is dedicated to biostatistical design and data management. This allowed the transformation from a primitive system with inconsistent and incompatible fields into a centralized relational database with unique identifiers, capable of data abstraction.

The Department of Decision Science and Information Systems is another example of collaboration within the School of Management, Gatton College of Business and Economics. Through CCTS efforts, the Department of Decision Science and Information Systems departmental faculty collaborate with CBI researchers bringing in the additional faculty with expertise in knowledge management and ontology.^14–20

Using the new central UK Biospecimens Core for tissue repository, the Medical Outcomes of Mothers and Infants registry will integrate clinical data from UK HealthCare’s EMRs with research specific data captured outside the university’s system and biospecimens. Structured data entry templates capture relevant obstetric clinical data within the EMR and reside within the HPADW with CBI administration. The tissue repository in the central Biospecimens Core will enable comprehensive annotation and tracking with CBI development of a web-based front research specific data entry. CBI will provide the reporting facilities for this registry as one of the reporting tools that support the entire HPADW. This coordinates processes for investigators’ access to clinical data as well as maternal and pregnancy related biospecimens.

A new IBM supercomputer was purchased by UK and is housed in the Center for Computational Sciences. One of the fastest university supercomputers in the world (peak performance 16.3 teraflops), it can analyze large datasets, such as genomic data, whose analysis requires a supercomputer. This center offers other high-performance computing resources to university researchers through the National Science Foundation TeraGrid. Using high-performance network connections, the TeraGrid integrates high-performance computers, data resources and tools, and high-end experimental facilities around the country.

Since the creation of CBI, seven grants directly affiliated with the CBI have been submitted with five receiving funding. Funding agencies include Health Resources and Services Administration, Institute of Museum and Library Services, Library of Medicine, Office of the National Coordinator for Health Information Technology, and National Institute of Diabetes and Digestive and Kidney Diseases. The total awards to date exceed $9,000,000. The applications yet to be funded included additional requests for NIH funding.

State

The NIH-funded Kentucky-Institutional Development Award Network of Biomedical Research Excellence (Bioinformatics Core) is a collaborative effort between UK, the University of Louisville and other four-year Kentucky institutions as outreach partners. Directed by UK, the Bioinformatics Core develops and maintains centralized genomics and bioinformatics facilities with open access tools and resources around bioinformatics issues, particularly DNA sequence and microarray data analyses via a core facility. The UK Microarray Core Facility provides data analysis for all microarray experiments and has processed thousands of Affymetrix (Santa Clara, CA) chips from over 100 basic and clinical labs in over 20 departments. Researchers are advised on experimental design and all chips are entered in the UK-Stanford Microarray Database. This unique combination of high quality basic and clinical science microarray data in an established database allows UK to integrate current and future clinical data into the data warehouse providing a national model for such integration.

The Markey Cancer Control Program supports the National Cancer Institute funded Surveillance, Epidemiology, and End Results population web based Kentucky Cancer Registry. This registry provides public access to cancer incidence and mortality Geographic Information System and is used by all Kentucky hospitals. The registry is implementing electronic pathology reporting in over 30 pathology laboratories. A CBI core member chairs the pathology data work group for the North American Association of Central Cancer Registries that defines the international Health Level 7 standard for e-path transmissions to cancer registries.⁸ The Kentucky Cancer Registry defines cancer registry data standards in National Cancer Institute’s Cancer Data Standards Repository and caTISSUE Suite where specimens are linked and annotated with registry, pathology, epidemiological and other datasets. The CCTS effort was the impetus that facilitated participation of the Markey Cancer Center in CBI research projects including statewide development of the UK Women’s Health Registry application.

A concerted effort to establish a statewide EMR system was undertaken among providers, payers, legislators, and other stakeholders in 2005 alongside UK’s efforts to create an EMR. Legislation created the Kentucky eHealth Network, a secure, interoperable statewide electronic health network. A CBI co-director serves as chair of the Kentucky eHealth Network Board charged with implementation and oversight of the state electronic health network. The state has assets to implement an innovative model and conduct evidence-based research to improve healthcare with the governing infrastructure with UK leading the research component in conjunction with the University of Louisville.

The Kentucky Health Information Exchange will permit the exchange of patient consented data among providers, and provide tools for decision support and disease management as well as bio-surveillance. As disease specific human genome markers become available, it will support predictive modeling for preventive health based on genetic information. The end result will allow physicians to access the patient’s complete medical information along with medical decision support tools, alerts and current evidence based research from an examination room. This approach will eliminate redundancy, prevent medical errors, and speed correct patient decisions. CBI faculty will study the impact, the return on investment, and improvements in disease outcomes with this system.

The CBI supports collaborative biomedical informatics research efforts with outreach missions using Internet-based training of community-based practitioners in subject recruitment and community-based research. The CBI is expanding clinical research support in a distributed rural network, including Appalachia, by building upon the regional project like the Marty Driesler and Kentucky Lung Cancer Network programs. Clinicians participating in the community-based practitioner Kentucky Ambulatory Network are provided with a link to the UK HealthCare EMR to assist providing quality patient care and facilitate the redesign of healthcare processes. The UK HealthCare EMR improves access to clinical information by nature of its web-based availability, improves patient care efficiency, and decreases medication errors with computer-based physician order entry¹⁴. Interdisciplinary teams continue to implement clinical applications to support a longitudinal EMR, provide data for clinical and administrative decision-making, and provide a common clinician user view of records location independent. This electronic process improves patient care, particularly the coordination of chronic care between primary care provider and specialist providers by facilitating information exchange and patient care coordination²². Although early on in this process with formal measurements yet to be done, our organizational experience suggests results similar to Menachemi et al. that users of IT applications, especially EMRs, are generally satisfied with the technology and are more likely to be satisfied with their medical practice²³. These efforts are directly linked to the CBI’s statewide EMR efforts. Understanding and addressing related workflow, change management, communication, human-computer interface issues and developing methods to evaluate models and systems, including health services research and data mining are all CBI roles.

The CBI links practitioners electronically to share information providing a secure link as well as support sites for subject and patient data, specimen collection, and follow up. Initial efforts have demonstrated the ability of current university-based systems (e.g., central tissue-banking support, secure local collection of patient medical information), community providers’ systems usage, and CCTS opportunities to efficiently and effectively supply these resources.

National

The CBI has established a collaborative relationship with several centers as a founding member in 2007 with Johns Hopkins University and University of Michigan of the Academic Medical Center Clinical Systems Collaborative. This collaborative brings together academic medical centers deploying Eclipsys applications to share non-confidential information and knowledge regarding strategies, experiences, and best practices for deploying Eclipsys software systems; brings forward to Eclipsys and other industry leaders strategies, ideas, and approaches that maximize successful deployment; and promotes collaboration through periodic meetings and research. Current members include Yale, NIH, Boston Medical, Memorial Sloane Kettering, Barnes Jewish, University of Pennsylvania, Hospital for Special Surgery, New York Presbyterian and New York University. Several existing CTSAs use Eclipsys facilitating future collaborations. The university engages in national benchmarking through participation in the University Health System Consortium, a consortium of academic medical centers organized, in part, around benchmarking quality and outcome performance.

UK supports bi-directional communication through secure, high bandwidth data transfer and collaboration portals for data sources of external institutions and partners, such as the National CTSA Consortium, the primary care practice-based research network Kentucky Ambulatory Network, and Cancer Biomedical Informatics Grid^™. This feature is critical to enable data and resource sharing. To support coordination of multi-site NIH trials and industry trials expanded clinical trials management systems, in August 2008 the CBI joined Vanderbilt University and other collaborating institutions in the research electronic data capture consortium project. Research electronic data capture is a software toolset and workflow methodology to manage research data for research studies. The consortium, initiated in August 2006, has shared the software and methodology with the larger research community of CTSA, General Clinical Research Center and Research Centers in Minority Institutions participating institutions. The consortium is open to any academic institution, and members are provided software and support at no charge with the expectation that participants will contribute back to the overall project. The product is a secure, web-based application that is flexible enough to be used for any type of research. It provides an intuitive interface for users to enter validated data, and an export mechanism for end-of-study export of data to common statistical packages (SPSS, SAS, Stata, R/S-Plus). The CTSA will be one entity utilizing research electronic data capture. At this writing UK has 117 research electronic data capture users, 98 separate research databases with 7,353 total fields and 130 surveys with nearly 11,000 survey responses (Table 3). The CBI will partner with private industries to form a public-private link – linking UK to the community to conduct research. To assist with these linkages, the CCTS developed a web site to provide information and a central repository for its constituents.

Table 3

Research Electronic Data Survey and User Statistics

Centralized Web Portal, eWorkbench

The CCTS innovative online web portal, eWorkbench, in addition to fostering CCTS communication also provides a cornerstone of communication and data sharing within a broad network of community collaborators. The eWorkBench assist UK’s statewide and regional engagement at community-based hospitals and participation in Kentucky eHealth initiatives by creating an integrated biomedical informatics infrastructure.

Application requests for CCTS pilot and collaborative project funding are issued through the eWorkbench, which also maintains a listing of other relevant pilot funds sources. The infrastructure is supported by enterprise and university IT and functions to provide critical information systems support to implement appropriate processes for funding categories, particularly in regards to investigator-specific research development support, ad hoc review, and award and administration.

Biomedical Informatics Related Education and Training

The CBI focuses on three areas for related education and training including informatics and bioinformatics classes offered through the university, outreach education and training, and collaborative educational programs. The university offers a wide variety of biomedical informatics classes. Grouped in contextual areas based on a prior needs assessment and the AMIA 10×10 program²¹; a sample of educational offerings is listed in Table 4.

Table 4

Sample of Informatics, Information Technology and Systems Courses Offered at the University of Kentucky

The CBI recently reviewed the university’s education and training resources and identified a high demand in biomedical informatics, signifying the need for a structured academic program. The CBI is expanding biomedical informatics by developing biomedical informatics courses, and outreach and collaborative training programs to address these academic demands. As a result, CBI faculty provides biomedical informatics instruction designed to educate a cadre of researchers and professionals with multidisciplinary backgrounds and with substantial understanding of the principles and applications of computational technology. The current interdisciplinary biomedical informatics certificate program represents the initial offering of a formal degree program. The CBI is maturing this nine credit hour certificate program to an interdisciplinary Master of Sciences level degree with future plans to offer a formal PhD degree program. Currently there are eight students enrolled in the initial three credit hour survey course.

Other educational offerings include the CBI Seminar Series where university biomedical informatics researchers from multiple colleges present their research and develop research collaborations. These seminars, with attendance of 20 – 50 participants, promote cutting edge biomedical informatics bringing the biomedical community together as a research hub. In addition, the university continues to develop research seminars and invite nationally and internationally renowned scholars and educators to enhance student, faculty, and staff learning in biomedical informatics. These efforts support clinical and nonclinical areas and bioinformatics and medical informatics. Collaborating with existing imaging facilities, the CBI also identified the need to develop short courses introducing clinical imaging modalities. The CBI supports the collaboration of several Colleges in developing information management classes offered through CBI training programs. In addition, the College of Communications and Information Studies is proposing an undergraduate information studies program within its School of Library and Information Science. This program would include the creation, organization, and preservation of digital materials related to health IT. By connecting three core components--health information, people, and technology--the CBI is the university’s educational hub of biomedical informatics.

The CBI is also expanding and formalizing training and education programs by collaborating with nationally recognized institutions and experts, and will support professional development in biomedical informatics. The CBI has created a collaborative education liaison with informatics faculty at the University of Pittsburgh with trainees participating in the annual Advancing Practice, Instruction and Innovation through Informatics Conference organized and held at the University of Pittsburgh. In addition, two members of the CBI have participated in the Marine Biological Laboratories at Woods Hole Biomedical Informatics fellowship program, a competitive fellowship sponsored by the National Library of Medicine to develop institutional leaders in biomedical informatics.

DISCUSSION

Barriers

The CBI and its integrative approach faced several barriers in its evolution from concept to durable program. An initial barrier was finding the funds for the informatics integrated road map. The most common barrier for researchers to acquire currently available technology tools has been shown to be related to the financial burdens caused by unmet institutional support or imposed on small laboratories for this technology.⁴ Development of a nexus for university-wide support for biomedical informatics, the CBI, was crucial to leverage financial resources to overcome this barrier. Early engagement of senior leaders helped ensure UK’s commitment to significant funding to leverage IT resources effectively. The consistent commitment of senior leadership in spite of emerging threats including financial constraints of the economy is crucial. UK committed significant funding exceeding $10 million over 5 years to expand and enhance biomedical informatics capabilities.

The federation of existing databases, a central CBI task, presented and continues to present challenges in terms of redundancy, inconsistency, uncertainty, and security. Use of detailed, standardized descriptive medical terminology (e.g., Health Level 7, current procedural and coding terminology, systematized nomenclature of medicine, logical observation identifiers names and codes) is required for meaningful analyses when combining data from multiple sites. The metadata framework for digitized pathologic images serves as a specific example of barriers in developing a single system to provide all the data elements to adequately describe pathologic images. Single system development involves integrating biomedical imaging information and addressing critically important needs of the biomedical imaging community for metadata tools supporting comprehensive biomedical image libraries and the data elements required to adequately describe pathologic images. Initial steps included a review to validate the strengths of disparate existing datasets, the determination of areas of overlap and duplication, and the provision of a foundation to collect, clean, and map potential candidate data elements. Project outcomes include significant translation of core concepts in information representation (cataloging, classification, authority and access control, subject analysis, arrangement and display, and vocabulary control) that have been developed, standardized, and practiced in libraries, into new and emerging information management needs. While management of individual databases is a well-studied problem, integration of this scale with the HPADW continues to present new challenges and barriers.

Additional challenges include the uneven distribution of data collection resources throughout rural Kentucky. Patient information obtained from different clinical databases and geographical sites represent one example. The depth and breadth in collection methodologies background of the CBI’s faculty and staff has helped alleviate this barrier. This data collection required dependence on IT personnel outside of CBI to help with databases’ design.

Addition of these IT personnel created data security issues as well as additional obstacles caused by a lack of statistical or data management expertise. In an effort to minimize and overcome these barriers, regular meetings with all involved parties concerning data collection occur to proactively address issues.

A larger challenge was presented for data security and ensuring patient privacy requiring central systems and processes to be refined. As detailed in the methods, this required creation of systems that allowed access to CBI Data to several groups (UK CCTS staff, researchers and collaborators and/or university faculty or staff) consistent with requirements of their project(s). The approach of the CBI in cooperation with the CCTS key function of research integrity (Table 2), centralizes the regulatory procedures to obtain information and allow tracking of data via the eWorkbench while ensuring privacy and security for all users.

As recognized in a previous report²¹ on biomedical informatics education, UK’s training program was focused on educating biomedical researchers rather than health care professionals. Recognizing these limitations, the CBI integrated educational programs with inclusion of health care professionals. This requires recruitment of appropriately trained instructors and development of teaching modalities specifically for health care professionals and trainees. Efforts have included improved access to operational informatics classes with the emphasis of EMRs, telemedicine, and data mining. An academic program or research practicum related to health IT is being developed by the CBI in order to expose students to the decision-making procedure based upon the use of the collected data. This program will hopefully overcome some of the barriers presented by developing an educational environment that links theory to actual work settings.

The CBI’s development of new academic informatics education programs faced a barrier related to the organizational structure and structure of degree-granting programs. University regulations require that degree-granting programs must be located within a single College. To provide an academic home for the overall discipline of biomedical informatics, the Division of Biomedical Informatics was created in the College of Public Health. The division serves as the academic home for training and education programs in informatics, which includes development of new certificate, MS, and PhD programs. Although separate from the CBI, the Division of Biomedical Information bridges the academic program and CBI.

The largest nonfinancial barrier to overcome for CBI was cultural milieu change. This change was deemed essential to the success of the redesign of information systems, linkage of EMRs with databases, and establishment of IT infrastructure. Providing appropriate tools and training, instituting and expanding educational offering, continuing integration with system upgrades, and eliminating waste and redundancy have all been instrumental in creating an overall IT/informatics functional structure.

Conclusions

This report describes one academic health center’s experiences with a transdisciplinary initiative integrating informatics and the clinical enterprise to form a comprehensive biomedical informatics service in response to the CTSA mechanism. One key approach for effective integration was senior leadership buy-in to provide the necessary financial resources and leveraging of resources to permit this integration to enhance translation of research discoveries and therapies. This strategy with integration of senior leadership for research and the clinical enterprise can be generalized to other universities and academic health centers. The CBI, working with the UK academic, clinical, and external partners is currently in the third year of implementing the university-wide biomedical informatics roadmap with the goal to establish an integrated biomedical informatics environment over five to seven years.

The CBI provides CCTS researchers with the resources to fulfill the objectives of the NIH CTSA program while the CBI possesses the core components of a biomedical informatics program. Within this structure, the CBI’s primary goal is to bring together this wealth of resources and expertise and effectively utilize new resources to create an enterprise that focuses on translational research. To ensure that the CBI continues to provide CCTS researchers with the appropriate informatics tools and training and breaks new ground through applied and empiric research, the CBI has adopted an interdisciplinary governance structure with participation of key stakeholders and leaders. This structure continues to evolve. The CBI directors have the critical data and the research expertise necessary to implement the roadmap developed in collaboration with IBM to guide the progress. This roadmap provides explicit benchmarks closely monitored by the senior leadership team and oversight group and adjustments are made as needed.

The efforts of the CBI have already and will continue to lead to new opportunities for areas of investigation. The federation of existing databases, a central CBI task, continues to provide new research problems in database management. Federation of databases will enable new research in knowledge discovery by development of methods for discovering patterns and knowledge buried in large amounts of data by data mining. The search for significant patterns located within high dimensional clinical and experimental data is one focus of ongoing efforts of CBI in collaboration with computer science.

This manuscript details the integration of IT to biomedical informatics and creation of an analytics environment at one academic health center with strategies employed to accomplish these goals. These strategies can be adopted by other universities desiring to leverage resources. Significant progress has been made with the CBI in regards to governance structure, HPADW, significant collaborations from the university level to the national level, eWorkbench, and expanded educational and training offerings. The following areas have been identified by the CBI for further development (1) EMRs; (2) decision support, health care quality, and assessment; (3) information storage, retrieval, and digital libraries; (4) imaging informatics and telemedicine; (5) organization and management issues in informatics; and (6) career and professional development.

Acknowledgments

The authors have received NIH financial support; the authors do not have any conflicts of interest.

Contributor Information

C. William Balke, Program Director, CTSI Clinical Research Center, Professor of Medicine, University of California San Francisco, 505 Parnassus Avenue, M-1203, UCSF Box 0126, San Francisco, CA 94143-0126, Phone: (415) 476-8127, Fax: (415) 476-8167.

Gloria H. Umberger, Assistant Professor, University of Kentucky Department of Neurology, Administrative Director, Center for Clinical and Translational Science, University of Kentucky, Kentucky Clinic, Lexington, KY 40536-0284, Phone: (859) 323-4779, Fax: (859)323-1080.

Jeffery Talbert, Director, Institute for Pharmaceutical Outcomes and Policy, Suite 185, College of Pharmacy, University of Kentucky, 789 South Limestone Street, Lexington, KY 40536-0596, Phone: (859)-323-7141, Fax: (859)-323-0803.

Denise Niles Canales, Biomedical Informatics Reporting Officer, Center for Biomedical Informatics, University of Kentucky, 789 South Limestone, Suite 182, Lex, KY 40536-0596, Phone: (859) 323-2127, Fax: (859)-323-1080.

Carol L. Steltenkamp, Chief Medical Information Officer Associate Professor of Pediatrics University of Kentucky College of Medicine 800 Rose Street, N-107 Lexington, KY 40536-0293 Phone: (859) 323-5845 Fax: (859) 323-2044.

Joseph Conigliaro, Director, Quality, Safety and Patient Rights Program, Professor of Medicine and Health Services Management, University of Kentucky, Colleges of Medicine and Public Health, Lexington VA Medical Center, 800 South Limestone Street, Suite N-109, Lexington, KY 40536, Phone (859) 323-5845, Fax (859) 323-.2044.

References

1. Zerhouni EA. Medicine. The NIH Roadmap. Science. 2003;302:63–72. [PubMed]

2. Zerhouni EA. Translational and clinical science – time for a new vision. N Engl J Med. 2005;353:1621–1623. [PubMed]

3. Zerhouni EA, Alving B. Clinical and translational science awards: a framework for a national research agenda. Trans Res. 2006;148(1):4–5. [PubMed]

4. Anderson NR, Lee ES, Brockenbrough JS, et al. Issues in biomedical research data management and analysis: needs and barriers. J Am Med Inform Assoc. 2007;14:478–88. [PMC free article] [PubMed]

5. DiLaura R, Turisco F, McGrew C, et al. Use of informatics and information technologies in the clinical research enterprise within US academic medical centers: progress and challenges from 2005 to 2007. J Investig Med. 2008;56:770–9. [PubMed]

6. Turisco F, Keogh D, Stubbs C, et al. Current status of integrating information technologies into the clinical research enterprise within US academic health centers: strategic value and opportunities for investment. J Investig Med. 2005;53:425–33. [PubMed]

7. [Accessed May 25, 2010];University of Kentucky Top 20 Business Plan. Available at: http://www.uky.edu/OPBPA/business_plan.htm.

8. Havener L, editor. Pathology Laboratory Electronic Reporting Version 2.2. V. North American Association of Central Cancer Registries; 2009. [Accessed May 25, 2010]. Standards for Cancer Registries. Available at: http://www.naaccr.org/filesystem/pdf/Standards_Volume_V_version_2.2_draftB_revised_February_2009.pdf.

9. Kim S, Gilbertson J. Information requirements of cancer center researchers focusing on human biological samples and associated data. Information Processing & Management. 2007;43:1383–1401.

10. Yagi Y, Gilbertson JR. Digital imaging in pathology: the case for standardization. J Telemed Telecare. 2005;11:109–16. [PubMed]

11. Kim S, Rasmussen E. Characteristics of tissue-centric biomedical researchers using a survey and cluster analysis. JASIST. 2008;59:1210–1223.

12. Wong S, Hoo K, Knowlton R, et al. Design and applications of a multimodality image data warehouse framework. J Am Med Inform Assoc. 2002;9:239–254. [PMC free article] [PubMed]

13. Conigliaro J, Justice AC, Gordon AJ, et al. How Harmful is Hazardous Alcohol Use and Abuse in HIV Infection: Do Providers Know Who is At Risk? J Acquir Immune Defic Synd. 2003;33:521–525. [PubMed]

14. Bradley VM, Steltenkamp CL, Hite KB. Evaluation of reported medication errors before and after implementation of computerized practitioner order entry. J Healthc Inf Manag. 2006;20:46–53. [PubMed]

15. Holsapple CW, Pakath R, Sasidharan S. A website interface design framework for the cognitively impaired: A study in the context of Alzheimer’s disease. JECR. 2005;6:291–303.

16. Holsapple CW. The inseparability of modern knowledge management and computer-based technology. J Know Manag. 2005;9:42–52.

17. Chi L, Holsapple CW. Understanding computer-mediated interorganizational collaboration: a model and framework. J Know Manag. 2005;9:53–75.

18. Holsapple C, Joshi KD. A formal knowledge management ontology: conduct, activities, resources, and influences. J Am Soc Inf Sci Tech. 2004;55:593–612.

19. Fang X, Holsapple CW. The usability of web sites for knowledge acquisition: a taxonomy of influences. Int J Elec Bus. 2003;1:211–224.

20. Holsapple CW, Joshi KD. A collaborative approach to ontology design. Communications of the ACM. 2002;45:42–47.

21. Hersh W, Williamson J. Educating 10,000 informaticians by 2010: The AMIA 10×10 program. Int J Med Inform. 2007;76:377–382. [PubMed]

22. Ireson CL, Slavova S, Steltenkamp CL, Scutchfield FD. Bridging the care continuum: patient information needs for specialist referrals. BMC Health Services Research. 2009;9:163. [PMC free article] [PubMed]

23. Menachemi N, Powers TL, Brooks RG. The role of information technology usage in physician practice satisfaction. Health Care Manage Rev. 2009;34:364–371. [PubMed]