We identified 360 research projects from the nine QUERI Centers and determined that 271 of these projects met criteria consistent with QUERI Steps 4–6 implementation projects. We excluded 185 projects that had been completed prior to October 2004 in order to focus on the projects with the most recently available information. The project-level analysis focused on the remaining 86 implementation research projects.
HIT Role in Providing Data and Information for Implementation Research
A key aspect of HIT use was to provide data that either directly guided implementation projects or informed the impact of an intervention. We identified four data collection approaches used among the 86 implementation research projects: (1) VA-wide databases built with extractions from local health-care facility data systems (35 projects), (2) local or remote extraction from local electronic health records (EHR) (59 projects), (3) collection of primary data directly from research subjects (55 projects), and (4) collection of data from non-VA databases (6 projects). Most projects used a combination of approaches.
The Role of Implementation Research in the Use and Development of HIT
QUERI researchers’ use of HIT was concentrated in five distinct areas: (1) clinical decision support systems (CDSS); (2) HIT tools targeted for specific users; (3) telehealth applications; (4) design and maintenance of VA-wide databases; (5) integration and innovative use of extant VA-wide data (Table ).
Examples of QUERI Projects for Each HIT Category
Use and development of CDSS in implementation research projects included a range of informatics tools that utilized the EHR as a platform for providing cues to clinicians to support clinical care. These tools included clinical reminders for specific actions and/or follow-up, alerts for medication issues, decision aids, and knowledge tools. QUERI researchers used CDSS in 23 implementation research projects (27%). Notably all of the QUERI programs had at least one research project in their portfolio that included use or development of CDSS. Use of CDSS tools included both the modification of VA’s existing clinical reminder software within the VA EHR16
and development of customized CDSS.
Two projects exemplify the range of HIT innovation with CDSS that QUERI teams developed. The Substance Use Disorder QUERI Center (SUD) evaluated a questionnaire to serve as a scaled marker for screening alcohol-related risks. The SUD placed the questionnaire in a new alcohol screening reminder within existing clinical reminders software and integrated with the VA EHR.17
The SUD convened researchers, VA clinical informatics experts from the Office of Health Information, and leadership from the clinical and policy services (Office of Mental Health Service, Office of Quality and Performance, and Patient Care Services) to develop and evaluate the alcohol screening clinical reminder.
A QUERI collaboration developed a new hypertension management CDSS outside of the VA EHR platform. The collaboration focused on the development of the Assessment and Treatment of Hypertension: Evidence-Based Automation (ATHENA) CDSS and involved multiple VA investigators including the Stanford University Medical Informatics group. ATHENA is a CDSS that utilizes clinical practice guidelines to assist clinicians in lipid management and blood pressure control in primary care.18
The researchers implemented and tested ATHENA within a clinical trial to identify difficulties in integrating the CDSS within the information architecture of VA EHR.
Another group of HIT tools used and developed in QUERI research projects provided generalized reference or educational material to clinician, patient, or administrative staff user groups. Unlike CDSS tools that link an event or processes with decisions or choice sets for action, these HIT tools were not linked to any specific patient information or specific clinical decision but instead left the decision about applicability to the user. We found 17 QUERI projects (20%) developed HIT tools to provide support to clinicians (nine projects), educate veterans (five projects), or monitor system performance (three projects). Internet-based toolkits were a particularly frequent example of HIT tools to support clinicians. For example, SUD conducted a study to identify acceptance by physicians of an Opiate Monitoring System used to help opiod agonist therapy clinics assess concordance with evidence-based practices. SUD developed and evaluated an online toolkit to help physicians understand key issues in prescribing buprenorphine for treatment of opioid addiction.19
Unlike a CDSS, this online toolkit was not linked to specific actions or processes of clinical care, yet it provided important information to clinicians in an easily accessible format. Projects focused on veterans included enhancement of the veterans’ personal health record Website. For example, the Stroke QUERI (STR) developed a Web page to alert users of the VA personal health record to stroke-related educational materials. The application allowed veterans to request the stroke educational materials from the Website.
We found eight QUERI projects (9%) that evaluated home telehealth initiatives. QUERI telehealth initiative examples included: Diabetes Mellitus QUERI center (DM) efforts to test telemedicine consultations among general medicine and diabetes specialists to improve health care utilization among diabetic patients, and a SUD randomized trial of recruitment into a VA-wide smoking cessation program that utilized a telephonic ‘Quitline’ and telephone care management20,21
. Studies involving veteran-focused telehealth utilized varying levels of technological sophistication with some studies using videoconferencing or robotic technology.22
We found seven QUERI projects (8%) focused on efforts to create new VA-wide databases. For example, the Ischemic Heart Disease QUERI center (IHD) used the development of a comprehensive clinical documentation and information system/database for the VA cardiac catheterization laboratories23
as an occasion to study the process of implementing HIT across the expanse of the VA.
The final category of researchers’ involvement in the development of HIT applications was efforts to integrate existing data into more useful formats. Implementation research projects in this category involved improving existing data. The HIV/Hepatitis QUERI Center (HIV) evaluated the use of a new case-finding algorithm to populate the VA HIV/AIDS Immunology Case Registry (ICR) to improve tracking of VA antiretroviral use and outcomes.24
A DM project studied the use of diabetes registry data to improve the assignment of patients to a clinic group responsible for diabetes care.25
HIT Barriers and Facilitators to Implementation Research
We discussed barriers and facilitators in the use and development of HIT in our semi-structured group interview with QUERI Center key personnel. We identified three major categories of barriers that, if resolved, would facilitate HIT use: (1) lack of local patient-level clinical data in VA-wide databases; (2) lack of VA-wide data in a standardized and analyzable format; (3) gaps in capture of clinical data in the EHR.
All of the QUERI Centers identified the need for additional patient-level clinical data that are captured locally in the VA EHR but were not included in standardized system-wide databases. Examples of specific types of such data included patient-level vital signs data, body mass index, weight, microbiology laboratory results, cancer pathology and diagnostic data, and smoking status. These data elements were identified as necessary for identifying target populations, tracking effects of implementation research interventions, and tracking national trends. Although the VA EHR is a VA-wide electronic health record, many clinical data elements are stored only at the local hospital level rather than in a national database. This situation creates challenges for multi-site studies because there may be no means for capturing data from multiple local EHR systems into a central repository for review and analysis during an implementation research project.
In our interviews, it was noted that two factors often made the option of extracting data from local EHR systems to a study-specific research repository untenable. First, regulations on human subject research and privacy protections required approvals from the Institutional Review Board of each participating site. The variability of IRB procedures introduces costs and delays that are particularly detrimental to “real-time” implementation research efforts.26
Second, coordination of multiple separate data extractions may also increase error rates compared to a single extract from a national database. Where possible, QUERI researchers attempted to utilize existing VA-wide patient-level clinical data, such as laboratory results and pharmacy data as noted previously. Another barrier we identified was the lack of standardized and analyzable data that make it “research-ready.” The most prevalent problem reported was that data were not standardized across facilities and thus were more difficult to collect, prepare, and analyze.
Finally, regarding gaps in capture of clinical information in the EHR, there were several examples shared that identified long-standing unmet needs for data collection. Implementation researchers petitioned to add additional data to the EHR that would then make nationwide clinical data available for research use. For example, researchers desired changes to the VA EHR to collect patient-centered outcomes data using standard assessment instruments, such as the SF-36 for health-related quality of life, and various specific mental health and substance use screening tools. Researchers reported that separate data collection outside of the EHR was necessary to meet research timelines.