|Home | About | Journals | Submit | Contact Us | Français|
Evaluations of computerized clinical decision support systems (CDSS) typically focus on clinical performance changes and do not include social, organizational, and contextual characteristics explaining use and effectiveness. Studies of CDSS for intensive insulin therapy (IIT) are no exception, and the literature lacks an understanding of effective computer-based IIT implementation and operation.
This paper presents (1) a literature review of computer-based IIT evaluations through the lens of institutional theory, a discipline from sociology and organization studies, to demonstrate the inconsistent reporting of workflow and care process execution and (2) a single-site case study to illustrate how computer-based IIT requires substantial organizational change and creates additional complexity with unintended consequences including error.
Computer-based IIT requires organizational commitment and attention to site-specific technology, workflow, and care processes to achieve intensive insulin therapy goals. The complex interaction between clinicians, blood glucose testing devices, and CDSS may contribute to workflow inefficiency and error. Evaluations rarely focus on the perspective of nurses, the primary users of computer-based IIT whose knowledge can potentially lead to process and care improvements.
This paper addresses a gap in the literature concerning the social, organizational, and contextual characteristics of CDSS in general and for intensive insulin therapy specifically. Additionally, this paper identifies areas for future research to define optimal computer-based IIT process execution: the frequency and effect of manual data entry error of blood glucose values, the frequency and effect of nurse overrides of CDSS insulin dosing recommendations, and comprehensive ethnographic study of CDSS for IIT.
The U.S. National Research Council recently endorsed the use of clinical decision support systems (CDSS) and “organizational systems-level research” of health information technology to help drive healthcare transformation . Historically evaluations of CDSS have focused on practitioner performance  rather than social, organizational, and contextual factors [3, 4]. Kaplan noted that CDSS evaluation studies measure CDSS effects on clinical performance, use experimental study designs or randomized controlled trials, disregard naturalistic study methods, ignore contextual issues surrounding system usage, investigate the perspectives of physicians rather than other clinical roles, and consider only the CDSS intervention, not other clinical information systems in use . The reporting of findings in the literature reflects a rationalist scientific orientation  and shows clinical and medical informatics investigators' preferences toward objectivist rather than subjectivist approaches to evaluation . Although subsequent reviews have identified dimensions of workflow integration as critical to CDSS success , researchers have yet to fully embrace the National Research Council's directives or address the gaps identified by Kaplan.
Studies of clinical decision support systems for intensive insulin therapy (IIT), a treatment combining frequent blood glucose monitoring and insulin drip adjustments to maintain tight glucose control , follow the general CDSS evaluation trend. Investigations using experimental designs have demonstrated improved clinician protocol adherence and achievement of target glucose levels using computer-based IIT protocols instead of paper-based versions [8-19]. However, these evaluations have paid little attention to the context of interventions, including the complex interaction between staff, testing devices, and computers that may result in inefficiency and error. Nurses use computer-based IIT advisors to document care and calculate insulin doses, but investigations mostly rely on anecdotal feedback to understand nurse perspectives of CDSS and rarely consider CDSS usage with respect to other care processes and clinical information systems. The literature describes paper-based IIT protocol implementation barriers  and effects on nurse work  but does not explore the complexity and organizational change related to computer-based IIT approaches.
Understanding the mechanisms of effective intensive insulin therapy CDSS is important because IIT is the standard of care for critically ill patients . In 2001 the Leuven study demonstrated morbidity and mortality improvements through an intensive insulin therapy protocol , and subsequent studies at other institutions have produced similar results [23, 24]. However, a 2008 meta-analysis of randomized trials raised concerns about the therapy's mortality benefit and safety . Differences in care protocols ranging from nutrition provisions  to target blood glucose ranges [26, 27], insulin administration , and intended patient populations  may explain variation in IIT outcomes, but researchers have not determined comprehensive solutions, especially ones that address computer-based approaches.
Although care protocols define the decision-making behavior clinicians should exhibit under certain conditions  and represent the evidence-based, formal structure of healthcare organizations, actual work activities usually differ from official practice definitions . In patients treated with computer-based intensive insulin therapy in the surgical intensive care unit at Vanderbilt University Hospital, researchers found fourteen percent of blood glucose measurements were not taken on time . Significant relationships between late blood glucose measurements and episodes of hyper- and hypoglycemia  as well as blood glucose variability and mortality  suggest that workflow may be a factor in computer-based IIT performance and patient outcomes . In sociology and organizational studies, institutional theory [31, 34-38] examines the way rules, policies, and procedures affect and are affected by “assumptions, norms, values, choices, and interactions” . This approach has informed investigations of information technology in law , banking , and research workplaces , and informatics researchers have focused on similar issues to influence system design [42-44]. To improve intensive insulin therapy protocol performance and patient outcomes, researchers and practitioners can use institutional theory to address care process execution issues related to human behavior.
This paper takes a subjectivist approach  to the study of computer-based intensive insulin therapy and illustrates the need for additional research in two parts: 1) a literature review, which uses institutional theory to take inventory of formal structure and social organization  reported in computer-based IIT evaluations, and 2) a case study that builds on the literature review and emphasizes social, organizational, and contextual aspects typically absent from computer-based IIT evaluations. The literature review can potentially serve as a source for other CDSS evaluators interested in social, organizational, and contextual elements, and the case study shares the experience of computer-based IIT at one institution so other institutions can make informed decisions. Overall the analysis shows a gap in the computer-based IIT literature concerning complexity of protocol execution, opportunity for error in staff-device-CDSS interaction, effects on other workflow and care processes, and the magnitude of organizational change necessary for implementation.
In May 2008 we searched ISI Web of Science for articles citing the Leuven study (1,783 articles) and containing the keyword “protocol” (129 articles). Because the Leuven study played a significant role in IIT protocols becoming the standard of critical care, we used it to focus our search. From the “protocol” corpus we identified fifteen evaluations of computer-based IIT protocols. Fourteen evaluations used experimental designs or randomized trials, and one was a practice report. The studies examined eighteen intensive care units in twelve healthcare organizations excluding the hundreds of sites evaluated in a longitudinal study of a commercial product .
One of the authors (TRC) reviewed the studies through the lens of institutional theory [31, 34-38] to identify aspects of computer-based intensive insulin therapy's formal structure—the prescribed, written policies established to govern and evaluate behavior—and social organization of computing—the interaction of people, process, and technology across different locations and over time . Researchers have used these dimensions to understand the interdependence of technology and human behavior in shaping organizational activity in banking , legal  and university research settings . For example, through the implementation of a locally hosted digital legal library, a metropolitan court system sought to improve attorneys' legal research and limit cost . Formal structure, manifested in system policies, defined access according to professional role and discouraged use of expensive remote subscription services in favor of the local digital library . The social organization of computing was critical to effective digital library usage: convenient terminal access, workflow integration, favorable attitudes toward computing, separate computer work areas for competitive attorney groups, individualized training, and the emergence of social norms regarding digital library usage in courts .
Kling describes three main components of the social organization of computing: equipment configurations, skills and roles, and support infrastructure . Equipment configurations involve the locations of hardware, software, functionality modules within software, and peripherals; skills and roles encompass the various members of an organization who use, supply, or affect an information system's data; and support infrastructure concerns the ways that system stakeholders (e.g. users and managers) obtain assistance and direction . Additionally, temporal aspects of system use, such as periodic (e.g. morning vs. evening) and long-term change over time (e.g. initial vs. established patterns of usage), are salient for analysis .
Formal structure, defined as the codified procedures intended to govern and evaluate behavior, is well-documented in the computer-based intensive insulin therapy literature. Researchers frequently reported protocol algorithm details as well as evaluation measures in terms of practitioner and patient outcomes, which have been previously defined .
Computer-based IIT approaches used two main algorithmic techniques to determine insulin dosing: linear equations [8-11, 17, 45] based on the work of Bode  and White  and conditional logic [15, 16, 18, 48]. Other approaches included model predictive control  and engineering control math . Most researchers disclosed the logic of computer-based insulin dosing algorithms [8-10, 14-18, 45, 48], and some researchers disclosed previously [8, 9] or concurrently used  paper-based IIT protocols.
Measures of practitioner outcomes included blood glucose target achievement (e.g. time in target range [8, 13-15, 19, 45, 48], percentage in target range [8, 9, 11, 13, 16, 19], time before reaching target range [9-11, 13, 14, 16, 19, 45]), blood glucose mean and median (e.g. overall [9, 11, 13, 15, 17, 18], after 24 hours [14, 19], per day [8, 15, 45]), total blood glucose measures (e.g. overall [9, 13, 15, 16, 18, 45] and per day [8, 14, 15, 19]), hyperglycemia [8, 9, 11, 14-16, 19, 45], hypoglycemia [8-11, 13, 16, 18, 19, 45], insulin administration totals [10, 19], and protocol compliance (e.g. time to initiation of protocol [8, 14, 19], measurement and dosing per protocol schedule [16, 19, 48], administration of recommended insulin dose [19, 48]). Evaluation measures and clinical performance varied between studies. Only one study noted a low percentage of blood glucose results in target range and high percentage of tests not performed on time . Based on rare occurrences of hypoglycemia and reductions in hyperglycemia, most studies deemed IIT protocols “safe and effective” for glucose management.
Few studies evaluated patient outcomes in addition to practitioner outcomes [9, 17, 18]. Despite demonstrating improved practitioner outcomes, two studies showed no difference in patient outcomes [9, 18] while another showed reduced morbidity and length of stay but increased mortality . Most studies were preliminary and lacked statistical power to detect patient outcome changes.
Compared to formal structure, social organization—the interaction of people, process, and technology—was less consistently reported in the computer-based intensive insulin therapy literature. Computer-based approaches to IIT involved various levels of computer systems integration and interaction with testing devices as well as impact on and influence of other care processes and hospital units. The following reviews the social organization of computer-based IIT implementations in terms of equipment configurations, skills and roles, and support infrastructure . Table I provides a summary.
Equipment configurations include the placement of computers, software, software functionality, and peripherals within large information systems in particular settings . For computer-based IIT, this includes decision support system location and integration, blood glucose testing device usage, and device-computer interface. Figure 1 depicts the interaction of these elements in computer-based IIT workflow reported in the literature.
Clinicians used three IIT CDSS mechanisms: 1) modules embedded within existing primary clinical information systems, including care provider order entry (CPOE) systems, that are accessible from hospital workstations and store blood glucose and insulin dosing data in clinical data repositories (CDR) [8, 9, 11, 14, 16, 48, 49]; 2) “calculators” accessible on a hospital network that do not store data in a CDR [15, 50] and may require additional documentation in a clinical information system  and/or use of a preprinted order set ; and 3) applications installed on standalone computers1 [14, 49]. Few studies reported location of hospital workstations , electronic data interchange with patient monitoring equipment , and use of other clinical information systems that are related to or may affect IIT CDSS, workflow, or care processes (e.g. nursing documentation) .
Clinicians used handheld glucometers [8, 9, 13, 15-17, 48, 50], non-handheld blood gas analyzers [14, 49], and a combination of both  to obtain blood glucose measurements. The importance of handheld glucometers was demonstrated by additional glucometer investment before implementation , shortages during implementation , and the mechanical failure of a single non-handheld blood gas analyzer temporarily halting protocol use .
Clinicians manually transcribed blood glucose values from testing devices to CDSS [8, 9, 11, 18, 45, 50], automatically transferred test results through docking stations in real time , or automatically transferred test results from non-handheld blood gas analyzers in real time [14, 19]. Depending on clinical information systems integration, nurses recorded blood glucose results and insulin rates in both CDSS and nursing documentation tools . Continuous monitoring technology was identified as a possible alternative in the future [10, 16, 49].
Skills and roles include the various members of an organization who use, supply, or affect an information system's data . For computer-based IIT, this involves nurses as well as care members engaged in other processes that influence system use.
Nurse feedback regarding IIT CDSS was mostly positive but evaluation methods lacked rigor. One study used a formal questionnaire to gauge nurse perceptions before and after implementation , whereas most studies reported anecdotal nurse feedback related to the interventions' ease of use [8, 18] and increased nursing workload [11, 16, 49]. In one study researchers identified increasing nurse autonomy as a goal of the implementation . Although nurses are the primary users of IIT CDSS, nurse feedback is not a focus of investigations.
Other care processes that may affect IIT and use of IIT CDSS were frequently overlooked. One study reported the concurrent activity of a diabetes disease management service . Demands from surgery and imaging occasionally interrupted IIT usage , and ICU nurses administered steroids [9, 18] and nutrition [9, 14, 16, 18] that may have affected patients' blood glucose levels. Description of IIT CDSS workflow integration with respect to the disruptive nature of healthcare was not present in the literature.
Support infrastructure concerns the ways in which system stakeholders (e.g. users and managers) obtain assistance and direction . For computer-based IIT, this focuses on the activities of information technology professionals, care team members, and hospital administrators.
Multidisciplinary teams consisting of physicians, nurses, pharmacists, and informaticians were responsible for the creation of IIT care protocols and computer-based advisors [9, 16, 18, 50]. Some approaches to computer-based IIT stressed the importance of embedding decision support systems in clinical workflow [8, 9, 14]. Training procedures included pre-implementation multidisciplinary instruction [9, 14] and web-based nurse training  as well as “continued need for staff instruction and compliance regarding the protocol” .
All computer-based IIT protocols originated in an ICU setting, and many diffused to other ICUs within the same institution [9-11, 19, 45] as well as medical-surgical floors [10, 11, 45], recovery , labor and delivery , and progressive care units . Two approaches diffused to multiple hospitals after initial usage [10, 11], and two research teams created organizations to advance research and adoption of their respective systems [10, 49].
Computer-based intensive insulin therapy studies reported formal structure consistently and social organization inconsistently, which reflects the objectivist approach predominating CDSS investigations  and the norms of the clinical literature. Most evaluations provided algorithm details and measurements of practitioner performance, but consideration of real world system usage and effects on healthcare organizations in terms of equipment configurations, skills and roles, and support infrastructure varied. Although most interventions relied on the use of handheld glucometers, none recognized the complexity and capacity for error of nurse-device-computer interaction. Studies irregularly described the effect of computer-based IIT on other care processes and clinical information systems usage and vice versa. Although nurses were the primary users of computer-based IIT interventions, most evaluations did not explicitly evaluate nurse feedback. Some studies described the importance of workflow integration and multidisciplinary cooperation, but the literature lacked a comprehensive description of unintended consequences and change management strategies. Evaluations did not address social, organizational, and contextual issues related to computer-based intensive insulin therapy.
Intensive insulin therapy represents a set of organizational changes involving the recursive relationship between formal structure and actual work practices, a process which the following case study demonstrates. Based on review of the literature, most computer-based intensive insulin therapy studies ignore the social, organizational, and contextual aspects that explain the effectiveness of interventions. By examining the transition from ad hoc sliding-scale insulin therapy to standardized intensive insulin therapy in the surgical intensive care unit at Vanderbilt University Hospital, this case study illustrates aspects usually omitted from evaluations of computer-based IIT: the importance of local leadership, the expenditure of labor and capital, the relationship between the ICU and other organizational entities, and the influence of technology on clinical process and vice versa. Additionally, the case draws attention to consequences of computer-based IIT—staff- device-computer interaction and the therapy's effect on other care processes—that represent opportunities for error and require additional research. Rather than treat the research setting as static, we aim to show how its dynamic properties change over time and affect and are affected by physicians, nurses, laboratory personnel, and informatics personnel.
We used naturalistic methods  to create a three stage chronological narrative of insulin therapy in the study site: glycemic regulation before IIT, paper-based IIT, and computer-based IIT. For stage one, we interviewed nurses, physician leadership, and laboratory personnel. For stage two, we reviewed colleagues' publications  and interviewed nurses and informatics support staff. For stage three, we interviewed nurses, physician leadership, informatics support staff, and laboratory personnel in addition to directly observing workflow and reviewing colleagues' publications . Preceding the narrative stages, we also gathered site background information based on review of internal documents and interviews with unit leadership. The Vanderbilt University Institutional Review Board approved this study.
At Vanderbilt University Hospital (VUH), a large academic urban tertiary care center consisting of 501 beds, the surgical intensive care unit (SICU) admits 1,300 patients each year. The SICU occupies a single floor of the hospital and has a horseshoe layout with a nurse station and supply room in the middle and 21 beds lining the exterior. Each patient room contains at least one clinical workstation connected to the hospital network. Additional workstations are located on mobile carts, at the central nurse's station, adjacent to isolation rooms in antechambers, and throughout the corridors. At VUH the use of electronic patient care information systems has a fifteen year history and is engrained in clinician culture. Clinicians use locally developed electronic medical record and provider order entry systems in addition to vendor applications for ancillary functions and nursing documentation.
Since 2001 the SICU has been under the leadership of a medical director focused on strengthening unit operations as well as promoting collaboration with other hospital units. Efforts include increasing the number of SICU beds from 14 to 21, expanding the use of evidence-based guidelines, shifting cardiovascular surgery patients out of the SICU into a new intensive care unit, collaborating more closely with trauma ICU, creating a full-time SICU critical care service to replace an elective service comprised of critical care and anesthesiology faculty, and facilitating the creation of a separate emergency general surgery service. During this period of growth, SICU experienced increased patient volume and illness severity compared to pre-2001 levels.
Dependence on clinical judgment, inconsistent care processes, and documentation difficulties characterized sliding scale insulin (SSI) therapy, the standard of care for SICU glycemic regulation prior to paper-based intensive insulin therapy. All diabetic patients, as well as non-diabetics with blood glucose issues caused by sepsis or medications, received SSI treatment. Although nurses generally contacted physicians when a blood glucose measurement exceeded 150-200 mg/dL, no explicit criteria defined the threshold of hyperglycemia and when a SICU patient should begin insulin therapy. Physicians' SSI orders defined blood glucose measurement intervals and specific insulin doses for blood glucose ranges. Less experienced nurses adhered to SSI orders whereas more experienced nurses would use clinical judgment (e.g. accounting for a patient's glucose-affecting therapies) in determining subcutaneous insulin injection dosing and subsequent blood glucose monitoring intervals (e.g. Q1H to Q6H) using LifeScan Basic® handheld glucometers. In addition to subcutaneous sliding scale insulin, patients received insulin infusions along with electrolytes as part of total parenteral nutrition. This dosing was also non-standardized and relied on physician discretion. Physicians and nurses depended on experience to initiate therapy, adjust subcutaneous insulin doses, and monitor blood glucose levels.
SSI data management was problematic. Following each blood glucose measurement and insulin administration, nurses documented data on the paper ICU flowsheet, daily glucose log, and medication administration record. Once per day a carbon copy of each patient's daily glucose log was transported to the laboratory for entry into the laboratory information system (LIS). Recording blood glucose (BG) results in the LIS enabled the institution to track resource utilization, assess point-of-care testing compliance, manage billing, and meet regulatory requirements. The LIS also interfaced with the clinical data repository, which clinicians accessed from hospital workstations to view lab results. However, blood glucose and insulin data appeared in the CDR only about 40% of the time: SICU staff were often too busy to transport logs, and the laboratory did not routinely send personnel to SICU to check compliance and collect log sheets. Physicians turned to paper charts instead to obtain blood glucose and insulin data.
A non-protocol-based approach to care, sliding scale insulin permitted variability in clinical decision making. Treatment using SSI was reactive rather than proactive in that it treated hyperglycemia instead of attempting to prevent it, which allowed fluctuation of blood glucose levels and risk of hyper- and hypoglycemia in patients . Non-standardized care and workflow breakdowns typified sliding scale insulin in the SICU.
In August 2003 the VUH SICU implemented a paper-based intensive insulin therapy protocol  based on the Leuven study , but labor requirements, task complexity, and workflow integration hindered protocol performance. The protocol increased nurse workload by requiring blood glucose measurements, insulin rate adjustments, and subsequent documentation at two hour intervals for both diabetic and non-diabetic patients. Under the new protocol, nurses initiated intensive insulin therapy when a patient's BG level exceeded 110 mg/dL instead of waiting for BG levels to reach a discretionary level as under the previous SSI standard of care. This increased the number of patients treated with insulin. After performing BG tests at the bedside using LifeScan Basic glucometers, nurses consulted the paper medication administration record for the protocol's instructions to manually calculate insulin titrations, a process which required nurse interpretation of the protocol (e.g. “increase infusion by 1-2 units/hr” or “decrease infusion by 25-50%”). Nurses then recorded BG and insulin data on the ICU flowsheet, daily glucose log, and medication administration record. Further complicating implementation was a local nursing shortage. Staff disagreed with the protocol, ignored recommendations, lacked time to perform calculations, and made mental mistakes. Although the purpose of the protocol was to improve care through standardization, variability persisted while demands on nurses increased and potential patient safety threats emerged.
The new protocol also created difficulty for the laboratory, which affected SICU staff. Because of the increase in blood glucose tests performed, laboratory personnel required more time to transcribe test values into the system, which resulted in a processing backlog. Illegible daily glucose logs caused laboratory personnel to occasionally transcribe BG results incorrectly, which caused values in the CDR to not match up with paper documentation. Nurses and physicians became frustrated because they were unable to access accurate BG values through the CDR in a timely fashion.
Overall physicians and nurses were not satisfied with IIT's impact on work processes, and auditing protocol performance was labor intensive due to manual chart review. The average patient blood glucose value, 140-150 mg/dL, exceeded the target of 80-110 mg/dL. Despite organizational changes to standardize patient care, nurse work processes varied and practitioner outcomes did not meet goals.
A multidisciplinary team implemented a computer-based advisor in the VUH SICU that improved intensive insulin therapy performance  and produced unintended consequences. In May 2004 an informatics faculty member approached the SICU medical director, a surgeon and critical care physician respected by staff, about developing a computerized IIT approach to improve protocol adherence and capture of process variables for subsequent analysis (e.g. blood glucose values, insulin doses). The SICU medical director commissioned a team of staff nurses, nursing leadership, pharmacists, physicians, and informaticians to assess the IIT process and develop the functionality and interface for a clinical decision support system. Because care provider order entry usage was a regular part of clinical workflow, the team decided to embed the decision support module in the institution's CPOE system. The team tested the intervention and worked with “super user” nurses to refine the tool's ease of use, validate its effectiveness, and assuage concerns about computer-based dosing recommendations. The team also created a training regimen for staff consisting of classroom training for nurses, physician training through orientation, pharmacist training through rounds, continuous informatics staff support, and ad hoc instruction from a SICU nurse practitioner educator.
A separate laboratory investment decision influenced nurse IIT workflow, CDSS design, and project timing. Independent of the SICU in September 2004, the laboratory replaced all glucometers across the institution with Lifescan® SureStep® Pro™ devices ($550 each) and installed a data infrastructure consisting of docking stations ($300 each) and a software interface ($90,000 5-year contract) to automatically transfer blood glucose results from testing devices to the LIS and CDR, thus alleviating the laboratory daily glucose log processing problem. However, test results took up to ten minutes to transfer from device to CDR. Furthermore, devices would not send results until errors were resolved, which occasionally lengthened the transfer process. Data transfer issues coupled with the time-sensitive nature of IIT affected CDSS design: nurses would manually transcribe the latest blood glucose value from the glucometer to the CDSS. To initiate a blood glucose measurement, a nurse used the SureStep® Pro's™ integrated barcode reader to scan barcodes attached to his name badge and the patient's bedside. If barcode scanning failed, a nurse manually entered identification numbers for himself and/or the patient. For legal and billing purposes, the laboratory required BG results entered directly into the LIS by laboratory personnel or automatic device transfer, not manual nurse transcription. Once per shift nursing assistants collected devices and placed them in one of two docking stations to transfer test results and accompanying identification information. After use of SureStep® glucometers became a regular part of workflow, the SICU team resumed its CDSS implementation effort in December 2004.
The computer-based IIT approach introduced a tool to assist nurses with glucose maintenance as well as a practice change to increase physician involvement in glycemic regulation. Instead of starting the protocol when a blood glucose reading exceeded 110 mg/dL, a nurse contacted a physician to initiate therapy using the CDSS module, which consisted of two parts : an initiation screen for a physician to specify care and “notify house officer” parameters, and an insulin rate adjustment screen for a nurse to manually enter blood glucose values (Figure 2). Following a physician's one-time use of the initiation screen, the nurse accessed the CDSS module according to the protocol schedule (usually Q2H) in order to document blood glucose results and calculate new insulin titrations. The CDSS module utilized a linear equation to determine an insulin titration , which eliminated the need for nurses to manually calculate insulin infusion rates. However, nurses could override the CDSS module's recommendations and enter an insulin titration using their clinical judgment when necessary (e.g. simultaneous administration of glucose-affecting medication). After using the CDSS, nurses manually adjusted rates of pharmacy-prepared regular insulin drips (150 units in 150mL normal saline solution with a 24 hour expiration) on Alaris® infusion pumps equipped with Guardrails® software, which was not configured to transfer infusion data to the CDR. Although several years later the institution implemented a barcode medication administration system integrated with other clinical software for administering intermittent medications, nurses did not use it for infusions.
Compared to its paper predecessor, the SICU computer-based IIT protocol increased protocol adherence, reduced time to initiate treatment, expanded the percentage of blood glucose readings in the target range, and simplified record keeping . The hospital's medical, neurological, cardiovascular, and trauma ICUs adopted the same computer-based approach to IIT, and the trauma ICU demonstrated glycemic regulation improvements using the intervention . Neither the SICU or trauma ICU studies were sufficiently powered to detect patient outcome improvements, but in terms of glycemic regulation improvements, the computer-based approach to IIT was a success at the institution.
In addition to improving practitioner performance, the intervention produced unintended consequences related to workflow and technology. First, IIT and other redesigned clinical activities contributed to increased overall CPOE usage, which resulted in clinicians waiting to use terminals in SICU. In response, the institution purchased additional workstations. Second, the clinical data repository's blood glucose and insulin data appeared in duplicate—one set of values entered manually by nurses into the CDSS module, the other captured from the glucometer—with slightly different timestamps and occasionally different values. This resulted in visual clutter in CDR data displays, which may have contributed to clinician confusion or cognitive overload. Third, nurses “double documented” blood glucose and insulin values in the CDSS module and an electronic nursing documentation system, which was implemented two years after the introduction of computer-based IIT and the completion of protocol evaluations in the SICU  and trauma ICU . This resulted in a third set of values appearing in the clinical data repository. Furthermore, the approach to computer-based IIT assumed nurses never made errors when transcribing blood glucose values to calculate and adjust insulin doses. Despite these issues, computer-based IIT remains the standard of care for critically ill patients at VUH.
In the transition from sliding scale insulin to paper-based IIT to computer-based IIT, Vanderbilt University Hospital enacted considerable organizational changes related to evidence-based protocol development, nurse workload, physician involvement, blood glucose testing and infrastructure, and informatics development and support. Forces beyond SICU control—a local nursing shortage, the laboratory's decision to upgrade glucometers, and the institution's decision to implement nursing documentation software—affected the trajectory of intensive insulin therapy efforts over time, but SICU leadership and multidisciplinary cooperation helped ensure the project's success. Other institutions may experience similar organizational changes as part of their computer-based IIT efforts. Changes to glycemic regulation, glucometer usage, and computerization occurred gradually over time at VUH, which conceivably enabled stakeholders to adapt to process modifications more easily. In contrast, other institutions may face greater change management challenges if abruptly shifting from sliding scale insulin to computer-based IIT. Computer-based intensive insulin therapy is a complex, multifaceted organizational undertaking that requires substantial commitment to change and presents opportunities for further inefficiency and error reduction.
Kaplan's themes of clinical decision support system evaluation  are present in evaluations of computer-based intensive insulin therapy. In order to optimize computer-based intensive insulin therapy process execution, researchers and practitioners should address social, organizational, and contextual issues determining how and why implementations are successful. From our literature review and case study, three aspects of computer-based IIT appear particularly salient: (1) the relationship between clinical information systems, CDSS, testing devices, users, and error; (2) nurse perspectives; and (3) organizational change.
The interaction of hardware, clinical information systems, clinical decision support modules, blood glucose devices, and clinicians is complex, time consuming, and susceptible to error, yet most evaluations of computer-based IIT take it for granted. For example, a study of computer-based IIT conducted at Vanderbilt University Hospital stated that “[blood glucose] values are downloaded directly from the glucometer to the computer order entry system” , which misrepresents the reality of manual data entry and possibility for error inherent in the process. Installation of additional docking stations at each bedside to facilitate data transfer may be cost prohibitive or hindered by slow data transfer times. The purchase of glucometers that transmit data wirelessly across a hospital network to clinical data repositories in a reliable fashion may also be cost prohibitive. Furthermore, controversy surrounds the use of handheld glucometers for intensive insulin therapy due to possible inaccurate results [53-56]. Some studies suggest continuous glucose monitoring technologies can replace handheld glucometers today [57, 58] while others propose additional refinement [59-61] or recommend against their usage . In contrast to computer-based IIT, computer-based anticoagulation therapy  relies on a central laboratory's activated partial thromboplastin time results, which are processed less frequently  and are arguably more accurate than handheld glucometer test results. For computer-based intensive insulin therapy, the optimal configuration of testing devices, computers, decision support interfaces, and personnel is not yet understood.
Few studies have focused on nurse perspectives regarding intensive insulin therapy, particularly for computer-based approaches, and additional study can potentially improve protocols and workflow. A direct observation study of a paper-based IIT protocol showed that nurses required between three and nine minutes (mean 4.72, SD 1.13, median 4.67) to obtain a testing device, measure blood glucose, and adjust insulin . A separate time-motion study found nurses required 20-30 minutes to complete IIT tasks and document care . Times varied due to treatment differences for hypoglycemia, hyperglycemia, and euglycemia  as well as nurses locating devices, troubleshooting devices, caring for patients with isolation precautions, and occasionally ignoring hygiene and safety requirements . Such issues may also influence provision of computer-based IIT, and CDSS and other computer system usage during IIT administration may have other unintended consequences that add to nurse work or detract from patient safety. In a study of computer-based IIT2, nurses indicated the following reasons for declining CDSS recommendations: patient blood glucose trends, concurrent administration of medications prepared in a glucose solution, nutrition changes, concurrent epinephrine administration, hypothermia, agitation, and previously entered incorrect data . The results of this study show some of the effects of CDSS on IIT and demonstrate the value of nurse-focused evaluation of computer-based IIT in making workflow and care barriers explicitly understood. New dosing algorithms can potentially incorporate such factors so that IIT protocols reflect the realities of clinical practice and judgment.
The same computer-based intensive insulin therapy protocol used in two hospitals, or two units in the same hospital, might produce variability in social processes and clinical performance. Examining the social organization of computer-based IIT evaluations shows that the effects of computer-based IIT implementations on healthcare organizations are not explicitly reported. Our case study demonstrates how a surgical intensive care unit with strong leadership and institutional informatics support overcame technological and organizational barriers to implement computer-based IIT. Although other ICUs in the institution now use the same computer-based IIT approach, the intervention may or may not appropriately match workflow, organizational, and clinical needs because it was designed for the SICU. A recent multi-site IIT trial , which showed increased mortality for patients treated with IIT versus those treated with conventional therapy, used the same computer-based IIT dosing calculator in all sites . The researchers did not explore computer-based IIT process execution across sites although such issues may have affected clinical performance.
To understand computer-based IIT usage, future studies should combine quantitative and qualitative methods. First, by comparing manually entered blood glucose data and automatically captured values from glucometers, we can determine the frequency and effect of incorrect data on insulin dosing and blood glucose variability. Most computer-based IIT studies utilize handheld glucometers and assume the transcription of blood glucose values from testing devices to CDSS is error free. A study examining ventilator settings automatically captured from a device versus manually entered into a computerized CDSS showed 3.9% of computer-generated recommendations contained incorrectly entered data . Blood glucose value errors may potentially contribute to blood glucose variability, which has been associated with mortality .
Second, investigating the impact of CDSS insulin dose overrides can assess the effectiveness of nurses' clinical judgment. Studies of medication-related CDSS embedded in CPOE systems show physician override rates of computer-based recommendations as high as 91% [70, 71], and researchers suggest using quantitative and qualitative methods to understand clinician-CDSS interaction . Nurses deviate from CDSS suggestions when a clinical situation is more complex than a computer algorithm's parameters . However, little is known about whether nurses' clinical judgment is appropriate under these circumstances. Quantifying the frequency and effect of insulin dose overrides on blood glucose variability can potentially answer this question. Examining medication administration records and clinical documentation corresponding to nurse overrides of CDSS recommendations may provide indication of additional variables for IIT dosing algorithms to consider (e.g. corticosteroids).
Third, the use of ethnographic methods to study computer-based intensive insulin therapy can potentially lead to software and process enhancements. An approach from anthropology, ethnography has been used in clinical research to improve surgical resident handoffs  and in informatics research to identify and resolve incorrect software design assumptions [42, 73]. Extensive direct observation of clinicians using CDSS for IIT in the field can reveal benefits and drawbacks of the current approach with respect to computer system usage, care processes, and issues currently unknown. Additionally, ethnographic study of computer-based IIT in multiple ICUs can potentially show site-specific differences in social organization of the intervention that may affect clinical performance. By understanding the use of computer-based IIT in real world settings, researchers and practitioners can make care workflow and protocol modifications to potentially achieve the morbidity and mortality improvements demonstrated in the Leuven study.
There are limitations to this study. First, we examined intensive insulin therapy in one intensive care unit at a single institution with sophisticated clinical informatics systems. Findings may not generalize to sites with less informatics development. Future research will examine additional ICUs at Vanderbilt University Hospital and then proceed to additional institutions. Second, other theoretical approaches might be more illuminative than institutional theory's social organization of computing in examining computer-based intensive insulin theory. For example, social interactionism, “fit,” and “the 4 C's” may be potentially useful methods .
Our analysis contributes to the understanding of computer-based intensive insulin therapy's social, organizational, and contextual aspects. More broadly, this paper addresses the underreported elements explaining how and why clinicians use CDSS interventions. We suggest future IIT CDSS research involve quantifying error, assessing clinical judgment in overriding CDSS recommendations, and directly observing nurse use of IIT CDSS with respect to other care processes and clinical information systems. Researchers and practitioners can use this study to approach computer-based intensive insulin therapy and clinical decision support system improvement projects.
What was already known:
What this study added to our knowledge:
Mr. Campion received support from National Library of Medicine Training Grant NLM T15 007450-07. The authors thank Gwen Holder, RN, MSN for internal document assistance.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Conflict of Interest Statement The authors have no conflicts of interest to report.
1In these studies the authors described use of the CDSS in embedded and standalone configurations.
2This study did not meet literature review criteria as it was published in June 2008 (and not yet indexed in ISI Web of Science) and did not cite the Leuven study.