PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of peerjLatest ArticlesFor AuthorsEditorial BoardPeerJPeerJ
 
PeerJ. 2017; 5: e3083.
Published online 2017 March 14. doi:  10.7717/peerj.3083
PMCID: PMC5354075

Comparison of methods of alert acknowledgement by critical care clinicians in the ICU setting

Academic Editor: Harry Hochheiser

Abstract

Background

Electronic Health Record (EHR)-based sepsis alert systems have failed to demonstrate improvements in clinically meaningful endpoints. However, the effect of implementation barriers on the success of new sepsis alert systems is rarely explored.

Objective

To test the hypothesis time to severe sepsis alert acknowledgement by critical care clinicians in the ICU setting would be reduced using an EHR-based alert acknowledgement system compared to a text paging-based system.

Study Design

In one arm of this simulation study, real alerts for patients in the medical ICU were delivered to critical care clinicians through the EHR. In the other arm, simulated alerts were delivered through text paging. The primary outcome was time to alert acknowledgement. The secondary outcomes were a structured, mixed quantitative/qualitative survey and informal group interview.

Results

The alert acknowledgement rate from the severe sepsis alert system was 3% (N = 148) and 51% (N = 156) from simulated severe sepsis alerts through traditional text paging. Time to alert acknowledgement from the severe sepsis alert system was median 274 min (N = 5) and median 2 min (N = 80) from text paging. The response rate from the EHR-based alert system was insufficient to compare primary measures. However, secondary measures revealed important barriers.

Conclusion

Alert fatigue, interruption, human error, and information overload are barriers to alert and simulation studies in the ICU setting.

Keywords: Implementation barriers, Alert studies, Simulation studies, Electronic health record, Intensive care unit, Sepsis

Introduction

Electronic health record (EHR)-based, automated sepsis alert systems have failed to demonstrate improvements in clinically meaningful endpoints, such as Intensive Care Unit (ICU)/hospital length of stay (LOS) and mortality (Hooper et al., 2012; LaRosa et al., 2012; Nelson et al., 2011; Sawyer et al., 2011). This includes studies of ICU-specific and non-ICU specific alert systems. This also includes studies ranging in variation regarding degree of distinction between the detection of sepsis, severe sepsis, and/or septic shock (Dellinger et al., 2013). Clinically meaningful endpoints range from compliance with the international Surviving Sepsis Campaign (SSC) guidelines to hospital LOS, ICU LOS, and mortality. There are ICU-based and hospital wide means to trigger an alert for the early recognition of sepsis. Most EHRs now have a built in system to support this alert

Time to alert acknowledgement has been validated as one proxy for time to recognition of sepsis by critical care clinicians (Dziadzko et al., 2016). The failure of EHR-based, automated sepsis alert systems to be directly correlated with improvements in clinically meaningful endpoints is frequently attributed to limitations of detection algorithms and/or the need for clinical decision support (CDS) systems (Semler et al., 2015). Human factors, such as the impact of workflow changes or the influence of method of alert delivery, are known to be barriers to the implementation of new alert systems in the clinical setting (Harrison, Herasevich & Gajic, 2015).

As “alarm hazards” have been ranked as the top health technology hazard in the United States (ECRI-Institute, 2013), it is important to explore the effect of implementation of new alert systems on workflow changes and other human factors in the clinical setting. Despite outcome improvements in recent decades (Kaukonen et al., 2014), sepsis remains one of the most expensive in-hospital conditions (Torio, 2013). As one of the most technologically sophisticated hospital environments, the critical care setting serves as a model to explore the impact of implementation of new alert systems. Despite these technological advances, including widespread utilization of text messaging and smartphones by clinicians, text paging remains standard practice in the hospital setting (HIMSS Analytics, 2016). We hypothesized that time to severe sepsis alert acknowledgement by critical care clinicians in the ICU setting would be reduced using an EHR-based alert acknowledgement system compared to a text paging-based system.

Methods

Study design and setting

This study was performed in February 2015 in the medical ICU at Mayo Clinic in Rochester, MN, USA (Fig. 1). This study was not performed in a laboratory setting or simulation environment. This medical ICU has been described previously as a benchmark for evaluation of institutional performance (Afessa et al., 2005). Severe sepsis alerts were delivered to critical care clinicians, including attending physicians, fellows, residents, and nurse practitioners/physician assistants (NPs/PAs) using traditional HIPAA-compliant text paging. This study was approved by the Mayo Clinic Institutional Review Board (IRB) for clinician-participant enrollment by oral consent.

Figure 1
Schematic illustration of study design.

Study participants and medical ICU workflow

The medical ICU at Mayo Clinic consists of two physically adjacent 12-bed units, in close proximity to a nearby 9-bed respiratory care unit (RCU). For any given month, there are approximately 15 critical care attending physicians, six critical care fellows, four postgraduate year three internal medicine residents (PGY-3), six PGY-1 interns, and nine dedicated medical ICU NPs/PAs. There are 2 shifts: 6 am–6 pm (AM) and 6 pm–6 am (PM). On any given day, the morning shift is further divided into two teams. Team 1 is assigned to the majority of the medical ICU patients. Team 2 is assigned the remaining patients, as well as the RCU, which is further staffed by an additional fellow and dedicated NP/PA from the same group of approximately 40 clinicians in the medical ICU that month.

AWARE (Ambient Warning and Response Evaluation)

AWARE is the ICU-specific EHR system used in this study for patient viewer/monitoring. It was developed at Mayo Clinic and has been in routine clinical use in the medical ICU at Mayo Clinic since July 2012 (Pickering et al., 2015; Pickering et al., 2010). AWARE has been demonstrated to improve clinician task load, errors of cognition, and performance (Ahmed et al., 2011). AWARE is accessible from every computer workstation in this medical ICU, including bedside desktops, nursing stations, and clinician workrooms.

Severe sepsis alert system

The severe sepsis detection algorithm was developed at Mayo Clinic and implemented into AWARE in December 2014 (Harrison et al., 2015). Within AWARE, the severe sepsis alert system displays a passive, yellow alert icon when severe sepsis is detected. This yellow alert icon can also be activated manually by clinicians for specific patients, when severe sepsis is suspected, but not detected by the automated alert system. This yellow alert icon is automatically updated to display a passive, green alert icon within AWARE after completion of the four elements of the 3-hour Surviving Sepsis Campaign (SSC) bundle (Dellinger et al., 2013). Once activated, the yellow alert icon will persist for at least 6 h, unless completion of the 6 h SSC bundle is detected (green alert icon) or manual deactivation by clinicians occurs. In the context of prolonged severe sepsis and/or septic shock, the yellow alert icon can persist indefinitely. The green alert icon automatically reverts back to “no sepsis detected” after 3 h, unless additional automatic (or manual) activation occurs.

Study procedures

Clinicians agreed to participate in this study from February 02 through February 28 in 2015. The evening before each AM shift and the next PM shift, clinician participants for these upcoming shifts received a detailed email reminder with instructions (Fig. 2). The number of severe sepsis system alerts per shift through AWARE (yellow or green icon alerts) was entirely dependent on the number of septic patients in the medical ICU during any specific shift. Clinician participants randomly received no more than 3 simulated severe sepsis alerts per shift via text paging. In both cases, clinician participants were instructed to acknowledge all AWARE and traditional text paging severe sepsis alerts by email response. The difference between the time to severe sepsis alert activation in AWARE (or alert delivery via text page) and email response was defined as the time to alert acknowledgement.

Figure 2
Detailed daily email reminder to clinician participants with complete instructions.

Survey design

To compare clinician satisfaction between the EHR-based alert acknowledgement system and text paging-based system, clinician participants completed a structured, mixed quantitative/qualitative survey, upon completion of the severe sepsis alert acknowledgement portion of this study (full survey facsimile in Results, Fig. 3). These questions were designed partially on existing clinician satisfaction surveys of alert methods for use in the hospital and critical care settings (Embi, Jain & Harris, 2008; Wagner et al., 1998).

Figure 3
Facsimile of the structured, mixed quantitative/qualitative survey provided to the clinician participants with all quantitative results overlaid: median (IQR).

Statistical analysis

Severe sepsis alert system data was extracted directly from AWARE using METRIC Data Mart, a near-real time relational database of the complete EHR, which was developed at Mayo Clinic and has been described previously (Herasevich et al., 2010). Data was queried using JMP Pro (SAS Institute, Inc). Data collection and statistical analyses, such as the two-sided Student’s t-test and the Chi-squared test, were also performed in JMP Pro. For all statistical analyses, a p-value of less than 0.05 was considered to be statistically significant. For all median values from the survey results, interquartile range (IQR) was reported.

Results

Prior to initiation of this study, a 1-day feasibility pilot was performed in January 2015 using seven medical ICU clinicians. Based on the result of this feasibility study (data not shown), it was determined that a sufficiently high clinician participant alert acknowledgement rate could be obtained from both severe sepsis system alerts through AWARE (yellow or green icon alerts) and simulated severe sepsis alerts through traditional text paging in the ICU setting. Based on the results of this feasibility pilot, participant instructions were optimized (Fig. 2).

Of the 40 clinicians staffing the medical ICU in February 2015, 13 (32%) were recruited to participate in this study. However, it was decided after two weeks (February 02 AM through February 15 PM) to prematurely terminate this study, due to sufficient statistical power for time to alert acknowledgement analysis, as well as feedback from clinician participants. As a result, it was necessary to exclude one NP/PA due to unavailability in the medical ICU during this shortened study period (RCU only). Ultimately, 12 clinicians participated: five NPs/PAs (out of nine), three attending physicians (out of 15), two fellows (out of six), two PGY-3s (out of four), and zero PGY-1s (out of six). The median number of potential AWARE alert acknowledgements per shift was two (IQR 1–4). The minimum and maximum numbers were zero and five. The number of patients who trigged at least 1 severe sepsis system alert through AWARE (yellow or green icon alert) was 28. Of the 28 shifts that occurred during this shortened study period, 23 shifts (82%) were covered by at least one participant (Table 1).

Table 1
Number of shifts per clinician participant and number of participants per shift.

The alert acknowledgement rate from the severe sepsis alert system through AWARE was 3% (N = 148) and 51% (N = 156) from simulated severe sepsis alerts through text paging (Table 2). Time to alert acknowledgement from the severe sepsis alert system through AWARE was median 274 min (N = 5) and median 2 min (N = 80) from simulated severe sepsis alerts through text paging. The 5 alert acknowledgements from the severe sepsis alert system through AWARE came from only three clinician participants (NP/PA #01, NP/PA #04, and NP/PA #05), while all 12 participants acknowledged at least one simulated severe sepsis alert through text paging.

Table 2
Comparison of alert response rate and median time to alert acknowledgement between the severe sepsis alert system through AWARE and simulated severe sepsis alerts through traditional text paging.

All participants completed a structured, mixed quantitative/qualitative survey. For the quantitative portion of the survey (Fig. 3), clinicians found alert by AWARE to be slightly less disruptive than alert by text paging. Clinicians found acknowledgement of AWARE and text paging alerts to be equally disruptive. When AWARE and text paging alerts were directly compared, a clear preference for text paging for both “urgent” and “non-urgent” alerts was present. When asked to “select one or more” (text paging, AWARE, email, phone call, text message, or other), the results for non-urgent alerts were mixed. However, when asked the same question for urgent alerts, the preference was once again clearly for text paging.

For the qualitative portion of the survey (Fig. 4), 11 out of 12 clinician participants provided “at least one suggestion for improving alert/notification delivery”. Clinicians commented on inhomogeneous overall use of AWARE in the medical ICU, despite implementation several years prior (July 2012). Of the same 11 clinicians, four provided “any additional comments”: the same three NPs/PAs who responded to at least one alert acknowledgement from the severe sepsis alert system through AWARE, as well as Attending #03. A clear theme concerning alert fatigue, interruption, human error, and information overload was present.

Figure 4
All qualitative responses to the structured, mixed quantitative/qualitative survey reproduced in their entirety, including typographical errors.

An informal group interview in the form of a noon pizza party was held to thank all clinician participants and gather additional feedback on the barriers to clinician participation and engagement in this implementation study. The four clinicians who attended were once again the same four clinicians who provided “any additional comments” on the survey. The statements regarding alert fatigue, interruption, human error, and information overload were reinforced, despite a strong interest from these clinicians to participate. Regarding inhomogeneous overall use of AWARE in the medical ICU, specific attention was drawn to a particular lack of interest from residents to use AWARE, as well as a lack of interest from both residents and fellows to participate in any research study during their required rotations through the medical ICU, including implementation of the severe sepsis alert system.

Discussion

We hypothesized that time to severe sepsis alert acknowledgement by critical care clinicians in the ICU setting would be reduced using an EHR-based alert acknowledgement system compared to a text paging-based system. Based on the limited alert acknowledgement response rate using the severe sepsis alert system compared to traditional text paging, it was not possible to answer this hypothesis. However, feedback from the structured, mixed quantitative/qualitative survey, as well as the informal group interview, provided invaluable insight into the sources of this limited acknowledgement response rate. Implementation barriers included human factors, such as alert fatigue, interruption, human error, and information overload.

With the implementation of increasingly sophisticated EHR systems, interest in the development of novel automated detection and alert systems has increased (Bourgault et al., 2014). However, investigation into best methods of alert delivery (text paging, EHR systems, email, phone calls, and/or text messaging) for urgent and non-urgent alerts in the hospital setting is limited (Gill, Kamath & Gill, 2012). Investigation into the most appropriate clinician for alert delivery is also limited (Zhang et al., 2003). Monitoring and alert systems have been developed for patient use in the home setting (Steinman et al., 2011; Tchalla et al., 2012). However, there has been comparatively limited investigation into methods of alert delivery to clinicians in the hospital setting (Loo et al., 2011). Interestingly, many of these studies have been performed in the geriatric patient population, but not in the ICU setting, where the average patient age is often 65 or older (Seferian & Afessa, 2006). Thus, there is a clear need for further systematic exploration of human factors barriers to the implementation of new alert systems in the ICU setting, such as the impact of workflow changes and the influence of method of alert delivery.

Implementation of automated detection and alert systems without consideration of these factors is known to have the potential to result in alert fatigue (Singh et al., 2013), interruption (Hodgetts & Jones, 2007), human error (Bates et al., 1998), and information overload (Stokstad, 2001). Recognition of the importance of alert fatigue in the hospital setting has increased significantly in recent years (Herasevich et al., 2013). However, implementation of automated alert systems generally must be performed in the context of information overload and complex task interruption (Eppler & Mengis, 2004). It is also known that information overload can alter alert perception in the medical setting (Glassman et al., 2006). This can cause clinicians to perceive alert systems negatively and deter future use (Harrison et al., 2016). Thus, the task of generating clinically meaningful alerts while concurrently minimizing information overload and task interruption is challenging.

Clinician-participant comments provided valuable insight regarding preferences for method of alert delivery. Although there was a clear preference for receiving urgent alerts through text paging, additional investigation is required to specifically explore the rationale for this preference. Understanding the rationale for this preference may reduce the barriers to answering the primary objective of this study, which was comparison of time to severe sepsis alert acknowledgement methods by critical care clinicians in the ICU setting. These secondary outcomes revealed important barriers to the inability to answer the primary outcome, which are applicable and generalizable to future studies.

Even the reluctance of clinicians to participate in this research study, as revealed during the post-study discussion, has the potential to confound interpretation of the results of this study. Whether this reluctance is the result of information overload, the Hawthorne effect, and/or some other factor(s), clinicians are currently struggling to find a balance between understanding of the importance of their participation in clinical research studies and “study fatigue.”

This study has several limitations: (1) This was a single-center study at an academic medical center. Well-established biases and potential confounders are known to be present with this particular study design (Straus et al., 2005); (2) Unlike the severe sepsis system alerts through AWARE (yellow or green icon alerts), the severe sepsis alerts through text paging were simulated. Comparing non-simulated alerts to simulated alerts may introduce additional confounders into the interpretation of the results of this study; (3) Although not investigated in this study, the feasibility of severe sepsis alert delivery using an EHR-based, automated mobile app for smartphones has been validated (Dziadzko et al., 2016); (4) The significant range of clinical experience of clinician-participants introduces study bias. The potential application of this technology for the future of clinical practice and clinical research should not be ignored. Ultimately, a multi-center, non-simulated study in the ICU setting is required to address various aspects of these limitations.

Conclusion

It could not be determined whether an automated alert for severe sepsis reduced time to alert acknowledgement by critical care clinicians in the ICU setting compared to text paging. This was due to an extremely limited alert acknowledgement response rate using the severe sepsis alert system compared to traditional text paging. Implementation barriers, including human factors—such as alert fatigue, interruption, human error, and information overload—were determined to be an important source of this finding.

Funding Statement

This work was supported by Center for Medicare and Medicaid Innovation (CMMI) Health Care Innovation Award “Patient Centered Cloud-based Electronic System: Ambient Warning and Response Evaluation (ProCCESs AWARE)” (1C1CMS330964-01-00). AMH is supported by a doctoral dissertation grant from the AHRQ (R36 HS022799). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

The following grant information was disclosed by the authors:

Center for Medicare and Medicaid Innovation (CMMI) 1C1CMS330964-01-00.
AHRQ R36 HS022799.

Additional Information and Declarations

Competing Interests

AWARE is patent pending (US 2010/0198622, 12/697861, PCT/US2010/022750). Drs. Herasevich, Gajic, and Pickering and Mayo Clinic have a financial conflict of interest relating to licensed technology described in this paper. This research has been reviewed by the Mayo Clinic Conflict of Interest Review Board and is being conducted in compliance with Mayo Clinic Conflict of Interest Policies.

Author Contributions

Andrew M. Harrison performed the experiments, analyzed the data, wrote the paper, prepared figures and/or tables, reviewed drafts of the paper.

Charat Thongprayoon, Christopher A. Aakre and Jack Y. Jeng performed the experiments, reviewed drafts of the paper.

Mikhail A. Dziadzko analyzed the data, reviewed drafts of the paper.

Ognjen Gajic, Brian W. Pickering and Vitaly Herasevich conceived and designed the experiments, reviewed drafts of the paper.

Human Ethics

The following information was supplied relating to ethical approvals (i.e., approving body and any reference numbers):

This study was approved by the Mayo Clinic Institutional Review Board (IRB 13-003325).

Patent Disclosures

The following patent dependencies were disclosed by the authors:

US 2010/0198622, 12/697861, PCT/US2010/022750.

Data Availability

The following information was supplied regarding data availability:

The raw data is included in the tables and figures of the manuscript.

References

Afessa et al. (2005) Afessa B, Keegan MT, Hubmayr RD, Naessens JM, Gajic O, Long KH, Peters SG. Evaluating the performance of an institution using an intensive care unit benchmark. Mayo Clinic Proceedings Mayo Clinic. 2005;80:174–180. doi: 10.4065/80.2.174. [PubMed] [Cross Ref]
Ahmed et al. (2011) Ahmed A, Chandra S, Herasevich V, Gajic O, Pickering BW. The effect of two different electronic health record user interfaces on intensive care provider task load, errors of cognition, and performance. Critical Care Medicine. 2011;39:1626–1634. doi: 10.1097/CCM.0b013e31821858a0. [PubMed] [Cross Ref]
Bates et al. (1998) Bates DW, Leape LL, Cullen DJ, Laird N, Petersen LA, Teich JM, Burdick E, Hickey M, Kleefield S, Shea B, Vander Vliet M, Seger DL. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA: The Journal of the American Medical Association. 1998;280:1311–1316. doi: 10.1001/jama.280.15.1311. [PubMed] [Cross Ref]
Bourgault et al. (2014) Bourgault AM, Heath J, Hooper V, Sole ML, Waller JL, Nesmith EG. Factors influencing critical care nurses’ adoption of the AACN practice alert on verification of feeding tube placement. American Journal of Critical Care. 2014;23:134–144. doi: 10.4037/ajcc2014558. [PubMed] [Cross Ref]
Dellinger et al. (2013) Dellinger RP, Levy MM, Rhodes A, Annane D, Gerlach H, Opal SM, Sevransky JE, Sprung CL, Douglas IS, Jaeschke R, Osborn TM, Nunnally ME, Townsend SR, Reinhart K, Kleinpell RM, Angus DC, Deutschman CS, Machado FR, Rubenfeld GD, Webb SA, Beale RJ, Vincent JL, Moreno R. Surviving sepsis campaign: international guidelines for management of severe sepsis and septic shock: 2012. Critical Care Medicine. 2013;41:580–637. doi: 10.1097/CCM.0b013e31827e83af. [PubMed] [Cross Ref]
Dziadzko et al. (2016) Dziadzko MA, Harrison AM, Tiong IC, Pickering BW, Moreno Franco P, Herasevich V. Testing modes of computerized sepsis alert notification delivery systems. BMC Medical Informatics and Decision Making. 2016;16:156. doi: 10.1186/s12911-016-0396-y. [PMC free article] [PubMed] [Cross Ref]
ECRI-Institute (2013) ECRI-Institute 2014 top 10 list of health technology hazards. Health Devices Journal. 2013;42(11):354–380.
Embi, Jain & Harris (2008) Embi PJ, Jain A, Harris CM. Physicians’ perceptions of an electronic health record-based clinical trial alert approach to subject recruitment: a survey. BMC Medical Informatics and Decision Making. 2008;8:13. doi: 10.1186/1472-6947-8-13. [PMC free article] [PubMed] [Cross Ref]
Eppler & Mengis (2004) Eppler MJ, Mengis J. The concept of information overload: a review of literature from organization science, accounting, marketing, MIS, and related disciplines. Information Society. 2004;20:325–344. doi: 10.1080/01972240490507974. [Cross Ref]
Gill, Kamath & Gill (2012) Gill PS, Kamath A, Gill TS. Distraction: an assessment of smartphone usage in health care work settings. Risk Management and Healthcare Policy. 2012;5:105–114. doi: 10.2147/RMHP.S34813. [PMC free article] [PubMed] [Cross Ref]
Glassman et al. (2006) Glassman PA, Belperio P, Simon B, Lanto A, Lee M. Exposure to automated drug alerts over time: effects on clinicians’ knowledge and perceptions. Medical Care. 2006;44:250–256. doi: 10.1097/01.mlr.0000199849.08389.91. [PubMed] [Cross Ref]
Harrison et al. (2016) Harrison AM, Gajic O, Pickering BW, Herasevich V. Development and implementation of sepsis alert systems. Clinics in Chest Medicine. 2016;37:219–229. doi: 10.1016/j.ccm.2016.01.004. [PMC free article] [PubMed] [Cross Ref]
Harrison, Herasevich & Gajic (2015) Harrison AM, Herasevich V, Gajic O. Automated sepsis detection, alert, and clinical decision support: act on it or silence the alarm? Critical Care Medicine. 2015;43:1776–1777. doi: 10.1097/CCM.0000000000001099. [PubMed] [Cross Ref]
Harrison et al. (2015) Harrison AM, Thongprayoon C, Kashyap R, Chute CG, Gajic O, Pickering BW, Herasevich V. Developing the surveillance algorithm for detection of failure to recognize and treat severe sepsis. Mayo Clinic Proceedings. 2015;90:166–175. doi: 10.1016/j.mayocp.2014.11.014. [PubMed] [Cross Ref]
Herasevich et al. (2013) Herasevich V, Kor DJ, Subramanian A, Pickering BW. Connecting the dots: rule-based decision support systems in the modern EMR era. Journal of Clinical Monitoring and Computing. 2013;27(4):443–448. doi: 10.1007/s10877-013-9445-6. [PubMed] [Cross Ref]
Herasevich et al. (2010) Herasevich V, Pickering BW, Dong Y, Peters SG, Gajic O. Informatics infrastructure for syndrome surveillance, decision support, reporting, and modeling of critical illness. Mayo Clinic Proceedings. 2010;85:247–254. doi: 10.4065/mcp.2009.0479. [PMC free article] [PubMed] [Cross Ref]
HIMSS Analytics (2016) HIMSS Analytics The hidden cost of pagers in healthcare: how outmoded technology is draining healthcare it budgets. http://www.tigertext.com/wp-content/uploads/Report-Hidden-Cost-of-Pagers.pdf 2016
Hodgetts & Jones (2007) Hodgetts HM, Jones DM. Reminders, alerts and pop-ups: the cost of computer-initiated interruptions. vol. 4550. Springer; Berlin: 2007. pp. 818–826. (Lecture notes in computer science).
Hooper et al. (2012) Hooper MH, Weavind L, Wheeler AP, Martin JB, Gowda SS, Semler MW, Hayes RM, Albert DW, Deane NB, Nian H, Mathe JL, Nadas A, Sztipanovits J, Miller A, Bernard GR, Rice TW. Randomized trial of automated, electronic monitoring to facilitate early detection of sepsis in the intensive care unit*. Critical Care Medicine. 2012;40:2096–2101. doi: 10.1097/CCM.0b013e318250a887. [PMC free article] [PubMed] [Cross Ref]
Kaukonen et al. (2014) Kaukonen KM, Bailey M, Suzuki S, Pilcher D, Bellomo R. Mortality related to severe sepsis and septic shock among critically ill patients in Australia and New Zealand, 2000–2012. JAMA: The Journal of the American Medical Association. 2014;311:1308–1316. doi: 10.1001/jama.2014.2637. [PubMed] [Cross Ref]
LaRosa et al. (2012) LaRosa JA, Ahmad N, Feinberg M, Shah M, Dibrienza R, Studer S. The use of an early alert system to improve compliance with sepsis bundles and to assess impact on mortality. Critical Care Research and Practice. 2012;2012:980369. doi: 10.1155/2012/980369. [PMC free article] [PubMed] [Cross Ref]
Loo et al. (2011) Loo TS, Davis RB, Lipsitz LA, Irish J, Bates CK, Agarwal K, Markson L, Hamel MB. Electronic medical record reminders and panel management to improve primary care of elderly patients. Archives of Internal Medicine. 2011;171:1552–1558. doi: 10.1001/archinternmed.2011.394. [PubMed] [Cross Ref]
Nelson et al. (2011) Nelson JL, Smith BL, Jared JD, Younger JG. Prospective trial of real-time electronic surveillance to expedite early care of severe sepsis. Annals of Emergency Medicine. 2011;57:500–504. doi: 10.1016/j.annemergmed.2010.12.008. [PubMed] [Cross Ref]
Pickering et al. (2015) Pickering BW, Dong Y, Ahmed A, Giri J, Kilickaya O, Gupta A, Gajic O, Herasevich V. The implementation of clinician designed, human-centered electronic medical record viewer in the intensive care unit: a pilot step-wedge cluster randomized trial. International Journal of Medical Informatics. 2015;84:299–307. doi: 10.1016/j.ijmedinf.2015.01.017. [PubMed] [Cross Ref]
Pickering et al. (2010) Pickering BW, Herasevich V, Ahmed A, Gajic O. Novel representation of clinical information in the icu—developing user interfaces which reduce information overload. Applied Clinical Informatics. 2010;1(2):116–131. doi: 10.4338/ACI-2009-12-CR-0027. [PMC free article] [PubMed] [Cross Ref]
Sawyer et al. (2011) Sawyer AM, Deal EN, Labelle AJ, Witt C, Thiel SW, Heard K, Reichley RM, Micek ST, Kollef MH. Implementation of a real-time computerized sepsis alert in nonintensive care unit patients. Critical Care Medicine. 2011;39:469–473. doi: 10.1097/CCM.0b013e318205df85. [PubMed] [Cross Ref]
Seferian & Afessa (2006) Seferian EG, Afessa B. Demographic and clinical variation of adult intensive care unit utilization from a geographically defined population. Critical Care Medicine. 2006;34:2113–2119. doi: 10.1097/01.CCM.0000227652.08185.A4. [PubMed] [Cross Ref]
Semler et al. (2015) Semler MW, Weavind L, Hooper MH, Rice TW, Gowda SS, Nadas A, Song Y, Martin JB, Bernard GR, Wheeler AP. An electronic tool for the evaluation and treatment of sepsis in the ICU: a randomized controlled trial. Critical Care Medicine. 2015 doi: 10.1097/CCM.0000000000001020. [PMC free article] [PubMed] [Cross Ref]
Singh et al. (2013) Singh H, Spitzmueller C, Petersen NJ, Sawhney MK, Sittig DF. Information overload and missed test results in electronic health record-based settings. JAMA Internal Medicine. 2013;173(8):702–704. doi: 10.1001/2013.jamainternmed.61. [PMC free article] [PubMed] [Cross Ref]
Steinman et al. (2011) Steinman MA, Handler SM, Gurwitz JH, Schiff GD, Covinsky KE. Beyond the prescription: medication monitoring and adverse drug events in older adults. Journal of the American Geriatrics Society. 2011;59:1513–1520. doi: 10.1111/j.1532-5415.2011.03500.x. [PMC free article] [PubMed] [Cross Ref]
Stokstad (2001) Stokstad E. Information overload hampers biology reforms. Science. 2001;293:1609. doi: 10.1126/science.293.5535.1609. [PubMed] [Cross Ref]
Straus et al. (2005) Straus SE, Richardson WS, Glasziou P, Haynes RB. Third edition. Edinburgh: Churchill Livingstone; 2005. Evidence-based medicine: how to practice and teach EBM.
Tchalla et al. (2012) Tchalla AE, Lachal F, Cardinaud N, Saulnier I, Bhalla D, Roquejoffre A, Rialle V, Preux PM, Dantoine T. Efficacy of simple home-based technologies combined with a monitoring assistive center in decreasing falls in a frail elderly population (results of the Esoppe study) Archives of Gerontology and Geriatrics. 2012;55:683–689. doi: 10.1016/j.archger.2012.05.011. [PubMed] [Cross Ref]
Torio (2013) Torio CMAR. National inpatient hospital costs: the most expensive conditions by Payer, 2011. HCUP Statistical Brief #160. 2013. https://www.hcup-us.ahrq.gov/reports/statbriefs/sb160.jsp https://www.hcup-us.ahrq.gov/reports/statbriefs/sb160.jsp
Wagner et al. (1998) Wagner MM, Eisenstadt SA, Hogan WR, Pankaskie MC. Preferences of interns and residents for E-mail, paging, or traditional methods for the delivery of different types of clinical information. Proceedings of the AMIA annual symposium; 1998. pp. 140–144. [PMC free article] [PubMed]
Zhang et al. (2003) Zhang J, Johnson TR, Patel VL, Paige DL, Kubose T. Using usability heuristics to evaluate patient safety of medical devices. Journal of Biomedical Informatics. 2003;36:23–30. doi: 10.1016/S1532-0464(03)00060-1. [PubMed] [Cross Ref]

Articles from PeerJ are provided here courtesy of PeerJ, Inc