|Home | About | Journals | Submit | Contact Us | Français|
The dynamic and distributed work environment in critical care requires a high level of collaboration among clinical team members and a sophisticated task coordination system to deliver safe, timely and effective care. A complex cognitive system underlies the decision-making process in such cooperative workplaces. This methodological review paper addresses the issues of translating cognitive research to clinical practice with a specific focus on decision-making in critical care, and the role of information and communication technology to aid in such decisions. Examples are drawn from studies of critical care in our own research laboratories. Critical care, in this paper, includes both intensive (inpatient) and emergency (outpatient) care. We define translational cognition as the research on basic and applied cognitive issues that contribute to our understanding of how information is stored, retrieved and used for problem-solving and decision-making. The methods and findings are discussed in the context of constraints on decision-making in real world complex environments and implications for supporting the design and evaluation of decision support tools for critical care health providers.
Several researchers have proposed that the healthcare system can be characterized as a complex system. In a series of articles in the British Medical Journal, Wilson, Plsek and colleagues introduced complexity science in a general medical literature  suggesting applications for healthcare organization  and clinical practice . Smith argues that an emergency department is a “paradigmatic complex system” . This argument rests on the unpredictability of both patients’ clinical conditions and clinicians’ work patterns, the vast decision space and incomplete evidence that complicate clinical decision-making, and the inherent unpredictability of the system as a whole. This methodological review paper focuses on the cognitive dimensions of the complex critical care environment  and their implications for decision support. Although important, we do not explicitly cover other non-cognitive dimensions in this paper.
It is well known that errors increase as a function of complexity. The phrase “error in evolution” denotes the progression of a series of small errors towards a cumulative adverse event. Carlson and Doyle argue that complexity confers behavior that is “robust yet fragile” , resulting in a system that is tolerant to common perturbations but vulnerable to failure of certain individual components. Rasmussen’s “error margins” refer to the limits of the cognitive capacity of a given system [7–9]. When these margins are approached, for example, when operating at maximum productivity, the system is driven toward the boundaries of safety.
Theoretical and methodological notions from the field of distributed cognition (discussed in Section 3.2) and concepts from the complex systems literature are pertinent to the study of the Intensive Care Unit (ICU) and Emergency Department (ED) as complex cognitive systems. There are four major properties of complex adaptive systems: aggregation, non-linearity, flow, and diversity . In the ICU, clinicians aggregate in response to changing patient priorities, for example in the event of an emergency resuscitation. The similarity between aggregate members is cognitive: they are united temporarily by a shared goal where “the local rules of interaction change as the system evolves or develops” (p. xii) , making it a nonlinear situation. Local interactions between clinician-agents and artifacts are fundamental to collaborative decision-making. Furthermore, with each decision-action event, the patient is perturbed, providing real-time feedback and prompting re-evaluation of “correct” decisions with unintended consequences. ICU decisions emerge from the flow of information between clinical team members. This team is diverse: it includes attending physicians, residents, nurses, and pharmacists, each with a particular expertise and perspective. Information also flows between artifacts and between artifacts and human agents, such as patient monitors, clinical images, medical records, and clinicians. Research studies need to address factors that push decision-makers toward unsafe boundaries, and then make these identified decision processes at crucial points explicit to decision-makers. Providing decision support in such an environment will rest upon findings from such cognitive studies.
Decision support in the clinical environment is defined as “advice and guidance offered by information and communication technology to aid the problem solving and decision making of health care providers” (p. 6) . Computer-based decision support can be seen as the use of information technology to bring relevant knowledge for the well being of the patient.
Computer-based decision support tools are supposed to help practitioners avoid errors, ensure quality and improve efficiency in healthcare. Yet, there appears to be resistance to the use of such systems. How well a system is accepted by the practitioners depends on the degree to which it supports them in achieving their immediate goals.
In this paper, we review cognitive methodologies for the study of medical cognition and their applications to critical care settings to understand the nature of clinical decision-making and use these findings to inform the design of computer-based decision support.
Translational research is typically defined as the transfer of knowledge from one domain to another. Translational clinical research is specifically about the translation of basic biomedical research findings from bench to bedside. We define translational cognition as the translation of research of basic and applied cognitive issues to the understanding of medical cognition and the evaluation, design, and implementation of decision support tools for healthcare. The translation can be for general principles, generic methodologies, and specific research findings.
In this paper, we include both intensive care (inpatient) and emergency care (outpatient) in our definition of critical care. We focus on the following aspects of translational cognition: the application of general cognitive principles to healthcare domains, and more specifically, how findings from studies of cognition and decision-making in critical care environments can be used to develop decision support tools. To address these issues, we will first discuss why we need to study real world decision-making in order to understand how people make decisions under various constraints. We then describe the framework of distributed cognition which provides a language, a frame of reference, and a perspective for cognitive studies of critical care. The theory of cognitive load, which lends itself to decision-making research, is subsequently introduced. We continue with a review of the cognitive foundation of medical errors, including how the nature of the critical care environment places a cognitive overload on clinicians, thus increasing chances for error, and a cognitive taxonomy for categorizing errors, such that we can outline systematic, principled methods for design of improved medical error reporting systems for the purpose of providing decision support. We then provide an overview of the methods and techniques used to study cognition by other researchers and by our research teams. In the second half of this paper, we discuss specific examples from our studies in critical care that provide support for the value of cognitive methods in understanding decision-making in these environments. These examples are for illustrative purposes only; they are not inclusive. We have selected our own examples because of our familiarity with them and our access to the details of these studies. Subsequently, we integrate our findings from the critical care studies and make recommendations for developing decision support tools, including information technology interventions, to improve quality and safety in critical care.
There are two frameworks that provide the foundation for our research on cognition and errors in critical care: naturalistic decision-making and distributed cognition. We argue that a naturalistic approach to understanding decision-making in medical settings, such as critical care, is necessary for the eventual development of decision support tools for error prevention and management. In conjunction with this naturalistic approach, we conceptualize the critical care environment in terms of the framework of distributed cognition, which is crucial for the understanding, identification, and management of the various factors responsible for the quality and safety of critical care.
A large body of empirical research on medical decision-making has accumulated over the past half century. Much of this work has been conducted within the classical paradigm of decision-making. However, it became increasingly apparent that there are several weaknesses to this paradigm that undermine the value of conclusions of this body of research for the development of effective decision-support technologies for healthcare settings. Thus, it is essential to develop a broader-based and more valid foundation of “basic science” decision research for the study of medical and other decision-making as it occurs in naturalistic settings. This has led to the emergence of the naturalistic decision-making (NDM) approach [5, 13–15].
In contrast to the classical decision-making (CDM) paradigm, the NDM approach focuses on developing in-depth, ecologically valid, descriptive models of decision-making performance, which necessitate the use of a wide range of qualitative (and quantitative) methodologies. Whereas CDM studies are usually controlled laboratory studies, NDM research is conducted in real-world settings, where multiple cognitive, social, affective and environmental factors influence decisions and behavior. In such settings, decision strategies are needed that adapt to the constraints of the particular situation, which include stress, time pressure, and risk, among other factors. These strategies may be the product of individuals or teams. Research has moved towards investigating team interaction and performance, as communication and collaboration in the medical environment is critical for successful continuity of the daily workflow.
Because technology mediates clinical performance, decision support technologies need to be conceptualized in the context of actual practice. Towards this end, there is a need for a deeper understanding of clinical performance in real-world settings (under sub-optimal conditions) by both novices and experts, the effects of technology propagating through the different layers of an organization, and the adaptiveness of health professionals to an increasingly technologically-mediated world. Therefore, acquired knowledge of decision-making in complex, real-world environments, based on a cognitive framework, is necessary for designing and implementing technologies that can facilitate decision processes in real-world clinical settings, such as in critical care. In conjunction with this naturalistic approach, we conceptualize the critical care environment in terms of the framework of distributed cognition, which is described in the next section.
Distributed cognition is the theoretical development in the distributed system approach, originally conceptualized by Hutchins and colleagues and later expanded by others [16–23]. It has previously been applied to the study of cognitive systems underlying task performance on naval vessels  and in the airplane cockpit . It is a scientific discipline that is concerned with how cognitive activity is distributed across internal human minds, external cognitive artifacts, and groups of people, and how it is distributed across space and time (see Figure 1) [16, 18, 22–30]. In this view, people’s intelligent behavior results from interactions with external cognitive artifacts and with other people, and people’s activities in concrete situations are guided, constrained, and to some extent, determined by the physical, cultural, social, and historical contexts in which they are situated [31, 32], as seen in a natural working environment. The unit of analysis is a distributed cognitive system composed of a group of people interacting with external cognitive artifacts (e.g., cockpit of a commercial airplane, emergency department in a hospital, and an air force squadron unit). In general terms, we describe the components of a distributed cognitive system as internal and external representations. Internal representations are the knowledge and structure in individuals’ minds; and external representations are the knowledge and structure in the external environment .
The following describes how cognition is distributed between an individual mind and an external artifact and between individual minds. There are a wide variety of complex tasks that require the processing of information that is distributed across internal minds and external artifacts. External artifacts are defined as objects (e.g., light switches), symbols (e.g., writing), tools (e.g., slide rule, abacus, calculator, computer), and other entities that change, support, or modify human cognitive behavior. It is the interwoven processing of internal and external information that generates much of a person’s intelligence. For example, let us consider multiplying 965 by 273 using paper and pencil. The internal representations are the meanings of individual symbols (e.g., the numerical value of the arbitrary symbol “5” is five), the addition and multiplication tables, and arithmetic procedures, which have to be retrieved from memory. The external representations are the shapes and positions of the symbols and the spatial relations of partial products, which can be perceptually inspected from the environment. To perform this task, people need to process the information perceived from external representations and the information retrieved from internal representations in an interwoven, integrative, and dynamic manner. Zhang & Norman  developed a framework of distributed representations to account for the behavior in these types of distributed cognitive tasks. One important aspect emphasized by distributed cognition research is that external representations are more than inputs and stimuli to the internal mind. External representations have many non-trivial properties. For many tasks, external representations are intrinsic components, without which the tasks either cease to exist or completely change in nature.
Cognition can also be distributed across a group of individuals. There are two different views of how this occurs. The reductionist view considers that the cognitive properties of a group can be entirely determined by the properties of individuals. In this view, to understand group behavior, all we need is to understand the properties of individuals. In contrast, the interactionist view considers that the interactions among the individuals can produce emergent group properties that cannot be reduced to the properties of the individuals. In this view, to study group behavior we need to examine not only the properties of individuals but also the interactions among the individuals. Examples of emergent group properties include group affect , collective efficacy , shared mental models, and transactive memory systems .
Hollnagel and Woods [36, 37] recently offered a systematic account of distributed cognition in what they called joint cognitive systems. They consider people and technology as a joint cognitive system for work. Technology and automation, they found, do not necessarily lead to simplification of work. Rather, they introduce more complexity and adaptation. In other words, when a new technology is introduced, people adapt their strategies and artifacts to work around difficulties and accomplish their goals as responsible agents.
In our research, we find that the concept of distributed cognition is very valuable for accounting for activities in critical care, where activities are distributed between people and artifacts, across members of groups, and across space and time [16, 24]. Therefore, the quality of patient care is measured as a function of how well the whole system operates (interaction of individuals, teams, information systems, and the critical care environment). In this framework, medical errors are viewed as inevitable but cognitively useful phenomena that cannot be totally eliminated. They are products of the distributed cognitive activities in the distributed systems that are grounded in complex physical, social, and cultural environments. In order to manage errors during clinical decision-making, it is critical to understand how decisions are made and what underlying cognitive mechanisms are used to process information during interactions with patients, colleagues, and technology in these systems. Albolino’s work on sensemaking (social understanding) in the intensive care unit [38, 39] is related to this area of a high tempo and highly uncertain environment. Results from this work have shown that clinicians in the intensive care unit balance their work between collaborative “sensemaking” episodes and routine work activities in order to organize future courses of action.
People make use of adaptive strategies to perceive, interpret, organize and communicate information, but their actions are constrained by functional characteristics of the system and constraints of the environment . Reasoning and interpretation of information are influenced by cultural expectations, for example, by the assumptions of responsibility attributed to specific professional roles (e.g., physicians, nurses, clerks), and by the limitations of human attention and memory on cognitive processing . In critical care, the complexity of performing even routine tasks is increased by the constraints of time, insufficient or unavailable information, by stress, and by frequent and unpredictable interruptions . Tasks are often completed in a non-linear progression, as equipment and people move around and need to be located, break-in-tasks need to be attended to as they arise , and staff need to temporarily resign from a task due to interruptions . Work in such a highly interruption-driven environment puts extraneous demands on the cognitive resources of each clinician.
As shown by France and colleagues , temporary interruptions appear to be a major source of inefficiency in emergency care, and likely a major threat to patient safety, as in other similarly demanding environments such as aviation . In a study comparing clinician workflow in an emergency department with that in a primary care clinic, researchers found that emergency physicians were interrupted at a much higher rate (9.7 times an hour) than primary care physicians (3.9 times an hour), and that emergency physicians were involved more often in simultaneous care of multiple patients . Although interruptions are necessary and important to maintain awareness of the continuously changing working environment, inappropriate management of interruptions can have a detrimental effect on performance, efficiency and error rate.
Coiera and others have studied how communication patterns contribute to the interrupt-driven environment of the ED [44–47]. Results from these studies indicate that healthcare providers’ communication preference for synchronous communication to obtain information contributes to the number of interruptions experienced by doctors and nurses. For example, in analysis of nearly 20 hours of observation data for doctors and nurses working in an Australian ED, one study found that 35.5% of the communication events were interruptions . This resulted in a rate of 14.8 interruptions per person per hour. In a study of doctors and nurses employed in a British general district, it was reported that communication behaviors contributed to an interruptive workplace . The researchers reported that while the medical staff received multiple paging interruptions, they generated twice as many outgoing calls. The medical staff also experienced interruptions through face-to-face contacts. Two studies specifically investigated paging as a source of interruption for clinicians working in a hospital, finding that a significant amount of interruptions were a result of pagers [48, 49].
Earlier in this paper, we stated that the goal of the distributed cognition approach to workflow, collaboration, reasoning, and human-computer interaction research is to understand the shifting meaning of information in the context of the environment  and explain how it is transformed as it propagates through the system and circulates among collaborating agents. This insight may guide the selection of appropriate technological interventions for specific problems and avoid adding an undesirable level of complexity to an already difficult process. For example, pager interruptions are major issues for healthcare and there is an urgent need to manage such interruptions , which exemplifies an information delivery system that needs to be better integrated into clinical work. The most effective interventions will likely center on three approaches: the use of technology to automate information flow, the elimination of unnecessary interruptions, and the development of optimal means of communication to manage unavoidable interruptions.
Recently, France and colleagues  conducted a study on the effects of implementing an electronic whiteboard in the ED on physician work, communication and workload. Results showed that physicians in this study performed more tasks and were interrupted less frequently with the introduction of the electronic whiteboard than physicians in previous studies in conventional EDs without such technology. In addition, only 9% of the interruptions affected direct patient care. Although the presence of the electronic whiteboard increased work and communication efficiency, interruptions continued to occur, suggesting the need for more comprehensive interventions, not solely limited to the introduction of information technology into the environment. It is also possible that some minimal interruptions are necessary for “efficiency”, since critical care personnel provide patient care to more than one patient at a time in this environment. Xiao and colleagues [50, 51] have also conducted work on cognitive artifacts (i.e., cognitive properties of a whiteboard) and its effects on collaboration in a Level I trauma center operating room unit. For example, the public display whiteboard was used as an efficient tool for supporting collaborative work and for inventing new ways of representing information, using the magnetic objects on the board. Such tangible aspects of highly collaborative healthcare work have profound implications for research and development of information and communication technology despite the tendency to model work as flow of abstract data items (see also ).
Based on findings from studies on interruptions and communication in critical care, Coiera and colleagues recommend focusing on support for better communication practices between clinicians as a way to increase the quality and safety of patient care [53, 54]. In fact, they emphasize that understanding communication patterns will improve our understanding of how decision support systems should be designed to support effective communication . In other words, the human factors involved in information exchange and interaction are fundamental to designing adequate support systems for work in critical care.
Technology alters the way individuals and groups collaborate and work. It may increase, enhance, or speed up performance  and reorganize task completion strategies. Its impact on the use of knowledge and reasoning has been evident, for example, in studies of electronic health record systems  and web-based patient tools for health management .
Many currently available healthcare information systems are not sufficiently sophisticated to operate effectively in highly complex environments and fail to provide adequate support to clinician users . Complex information systems in combination with stressful, high-velocity work environments may add to the extraneous cognitive demand and create ample opportunities for error . Highly complex system interfaces, for example, text-laden, dense and cluttered screens of many information systems, raise considerably the level of cognitive workload and add to the number of cognitive tasks required to monitor and manage a computer-driven work environment [61, 62]. A particularly detrimental aspect of cognitive overload in clinical work is the diversion of attention away from the main medical task. A physician whose attention is constantly shifting and who needs to mentally integrate data from disparate displays may not be able to formulate a complete and coherent picture of the current state of the system . However, a systematic and robust conceptual understanding, or situational awareness, is necessary to recognize unusual or abnormal system states signifying a possible failure. It is therefore necessary that systems present perceptual cues that do not require the conscious effort of drawing meaning and interpretation of screen objects or systems but support integrative views and perceptual judgments . The increasing versatility and complexity of clinical information systems also require users to develop a high level of skill that can be acquired only through hours of training and extended work experience. Such an effort is often unrealistic to demand from clinicians whose time is scarce and expensive . Most users of clinical information systems therefore never achieve a high level of proficiency. Cursory training and an only vague familiarity with a new system then leaves users to rely on opportunistic learning during actual clinical work and may result not only in delays but ultimately translate into medical errors .
Careful design may help to minimize routine tasks by automating them and by displaying context-relevant information in formats that require minimal further interpretation or mental manipulation for immediate, direct use [66, 67]. For example, studies in aviation and power plant management have shown that intuitive monitors can improve detection, control, and prediction of future system states  and control staff can therefore avoid errors by making more accurate decisions. In medicine, clinical performance can be improved when displays are consistent with the user’s clinical processes and mental models . Clinicians will then be able to conserve their attentional resources and focus fully on higher-order mental activity, such as clinical reasoning, strategy and treatment planning, and devote more time to unusual or non-routine cases .
In order to understand medical errors, we need to categorize them along different dimensions. Most of the medical error taxonomies [71–79] are based on clinical, administrative, and other non-cognitive dimensions. They are mostly useful in documenting errors but not useful for explaining, managing, and preventing errors. Medical error is largely a cognitive phenomenon caused by many cognitive as well as non-cognitive factors. In order to understand the cognitive mechanisms underlying various errors, we need to categorize the errors along cognitively meaningful dimensions. To address the need of a cognitive framework specifically developed for medical errors, Zhang, Patel, Johnson, & Shortliffe  developed a cognitive taxonomy based on Reason’s theoretical framework of human error and Norman’s “Action Theory” [81, 82].
Cognitive factors are important for understanding medical errors at various levels of the healthcare system hierarchy. At the level of the individual, cognitive factors of individuals (e.g., knowledge, attention, memory, perception, action, reasoning, decision making, etc.) play a critical role . At the next level, errors can occur due to interactions between an individual and technology. This is an issue of human-computer interaction, where cognitive properties of interactions between human and technology affect and sometimes determine human behavior and task complexity [16–23, 81, 84, 85]. For example, poor design of the control displays of infusion pumps could lead to serious medication errors. At the next level, errors can be attributed to the social dynamics of interactions between groups of people interacting with complex technology in a distributed cognitive system. For example, errors can emerge in many scenarios such as the failure of coordination and communication between overnight and daytime nurses who must achieve mutual understanding about the state of a patient for whom they both care. At the next few levels up, errors can be attributed to factors of organizational structures (e.g., coordination, communications, standardization of work process), institutional functions (e.g., policies and guidelines), and national regulations. At these higher levels, cognitive factors also play important roles in the forms of organizational memory , decision-making , problem-solving [56, 88] and communication . For example, at the organizational and institutional levels, the high-urgency nature of the decision-making environments, such as in intensive care, makes them vulnerable to multiple kinds of errors .
Many errors in healthcare are systemic institutional errors caused by problems that are not due to any individual or team of individuals, but rather are caused by some fault in a system. This category may include problems with technological systems , the physical design of the workspace, or the use of institutionally sanctioned, but faulty protocols. Although the properties at various levels can be to some extent studied independently, a cognitive foundation for the system is essential for a comprehensive and in-depth understanding of medical errors.
Figure 21 shows the cognitive taxonomy developed by Zhang et al  by integrating Reason’s taxonomy and Norman’s action theory, where errors are divided into slips and mistakes, which are further divided into two more levels. An example of an execution slip is when a nurse intends to decrease a value using the decrement function, but pushes the down arrow key (which moves to the next field) instead of the minus key. An example of an evaluation slip is when a user presses the start button on an infusion pump after which the pump indicates that it has started infusing, so the user assumes the patient was receiving the drug; however, the user had forgotten to open the clamp on the hose, so no drug was being delivered to the patient.
This cognitive taxonomy can cover major types of medical errors because a medical error is a human error in an action and any action goes through the seven stages of the action cycle, described by Norman : establishing the goal, forming intentions, specifying the action sequence, executing the actions, perceiving outcomes, interpreting outcomes, and evaluating the outcomes against the goal. Most reasoning and decision-making errors in medicine are under the category of mistakes in the taxonomy. They are due to incorrect or incomplete knowledge, or other factors. This taxonomy also provides preliminary analyses of underlying cognitive mechanisms for each category of errors and recommendations for intervention strategies. More recently, Malhotra and colleagues  extended this model to include communication between multiple healthcare providers.
Recently, as a departure from the traditional approaches to human errors, Hollnagel, Woods, and Levenson  proposed a Resilience Engineering approach to medical error. One of the key points they make is that past efforts for improving system safety are commonly based on hindsight. The resilience engineering approach proposes a completely new vocabulary, and therefore a completely new way of thinking about safety. They argue that people are usually resilient when adapting to different situations, and therefore, research should focus on the processes and support systems that will allow individuals to be successful and avoid error.
Ethnography, as a research method, is commonly used in sociology and anthropology to acquire detailed accounts of a particular environment, the people involved, and individuals’ interactions within the environment. Though traditional ethnography and cognitive methodology are drawn from different disciplines, and thus have different goals, they can be integrated to develop an innovative, more optimal technique to be used in the study of cognition. Cognitive ethnography (CE) emerged from the adaptation and modification of three of ten principles of prototypical ethnography outlined by Ball and Ormerod . These include replacing the principle of (1) “intensity” with “specificity” of data collection; (2) “independence,” which states that the researcher must not have any existing theories, goals, or beliefs prior to observation, with “purposive” techniques involving specific research goals and theoretical interests; and (3) “personalization” (which requires researchers to make note of their thoughts and feelings on the observations) with “verifiability” (validation of the results across various settings and triangulation across observers). The purpose of this adaptation is to constrain the amount of data to be analyzed, with a more specific goal in mind.
Ethnographic methods used in our studies in critical care include shadowing representative physicians and nurses and audio- or video-recording all of their interactions with each other as well as think-aloud protocols while doing identified tasks (such as medical rounds). It also includes note-taking of non-verbal cues and interactions, while passively observing the clinical workflow, performance of routine and non-routine tasks and the nature of communication between clinicians. Think-aloud tasks are used to capture the individual’s thoughts and reasoning processes during problem-solving and decision-making, as these processes unfold [95, 96]. The collected data also provide information about the style and content of verbal interactions among all members of the team as well as individual reasoning processes. In addition, the dynamics of interaction during weekly meetings are recorded, with the aim of identifying the role of communication in making decisions. Semi-structured interviews with physicians, nurses and other clinical staff are also conducted to inquire into the nature of the interactions observed and to examine error-prone situations that may have occurred during each session.
Data are analyzed to represent the workflow of the critical care environment, which emphasize the importance of representation in the strategizing process of encoding information for making decisions. Observation and interview data are used to build individual pieces of the workflow, depending on the individual and the activity concerned. For example, Malhotra and colleagues  identified seven key generic activities (i.e., re-orientation and preliminary planning at the beginning of the workday, goal formulation, goal execution, transfers, admissions, re-assessment, and evening sign-out) in the patient care process, called critical zones, for dividing up the workflow. Next, these individual pieces are integrated according to the critical zones to develop a generalizeable cognitive model of the workflow (see Figure 32), which can be used to identify, characterize and predict medical errors in the ICU.
There are four levels of abstraction from which the model can be interpreted. The first and top level is the model as a whole. The second level includes the three groupings of critical zones (CZs), with the different background colors (yellow, blue, and grey). At this level, we can see indications of where and when medical errors are likely to occur if an individual is multitasking between different CZs and is cognitively overloaded. Within these groupings are the seven individual CZs (green boxes) and the adjacent blue boxes, which are the activities that co-occur in these CZs, and which make up the third level. Here, the interactions between the different members of the healthcare team as well as the flow of information are evident. The fourth level is the individual level, although it is not shown in Figure 3. At this level, we can follow the individual team member and identify his or her incumbent dependencies as well as outcomes related to activities of knowledge acquisition, information processing, task execution, and communication. Depending upon the team member, we may predict which part of the workflow may break down because of a faltering dependency. This workflow model was created as a simplified template that can be modified for use in other critical care settings to identify weaknesses and potential for errors in the clinical workflow. Such data can guide the development and implementation of information and communication technologies that can be targeted as support of each of the areas of weakness.
Cognitive Task Analysis (CTA) is a core methodology used in cognitive science and engineering in both laboratory and real-world settings . Using CTA, individuals’ performance can be studied by examining the quality and quantity of domain-specific knowledge required for a task, the information-processing demands of a task, the effectiveness of available technology-based support to perform the task, and decision points . CTA can be used to characterize tasks that place cognitive demands on the individual (e.g., diagnostic reasoning) and require similar reasoning strategies because of a shared underlying structure, and this can be generalized to clinicians across clinical settings [5, 99].
There are different types of task analysis that can be used depending on what is being analyzed and the purpose of the analysis. Hierarchical task analysis is the basic analysis for any task, in which high level tasks are broken down into their constituent subtasks and operations. This process is useful for the understanding and design of user interfaces. Action cycle analysis is based on the 7-stage model of human action by Norman  (i.e., establishing the goal, forming intentions, specifying the action sequence, executing the actions, perceiving outcomes, interpreting outcomes, and evaluating the outcomes against the goal). This method is used to analyze key subtasks that are critical for the usability of any device. Important to this analysis is the identification of the points where the action cycle can break down, which are primarily at the interface of execution and evaluation of the task. The execution is influenced by the difference between the goals and intentions of the user and the actions enabled by the system. The evaluation is influenced by the degree to which the user can perceive and interpret the state of the system and determine how well the user’s expectations have been met (e.g., feedback). After identification, ways to improve the interface can be suggested through changes in system design and education of users.
GOMS analysis (Goals, Operators, Methods, and Selection Rules) is a keystroke-level computational model that attempts to predict performance times for error-free expert performance of tasks by summing up the time for key-stroking, pointing, homing, drawing, thinking, and waiting for the system to respond [100, 101]. It is useful for the analysis of tasks that have complex goal-subgoal structures. It is also useful for modeling task performance levels of alternative designs without actually implementing the designs. One end product of the task analysis is the identification of the ideal task structure for good performance, interactions among procedures, and the information flow of the task. Another end product, which is more important, is a taxonomy of tasks. For example, in a task taxonomy based on the types of information processing, there are information tasks for retrieval, encoding, transformation, calculation, and comparison, as well as other information tasks.
The traditional taxonomies of medical errors as isolated constructs do not show much utility in the understanding, explanation, management, and reduction of medical errors. The major weakness in the traditional taxonomies is that the concepts in one taxonomy are isolated, and not semantically linked and integrated with the concepts in other taxonomies.
One new approach to medical error is to use ontology engineering tools to develop a meta-taxonomy of medical errors that integrate taxonomies that were created for different purposes. As a whole, the meta-taxonomy will have much more utility in categorizing, explaining, and managing errors. Along this line of thinking, we have developed a comprehensive medical error ontology to serve as a standard representation for medical error concepts gleaned from various existing published taxonomies [102, 103]. Eight candidate taxonomies were selected from published literature and merged to create a reference ontology consisting of 12 multi-dimensional axes that encompass the major aspects of a medical error event. A general ontology of medical errors is crucial for the following reasons: (1) to provide formal definitions and coverage of an entire range of concepts and relationships about medical errors; (2) to resolve present difficulties in pooling medical error information from varied data sources and classifications; (3) to enable analysis, interpretation, understanding and sharing of “medical errors” in a single, standard framework; (4) to enable identification of strategies for improvement to prevent medical errors; and (5) to provide systematic, principled methods for the design of improved medical error reporting systems.
The theoretical and methodological frameworks of naturalistic decision-making and distributed cognition described earlier provide a foundation for research in critical care. In this section, we describe how research studies in critical care support the value of using cognitive theory and methods for understanding decision-making and errors in critical care environments, and have implications for the design and implementation of decision support at the time and place when needed. The following four themes are elaborated with specific examples from our studies: (1) the clinical workflow in the ED and ICU, with an emphasis on cognitive overload and team decision-making; (2) expert-novice differences in comprehension of Psychiatric ED medical records; and a (3) cognitive analysis of a provider order entry interface and medication support in the ICU.
There are several factors that contribute to inefficiency and complexity in the clinical workflow, namely, multitasking, shift changes and handoffs, and interruptions. Such factors provide more opportunities for error, and place a higher cognitive load on each individual clinician. There are several steps in the patient care process through critical care, from triage to registration to the main emergency department to an inpatient intensive care unit. In this section, we use examples from our studies of different areas in the ED to describe the clinical workflow factors that contribute to cognitive overload of clinicians, which result in inefficiencies and delays in the clinical workflow and in patient care. Next, we describe how team interactions affect decision-making and error with examples from our studies of the workflow of the ICU.
Using cognitive methods of data collection and analysis described earlier (Section 3.5), several studies were conducted in various areas of the Emergency Department (ED) at a major Medical Center in New York City and another major hospital in the gulf coast region of the United States in view to identify problem areas and develop technological interventions.
Studies of the workflow in ED Triage were aimed at identifying the task and information flow and the use of information resources throughout the patient care process. Figure 4 shows the model that was constructed based on a cognitive task analysis (see Section 3.5.2) of triage workflow observations, as well as information from questionnaires and interviews with key clinical personnel . This model shows the actions, agents (clinical team members), information resources available and their interactions during a typical patient encounter in triage. The description of agent abbreviations and associated tasks as used in the figure and text is in Table 1. The series of tasks (boxes in Figure 4) follow a chronological order from a patient’s entrance into the ED to either discharge or admission to the hospital.
Upon entering the ED, patients proceed through a pre-triage and triage process. Pre-triage (tasks 1–3) occurs when the patient enters the ED and gives the pre-triage nurse (PTN) his or her chief complaint for coming to the ED. Here, the PTN begins a paper chart (task 1) for the patient, as there is no access to the electronic medical record system at this point. Depending on the severity of the case (task 2), the PTN may have a triage nurse (TN) evaluate the patient immediately or initiate a chart and place it in a cue for the TNs to process in time order (task 3). The next step is the triage of the patient (tasks 4–7), where minimal electronic decision support is utilized. The TN assesses the patient’s complaint (task 4), examines the patient’s vital signs (task 5) and conducts point-of-care tests such as peak expiratory flow volume, finger-stick blood sugar or hemoglobin assessment, or an EKG depending on the patient’s complaint and medical history (task 6). Then, the TN uses the electronic tracking system, “e-track”, to assign the patient to a treatment area of the ED and a district nurse (DN), depending on the current workload of each nurse (task 7).
After the TN completes the triage process and records the patient’s assignment on the chart, the patient is taken to the assigned area in the ED or told to wait in the waiting room, ideally by the Emergency Room Technician (ERT) (task 8). The ERT, or TN, locates the DN (task 9) and gives the DN a brief verbal report about the patient (task 10). The TN then gives the patient’s chart to the registration department (task 11), which is the last step in the triage process. Finally, the TN returns to the triage area and repeats this process with the next patient. The overall process requires the TN to physically move to various areas in the ED. The workflow continues into the Registration area (task 12) and then into the main ED, where patients are assessed by the DN and a physician, diagnosed, and treatment is initiated resulting in either discharge or admission to an inpatient unit (tasks 13–16).
Results of analysis of this data by Horsky and colleagues  show significant delays in the triage process that may carry over to the main ED, thus increasing inefficiency and opportunities for error in making patient care decisions. One main reason for delay and an inefficient workflow is the existence of three different electronic information systems and one circulating paper-based chart, which requires the clinicians to access, aggregate and cross-match patient information across the systems.
All patient encounters were categorized according to the main reason for delay in the triage encounter. Twenty percent of encounters were classified as typical findings as they did not include any events uncharacteristic for the triage task and were therefore used as a reference in estimates of triage delay. The mean time of triage encounters in the Reference category was 8 minutes and 38 seconds. There were five types of events found to prolong triage (Interpreting, Workflow, Locating, Extra Tasks and Patient-related) and one type of event that shortened it (Fewer Tasks). For example, 18% of encounters included delays associated with obtaining an interpreter, such as repeated paging and long waits. In addition, 14% of encounters included delays due to difficulty tracking clinical personnel and equipment needed to assess the patient. The longest average encounter time (17 minutes, 47 seconds) occurred in 9% of triage cases where the delay was due to patient-related medical reasoning and consultations about acuity level, or in determining the institutional policy for treating patients who were intoxicated or present with psychiatric symptoms. When aggregating the categories into Reference, Workflow, and Patient-related categories, it was estimated that 23% of nurse contact time with patients was estimated as delay. Most of that delay (79%) was categorized as workflow-related.
Due to the time-pressured and urgent nature of the patient care process in the ED, clinical personnel tend to use shortcuts when they are able in order to decrease time to patient care by the physician. Hakimzada et al.  traced four cases of errors, related to patient misidentification, back to ED Registration, which were ultimately due to the tendency for registration staff to use workarounds and shortcuts during times of high patient volume.
The process of patient care continues into the main ED. An ethnographic study of the clinical workflow in an Adult ED  identified several key tasks in the workflow that contribute to cognitive overload on the clinicians. These tasks include shift changes and handoffs, multitasking, interruptions, and documentation. For example, there was an interruption every 9 and 14 minutes on average for the attending physicians and the residents, respectively, making the communication process more difficult and cognitively taxing for the clinicians. Observed sources of interruption in the ED included other patients, other staff, telephones and pagers. The data indicate a higher frequency of interruptions that are resolved quickly by the attending physician. On the other hand, interruptions caused by the ED residents, while less frequent, have a much longer duration. One additional observation was the consistent need expressed by clinicians for a computer-based tracking system, which would help them monitor and find patients throughout the ED. During the period of observation, a tracking system was implemented within the ED and although it resolved some of the problems, the system fell short of expectations due to its inability to communicate with other information systems within the ED.
A high prevalence of interruptions was also documented in another study focusing on ED nurses working in a Level One Trauma Center at a different location [107, 108]. According to Brixey and colleagues, an interruption is defined as “a break in the performance of a human activity initiated by a source internal or external to the recipient with occurrence situated within the context of a setting or location. This break results in the suspension of an initial task to perform an unplanned task with the assumption that the initial task will be resumed” (p. E38) . Brixey et al.  categorized ED interruptions and activities using the HyMCIA (Hybrid Method to Categorize Interruptions and Activities) method through the collection of ethnographic data. Analysis of observations resulted in the development of a taxonomy of interruptions, a non-dynamic representation of the phenomenon. Based on this taxonomy, a timeline of activities and interruptions was constructed, which served to place the discontinuities in the workflow caused by interruptions into context.
In the Brixey et al. study, it was observed that nurses received slightly more interruptions per hour than physicians (an average of 12 vs. 10). In addition, physicians were most frequently the interruption initiators (63% of the time). Interruptions in the workflow were initiated by people, pagers, and telephones, as well as the physical environment when supplies were not available. After the interruption, physicians and nurses usually returned to the original, interrupted activity more often than leaving the activity unfinished. The efficient return to interrupted activities can be supported by information technologies such as memory aids, which would decrease the cognitive load burden on the clinicians and facilitate patient care decisions that are delayed due to interruptions.
The development of a graphical representation of the clinical workflow in the ED (see Figure 4) helped in identifying the problems in communication flow, bottlenecks and repetitive tasks in the ED process of care . Figure 5 is a graphical representation of the observed task flow, communication patterns and patient tracking (top portion of figure) and the proposed technological changes (lower portion of figure), which are aimed at improving the clinical workflow.
The data suggest that the use of a solely electronic-based medical record system, including electronic patient tracking, can facilitate the management of patient information and patient care decisions within the limited time frame. An integrated, connected support system would reduce repeated manual copying of information at various stages in the care process, and eliminate the need to physically locate nurses, interpreters, and patients in the busy ED, which only results in increased delays in care. As Horsky and colleagues  suggest, in order to ensure interoperability, specific tasks, information sharing and decision support may require different modalities of communication delivered by different technologies. Implementing an integrated system with real-time updates of patient information in the computer system makes the registration desk redundant and unnecessary. For example, during pre-triage, patients are asked to present identification from which personal data are hand-copied on a paper form. The PTN could instead have a workstation networked to the hospital EMR and initiate integrated paperless charting and tracking by searching for an existing record so that returning patients would have history, allergies and other pertinent data ready for the triage nurse. Patients could be issued a bracelet with their name and encoded basic data (e.g., an RFID [radio-frequency identification] tag) for quick identification later in the process. Asynchronous, less interruptive means of communication, differentiated by urgency and priority, could replace the current pattern that relies mostly on verbal or personal contact. These support measures would increase efficiency, create a better task division between the nurses, and decrease the cognitive load on the triage nurses’ memory during patient assignment to nurses.
In summary, these recommendations allow for the management of patient information with decreases in time taken for clinical tasks to increased efficiency of the ED process, from triage to admission or discharge. It should be noted that when implementing new technology, there needs to be a close, careful and ongoing monitoring of the process as new challenges and problems may be introduced.
The psychiatric emergency department (Psych ED) functions similarly to the general ED; however, there are several characteristics unique to this environment. Cohen and colleagues  used the framework of distributed cognition to develop a model of the clinical workflow in the Psych ED.
Findings from this study show that cognitive work is distributed across both agents (individual clinicians) and artifacts. Figure 63 gives a graphical representation of this distribution, showing the various members of the multidisciplinary clinical team, their tasks, including information- gathering tasks (e.g., taking a patient history) and action-execution tasks (e.g., administering a medication), and the artifacts used for recording patient information (e.g., whiteboard, clinical notes) . Although this distribution is functional, analysis of the distribution has revealed several latent flaws in the system related to the underlying distribution of cognition across teams, time, space and artifacts. Errors and near misses derived from the observation and interview data were interpreted in relation to the level of the distribution in which they occurred (see Table 2 for examples of latent flaws identified in this study) .
The analysis of verbal protocols was used to characterize decisions taken in the Psych ED, and to determine what information content was used to support these decisions. The data suggest that when developing systems that provide support for making decisions during these crucial periods of “near misses”, there needs to be consideration for the factors that lead to pushing decision-makers towards error boundaries and how these factors can be monitored.
The intensive care unit (ICU) is another dynamic and complex environment, with high stakes for patient safety and minimal room for error. Patient rounds are one of the most important activities that occur in the ICU, as individual patients are visited and evaluated by the team of clinicians. Patel, Kaufman and Magder  investigated the collaborative decision-making and team interactions in a medical ICU, focusing on observations and recordings of morning patient rounds and related information from one patient’s charts. During morning rounds, team members give patient reports that are then discussed as a group for the evaluation of patients’ status, previous decisions and actions made and for the planning of next actions to take.
Analysis showed that the rounds are characterized by three phases . The first phase involves a report from the overnight resident. The report is used to describe to the team the patient’s condition during the previous 24 hours, including critical decisions that were made and actions that were taken. The team then critiques and evaluates these decisions as to their efficacy and appropriateness for stabilizing the patient. The second phase involves a report from the overnight nurse. This report includes an assessment of the patient’s situation specifically regarding the vital signs and symptoms related to fluid balance and food intake, as well as the psychological status of the patient. Then, the resident makes suggestions as to the collection of more information in order to decide on the next course of action. At this point, there is a shift change, with another resident replacing the overnight resident. This phase ends with the expert’s (attending physician) evaluation of the patient’s status to make the next round of decisions for the ICU. The third phase involves a dialogue between all team members, where gaps in information are filled, further information is requested, and sensitivities regarding specific issues are discussed and resolved. In this phase, the pharmacist and nutritionist evaluate the patient’s medication and dietary requirements, respectively. The expert physician concludes with a summary of the actions to be taken during the subsequent 24 hour period, and all team members are updated on the patient’s status and are made aware of their individual responsibilities.
Throughout this process, the expert physician manages the flow of information so that there is a reduction in the cognitive complexity and effort for the team, with an increase in cognitive complexity for himself, as he integrates the multiple pieces of information about the patient, using basic science concepts as needed. The expert relies on the team for maintaining the shared knowledge in a distributed working memory, and for analysis of patient data, which is primarily done by the resident. This balanced process of team-individual decision-making and data synthesis works to make the patient care process efficient.
Patel, Kaufman and Magder  also conducted a dialogue analysis of the morning rounds for three days in terms of episodes corresponding to topics of discussion. The analysis included identifying the number of propositions (concepts) and clinical findings (useful or relevant concepts for making decisions) for the three morning rounds. The results of the analysis show several patterns of interaction that change over time, shifting from a focus on the patient’s condition on the first day in the ICU (generating 60 findings), to the effects of medication and adjustment decisions on the second day, and to longer-term therapeutic management issues, resulting in patient discharge on the third day. On the first day, 78% of the 185 concepts used in the discussion reflected distinct or new information. The concepts raised by the resident became the basis of most of the subsequent discussion, and were repeatedly reviewed and evaluated during the discussion. On the second day, 65% of the 145 concepts used in the dialogue reflected new information not previously discussed. As the patient’s condition was quickly changing, there were more focused communication exchanges, suggestions and decisions made during discussions. By the end of the second day, the team had acquired substantial shared knowledge about the patient, which was reflected in the deeper level of analysis and synthesis of information. On the third day, the attending physician (expert) planned to discharge the patient. As such, there were many fewer exchanges between team members, including summarizing the information from the patient record and the expert advising on future management of the patient’s condition. Likewise, there were only a total of 76 concepts used in discussion, of which 61% were new information.
Information used during rounds was at various levels of granularity, from basic medical sciences to pathophysiology to medical information. In addition, the level of information processing was different for nurses, residents and attending clinicians in providing patient care. Knowledge–based decision support during the rounds will need to take into account these differences and be able to deliver information “just in time” during practice.
There is evidence that shows that poorly designed electronic medical records (EMR) may decrease productivity and increase errors in clinical practice [111, 112]. Sharda and colleagues  investigated the effects of expertise on the comprehension of psychiatric narratives by clinicians. They used cognitive methods to determine design criteria for EMRs. Data were collected using think-aloud protocols (see Section 3.5.1) from expert and novice psychiatrists as they read clinical narratives, based on real discharge summaries. The transcribed protocols were then analyzed using one of the natural language representational methods, propositional analysis [114–116], and semantic analysis (see  for an extensive review of this methodology). Results showed that novices (2nd year psychiatry residents) (1) were less able to distinguish relevant from irrelevant information in the EMR despite recalling similar quantities of information, and (2) made less accurate inferences than did the expert psychiatrists . In addition, expert subjects were more precise than non-expert subjects both in their use of language and in the accuracy of inferences drawn. On occasion, non-expert subjects would reach correct conclusions, but for the wrong reasons .
However, when the discharge cases were restructured, the novice subjects were able to make more inferences from relevant material. The authors note that this has implications for the design of EMR interfaces. It has been shown that such interfaces can affect knowledge organization and reasoning , and as such can be considered cognitive artifacts . Paper records can also be considered cognitive artifacts. However, because of their dynamic nature, electronic medical records have the potential to present information in a manner that affects human cognitive performance. The results show that electronic data organization support through structured text helps novices in reducing cognitive load of sifting through massive narrative data and guides them in focusing on relevant data. This can be done relatively efficiently for purposes of screening and managing patients in emergency care.
Computer-assisted provider order entry is a support tool that is designed for expediting medication ordering. The structure of the order entry system needs to be designed to take into account physicians’ interactions with the system with the aim of reducing the cognitive demands on the individual to facilitate decision-making. Horsky and colleagues  developed a methodology for the characterization of cognitive demands of a medical information system, which was based on the distributed resources model, an approach that describes the dimensions of user interfaces that introduce unnecessary cognitive complexity. This method evaluates the relative distribution of external (system) and internal (user) representations embodied in system interaction (see Section 2.2 and [16–23]). An expert “walkthrough” evaluation of a commercial order entry system was conducted, followed by a simulated clinical ordering task performed by seven clinicians. This type of analysis involves usability inspection of data collected from tasks performed by the experts, including the visual contents of the computer monitors that are recorded, along with their verbalizations . Use of both verbal and visual data facilitates the identification and characterization of the user’s interaction strategies with the order entry system. The analysis revealed that the configuration of resources in this ordering application placed unnecessarily heavy cognitive demands on the user, especially on those who lacked a robust conceptual model of the system. The distributed resources model also provided some insight into clinicians’ interactive strategies and patterns of associated errors.
A further series of studies explored the relationship between computer-based provider entry (CPOE) systems and medical error [70, 111]. A novel approach to error analysis was used to interpret a dosing error related to computer-based ordering of potassium chloride (KCl) . The sequence of events leading to this error was chronologically reconstructed from disparate sources including usage logs, interviews and usability inspection. Errors in several aspects of the drug ordering process were identified, including system usability difficulties, user training problems and suboptimal clinical system safeguards. Results of the analysis were used to formulate specific recommendations for interface layout and functionality modifications, suggesting new user alerts, proposed changes to user training, and attention to the error-prone steps of the KCl ordering process to reduce the risk of future medication dosing errors.
The characterization of work and information flow in each clinical context provides a foundation from which to develop and implement decision support in critical care aimed at improving the efficiency of the clinical workflow with a redistribution of cognitive tasks and better communication and collaboration between clinicians. Improved workflow and team interaction will serve to reduce and prevent errors and thus increase patient safety in a complex, dynamic and time-pressured environment. For example, the generalizable cognitive model of the clinical workflow developed by Malhotra and colleagues  can be used to inform the development and implementation of decision support tools, such as cognitive aids and other technological information systems, that are responsive to the nature of the clinical workflow. However, unlike the popular goal of achieving flawless performance (through development of error-free systems), the results from these studies will have implications for developing adaptive systems that anticipate errors, respond to them, or substitute less serious errors that allow subsequent interventions before they result in an adverse event.
In each of the critical care environments (ICU, ED and Psych ED), we have identified bottlenecks in the workflow and systemic flaws that leave the system vulnerable to error. These include the loss of information at shift-change, inefficient patient tracking and cognitive overload as a consequence of multi-tasking and frequent interruptions. Furthermore, as part of our distributed cognitive analysis of each environment, we have characterized interactions between human and technological agents (or lack thereof) that underlie the process of patient care. The errors and problems we identified in these critical care settings are likely to occur within similar systems at other hospitals. Increasingly complex systems of care delivery require comprehensive analyses of human actions and errors for design changes that emphasize clarity of communication and the implementation of technology that supports specific user tasks
In this methodological review paper, we attempted to cover the cognitive methodologies and their applications in translating the findings of cognitive research into implications for providing efficient, effective and safe decision support for critical care settings. We illustrated these methodologies with examples mostly from our own research. The focus was on the following aspects of translational cognition: from general cognitive principles and methods to their applications in healthcare domains, including the nature of problem-solving and decision-making, and distributed team cognition. These cognitive studies address the issue of where and when there are limitations to human memory, problem-solving and decision-making strategies as well as communication failures, both individually and in teamwork, which could be circumvented using specific information and communication decision support tools.
Support from the US National Library of Medicine (R01 LM07894) grant to Vimla Patel is greatly acknowledged.
1Reprinted from Journal of Biomedical Informatics, Vol 37, Zhang J, Patel VL, Johnson TR, Shortliffe EH, A cognitive taxonomy of medical errors, pp, 193–204, Copyright Elsevier Limited 2004.
2Reprinted from Journal of Biomedical Informatics, Vol 40, Malhotra S, Jordan D, Shortliffe E, Patel VL. Workflow modeling in critical care: Piecing together your own puzzle, pp. 81–92, Copyright Elsevier Limited 2007.
3Reprinted from Artificial Intelligence in Medicine, Vol 37, Cohen T, Blatter B, Almeida C, Shortliffe E, Patel V, Distributed cognition in the Psychiatric Emergency Department: A cognitive blueprint of a collaboration in context, pp. 73–83, Copyright Elsevier Limited 2006.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.