|Home | About | Journals | Submit | Contact Us | Français|
Building upon the foundation of the Structured Narrative electronic health record (EHR) model, we applied theory-based (combined Technology Acceptance Model and Task-Technology Fit Model) and user-centered methods to explore nurses’ perceptions of functional requirements for an electronic nursing documentation system, design user interface screens reflective of the nurses’ perspectives, and assess nurses’ perceptions of the usability of the prototype user interface screens. The methods resulted in user interface screens that were perceived to be easy to use, potentially useful, and well-matched to nursing documentation tasks associated with Nursing Admission Assessment, Blood Administration, and Nursing Discharge Summary. The methods applied in this research may serve as a guide for others wishing to implement user-centered processes to develop or extend EHR systems. In addition, some of the insights obtained in this study may be informative to the development of safe and efficient user interface screens for nursing document templates in EHRs.
Our research team at Columbia University developed the Structured Narrative Electronic Health Record (EHR) model, primarily aimed at using narrative data to enrich EHRs. The model integrates structured data and free text data (narratives) into a single gross structure, a template . Designed to extend an existing EHR system, the model includes a clinical document database, document ontology, natural language processing (NLP), and inference engine . The Structured Narrative EHR model was operationalized in a physician documentation system called eNote . Building upon the foundation of the Structured Narrative EHR model, we applied theory-based (Technology Acceptance Model (TAM) and Task-Technology Fit (TTF) Model) and user-centered methods to explore nurses’ perceptions of functional requirements for an electronic nursing documentation system, design user interface screens reflective of the nurses’ perspectives, and assess nurses’ perceptions of the usability of the prototype user interface screens. In this paper, we emphasize the theoretical and methodological aspects of our research focused on developing user interface screens for an electronic nursing documentation system.
The Structured Narrative EHR model is a hybrid model that supports structured data entry and narrative text that is “wrapped” in formal semantic structures. A primary motivation for developing the Structured Narrative EHR model was to enable extraction of data documented as free text through application of formal semantic structures and NLP.
Figure 1 graphically illustrates the four major functional components of the Structured Narrative EHR model: Import, Authoring, Browsing, and Export. The Import component manages acquisition of documents from external sources whereas the Export component manages sending of clinical documents to an external system. The Authoring component supports creation of new documents from document templates. The Browsing component supports document navigation and search for specific document templates or documents .
In the model, clinical documents are represented using the Health Level Seven (HL7) Clinical Document Architecture (CDA) standard and stored in Tamino, a native XML database that enables direct access and manipulation of XML documents. CDA specifies the structure and semantics of clinical documents and enables clinical document exchange across and within organizations . Using XML, clinical information can be coded at the time of capture and processed electronically resulting in an information resource that is retrievable by clinicians for patient care, but also useful for other purposes such as research and education [4, 5]. NLP is used to produce XML documents by identifying semantic structures and also to encode free text portions of clinical documents with detailed XML . The HL7/Clinical Logical Observation Identifier Names and Codes (LOINC) Document Ontology is used for naming, organizing, and managing clinical documents in the model . The Inference module supports a number of document services. For example, it supports locating a specific type of document then narrowing down to a specific document and then a particular section or element in the document by filtering relevant information. The Structured Narrative EHR model provided an important technical foundation for designing user interface screens for nursing document templates, however, it was imperative that we apply theory-based and user-centered methods to move from the model to actual prototype design and refinement so that the design matched user mental models and needs.
The theoretical foundation for the study is the combined model of TAM and TTF. The TAM has been extensively tested and used to represent theoretical understanding about information system (IS) usage and IS acceptance behaviors. The theoretical basis of the TAM is a theory of human behavior from social psychology . The TAM hypothesizes that two beliefs, perceived usefulness and perceived ease of use, are relevant to system acceptance behaviors. Perceived usefulness is ‘the degree to which a person believes that using a particular system would enhance his/her job performance’ and perceived ease of use is ‘the degree to which a person believes that using a system would be free of effort’ . The TTF model addresses the relationship between task, user acceptance, and utilization, which has four constructs: task characteristics, technology characteristics, task-technology fit, and performance/utilization . The construct, task-technology fit is the matching of the capabilities of the technology to the demands of the task and the ability of information technology to support a task [9–11]. Thompson et al. specifically delineate user perceptions of the match in regards to job-related efficiency, effectiveness, quality, and overall performance improvement .
The combined model for this study addresses technology beliefs and fitness of technology for task . Task characteristics, technology characteristics, and task-technology fit constructs serve as antecedents to the TAM model constructs, perceived usefulness and perceived ease of use, and as direct effects on system utilization. In Dishaw and Strong ’s study, the combined model provided a better explanatory model of information technology utilization than either the TAM or the TTF alone since there was a significant relationship between task-technology fit and perceived ease of use and perceived usefulness. In addition, the TAM model indirectly considers how the new system supports the user’s task through the perceived usefulness construct, whereas task characteristics and task-technology fit from the TTF model directly affect utilization. The amount of utilization variance explained by the models was TAM - 36%, TTF - 41%, and TAM and TTF combined model – 51% .
Consistent with the tenets of TTF, nursing informaticians have long recognized the importance of the match between nursing practice and the systems designed to support it. Toward this goal, the criteria/requirements for nursing information systems have been studied for several decades and formalized by a number of investigators. Gassert  developed a model to define information system requirements and applied the model to identifying requirements for a mobile computing system that supported cardiac surgery patient care. The model suggested specific categories, such as Information Processing Requirements (e.g. activation of Internet connection from menu), Data Requirements (e.g., diet information), and Information Functions (e.g., checking activity schedules for patient Information Functions and providing discharge information as needed for nursing Information Functions). Lundgren and Wisser  argued the need for a standard for functional requirements for EHR systems that support nursing activities. They illustrated examples of potential requirements for standardization, such as availability of information (e.g., simultaneous access), standard symbols for user interface, information exchange, security, data integrity, and term dictionary. Goossen et al.  conducted a study to identify NIS criteria using a Delphi study with a panel of 44 nursing informatics experts from Europe and North America. These included decision support functions, free text data entry, and single data entry/multiple data views. Hardiker and Bakken  specified requirements for tools and techniques to support structured nursing data entry, such as supporting data re-use and multiple narratives, reflection of the clinical process related to documentation of nursing practice, presentation of documentation components in a computer screen (e.g., forms, order of elements, candidate values). A study by Ireland et al.  that developed a user interface screen for an Intensive Care Unit (ICU) patient database management system using a user-centered approach is somewhat similar to our study in terms of goals and methods. During the design process, ICU nurses identified components for the screen that they considered priority parameters for performing intensive nursing care tasks. The nurses’ cognitive organization of information was used for designing the layout of the screen. A questionnaire (n=17 ICU nurses) was used to assess whether the screen provided “sufficient information” on a patient’s status.
With the exception of Ireland, the prior research has primarily focused on high-level requirements specification rather than direct work with intended end-users to define specific templates using formal methods. However, these requirements, particularly those specified by Goossen  and Hardiker  are relevant and foundational to the research that we report in this paper. In contrast to Ireland’s work, our data collection and analytic methods were informed by theoretical models (TAM and TTF).
We applied theory-based and user-centered methods to elicit nurses’ perceptions of functional requirements for an electronic nursing documentation system, design user interface screens reflective of the nurses’ perspectives, and assess nurses’ perceptions of the usability of the prototype user interface screens.
We invited two nurses from an academic institution to the design procedure as domain experts. The nurses worked in an oncology unit and had a minimum of two years experience in oncology care. Two nursing informaticians participated as usability experts during the procedures. The informaticians were experts in nursing, knowledgeable about human-computer interaction, and had publications in the field of informatics. We conducted two sessions in the usability laboratory at Columbia University School of Nursing. The first session started with brainstorming about nursing documentation tasks and features and functions of a desirable electronic nursing documentation system. We used structured interview questions to facilitate the brainstorming (Table 1). The sessions were recorded using an mp3 recorder for the investigators’ record.
In the second session nurses and informaticians collaborated on the design of user interface screens for three nursing document templates (Nursing Admission Assessment, Blood Administration Documentation, and Nursing Discharge Documentation) using a white board and Post-it® notes. A white board represented a computer screen and several different colored and sized Post-it® notes represented document components for each screen (e.g., sections, subsections, data elements). We provided the nurses a preliminary draft of the Nursing Admission Assessment screen as a starting place. The nurses modified the draft by changing: (1) the name of a component to one that is more familiar to nurses, (2) the type of data, e.g., red Post-it® = free text data, yellow Post-it® = picklist data, (3) the size of a Post-it® (i.e., component versus subcomponent), and (4) the location of a particular component in a screen. During the design session, the informaticians served as consultants to the nurses in terms of confirming types of data format for document components and desired system functionality for supporting nursing documentation. The nurses subsequently repeated this process for a draft Blood Administration Documentation screen and a draft Nursing Discharge Summary screen. The proposed designs were recorded using a digital camera.
We prototyped user interface screens using Dreamweaver, a Web development tool , based upon the results of the design session. Discharge documentation was separated into Discharge Summary and Patient Discharge Instructions; the latter is beyond the scope of this paper. We also added data elements that supported the Joint Commission  requirements regarding Nursing Admission Assessment, Blood Transfusion, and Discharge Summary documentation to the screens designed, for instance, Height and Weight on Nursing Admission Assessment; Informed Consent for Blood Administration; and Pain Management for Nursing Discharge Summary.
We created a CDA template for each screen based on the CDA schema using <oXygen/>® XML Editor [21–23]. The HL7 - LOINC Document Ontology was used to represent document names [24, 25]. Section headings were represented using LOINC database codes [26, 27] when available and in other instances, Systematized Nomenclature of Medicine (SNOMED CT) terminology codes were used .
In this phase, a convenience sample of five nurses was recruited from an academic institution to participate in formative evaluation of the prototype user interface screens. The nurses had at least one month’s experience in using an electronic nursing documentation system (i.e., Eclipsys XA). The protocol was approved by the Institutional Review Board. We provided the five nurses with clinical scenarios and asked them to perform scenario-related documentation tasks using the prototype user interface screens in the usability laboratory. The documentation tasks were selected for relevance to the three user interface screens that we developed, i.e., nursing care on admission, nursing care for blood transfusion procedure, and nursing care for patient discharge.
After completing the documentation tasks, the nurses completed study questionnaires. The original Ease of Use (PEU) and Usefulness (PU) Questionnaire included 6 items for each construct rated on a 7-point Likert scale . Later, Dillon et al. adapted the questionnaire to include 10 items for each construct rated using a 5-point Likert scale . Research supports convergent, discriminant, and factorial validity and internal consistency (Cronbach’s alpha reliability = 0.94 for PEU and 0.98 for PU) of the questionnaire . The TTF measure was adapted from Factors Influencing the Utilization of Personal Computers instrument . The scale developers reported that fit between the job task and personal computer capabilities had significant influence on utilization. Cronbach’s alpha reliability was 0.82 for the TTF measure. For this study, 8 items for PEU and 9 items for perceived potential usefulness (PPU) were adapted from Davis and Dillon et al.’s instruments. The scale for each item was scored as 0=strongly disagree, 1=disagree, 2=neutral, 3=agree, and 4=strongly agree. Total scores range from 0 to 32 for PEU and from 0 to 36 for PPU. Thirteen TTF items plus an optional question about overall comments on the prototype were used. The scale was 0=strongly disagree, 1=disagree, 2=neutral, 3=agree, and 4=strongly agree. Total score ranges from 0 to 52. Descriptive statistics were used to summarize the nurses’ responses to the PEU, PPU, and TTF measures. Nurses’ responses to the optional open question were summarized.
During the brainstorming session, documentation efficiency and patient safety were addressed as functional requirements for an electronic nursing documentation system. Nurses wanted to document once and use the data for multiple purposes. For instance, the nurses requested a system function that automatically generates a clinical summary (e.g., patient lists for wound care, central line care, or social work) for both bedside nurses and charge nurses using the data that were already entered and stored in the system. Nurses desired parsimonious and unit-customized user interface screens that omitted items irrelevant to care in a particular specialty. For example, assessing whether a patient wants his/her baby to be baptized is not relevant to a patient admitted for a vasectomy.
Thematic analysis of the brainstorming revealed five categories of functional requirements: Information Display (e.g., system should display document list by discipline and then by subject domain/profession/type of service), Data Entry (e.g., system should provide only one data entry for identical data [i.e., Record once, re-use in many places]), Alerts/Reminders (e.g., system should provide a reminder for continuation of particular treatments [e.g., Fentanyl patch q 7 days]), Carry Forward Data (e.g., system should carry forward the patient’s constant data from the previous record so nurses do not have to repeat the same questions when the patient is readmitted for another cycle of chemotherapy, such as surgical history or chemo history that do not change over time), and Miscellaneous (e.g, tab function for more convenient data entry). The majority of requirements related to Information Display and Data Entry. Final user interface screens and their CDA representations are shown in Figures 2, ,3,3, and and4.4. CDA was sufficient to represent the three nursing documentation templates.
All nurses responded either Agree or Strongly Agree for 28 of 30 items related to PEU, PPU, and TTF (Table 2). Exceptions were: The system would enable me to complete my documentation tasks more quickly and The system is not missing critical data elements that would be useful to me in my documentation tasks which were rated in the range of neutral to agree. PEU scores ranged from 24 to 32 (possible 0 to 32), PPU scores from 24 to 32 (possible 0 to 36), and TTF from 35 to 51 (possible range 0 to 52).
Nurses’ narrative comments supported ease of use (e.g., “No training is needed to get accustomed to using the system”) and usefulness of the user interface screens as a learning and reminder tool (e.g., check boxes for transfusion reactions). Other comments suggested adding content such as travel information and current diet for Nursing Admission Assessment screens. One comment related to user preference for scrolling as compared to two columns of information.
Building upon the foundation of the Structured Narrative EHR model, we applied a set of theory-based and user-centered approaches to develop and evaluate user interface screens for a nursing documentation module based upon the premise that an EHR system designed from a nursing perspective is more likely to support nursing practice. Prior research reported that nurses perceive EHRs as having the potential to improve the quality of patient care and patient safety. However, the majority of nurses believe that EHR systems do not optimally function or match nurses’ workflow [30–33]. Issues with user frustration and errors can be related to difference between users’ and the system designers’ mental images of a system [34, 35]. The design of nursing documentation systems to support quality management and research as well as practice is a complicated task as it requires understanding of nursing practice and informatics [36–38]. In this study, nurses and nursing informaticians worked together. We explored nurses’ needs for an electronic nursing documentation system and designed user interface screens reflective of the nurses’ perspectives. We matched characteristics of nursing practice with content and functions of the user interface screens. Some important details of particular user interface screens were identified through the user-centered methods including answers to questions such as: What documentation components are necessary for user interface screens related to nursing admission assessment, blood administration record, and nursing discharge summary?; What type of data entry is proper for each component in the user interface screen?; Are the components in screens presented in a manner consistent with typical documenting order? This level of understanding is necessary to help system designers develop a system that better reflects nurses’ needs [39, 40].
Several findings from our study were consistent with previous research. As indicated in Hardiker and Bakken ’s research, presentation order in a screen mattered to nurses in our study. In fact, a nurse requested that the presentation of components in the user interface screen should be consistent with the order of patient assessment to assist nurses’ mental process for plan of care and clinical judgment. Another example is free text data entry format. Goossen et al.  considered free text data entry as one of important criteria for a NIS system. Nurses in our study indicated specific types of documentation components that they would like to have free text entry format over the structured format for patient safety and accuracy of documentation reasons. Structured data entry format (e.g., picklist, checkboxes) was considered to be ill-matched to user needs in some cases (e.g., respiratory distress) because the format may lower the quality of documentation. Workarounds such as an addendum to the structured documentation may lead to incomplete communication among nurses and other healthcare providers. Moreover, previous research indicates that nurses experienced difficulties in retrieving information for nursing practice from an EHR system [41, 42]. This is consistent with our finding that nurses wanted an easier and more efficient method for searching for information in the system including that related to patient safety (e.g., finding out the last time a Fentanyl patch was applied to a patient).
The end-user evaluation was informed by TAM and TTF as the theoretical foundation. The survey findings and narrative comments provided preliminary evidence that the prototype was designed in conjunction with nurses was perceived to be easy to use. The findings related to ease of use were also consistent with the heuristic evaluation that we conducted with five usability experts . In addition, nurses perceived the user interface screens as potentially useful and well-matched to nursing documentation tasks associated with admission assessment, blood trans-fusion, and patient discharge.
As the prototype user interface screens were built in the context of the Structured Narrative EHR model, we were able to explore some aspects dependent upon functionality not typically present in nursing documentation systems today. First of all, NLP enables processing of free text marked up as XML in a CDA document. This supports the nurses’ requirement to document some items as free text while making it available for re-use of nursing data/information for other purposes, such as quality management and decision support. Future research should focus on the feasibility and subsequent impact of incorporating advanced documentation features such as the ones explored in our study on efficiency and patient safety. Secondly, nursing informatics researchers have made efforts to integrate nursing data and information into more comprehensive data standards [44–50]. However, no previous research has focused on the application of CDA nursing documents. Our research provides evidence of the applicability and usefulness of CDA in representing nursing documents and provides an exemplar that may motivate others to further test it for the nursing domain. CDA-based nursing documents can support the exchange of nursing information in EHRs.
Our research focused on a single nursing specialty task and documentation, with a small sample of oncology nurses from a single clinical setting. The findings of design and evaluation must be considered in context of the limitations of our study. However, the methods that we applied may be useful to others. A nursing documentation system built upon the foundation of the Structured Narrative Model, with user interfaces designed from the nurses’ perspective, and evaluated using methods from a user-oriented theoretical foundation can take a full advantage of healthcare information and communication technology for the nursing domain and may facilitate re-engineering of nursing workflow.
The increased influx of resources into health information technology is aimed at improving the quality, safety, and efficiency of the U.S. healthcare system. It is imperative that the EHR systems that will be implemented are well-matched to the users’ needs and mental models, take advantage of features such as those in the Structured Narrative EHR model, and are based upon data standards. The multiple theory-based and user-centered methods that we applied may serve as an example for others who are designing new EHR components. Moreover, some of the insights obtained in this study may be informative to the development of safe and efficient user interface screens for nursing document templates in EHRs.
This study was supported by grant 1R01LM007268 (Stephen B. Johnson, Principal Investigator) and P20NR007799 (Suzanne Bakken, Principal Investigator). We would like to thank Drs. Elaine Larson and Mary Byrne for their guidance and feedback on the dissertation. We would like to also thank to the nurses from New York Presbyterian Hospital and Dr. Leanne Currie for participating in this study.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.