This project was performed as a quality improvement project. No records were maintained of the names of any participants. Results were used for the design of our final AIMS order entry screen. The University of Iowa Institutional Review Board reviewed our application for publication of the results and concluded “this is not human subjects research.”
Two touch-screen display formats were programmed on an iPad (Apple Inc., Cupertino, CA) to resemble the proposed AIMS medication order entry touch screens. The iPad program was Web-based, utilizing ASP.net, jQuery and SQL Server. Each time a participant selected a drug, the server recorded the result of that selection. Each trial lasted 2 minutes. The number of entries completed within the 2 minutes was recorded, along with any entry errors.
The two formats were alphabetical and categorical. For the first, the entry screen was constructed with “tabs” across the top labeled A-C, D-H, I-O and P-Z. Under each tab, individual medication “buttons” were listed in alphabetical order (e.g. acetaminophen, albuterol, alfentanil, atropine etc.). There were roughly 25–30 medications under each alphabetical tab. For categorical format, the entry screen was constructed with tabs across the top for Common Drugs, Fluids and Electrolytes, Cardiovascular, Coagulation, Regional Anesthetic, Antibiotics and Other Drugs. Under each category, drugs were grouped by category (e.g. all neuromuscular blocking drugs together, all opioids together etc.) Drugs listed under Common Drugs were chosen based on an extensive review of anesthetic records, and represented the 29 medications used most commonly in our operating rooms. Colors were used to label medications to match American Society for Testing and Materials (ASTM) standards as much as possible. Screen shots of example screens for both formats are shown in Figures and .
Screen shot of alphabetical display format. This drug entry screen was constructed with "tabs" across the top labeled A-C, D-H, I-O and P-Z. Under each tab, individual drugs were listed in alphabetical order.
Figure 2 Screen shot of categorical display format. The drug entry screen was constructed with “tabs” across the top for Common Drugs, Fluids and Electrolytes, Cardiovascular, Coagulation, Regional Anesthetic, Antibiotics and Other Drugs. Under (more ...)
There were 132 medications included in each of the study formats.
The study was conducted in the operating room environment but not during periods of patient care. Three groups of participants were chosen, based on their years of clinical anesthesia experience: anesthesia residents and student registered nurse anesthetists (SRNA) with <1
year of clinical anesthesia experience; anesthesia residents, fellows, and SRNA with 1 to 3
years of anesthesia experience; and anesthesiologists and Certified Registered Nurse Anesthetists, all with ≥ 4
years of experience. The convenience samples were N
20 participants per group, selected by the first author. Everyone invited to participate chose to do so. Each participant participated in the study individually, not in groups.
Participants were handed the iPad, which displayed a brief introduction of the study and confirmed their willingness to participate in the study. Each participant entered his or her years of clinical experience. Successive participants in each group were assigned in alternating order to an initial format, either alphabetical or categorical. Next, the participants underwent a training exercise to familiarize them with the system. The participants were presented with the names of countries, grouped either alphabetically or categorically by continent based on the initial assignment. The participant was prompted with the name of a country in the lower right corner of the screen and was then required to locate that country under one of the tabs, and tap the button for that country. If correct, the country moved automatically to the left side of the screen in a vertical list. A different country name then replaced the current country in the lower right portion of the screen. The demonstration ended when the participant successfully tapped four countries.
At the end of this training period, the participant tapped the Start Trial button. Based on the assignment either the alphabetical or categorical format appeared. The name of a drug appeared on the lower right corner of the screen and the participant was required to find that drug under the various tabs and touch the correct button to complete its entry. When the correct drug was selected, a new drug name would appear on the screen. Participants had 2 minutes to find and enter as many drugs as possible from a list of 25 drugs that were displayed in an order (Table ). Should they complete the 25 drugs within the allotted time, the list would cycle back to the first drug. At the end of the 2 minutes trial, they were shown their results (number of drug entered, number of errors) and directed to begin the next trial. The second trial used the same format and requested the entry of the same drugs as the first trial; however, the order with which drugs were listed was changed. This same process was repeated for a third trial. Participants were next instructed that the format would change from the previously viewed alphabetical format to a categorical format, or vice-versa. They repeated the demonstration with countries using the new format. They then completed an additional three 2 minute long drug-entry trials using the new format.
Order of Drugs in Trial 1, Trial 2 and Trial 3
The drugs were displayed in the same sequence for corresponding trials between the different formats (i.e. the sequence of drugs was the same for Trial 4 as it was for Trial 1). The total time for each participant was approximately 20 minutes.
To avoid any problems with a “learning curve,” only the number of drugs entered on the third trial with each format was used in our primary analysis. Numbers of drugs entered in each 2 minute period were compared pairwise by participant using two-sided one-group Student’s t tests. The pairwise differences were reported as mean ± standard deviation. Analyses were repeated after stratifying by the initial format viewed, alphabetical or categorical. Those differences were consistent with normal distributions (Shapiro-Wilk P
Statistical analyses of the numbers of medications entered incorrectly were performed using two-group permutation tests. This test is like the Wilcoxon signed-ranks test, but includes the 0’s. The zeros mattered because most participants had no errors for most trials. Confidence intervals for median differences were calculated using the Hodges-Lehman method. All results were calculated using exact methods (StatXact-9, Cytel Software Corporation, Cambridge, MA).