ReNAP was released into general use in March 1999, when an e-mail announcement describing the new feature was sent to all house staff and attending physicians. Access was provided via two of the main BICS menus—the inpatient order entry menu, for use by inpatient physicians, and the ambulatory record menu, for use by clinic physicians. Use of ReNAP is entirely voluntary.
Since its release in March 1999, ReNAP has gained tremendous popularity at the Brigham and Women's Hospital. In the first 24 months of its release, the usage rate has steadily climbed, and as of January 2001, this feature is used about 2,300 times per month (Figure 3). During the 12-month period between February 2000 and January 2001, 780 different clinicians used this feature. The top 50 users made 52.5 percent of requests, and the top 100 users made 74.9 percent of requests*
Laboratory notification requests per month (March 1999 to January 2001).
Of the 22,775 requests made between February 2000 and January 2001, 75 percent (17,204 requests) were made for hospitalized patients, and 24 percent (5,352 requests) were made for patients seen in the ambulatory setting (Figure 4). The vast majority of users (99.2 percent) preferred notification by alphanumeric pager to notification by e-mail.
Laboratory notification requests by patient location (22,775 total requests; February 2000 to January 2001).
Figure 5 shows the different categories of tests requested for notification with this feature. Data shown again reflect the 12-month period between February 2000 and January 2001. Of all requests in this time period, the combination of electrolytes, complete blood count, and coagulation accounted for 78 percent (17,766 requests).
Laboratory notification requests by test category (22,775 total requests; February 2000 to January 2001).
The Departments of Medicine and Surgery were the top users of this feature, accounting for 84 percent of requests. The Departments of Emergency Medicine and Obstetrics–Gynecology ranked third and fourth in usage. Overall, 96.9 percent of all notification requests were made by house staff.
We assessed the effects of training level on the usage of ReNAP. Over a 6-month period (August 2000 to January 2001), 84.4 percent of the 90 medicine and surgery interns (PGY1) used the system at least once a month, compared with 57.8 percent of 135 medicine and surgical residents (PGY2–3 for medicine and PGY2–5 for surgery; chi-square test, p<0.001). Medicine interns were more likely to use ReNAP than medicine residents—73 medicine interns, or 84.9 percent, used ReNAP, compared with 105 residents, or 58.1 percent (chi-square test, p=0.0001). A similar trend was observed with surgery interns—17 interns, or 82.3 percent used ReNAP compared with 30 residents, or 56.7 percent (chi-square test, p=0.074). These data suggest that supervising residents are less likely to use ReNAP than interns, who are primarily responsible for reviewing laboratory data. Other departments were not studied because of relatively small sample sizes.
We also assessed the difference in ReNAP usage between the medicine and surgery house staff. The percentage of house staff who used ReNAP at least once a month was not statistically different between medicine and surgery—73 medicine interns, or 84.9 percent, compared with 17 surgery interns, or 82.3 percent (chi-square test, p=0.79); and 105 medicine residents, or 58.1 percent, compared with 30 surgery residents, or 56.7 percent (chi-square test, p=0.89). We also sampled 50 requests each from medicine and surgery to assess the usage of ReNAP for patients in the intensive care units (ICUs) compared with patients on the wards. After adjusting for the higher proportion of ICU patients in the surgery department, we found that surgery house staff were not statistically more likely than medicine house staff to use ReNAP on ICU patients (weighted ICU usage rate, 18.3 percent medicine compared with 23.5 percent surgery; chi-square test, p=0.53).
A user satisfaction survey was sent to users of ReNAP. Users were asked to grade the reliability, ease of use, and helpfulness of ReNAP on a scale of 1 (best) to 4 (worst). Users were also asked to suggest changes to the feature and to report problems encountered when using ReNAP.
Altogether, 150 e-mail surveys were distributed to recent and unique users of ReNAP. Findings based on the 47 responses received (31 percent) showed that users were most satisfied with the helpfulness of ReNAP, although ease of use and reliability also received high scores (Table 1).
The comments, in general, were very positive. These comments include “works great,” “has significantly improved the quality of life for interns,” and “an excellent feature that more people should know about.” Other user comments have prompted us to improve the display of messages on the pager screen and to expand our test name dictionary.
Users were also asked on the satisfaction survey to describe some typical scenarios in which they had used this feature. Responses to this question include:
- Awaiting cardiac troponin results for patients in the emergency room, before deciding on their disposition
- Awaiting final creatine kinase results before sending a patient to the cardiac exercise treadmill test
- Forwarding pending laboratory results to cross-covering house staff (who are often busy admitting patients) so that they do not need to look up individual laboratory results
- Receiving the latest laboratory results even when away from a computer terminal (e.g., during attending rounds).