We implemented and evaluated a handheld computer procedure logging system, with internet-based data transfer to a central data repository. The system was found to be technically feasible, although initial problems were encountered related to the internet uploading process. All users successfully documented procedures on their devices and uploaded them to the central database. The database was a potentially valuable resource and it provided the Program Director with insight into the scope of procedural training experienced as well as the sites and clinical teachers involved. It should be noted that the procedures were entirely self-reported; we made no attempt to evaluate the accuracy of this information.
This procedure logging system has the potential advantage over other handheld systems [2
] in that it combines mobile data entry on the handheld with centralized data storage on an internet-based server. The centralized data storage allows access to the database in real time, allowing continual evaluation of trainees. However, the most common technical problem encountered was related to installation and setup of this communication software. The ability to upload data from home was considered a useful feature but required additional technical support
We found that only 55% of our small group of trainees used the logging system on a regular basis, with a decrease in procedures logged over time. The 914 procedures logged therefore represent only a proportion of the procedures performed by our trainees during the academic year. Of note, procedure logging was optional; mandatory use of the system may be an important consideration if training requirements change to mandate a procedure log. Other studies have reported variable compliance with similar systems. Garvin and coworkers [2
] found that 88% of their family medicine residents collected data on their handheld computer and 73% of them reported daily use. We previously reported a 38% regular use rate 5 months after the introduction of the procedure logging program in a general surgery program of 69 trainees [5
]. Others have reported difficulties in acceptance when introducing handheld computing technology, especially among the subset of staff/faculty physicians [8
]. This may partially be related to user seniority or age. Handheld computer use by physicians is increasing, particularly in younger age groups, in which utilization is greater than 50% [9
]. Compliance with such procedure logging systems may improve in the coming years as this younger cohort moves into senior positions. As technology improves based on lessons learned from experiences such as that gained in this study, increased acceptance is likely. Training in the use of the handheld device and software is critical [10
], and although we provided an initial training session and follow-up support, this may not have been adequate.
Although logging of procedures may not be required by all licensing authorities and hospitals, there are clearly benefits to having these data available [11
]. At the present time, documentation of procedural experience is not a requirement for critical care trainees in Canada, although the Program Director is required to ensure that trainees are competent in certain core procedures. In the future, such documentation may become increasingly important. Given current concerns over medical errors [12
] and the fact that many of these errors may be occurring in the critical care environment, documentation of procedure performance in training and during maintenance of competency programs is likely to gain importance. As we face a shortage of critical care medicine practitioners, it may be necessary to better define those multidisciplinary practitioners who are able to function in this capacity. Procedure logging in some form may be a valuable component of such an effort.