The study shows that our intervention aimed at Material resources, Training and Staffing resources resulted in demonstrable changes of scores on two of the relevant LOTICS scales. This type of intervention can provide direct benefits to the staff of an OR, because the changes on the working environment were both visible and resulted in improvement in task performance and are therefore likely to be accepted.
The philosophy underlying the development of the LOTICS scale is that interventions should address broad categories of error types (the underlying pathology) rather than individual symptoms. Given this approach, the intervention aimed at improving material resources was based on the concept of standardization. Standardization is a concept well understood by other safety critical industries that value the benefit of lightening the mental burden on staff and users to allow them to concentrate better on the job at hand [30
]. In aviation the standardization and disciplined use of procedures, termed SOPs (Standard Operating Procedures) is widely argued to be the most critical factor distinguishing between good and poor outcomes in aviation incidents [31
] and could be adapted to the OR to develop protocols that minimize the influence of competing tasks and high workload. Standardization of material and equipment further results in the reduction of costs of operation, in maintenance, repair, storage, and simplified issue procedures. As part of the process in I-OR to standardize and streamline instrumentation and equipment, including locations, old and/or less user-friendly apparatus was replaced, missing items were purchased and manuals with a uniform design were developed. This improvement (at a general level) should and did affect responses to specific test items referring to, amongst others, the availability of equipment, their quality, timely repair and replacement. Moreover, after the intervention PRISMA identified technical factors to be significantly less important as causes of incidents.
Understaffing is one of the greatest threats to patient safety. Staff are often the last layer of defence for any error occurrence and particularly the proportion of professional nursing staff has an effect on patient safety [25
]. At the time of the pre-test there were shortages in OR personnel in 14 out of the 60 (23
%) Dutch hospitals investigated [33
]. One of the reasons for understaffing in the Netherlands is that working in healthcare is found to be less appealing [34
]. To limit turnover and to attract new personnel we need to enhance the attractiveness of the profession. To investigate how this can be achieved we designed and evaluated a number of intervention programs. These programs focused on the enhancement of well-studies work climate characteristics: participation in decision making, job autonomy and social support. Employee perceptions of these characteristics have been linked to various stressors, and a number of individual and organizational outcome variables [36
]. In addition to the focus on work climate characteristics more training opportunities were created so that more trainees could be qualified. As expected the interventions turned out to result in higher scores in I-OR compared to C-OR on aspects like the amount of staff to provide good care and the amount of experienced staff.
Staff turnover rate in I-OR decreased from 9.4
% in the year before the intervention to 5.1
% in the year after the intervention. Although we realize that turnover is determined by many factors, including labor market, it is likely that some of this decline can be attributed to the interventions.
Change can be a complex and drawn-out process that depends on a variety of contextual factors. The OR is a highly compartmentalized department structure which brings together members from multiple disciplines whose training and professional goals vary. Lack of communication between operating room personnel is common [37
]. Most surgical errors are not attributable to an individual but involve multiple personnel and steps; approximately 43
% of errors are due to poor communication [20
]. During the intervention in the OR we actually saw an increase in reported problems with communication. When communication problems do occur, they are found most often between different professional members of a team, such as between anaesthesiologist and surgeon or between nurses and doctors [38
].The staff of the I-OR indicated that they needed more information to do their tasks. A tentative explanation for this result could be that having created heightened awareness about safety issues, the staff was more alert to the communication problems they experienced.
The importance of incident reporting is widely recognized [10
]. Unfortunately, reporting is grossly incomplete. After the intervention, incident reporting rates in I-OR increased significantly compared with pre intervention rates. We realize that it is difficult to deduce from this result whether the 2.4x change in error reporting reflects a change in report behaviors with actual rates remaining constant or whether the 2.4x change in error reports reflects an increase in error rates despite the intervention. Various studies, however, showed that as an institution improves in the care it delivers and its safety culture more problems may be reported since open reporting is a tenet of safe practice [40
]. Increased incident reporting rates may not be indicative of an unsafe organization, but may reflect a shift in organizational culture [41
]. In this context it is important to note that the total number of reported incidents more than doubled while the contribution of technical factors to incident causation remained constant.
The propensity to report is probably further strengthened in our study by the implementation of the electronic report system. Various studies showed that an accessible and easy to use reporting system [42
], the understanding that the reports will be handled in a non-punitive manner [43
], and the notion that the reports are taken seriously and will lead to enhanced learning and systematic changes which will prevent it from recurring [44
], positively affects the willingness to report incidents. The empirical findings in this and other studies, taken as a whole, suggest that our result, an increase in incident reporting in I-OR, reflects a change in report behaviors rather than an increase in incident rates.
We believe that this work can contribute to patient safety initiatives and research in two ways: (1) our experience provides detailed insight in the latent risk factors, (2) our findings suggest that the methodology used in the study shows promise as a method for evaluating changes in the quality and safety of care in the operating rooms. Changing culture is a new watchword in patient safety [45
]. The willingness of staff to speak up about a patient-safety concern is an important part of safety in the operating room [46
]. Therefore there needs to be a culture of openness [47
]. We think a first step is this approach is to build a strong foundation of safety awareness among your staff and this may best be done by implementing concrete and visible improvements. We think staff perceptions of safety are a high priority issue within the OR, which will eventually motivate staff to take greater ownership of and responsibility for patient safety.
In the present study the intervention addressing training did not result in a significant improvement. This may have been due to a failure to address the problem at a deeper level, that is, the deficiencies in the business process behind the detected indicators. It is conceivable that the intervention attacked the problem at a ‘symptom curing’ level the training of the use of new equipment. As a result, this intervention may not have remedied problems at a systemic level, as revealed by the responses to test items referring to various other aspects of the training procedure.
Safety questionnaires are increasingly used in healthcare for assessment of safety issues, but they differ in the scope and extent. Sexton and co-workers developed a safety attitudes questionnaire that was validated over a wide range of clinical areas (ICU, OR, inpatient settings and ambulatory clinics) and 3 countries and administered to a large study group [48
]. The factors identified by their questionnaire were teamwork climate, safety climate, perception of management, job satisfaction, working conditions and stress recognition. They claim that the results could be used to benchmark organizations and to measure effectiveness of interventions. Similar safety questionnaires have been used by others to access teamwork and safety climate in hospitals and nursing units [49
Compared to their study our study was limited to a smaller group of disciplines and settings. Furthermore our questionnaire was more limited in scope and more directed to a limited set of factors that we connected to latent risk factors (LRFs), as identified in incident analysis. But a major difference is that those LRFs assessed enabled a much more concrete identification of measures for intervention, as compared with abstract factors like the perception of management, job satisfaction and safety climate, while still providing a way of assessing pre- and post-intervention values. There is still much work required before we are able to understand the full value of using climate questionnaires in health care, as Pronovost and Sexton have [51
] have recently pointed out.