|Home | About | Journals | Submit | Contact Us | Français|
The monitoring of adverse events in clinical care can be an important part of quality assurance. There is little evidence on the monitoring of re‐exploration after cardiac surgery.
To apply statistical monitoring techniques to the rate of re‐exploration for excessive bleeding in adult patients undergoing cardiac surgery procedures using cardiopulmonary bypass at Geelong Hospital, Victoria, Australia, between 1997 and 2003.
Shewhart charts, moving average plots and cumulative sum (CUSUM) charts were used to demonstrate changes in the rate of re‐exploration over time.
A CUSUM chart was used retrospectively at a time of perceived deteriorating clinical outcomes in patients of the cardiac surgery service. At this time, an intervention aimed at reducing the re‐exploration rate was performed, and subsequent CUSUM charts indicated an improvement in this rate. The CUSUM chart has become an important part of the quality feedback of clinical care outcomes within the Anaesthesia & Pain Management unit of Geelong Hospital.
Statistical monitoring techniques for quality assurance can identify important changes in clinical performance, and their adoption by clinicians is recommended.
Several authors have described the importance of the involvement of clinicians in quality assurance activities.1,2 One component of quality assurance that has been underutilised is statistical monitoring of the rates of specific undesirable patient outcomes. High‐profile cases have helped to illustrate possible methods for the statistical monitoring of adverse clinical outcomes.3 However, examples closer to routine clinical care are required to demonstrate further the role of statistical monitoring as part of a quality assurance process.
An adult cardiac surgery service was commenced at the Geelong Hospital (Geelong, Victoria, Australia) in 1997.4 Re‐exploration for excessive bleeding after cardiac surgery is an important adverse event that has been shown to increase the risk to patients of longer stay at hospital and in the intensive care unit, wound infection, other morbidity and death.5,6,7,8 At the Geelong Hospital, operating theatre staff, anaesthetists, intensive care staff and cardiac surgeons formed a subjective impression that there was an increased rate of re‐exploration for excessive bleeding in cardiac surgery cases. This occurred at a time when there was international evidence of declining rates of re‐exploration.9
In order to determine whether the perception of an increase in the rate of re‐exploration in cardiac surgery cases on cardiopulmonary bypass was warranted, we applied statistical monitoring techniques to the Geelong Hospital Cardiac Database.
The details of all adult patients undergoing cardiac surgery at the Geelong Hospital between late 1997 and early 2003 were recorded on a cardiac surgical database.4 Postoperative complications such as re‐exploration for bleeding were mandatory fields in the database.
The outcome of interest was the need for re‐exploration (for excessive bleeding) warranting return to the operating theatre. The service's experience with re‐exploration was summarised using Shewhart and moving average charts in batches of 100 patients. With equal‐sized batches, the Shewhart chart is a special case of the moving average chart that plots the moving average only for the 100th, 200th and so on patients. As 100 patients is an arbitrary choice, moving average charts were also plotted for batches of 50 and 150 patients. Probabilities from the binomial distribution were used to define 0.1–99.9% confidence limits around an expected rate of 5% for these charts. The inclusion of these limits converts the plots into control charts, with the process being defined as out of control if the plotted rate in the chart exceeds the limits. If re‐exploration continued at a rate of 5%, then this monitoring would on average take 500 batches of 100 patients before being incorrectly identified as out of control. The smaller the batch size, the shorter this average run length—that is, we expect a false alarm sooner with smaller batch size, but also expect a true alarm sooner.
Statistical monitoring of re‐exploration was performed using cumulative sum (CUSUM) charts.10,11,12,13 Firstly, a simple CUSUM chart, the cumulative number of excess cases of re‐exploration for bleeding, was constructed, assuming that the normal re‐exploration rate was 5%, based on experiences elsewhere.5,7 This was calculated sequentially in date order for each patient as the number of re‐explorations up to that time minus 5% of the number of patients up to the same time. Horizontal movement of this chart for any run of patients indicates performance at a 5% rate.
A CUSUM control chart with limits of acceptable and unacceptable performance was also calculated on the basis of sequential probability.10,12,13 For this chart, an important improvement was defined as a shift in the re‐exploration rate to 4%, and an important deterioration was defined as a shift to 6%. This CUSUM was a sum of contributions with value −s from successful outcomes and (1−s) from unsuccessful outcomes, where s=q/(p+q) and p=ln(0.06/0.04), q=ln(0.96/0.94). Coincidentally, s was close to 0.05, and hence this CUSUM control chart was similar to the number of excess re‐explorations explored in the first simple CUSUM chart. Control limits were set at −6.9 for accepting the null hypothesis of a 4% rate, and at +6.9 for accepting the alternative hypothesis of a 6% rate, by specifying a type 1 error rate of 0.05 and 95% power. This control chart had a reasonable average run length if performance remained close to a rate of 5%, but if performance improved (to 4% as may be possible6,8,9) or deteriorated (to 6%) then the limits would be crossed reasonably quickly.
The analysis of the cardiac surgical database was approved by the Geelong Hospital Research & Ethics Committee.
As of early 2003, 2148 adult patients had undergone cardiac surgery on cardiopulmonary bypass (mean age 66 years). The overall re‐exploration rate for bleeding was 4.9% (106/2148). The service was provided by three surgeons (A, B and C), who operated on 57, 23 and 20% of patients, respectively, and by 10 anaesthetists, who shared the workload quite evenly. In all, 76% of patients underwent coronary artery bypass graft surgery alone, 7% aortic valve surgery alone, 7% coronary artery bypass graft surgery and aortic valve repair, and 4% underwent mitral valve surgery alone. The remaining 6% of patients underwent various combinations of these and other procedures. The first two columns of table 11 contain further description of the patients.
The Shewhart chart in fig 11 illustrates that rates of re‐exploration remained low until the 13th batch, in which there were 16 re‐explorations, exceeding the control limit. The moving average chart in fig 22 shows that the re‐exploration rate reached 0% in the preceding 100 patients at patient 686, and that there were no further re‐explorations from patient 686 to 732, in about 1 month. Figures 3A and BB,, respectively, show the moving average for every 50 and 150 consecutive patients. Figure 3A3A does not suggest the process being outside control limits, whereas fig 3B3B indicates sustained performance outside control limits. These findings are disconcerting because the conclusion is dependent on the arbitrary choice of batch size. With increasing batch size comes increasing statistical power to detect an out‐of‐control process, but decreased ability to detect short‐term departures from control.
In 2000, the healthcare professionals responsible for the cardiac surgery service formed the subjective impression that there was an increased rate of re‐exploration. Two recognisable changes to the service could have contributed to this: (1) introduction of the anti‐platelet drug clopidogrel for the management of stable and unstable angina; and (2) appointment of a new cardiac surgeon (surgeon C) and a reduction in the number of cases handled by another cardiac surgeon (surgeon B).
The CUSUM chart of re‐exploration outcomes in fig 4A4A suggested an underlying re‐exploration rate of 5% until (approximately) patient 600. The subsequent run of good outcomes shown in infigsfigs 1 and 22 is evident in fig 4A4A as an accumulation of negative excess re‐explorations. The chart in fig 4A4A remained low until approximately patient 1200, from whom onwards the performance began to deteriorate. In fig 4B4B,, the CUSUM control chart crossed the lower limit at around patient 800. It would be reasonable to restart monitoring from that time using criteria based on expecting a re‐exploration rate of 4%. However, with the continuation of monitoring using the chart in fig 4B4B,, the upper limit was crossed at around patients 1400, indicating that an investigation into an increased rate of re‐exploration was warranted on statistical grounds.
In response to the concerns among team members of a perceived increase in the re‐exploration rate, there was a meeting held in early 2001 (at about patient 1300). Data on outcomes including a CUSUM chart similar to fig 4B4B were presented using projected slides and reproduced on handouts at a meeting of all professional groups, specialists and junior staff involved in the care of patients who underwent cardiac surgery. The presentation confirmed to staff that the subjective concerns about the increasing re‐exploration rate for bleeding had been justified. The meeting evolved into a forum for developing solutions to the problem.
The solutions developed included protocols for the following:
The data on patient outcomes were seen as a useful tool for monitoring progress. There was a commitment to the continued collection and open review of these data.
Following the review meeting, the number of excess re‐explorations for bleeding continued to increase, reaching a peak of 9.9 at patient 1502 at the end of July 2001 (fig 4A4A).). A subsequent combined case note and pharmacy review indicated that a new antiplatelet drug (not included in the perioperative protocols) had been introduced into clinical practice. This was a possible cause of several of the re‐explorations for bleeding around patient 1400 to 1500. From this time, particular attention was focused on the details of management of patients with potential problems scheduled to undergo cardiac surgery. From its peak, the chart (fig 4A4A)) maintained a steady decrease towards and then below zero excess re‐explorations.
Table 11 shows the results of further analysis of the 2148 patients for factors related to re‐exploration for bleeding. The results in table 11 indicate that urgency of surgery, renal impairment, aortic valve replacement and possibly the sex of the patient were associated with re‐exploration risk when considering patient and operation type information only. A risk‐adjusted CUSUM taking into account patient, operation type and operation process information would need to adjust for the use of blood products, prior cardiac surgery and impaired renal function. Such risk‐adjusted analysis can be carried out only when a risk index is available, and so was not possible while monitoring.
The impact of the aforementioned change in surgical staff is illustrated in CUSUM charts for each surgeon in fig 5A5A,, and a univariate comparison of surgeon rates is presented in table 11.. The cumulative number of excess re‐explorations for operations conducted by surgeon C, after an initial jump, settled down, to indicate similar performance to surgeon A (fig 5A5A).). For surgeon B, on the other hand, from about his 100th to 350th operation, there was a re‐exploration rate appreciably lower than 5%. When these surgeon‐specific charts are plotted in fig 5B5B against the patient numbers, it is clear how the shift from surgeon B to surgeon C around patient numbers 1100–1200 coincided with an upward shift in the service's chart of fig 4A4A.. The univariate comparisons of rates among surgeons was changed little by adjusting for the patient and operation type variables used in multivariate analysis in table 11.. However, additional adjustment for the use of blood products as in the second multivariate analysis in table 11 diminished the differences in re‐exploration rates between surgeons (odds ratio (OR) for surgeon B compared with surgeon A=0.6, 95% CI 0.3 to 1.3, p=0.2; OR for surgeon C compared with surgeon A=1.1, 95% CI 0.6 to 1.8, p=0.8).
The case study we present illustrates clearly how the rate of adverse outcomes of a clinical procedure can change over time. The benefits to patients from maintaining a reasonably low level of re‐exploration for bleeding after cardiac surgery include keeping the risk of blood product transfusion and wound infection low, and limiting durations of stay in the intensive care unit and in hospital.5,6,7,8 Although monitoring of mortality after cardiac surgery is not uncommon, we are unaware of any previous published reports of statistical monitoring of re‐exploration for bleeding after open heart surgery, despite its importance and accounts of changing rates.9
This case study could be one of a portfolio of examples used to convince clinicians that statistical monitoring has a place in quality assurance. Control charts are becoming more commonly used for monitoring clinical care and disease surveillance.14 We used one version of CUSUM, but we encourage clinicians to adopt any statistical monitoring method from the most basic Shewhart chart to case‐mix adjusted regression‐based methods.3,14,15,16 The associations shown in table 11 could form the basis of a future risk‐adjusted CUSUM for re‐exploration for bleeding after cardiac surgery.6,7,8 The value of any monitoring chart depends on subjective elements as we have demonstrated, but monitoring is clearly preferable to no monitoring at all. If monitoring is to take place, the data used must be of good quality.17,18
We used a CUSUM chart to confirm a suspected period of deteriorating performance. This was invaluable because it ended speculation and focused staff efforts on resolving the problem.19,20 The positive commitment of staff ensured rigorous adherence to changed protocols, and hence, in our view, contributed to the observed improvements in performance later in the series. The feedback of re‐exploration rates at monthly meetings of the anaesthetic, intensive care and operating theatre teams helped to maintain commitment to the new protocols. These are important principles in the monitoring of any clinical service to achieve cost‐effective outcomes for the organisation.21,22
CUSUM - cumulative sum
Competing interests: None declared.