PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of qualsafetyQuality and Safety in Health CareCurrent TOCInstructions for authors
 
Qual Saf Health Care. 2007 June; 16(3): 192–196.
PMCID: PMC2464986

Monitoring the rate of re‐exploration for excessive bleeding after cardiac surgery in adults

Abstract

Background

The monitoring of adverse events in clinical care can be an important part of quality assurance. There is little evidence on the monitoring of re‐exploration after cardiac surgery.

Objective

To apply statistical monitoring techniques to the rate of re‐exploration for excessive bleeding in adult patients undergoing cardiac surgery procedures using cardiopulmonary bypass at Geelong Hospital, Victoria, Australia, between 1997 and 2003.

Methods

Shewhart charts, moving average plots and cumulative sum (CUSUM) charts were used to demonstrate changes in the rate of re‐exploration over time.

Results

A CUSUM chart was used retrospectively at a time of perceived deteriorating clinical outcomes in patients of the cardiac surgery service. At this time, an intervention aimed at reducing the re‐exploration rate was performed, and subsequent CUSUM charts indicated an improvement in this rate. The CUSUM chart has become an important part of the quality feedback of clinical care outcomes within the Anaesthesia & Pain Management unit of Geelong Hospital.

Conclusion

Statistical monitoring techniques for quality assurance can identify important changes in clinical performance, and their adoption by clinicians is recommended.

Several authors have described the importance of the involvement of clinicians in quality assurance activities.1,2 One component of quality assurance that has been underutilised is statistical monitoring of the rates of specific undesirable patient outcomes. High‐profile cases have helped to illustrate possible methods for the statistical monitoring of adverse clinical outcomes.3 However, examples closer to routine clinical care are required to demonstrate further the role of statistical monitoring as part of a quality assurance process.

An adult cardiac surgery service was commenced at the Geelong Hospital (Geelong, Victoria, Australia) in 1997.4 Re‐exploration for excessive bleeding after cardiac surgery is an important adverse event that has been shown to increase the risk to patients of longer stay at hospital and in the intensive care unit, wound infection, other morbidity and death.5,6,7,8 At the Geelong Hospital, operating theatre staff, anaesthetists, intensive care staff and cardiac surgeons formed a subjective impression that there was an increased rate of re‐exploration for excessive bleeding in cardiac surgery cases. This occurred at a time when there was international evidence of declining rates of re‐exploration.9

In order to determine whether the perception of an increase in the rate of re‐exploration in cardiac surgery cases on cardiopulmonary bypass was warranted, we applied statistical monitoring techniques to the Geelong Hospital Cardiac Database.

Methods

The details of all adult patients undergoing cardiac surgery at the Geelong Hospital between late 1997 and early 2003 were recorded on a cardiac surgical database.4 Postoperative complications such as re‐exploration for bleeding were mandatory fields in the database.

The outcome of interest was the need for re‐exploration (for excessive bleeding) warranting return to the operating theatre. The service's experience with re‐exploration was summarised using Shewhart and moving average charts in batches of 100 patients. With equal‐sized batches, the Shewhart chart is a special case of the moving average chart that plots the moving average only for the 100th, 200th and so on patients. As 100 patients is an arbitrary choice, moving average charts were also plotted for batches of 50 and 150 patients. Probabilities from the binomial distribution were used to define 0.1–99.9% confidence limits around an expected rate of 5% for these charts. The inclusion of these limits converts the plots into control charts, with the process being defined as out of control if the plotted rate in the chart exceeds the limits. If re‐exploration continued at a rate of 5%, then this monitoring would on average take 500 batches of 100 patients before being incorrectly identified as out of control. The smaller the batch size, the shorter this average run length—that is, we expect a false alarm sooner with smaller batch size, but also expect a true alarm sooner.

Statistical monitoring of re‐exploration was performed using cumulative sum (CUSUM) charts.10,11,12,13 Firstly, a simple CUSUM chart, the cumulative number of excess cases of re‐exploration for bleeding, was constructed, assuming that the normal re‐exploration rate was 5%, based on experiences elsewhere.5,7 This was calculated sequentially in date order for each patient as the number of re‐explorations up to that time minus 5% of the number of patients up to the same time. Horizontal movement of this chart for any run of patients indicates performance at a 5% rate.

A CUSUM control chart with limits of acceptable and unacceptable performance was also calculated on the basis of sequential probability.10,12,13 For this chart, an important improvement was defined as a shift in the re‐exploration rate to 4%, and an important deterioration was defined as a shift to 6%. This CUSUM was a sum of contributions with value −s from successful outcomes and (1−s) from unsuccessful outcomes, where s = q/(p+q) and p = ln(0.06/0.04), q = ln(0.96/0.94). Coincidentally, s was close to 0.05, and hence this CUSUM control chart was similar to the number of excess re‐explorations explored in the first simple CUSUM chart. Control limits were set at −6.9 for accepting the null hypothesis of a 4% rate, and at +6.9 for accepting the alternative hypothesis of a 6% rate, by specifying a type 1 error rate of 0.05 and 95% power. This control chart had a reasonable average run length if performance remained close to a rate of 5%, but if performance improved (to 4% as may be possible6,8,9) or deteriorated (to 6%) then the limits would be crossed reasonably quickly.

The analysis of the cardiac surgical database was approved by the Geelong Hospital Research & Ethics Committee.

Results

As of early 2003, 2148 adult patients had undergone cardiac surgery on cardiopulmonary bypass (mean age 66 years). The overall re‐exploration rate for bleeding was 4.9% (106/2148). The service was provided by three surgeons (A, B and C), who operated on 57, 23 and 20% of patients, respectively, and by 10 anaesthetists, who shared the workload quite evenly. In all, 76% of patients underwent coronary artery bypass graft surgery alone, 7% aortic valve surgery alone, 7% coronary artery bypass graft surgery and aortic valve repair, and 4% underwent mitral valve surgery alone. The remaining 6% of patients underwent various combinations of these and other procedures. The first two columns of table 11 contain further description of the patients.

Table thumbnail
Table 1 Characteristics of patients and operations; associations between these characteristics and the risk of re‐exploration for bleeding analysed by univariate and multivariate logistic regression

The Shewhart chart in fig 11 illustrates that rates of re‐exploration remained low until the 13th batch, in which there were 16 re‐explorations, exceeding the control limit. The moving average chart in fig 22 shows that the re‐exploration rate reached 0% in the preceding 100 patients at patient 686, and that there were no further re‐explorations from patient 686 to 732, in about 1 month. Figures 3A and BB,, respectively, show the moving average for every 50 and 150 consecutive patients. Figure 3A3A does not suggest the process being outside control limits, whereas fig 3B3B indicates sustained performance outside control limits. These findings are disconcerting because the conclusion is dependent on the arbitrary choice of batch size. With increasing batch size comes increasing statistical power to detect an out‐of‐control process, but decreased ability to detect short‐term departures from control.

figure qc12435.f1
Figure 1 Number of re‐explorations in batches of 100 consecutive patients (rightmost open circle represents the final 48 patients); horizontal lines represent 0.1% and 99.9% binomial confidence limits around an expected ...
figure qc12435.f2
Figure 2 Moving average re‐exploration rate; the re‐exploration rate in the 100 patients up to and including the patient number shown; horizontal lines represent 0.1% and 99.9% binomial confidence limits around ...
figure qc12435.f3
Figure 3 (A) Moving average re‐exploration rate; the re‐exploration rate in the 50 patients up to and including the patient number shown; horizontal lines represent 0.1% and 99.9% binomial confidence limits around ...

In 2000, the healthcare professionals responsible for the cardiac surgery service formed the subjective impression that there was an increased rate of re‐exploration. Two recognisable changes to the service could have contributed to this: (1) introduction of the anti‐platelet drug clopidogrel for the management of stable and unstable angina; and (2) appointment of a new cardiac surgeon (surgeon C) and a reduction in the number of cases handled by another cardiac surgeon (surgeon B).

The CUSUM chart of re‐exploration outcomes in fig 4A4A suggested an underlying re‐exploration rate of 5% until (approximately) patient 600. The subsequent run of good outcomes shown in infigsfigs 1 and 22 is evident in fig 4A4A as an accumulation of negative excess re‐explorations. The chart in fig 4A4A remained low until approximately patient 1200, from whom onwards the performance began to deteriorate. In fig 4B4B,, the CUSUM control chart crossed the lower limit at around patient 800. It would be reasonable to restart monitoring from that time using criteria based on expecting a re‐exploration rate of 4%. However, with the continuation of monitoring using the chart in fig 4B4B,, the upper limit was crossed at around patients 1400, indicating that an investigation into an increased rate of re‐exploration was warranted on statistical grounds.

figure qc12435.f4
Figure 4 (A) Cumulative number of excess re‐explorations, given an expected rate of 5%. (B) Cumulative sum chart for testing the null hypothesis of risk of re‐exploration for bleeding  = 4%, against ...

In response to the concerns among team members of a perceived increase in the re‐exploration rate, there was a meeting held in early 2001 (at about patient 1300). Data on outcomes including a CUSUM chart similar to fig 4B4B were presented using projected slides and reproduced on handouts at a meeting of all professional groups, specialists and junior staff involved in the care of patients who underwent cardiac surgery. The presentation confirmed to staff that the subjective concerns about the increasing re‐exploration rate for bleeding had been justified. The meeting evolved into a forum for developing solutions to the problem.

The solutions developed included protocols for the following:

  • Preoperative management of patients taking antiplatelet drugs
  • Operative management of antifibrinolytic drugs
  • Investigation of patients with excessive blood loss
  • Perioperative management of patients with excessive blood loss
  • Postoperative management of patients with excessive blood loss

The data on patient outcomes were seen as a useful tool for monitoring progress. There was a commitment to the continued collection and open review of these data.

Following the review meeting, the number of excess re‐explorations for bleeding continued to increase, reaching a peak of 9.9 at patient 1502 at the end of July 2001 (fig 4A4A).). A subsequent combined case note and pharmacy review indicated that a new antiplatelet drug (not included in the perioperative protocols) had been introduced into clinical practice. This was a possible cause of several of the re‐explorations for bleeding around patient 1400 to 1500. From this time, particular attention was focused on the details of management of patients with potential problems scheduled to undergo cardiac surgery. From its peak, the chart (fig 4A4A)) maintained a steady decrease towards and then below zero excess re‐explorations.

Table 11 shows the results of further analysis of the 2148 patients for factors related to re‐exploration for bleeding. The results in table 11 indicate that urgency of surgery, renal impairment, aortic valve replacement and possibly the sex of the patient were associated with re‐exploration risk when considering patient and operation type information only. A risk‐adjusted CUSUM taking into account patient, operation type and operation process information would need to adjust for the use of blood products, prior cardiac surgery and impaired renal function. Such risk‐adjusted analysis can be carried out only when a risk index is available, and so was not possible while monitoring.

The impact of the aforementioned change in surgical staff is illustrated in CUSUM charts for each surgeon in fig 5A5A,, and a univariate comparison of surgeon rates is presented in table 11.. The cumulative number of excess re‐explorations for operations conducted by surgeon C, after an initial jump, settled down, to indicate similar performance to surgeon A (fig 5A5A).). For surgeon B, on the other hand, from about his 100th to 350th operation, there was a re‐exploration rate appreciably lower than 5%. When these surgeon‐specific charts are plotted in fig 5B5B against the patient numbers, it is clear how the shift from surgeon B to surgeon C around patient numbers 1100–1200 coincided with an upward shift in the service's chart of fig 4A4A.. The univariate comparisons of rates among surgeons was changed little by adjusting for the patient and operation type variables used in multivariate analysis in table 11.. However, additional adjustment for the use of blood products as in the second multivariate analysis in table 11 diminished the differences in re‐exploration rates between surgeons (odds ratio (OR) for surgeon B compared with surgeon A = 0.6, 95% CI 0.3 to 1.3, p = 0.2; OR for surgeon C compared with surgeon A = 1.1, 95% CI 0.6 to 1.8, p = 0.8).

figure qc12435.f5
Figure 5 (A) Individual charts of cumulative number of excess re‐explorations for bleeding for surgeons A, B and C; number of excess re‐explorations for a surgeon, given an expected rate of 5% of patients. (B) Surgeons' ...

Discussion

The case study we present illustrates clearly how the rate of adverse outcomes of a clinical procedure can change over time. The benefits to patients from maintaining a reasonably low level of re‐exploration for bleeding after cardiac surgery include keeping the risk of blood product transfusion and wound infection low, and limiting durations of stay in the intensive care unit and in hospital.5,6,7,8 Although monitoring of mortality after cardiac surgery is not uncommon, we are unaware of any previous published reports of statistical monitoring of re‐exploration for bleeding after open heart surgery, despite its importance and accounts of changing rates.9

This case study could be one of a portfolio of examples used to convince clinicians that statistical monitoring has a place in quality assurance. Control charts are becoming more commonly used for monitoring clinical care and disease surveillance.14 We used one version of CUSUM, but we encourage clinicians to adopt any statistical monitoring method from the most basic Shewhart chart to case‐mix adjusted regression‐based methods.3,14,15,16 The associations shown in table 11 could form the basis of a future risk‐adjusted CUSUM for re‐exploration for bleeding after cardiac surgery.6,7,8 The value of any monitoring chart depends on subjective elements as we have demonstrated, but monitoring is clearly preferable to no monitoring at all. If monitoring is to take place, the data used must be of good quality.17,18

We used a CUSUM chart to confirm a suspected period of deteriorating performance. This was invaluable because it ended speculation and focused staff efforts on resolving the problem.19,20 The positive commitment of staff ensured rigorous adherence to changed protocols, and hence, in our view, contributed to the observed improvements in performance later in the series. The feedback of re‐exploration rates at monthly meetings of the anaesthetic, intensive care and operating theatre teams helped to maintain commitment to the new protocols. These are important principles in the monitoring of any clinical service to achieve cost‐effective outcomes for the organisation.21,22

Key messages

  • The overall performance with respect to the rate of re‐exploration in adult patients undergoing cardiac surgery was 4.9%, close to that expected.
  • Retrospective application of statistical monitoring techniques showed that there had been a significant improvement in performance during the study period, followed by a significant deterioration in performance.
  • The deterioration in performance was noticed by staff in the absence of prospective statistical monitoring. However, this perceived change remained speculative, until confirmed by evidence in the form of a statistical control chart presented at a staff meeting.

Abbreviations

CUSUM - cumulative sum

Footnotes

Competing interests: None declared.

References

1. Shortell S M. Physician involvement in quality improvement: issues, challenges, and recommendations. In: Blumenthal D, Scheck A, eds. Improving clinical practice. San Francisco: Jossey‐Bass, 1995. 205–228.228
2. Weiner B J, Shortell S M, Alexander J A. Promoting clinical involvement in hospital quality improvement efforts: the effects of top management, board, and physician leadership. Health Serv Res 1997. 32491–510.510 [PMC free article] [PubMed]
3. Spiegelhalter D, Grigg O, Kinsman R. et al Risk‐adjusted sequential probability ratio tests: applications to Bristol, Shipman and adult cardiac surgery. Int J Qual Health Care 2003. 157–13.13 [PubMed]
4. Bucknell S J, Mohajeri M, Low J. et al Single‐versus multiple‐dose antibiotics prophylaxis for cardiac surgery. ANZ J Surg 2000. 70409–411.411 [PubMed]
5. Bolsin S N. Postoperative organ dysfunction after cardiac surgery. In: Prys‐Roberts C, Brown BR, eds. International practice of anaesthesia. Oxford: Butterworth‐Heinemann, 1996, 1/55/1–8
6. Karthik S, Grayson A D, McCarron E E. et al Reexploration for bleeding after coronary artery bypass surgery: risk factors, outcomes, and the effect of time delay. Ann Thorac Surg 2004. 78527–534.534 [PubMed]
7. Moulton M J, Creswell L L, Mackey M E. et al Reexploration for bleeding is a risk factor for adverse outcomes after cardiac operations. J Thorac Cardiovasc Surg 1996. 1111037–1046.1046 [PubMed]
8. Dacey L J, Munoz J J, Baribeau Y R. et al Reexploration for hemorrhage following coronary artery bypass grafting: incidence and risk factors. Arch Surg 1998. 133442–447.447 [PubMed]
9. Munoz J J, Birkmeyer N J O, Dacey L J. et al Trends in rates of reexploration for hemorrhage after coronary artery bypass surgery. Ann Thorac Surg 1999. 681321–1325.1325 [PubMed]
10. Bolsin S N, Colson M. Methodology matters; CUSUM. Int J Qual Health Care 2000. 12433–438.438 [PubMed]
11. Bolsin S. Routes to quality assurance: risk adjusted outcomes and personal professional monitoring. Int J Qual Health Care 2000. 12367–369.369 [PubMed]
12. Kestin I G. A statistical approach to measuring the competence of anaesthetic trainees at practical procedures. Br J Anaesth 1995. 75805–809.809 [PubMed]
13. Davies O L. The design and analysis of industrial experiments. 2nd edn. London: Longman, 1978. 57–93.93
14. Woodall W H. The use of control charts in health‐care and public‐health surveillance (with Discussion). J Qual Technol 2006. 3889–134.134
15. Poloniecki J, Sismanidis C, Bland M. et al Retrospective cohort study of false alarm rates associated with a series of heart operations: the case for hospital mortality monitoring groups. BMJ 2004. 328375–379.379 [PMC free article] [PubMed]
16. Steiner S H, Cook R J, Farewell V T. et al Monitoring surgical performance using risk‐adjusted cumulative sum charts. Biostatistics 2000. 11441–1452.1452 [PubMed]
17. Bird S M, Cox D, Farewell V T. et al Performance indicators: good, bad, and ugly. J R Stat Soc Ser A Stat Soc 2005. 1681–27.27
18. Spiegelhalter D J, Aylin P, Best N G. et al Commissioned analysis of surgical performance using routine data: lessons from the Bristol inquiry. J R Stat Soc Ser A Stat Soc 2002. 165191–231.231
19. Walshe K, Shortell S. When things go wrong: how health care organisations deal with major failures. Health Aff 2004. 23103–111.111 [PubMed]
20. Shortell S M, Bennet C L, Byck G R. Assessing the impact of continuous quality improvement on clinical practice: what it will take to accelerate progress. Milbank Q 1998. 76593–624.624 [PubMed]
21. Bolsin S, Colson M. Making the case for personal professional monitoring in health care. Int J Qual Health Care 2003. 151–2.2 [PubMed]
22. Bolsin S, Solly R, Patrick A. The value of personal professional monitoring performance data and open disclosure policies in anaesthetic practice: a case report. Qual Saf Health Care 2003. 12295–297.297 [PMC free article] [PubMed]

Articles from Quality & Safety in Health Care are provided here courtesy of BMJ Publishing Group