SPC has been applied to healthcare improvement in a wide range of settings and specialties, at diverse levels of organisations and directly by patients, using many types of variables (fig 1, tables 2–5). We found reports of substantial benefits of SPC application, as well as important limitations of, barriers to and factors that facilitate SPC application (tables 6–9). These findings indicate that SPC can indeed be a powerful and versatile tool for managing changes in healthcare through QI. Besides helping diverse stakeholders manage and improve healthcare
processes, SPC can also help clinicians and patients understand and improve patients' health
when applied directly to health indicators such as PEFR in asthma or blood sugar concentrations in diabetes. In healthcare, the “study subject” can thus also be an active agent in the process, as when patients apply SPC to their own health. Several studies indicated the empowering effects this may have on patients.35,38,40,50
SPC application thus has therapeutic potential as it can help patients manage their own health. We agree with Alemi and Neuhauser70
that this potential merits further investigation.
Most of the included articles concerned application of SPC in healthcare improvement in the USA. Articles from other countries appeared only towards the end of the study period (fig 1). We have no explanation for this finding, but we speculate that it is related to differences between US and other healthcare systems with regard to QI awareness and implementation.73
Only 22 studies included in the review were intended to test the utility of SPC (table 1). Of the four controlled studies, only one included a control chart in the intervention (as a minor component which did not fully exploit the features of SPC). In 35 articles we did not find an intention to test the utility of SPC application. In those cases, SPC was applied for other reasons (ie, to evaluate the impact of other interventions). Even though many articles thus did not address the utility of SPC, all studies offered information—to varying degrees—relevant to our review's question of how SPC has been applied to healthcare. The utility of SPC is reflected in benefits reported regarding SPC application (table 6).
SPC has been applied in over 20 specialties or fields of healthcare, at a wide range of levels (tables 3 and 4), suggesting that SPC has broad applicability in healthcare. The dominance of anaesthesia and intensive care can be explained in large part by the fact that many studies included their services in conjunction with other specialties. This reflects the way in which anaesthesia has a vital supporting role in many clinical care processes. The 97 SPC variables reported (table 5) demonstrate a diversity of situations in which SPC has been applied, ranging from process indicators of patients' health to health outcomes and many aspects of healthcare processes and organisational performance. This indicates that SPC is a versatile QI tool.
The benefits of SPC application (table 6) mirror those given in books and tutorials on SPC (exemplified by the quote in the Introduction to this review). As noted in a report from a top‐ranked healthcare system which has applied SPC widely:
“Among the most powerful quality management tools that IHC [Intermountain Health Care, USA] has applied is statistical process control, SPC. Most notable among those tools are control charts. Under optimal conditions, these graphical depictions of process performance allow participants to know what is happening within their processes as ‘real time' data enable them to make appropriate decisions. The capability of truly understanding processes and variation in a timely manner has resulted in the most dramatic, immediate, and ongoing improvements of any management technique applied at IHC.” (Shaha,26
The limitations of SPC application (table 7) identified by this review are important, and yet perhaps less emphasised than the benefits in books and tutorials on SPC. SPC cannot solve all problems and must be applied wisely. There are many opportunities to “go wrong”, as illustrated by the case where incorrect application was highlighted by other authors (limitation number 5 in table 7). In several cases, our own understanding of SPC suggested that investigators had not used it correctly or fully (eg, standard decision rules to detect special causes were not applied to identify process changes). In the worst case scenario, incorrect application of SPC could lead to erroneous conclusions about process performance and waste time, effort and spirit and even contribute to patient harm. In the more authoritative studies we reviewed, co‐investigators included experts in industrial engineering or statistics or authors who otherwise had developed considerable expertise in SPC methodology. On the basis of these observations, we conclude that although SPC charts may be easy to use even for patients, clinicians or managers without extensive SPC training, they may not be equally simple to construct correctly. To apply SPC is, paradoxically, both simple and difficult at the same time. Its power hinges on correct and smart application, which is not necessarily a trivial task. The key, then, is to develop or recruit the expertise necessary to use SPC correctly and fully and to make SPC easy for non‐experts to use, before using it widely.
Autocorrelation is another limitation of SPC highlighted by this review. Our review, and published books, offer limited advice on how to manage it:
“There is no single acceptable way of dealing with autocorrelation. Some would say simply to ignore it. [Others] would disagree and suggest various measures to deal with the phenomenon. One way is to avoid the autocorrelation by sampling less frequently. ... Others argue against plotting autocorrelated data on control charts and recommend that the data be plotted on a line chart (without any centerline or control limits).” (Carey,4
Just over a quarter of the articles reported barriers to SPC application (table 8). The three broad divisions of barriers—people, data and chart construction, and IT—indicate where extra care should be taken when introducing SPC in a healthcare organisation. Ideas on how to manage the limitations of and barriers to SPC application can be found among the factors reported to facilitate SPC application (table 9). They deal with, and go beyond, the areas of barriers we found. We noted the prominence of learning and also of focusing on topics of interest to clinicians and patients. The 11 categories under the heading “Smart application of SPC can be helpful” contain valuable approaches that can be used to improve SPC application. Examples include risk adjustment51,52,71
to enable correct SPC analysis of data from heterogeneous populations of patients (or organisational units). Basic understanding of SPC must be taught to stakeholders and substantial skill and experience is required to set up successful SPC application. Experts, or facilitators, in healthcare organisations can help, as indicated in table 9, and as we have described for other QI methods.74
We found more information on SPC benefits and facilitating factors than on limitations and barriers, and this may represent a form of publication bias, as indicated by the quote in the Introduction.11
We did not find any study that reported failed SPC application. We can speculate that there have been situations when SPC application failed, just as there must be many cases of successful SPC application that have not been reported in the literature. Studies of failed SPC application efforts, as well as studies designed to identify successful ways to apply SPC to manage change, would help inform future SPC application efforts. On the basis of this review, we agree with the argument that “medical quality improvement will not reach its full potential unless accurate and transparent reports of improvement work are published frequently and widely (p 319),”75
and also that the way forward is to strengthen QI research rather than to lower the bar for publication.76
Methodological considerations regarding the included studies
None of the studies we found was designed to evaluate the effectiveness quantitatively—that is, the magnitude of benefits—of SPC application. This would have required other study designs such as cluster randomised trials or quasi‐experimental studies.12
Although the “methods of evaluating complex interventions such as quality improvement interventions are less well described [than those to evaluate less complex interventions such as drugs]”, Eccles et al
argue that the “general principle underlying the choice of evaluative design is ... simple—those conducting such evaluations should use the most robust design possible to minimise bias and maximise generalisability. [The] design and conduct of quantitative evaluative studies should build upon the findings of other quality improvement research (p 47).”77
This review can provide such a foundation for future evaluative studies.
An important distinction is warranted here: we believe that SPC rests on a solid theoretical, statistical foundation and is a highly robust method for analysing process performance. The designs of the studies included in this systematic review were, however, not particularly robust with regard to evaluating the effectiveness of SPC application, and that was not their objective. This does not mean that SPC is not a useful tool for QI in healthcare, only that the studies reviewed here were more vulnerable to bias than more robust study designs, even if they do indicate many clear benefits of SPC application (table 6). Despite the studies not being designed to evaluate the effectiveness of SPC, many used SPC to effectively show the impact of QI or other change initiatives. In this way, SPC analysis can be just as powerful and robust as study designs often deemed superior, such as randomised controlled trials (RCTs).77
The key to this power is the statistical and practical ability to detect significant changes over time in process performance when applying SPC.9
On the basis of a theoretical comparison between control charts and RCTs, Solodky et al38
argue that control charts can complement RCTs, and sometimes even be preferable to RCTs, since they are so robust and enable replication—“the gold standard” for research quality—at much lower cost than do RCTs. These points have been further elaborated in subsequent work.78,79
A curious methodological wrinkle in our review is: can you evaluate the application of a method (eg, SPC) using that same method for the evaluation? Several of the included studies used SPC both as (part of) an intervention and as a method to evaluate the impact of that intervention. For example, Curran et al
used annotated control charts to feed information on MRSA acquisition rates back to stakeholders and used these same control charts to show the effectiveness of the feedback programme.61
Relationship between monitoring and improvement
When SPC is applied for monitoring, rather than for managing change, the aims are different—for example, to detect even small but clinically important deviations in performance—as are the methodological challenges.80,81
This review focused on the latter. Thus although studies on SPC application for monitoring healthcare performance were excluded from this review, we recognise the importance of such monitoring. The demarcation between monitoring and improvement is not absolute. Indeed, there are important connections between measurement, monitoring and improvement, even if improvement does not follow automatically from indications of dissatisfactory performance. “To improve performance, organizations and individuals need the capability to control, improve, and design processes, and then to monitor the effects of this improvement work on the results. Measurement alone will not suffice (pp 1–35).”82
Monitoring performance by way of control charts has been suggested as a better approach to clinical governance in the British National Health Service. Through six case studies, Mohammed et al
demonstrate how control chart monitoring of performance can distinguish normal performance from performance that is either substandard or better than usual care. “These case studies illustrate an important role for Shewhart's approach to understanding and reducing variation. They demonstrate the simplicity and power of control charts at guiding their users towards appropriate action for improvement (p 466).”83
Comments on the review methodology
No search strategy is perfect, and we may well have missed some studies where SPC was applied to healthcare QI. There are no SPC specific keywords (eg, Medical Subject Headings, MeSH) so we had to rely on text words. Studies not containing our search terms in the title or abstract could still be of potential interest although presumably we found most of the articles where SPC application was a central element. We believe the risk that we systematically missed relevant studies to be small. Therefore, our findings would probably not have changed much due to such studies that we might have missed.
The review draws on our reading, interpretation and selection of predominantly qualitative data—in the form of text and figures—in the included articles to answer the questions in our data abstraction form. The questions we addressed, the answers we derived from the studies, and the ways we synthesised the findings are not the only ways to approach this dataset. Furthermore, each member of the review team brought different knowledge and experiences of relevance to the review, potentially challenging the reliability of our analysis. An attempt was made to reduce that risk by having one investigator read all data abstraction forms, and obtain clarifications or additional data from the original articles when needed. That investigator also conducted the initial data synthesis, which was then reviewed by the entire team and the two outside experts. Although other interpretations and syntheses of these data are possible, we believe that ours are plausible and hope they are useful.
The methods for reviewing studies based primarily on qualitative data in healthcare are less well developed than the more established methods for quantitative systematic reviews, and they are in a phase of development and diversification.13,84,85
Among the different methods for synthesising evidence, our approach is best characterised as an interpretive (rather than integrative) review applying thematic analysis—it “involves the identification of prominent or recurrent themes in the literature, and summarising the findings of different studies under thematic headings”.86
There is no gold standard for how to conduct reviews of primarily qualitative studies. Our response to this uncertainty has been to use the best ideas we could find, and to be explicit about our approach to allow readers to assess the findings and their provenance.
The main limitation of this review is the uncertainty regarding the methodological quality of many of the primary studies. Assessment of quality of qualitative studies is still under debate, and there is no consensus on whether
at all, or, if so, how
to conduct such assessments.84
We reviewed all the studies that satisfied our inclusion criteria and made no further quality assessment. Therefore our findings should be considered as tentative indications of benefits, limitations, etc to be corroborated, or rejected, by future research. The main strength of this review is our systematic and explicit approach to searching and including studies for review, and to data abstraction using a standardised form. It has helped generate an overview of how SPC has been applied to healthcare QI with both breadth and depth—similar to the benefits of thematic analysis reported by investigators reviewing young people's views on health and health behaviour.87
In conclusion, this review indicates how SPC has been applied to healthcare QI with substantial benefits to diverse stakeholders. Although there are important limitations and barriers regarding its application, when applied correctly SPC is a versatile tool which can enable stakeholders to manage change in healthcare and improve patients' health.