Phase one: theoretical framework, evidence review to propositions
For this study, the process of theory formulation began as a synthesis of policy and research literature; the theories and working propositions (i.e.
, CMOs) were then refined through data analysis and interpretation. We conducted the evidence review using the principles of realist synthesis [26
]. Using this approach ensured the study had methodological and theoretical integrity.
The first stage of the synthesis involved the identification of concepts, programme theories, and subsequent framework development (Figure ). The construction of the framework was informed by the funder's requirement, an initial review of the literature undertaken for the proposal [6
], and key policy developments. The study's theoretical framework integrates various components, including the four areas that play a role in protocol-based care and related impact on stakeholder outcomes: patients, staff, organisations, and policy makers:
1. What are the properties of protocol-based care and protocols?
2. How are protocols developed?
3. What is the impact of protocol-based care?
4. How is protocol-based care implemented and used?
Additionally, implicit in the framework is the notion that protocol-based care is about introducing new practices, which is a function of the nature of the evidence underpinning the new practice (protocol, guideline), the readiness and quality of the context into which they are to be implemented and used, and the processes by which they are implemented. Therefore, the Promoting Action on Research Implementation in Health Service (PARIHS) framework was also embedded into the framework [9
]. The four theoretical areas needed to be related to outcomes and stakeholder issues; as such each area contained additional review questions:
1. Properties of protocol-based care and protocols:
1a. What is protocol-based care?
1b. What are protocols and what types/models of protocol based care are used in practice?
1c. What patient care issues/topics are covered by protocol-based care?
2. Development of protocols:
2a. How are protocols developed?
2b. What forms of evidence underpin the development of protocols?
2c. How does the method of protocol development affect use?
3. Impact of protocol-based care:
3a. How does protocol-based care impact on patient and organisational outcomes?
3b. How does protocol-based care impact on nurses and midwives?
3c. How does protocol-based care impact on nurses' and midwives' decision-making?
3d. How does protocol-based care impact on multi-disciplinary decision-making and interaction?
4. Implementation and use:
4a. What approaches are used to implement protocols, and how does this impact on their use?
4b. What are the facilitators and barriers to protocol-based care?
These questions were addressed by referring to available literature. Electronic searching including the Cochrane Trial Register, Medline, Embase, Cinahl, Assia, Psychinfo and hand searching was also used. As this literature about standardising care is vast and applying the principle suggested by Pawson [27
], searching and retrieval stopped when there was sufficient evidence to answer the questions posed. Literature was reviewed and information extracted using a proforma designed to capture data about the questions in each theory area, and their impact on patients, organisations, and staff.
As part of the review process, propositions were developed to be evaluated in phase two. Propositions were developed by searching for patterns within the literature about a particular theory area related to CMO. For example, in relation to properties of protocol-based care, looking for patterns about what types of properties (mechanisms) of standardised care approaches might impact (outcome) on their use in particular care settings (context)? In practice, because the literature was so variable, it was difficult to trace clear CMO threads, therefore some of the resultant propositions were fairly broad.
By way of illustration the following sections provide a brief summary of the literature within each theory area and linked propositions [29
Theory area one: Properties of protocol-based care and protocols
Standardised care approaches are widely used in service delivery and care; however, the term protocol-based care is absent. Similarly, there is little clarity about standardised care approaches, what they are, and a lack of agreement and consistency in the way terms are used. We found that standardised care approaches: localised care delivery through the use of care pathways, protocols, guidelines, algorithms (and other approaches such as patient group directives), and by particularising evidence to the local context; varied in the degree of specificity and prescriptiveness of formalised and/or codified information, and have the potential to involve all members of the health care team, and facilitate the sharing of roles and responsibilities. The following propositions resulted:
1. A clear understanding about the purpose and nature of protocol-based care by potential users will determine the extent to which standard care approaches are routinely used in practice.
2. The properties of standardised care approaches, such as degree of specificity and prescriptiveness, will influence whether and how they are used in practice.
Theory area two: Development of protocols
Whether standardised care approaches impact on practice and patient care is likely to be partly dependent on the way in which they are developed and the evidence base used in the development process. There is some available guidance on development processes; however this is general, and it is not clear how this has been used to develop standardised care approaches locally. Furthermore, authors who have developed protocols locally tend to provide limited information about development processes. It is therefore unclear how the development process might affect the subsequent use of resulting standardised approaches to care because of limited empirical evidence. The following propositions resulted:
1. Standardised care approaches that are developed through a systematic, inclusive, and transparent process may be more readily used in practice.
2. Standardised care approaches that are based on a clear and robust evidence base are more likely to impact positively on outcomes.
3. Locally developed standardised care approaches may be more acceptable to practitioners and consequently more likely to be used in practice.
Theory area three: Impact of protocol-based care
The evidence for the impact of standardised care processes on practice, patient and staff outcomes is variable. Even within studies there may be a demonstrable effect on one type of outcome, but no significant changes to others. There are questions about whether it may be the components or characteristics of the particular protocol, or the process of implementation that influence impact, or both. However, there is evidence to indicate that standardised care approaches can be influential, if only to raise awareness about particular issues or as an opportunity to bring clinical teams together [30
]. Findings from research also show that protocols can enable nurses' autonomous practice, support junior or inexperienced staff, and can be a vehicle for asserting power [31
]. The following propositions resulted:
1. The impact of protocol-based care will be influenced by the type of protocol being used, by who is using it/them, how, and in what circumstances.
2. More senior and experienced clinical staff will be less positive than junior and/or inexperienced nurses about using standardised care approaches.
3. The impact on decision making will be influenced by practitioners' perceived utility of standardised approaches to care.
4. Protocol-based care will impact on the scope and enactment of traditional nursing roles. Protocol-based care has the potential to enhance nurses' autonomy and decision-making latitude.
5. The impact on patient care will be influenced by the characteristics and components of the protocol and factors in the context of practice.
Theory area four: Implementation and use
Approaches to implementation, including clear project leadership, that have the scope to identify and address the complexities of use may be more successful in encouraging uptake than those that do not. Furthermore, integrating standardised care approaches within existing systems and processes may facilitate their use. In addition, certain contextual factors may facilitate or inhibit the use of standardised care approaches, although what these factors are requires further investigation. The following propositions resulted:
1. Interactive and participatory approaches and strategies to implement standardised approaches to care may influence whether or not they are used in practice.
2. The support of a project lead may increase the likelihood of the ongoing use of standardised care approaches.
3. Embedding the standardised care approach into systems and process may facilitate use, but there is a lack of evidence about how this might work for different groups and in different contexts.
4. Some contexts will be more conducive to using standardised care approaches than others, but it is unclear what might work in what circumstances and how.
Phase two: Testing propositions through case studies
Case study [32
] was used because it is methodologically complementary to realistic evaluation, which advocates the use of multiple methods to data collection, and recognises the importance of context. As with case study, realistic evaluation calls for making sense of various data sets (i.e.
, plurality) to develop coherent and plausible accounts. The refinement of the propositions required descriptive and explanatory case study. Additionally, in order to assist in explanation building and transferability of findings, multiple comparative case studies were included.
A 'case' was defined as a particular clinical setting/context, for example, a cardiac surgical unit (CSU), and the 'embedded unit' of that case the use of a particular standardised care approach, for example, the care pathway. Sites were purposively sampled in order to maximise rigour in relation to applicability and theoretical transferability [34
]. Criteria for selection included reported active engagement in protocol-based care activity, a requirement to study the use of a variety of standardised care approaches, and to study this use in different clinical settings in depth over time. Sites selected within England are listed in Table .
Clinical sites selected for study.
Pawson and Tilley [21
] argue that realistic evaluators should not be pluralists for pluralism's sake, but that methods should be chosen to test the hypotheses/propositions. Given the broad scope of the initial propositions and a desire to capture how standardised care approaches worked in situ
, we used a combination of methods, including those from ethnography:
1. Non-participant and participant observation of nursing and multi-disciplinary activities related to the use of standardised care approaches. Observations and discussions were recorded in field notes and/or audio-recorded as appropriate.
2. Post-observation interviews guided by issues arising from observations.
3. Key stakeholder interviews exploring views in general about the use, influences on use, and impact of standardised care approaches. Interviews were audio-recorded and later transcribed in full.
4. Interviews with patients about their experiences of standardised care.
5. Tracking of patient journeys in which patients were interviewed a number of times during their contact with the service.
6. Review of relevant documentation, such as copies of guidelines, protocols, and pathways.
7. Field notes written during and after each site visit.
Data were collected in sites for between 20 and 50 days. Study participants and data collected are presented in Tables and .
Data collected within and across sites.
Multi-site Research Ethics Committee (MREC) approval was sought and given. Each potential participant was given information about the study and an appropriate period of time allowed to lapse to before written consent was sought. Anonymity was assured by each site and all participants were given an identity code.
Approach to analysis
As this evaluation was a 'snap shot' of the use of standardised care approaches within sites, we used the analysis stage to test and refine propositions between site visits, and then in the final stages across data sets and sites, i.e., we did not capture any changes within sites over time.
Using a process of pattern matching and explanation building for each CMO, evidence threads were developed from analysing and then integrating the various data. The fine tuning of CMOs was a process that ranged from abstraction to specification, including the following iterations:
1. Developing the theoretical propositions at the highest level of abstraction -- what might work, in what contexts, how, and with what outcomes, and are described in broad/general terms above. For example, 'embedding the standardised care approach into systems and process (M1) may facilitate use' (O1) at least in some instances (C1, C2, C3...).
2. Data analysis and integration facilitated CMO specification ('testing'). That is, we refined our understanding of the interactions between M1, O1, C1, C2, C3. For example, data analysis showed that in fact there appeared to be particular approaches to embedding standardised care approaches (computerisation) (now represented by M2), that had an impact on their routine use in practice (now represented by O2), in settings where nurses were autonomous practitioners (an additional C, now represented by C4). These new CMO configurations (i.e., propositions) were then 'tested' with data from other sites to seek disconfirming or contradictory evidence.
3. Cross-case comparisons determined how/whether the same mechanisms played out in different contexts to produce different outcomes.
This process resulted in a set of theoretically generalisable features addressing our overarching evaluation question: Protocol-based care: what works, for whom, why, and in what circumstances? The following sections describe some of the findings that emerged from the analysis.
The nature of protocol-based care
Protocol-based care encompassed a variety of different standardised care approaches, patient conditions, and care delivery often within single sites; however, it was not a term that participants recognised. Data shows that protocol-based care was no greater than delivering (some) care with the use of particular standardised care approaches. In the reality of practice, the use of standardised care approaches was patchy, and influenced by individual, professional, and contextual factors. The most commonly used approaches were care pathways, local guidelines, protocols, algorithms, and patient group directives (PGD; medication prescribing protocol). Each of these was perceived, and did in practice, have differing levels of prescriptiveness, specificity, and applicability. These approaches and their characteristics have been plotted in Figure .
Conceptualisation of frequently used standardised care approaches.
Data shows that protocol-based care appeared not to be greater than the sum of its parts [8
]. The initial propositions (conjectured CMOs) that were developed from the evidence review only partially corresponded to the findings that emerged during analysis. From the iterative analysis process of scrutinising mechanisms, context, and outcomes (i.e.
, propositions), we were able to draw out what works, for whom, how, and what circumstances in relation to the use of standardised care approaches (refined CMOs). This is summarised in Table and elaborated on in the text below by integrating data to provide some illustrative examples of what worked, for whom, how, and in what circumstances (see full report for a comprehensive account of the findings with data excerpts [29
What works, for whom, how, and in what circumstances.
Example one: What works, for whom, how, in what circumstances -- extending roles and autonomy
There was clear evidence to show that standardised care approaches enabled the extension of traditional roles, and facilitate autonomous practice, which in turn resulted in more nurse and midwifery led care and services. These were perceived to be positive developments by doctors, nurses, and midwives. This finding came from data collected in the walk-in-centre (WIC), pre-assessment clinics (PAC), birth centre (BC), GP surgery (GPS), and diabetes clinic (DC), in the following ways:
WIC -- The clinical guidelines and algorithms facilitated the development of nurses' skills in examining and diagnosing. The patient group directives enabled them to extend their role to treating patients without the need to consult GP colleagues to obtain prescriptions.
PAC -- The pre-operative assessment guidelines and protocols supported nurse-led clinics enabling them to make decisions about what tests to order, how to interpret results, and ultimately to make decisions about fitness for surgery.
BC -- The normal labour pathway supported the development of a midwifery-led service for healthy pregnant women.
GPS -- Protocols enabled nurses to independently run clinics on the management of chronic diseases such as asthma, diabetes, and hypertension. Nurses were responsible for diagnosing, monitoring patient status, and recommending appropriate medications.
DC -- Protocols facilitated clinical nurse specialists to run clinics and performing tests and procedures independently.
It is difficult to determine whether it was the standardised care approaches that facilitated autonomous practice or the practice environment that supported nurses' practising autonomously. In this study, nurses were able to practice autonomously because of their role (they tended to be more senior, and/or be independent practitioners, e.g., clinical nurse specialists, midwives and health visitors) and because services were nurse-led. The development and introduction of standardised care approaches facilitated the enactment of both nurse-led service delivery and to work outside their traditional scope of practice. Findings showed that where nurses practised autonomously they were able to deliver more streamlined care because on a patient-by-patient basis they did not have to refer to, or follow up with doctors. A perhaps unintended consequence was the perceived protection value available standardised care approaches offered if nurses' judgements were questioned; they were considered to be a 'safety net.' In contrast, some doctors interviewed felt they provided a 'false sense of security.'
Example two: What works, for whom, how, in what circumstances -- use and visibility
Observing practice was useful in determining how and if standardised care approaches were being used in the practice settings. Overall, the use of standardised care approaches across all sites could be described on a continuum ranging from implicit to explicit use (see Figure ). For example, there were instances where during their interactions with patients, nurses, and doctors explicitly referred to protocols (e.g., as a checklist or reference). In contrast, there were many occasions where it was not obvious that available standardised care approaches were being used to explicitly guide care. For example, in the PAC clinics whilst there were protocols for ordering patient tests, nurses did not always refer to them, but used principles from them to apply to particular patients, justifying why they had not used the protocol in those instances.
Examples of how standardised care approaches were used.
The location of the standardised care approach and its level of visibility influenced how and whether it was used. In settings where they were more visible, physically close to the patient-practitioner interaction, and/or easily accessible, they tended to be referred to more often. For example, algorithms in the walk in centre were computer-based and were often used as an onscreen-prompting tool during interactions with patients. A similar finding emerged from GP site data where most staff routinely used the onscreen protocols (SOFIs) related to the Quality and Outcomes Framework (QOF). In the walk-in centre some nurses had copies of PGDs that fitted into their pockets or bags so that they could be quickly and easily referred to at the point of care. Furthermore, embedding the care pathways in documentation in both the cardiac surgical unit and the birth centre ensured that they were used routinely by the relevant professionals. In sites where these mechanisms were not in place, the explicit use of the standardised care approaches was patchy. For example, in the cardiac-thoracic unit, nurses described the location of guidelines, policies, and protocols as scattered in various areas, and mainly hidden from view. Similarly, in the pre-operative assessment clinics where the guidelines and protocols were in a paper-based manual, they were rarely referred to.
Example three: What works, for whom, how, in what circumstances -- making a difference
Where practitioners could see that the use of the standardised care approaches were making a difference to their practice, patient care, or service delivery, they tended to be more consistently used. In the GP site, opinion was unanimous that the use of the QOF-related protocols had improved the standard of patients' care; this perception was supported by the consistent achievement of targets and high QOF points, which provided a financial incentive to continue use.
In other sites, the ability of nurses to be able to practise autonomously and in extended roles appeared to provide a motivation to continue to use available protocols and guidelines. This was particularly the case in the walk-in centre with the use of the PGDs and algorithms, in the birth centre where care was completely midwifery led, and in the GP practice where nurses, midwives, and health visitors were running clinics.
Example four: What works, for whom, how, in what circumstances -- prescriptiveness versus flexiblity
The flexibility of the standardised care approaches appeared to impact on the way that they were used; however there are contradictory findings with respect to flexibility. For example, interviewees in the cardiac surgical unit felt that the care pathway was inflexible because it could not be used with patients who were complex cases (the care pathway had been developed for 'straightforward' cases). In contrast, nurses in the walk-in centre were using algorithms, which they described as prescriptive (and so not flexible) and apart from a small number of nurses, they were consistently used, even if only as a checklist at the end of a procedure or patient interaction. Similarly, protocols related to QOF, whilst prescriptive, were used by most staff in the practice. Whether it was the flexibility of the standardised care approach per se that influenced the type and amount of its use, or factors such as the motivation for using them --for example, incentives and being able to run a nurse-led service independently -- is difficult to unravel. However, this finding highlights that context of use is important, what might work in one setting may work differently in another.
Example five: What works, for whom, how and in what circumstances -- information sources
For new and/or junior doctors, nurses, and midwives, standardised care approaches of all types were perceived to be useful information resources. In contexts in which there were frequent staff changes, and/or reliance on agency practitioners, local standardised care approaches provided information about what was expected in terms of care delivery and standards in that particular setting. As a result, in some sites they were included in induction materials and formed part of competency assessments. In contrast, there was an expectation that more senior staff, by virtue of their experience, should already know that information contained in such tools. Nurses and midwives in this study, particularly those with more experience, either did not refer to them, or used them flexibly. They tended to privilege their own experience, or the experience of others, instead of referring to available standardised care approaches. Nurses, if unsure, tended to refer to human sources of information (rather than available standardised care approaches), such as a credible and knowledgeable colleague.
Example six: What works, for whom, how and in what circumstances -- team functioning
Findings show that standardised care approaches had no obvious effect on team functioning. In fact, there is evidence to suggest that standardised care approaches formalised respective roles, rather than enhanced teamwork. For example, within the cardiac surgical unit, the integrated care pathway, whilst it had been designed to become a permanent part of the multi-disciplinary record of care, had been colour coded so that each professional's section was easily identifiable. This resulted in the different professionals rarely consulting sections that were not their own; a practise seen during observations. An alternative view is that this approach clarified the contribution that each team member made to the patient's journey through cardiac surgery (even if it did not appear to enhance team working), and indeed the development of health visiting guidelines within the GP surgery had been viewed as an opportunity to clarify roles and responsibilities around skills.
In other sites with the exception of the GP surgery and the use of QOF-related protocols, and some junior doctors, generally medics were not using available standardised care approaches even if they were applicable to them. The common perception amongst both doctors and nurses/midwives was that the use of standardised care approaches was a nursing and midwifery initiative.