‘Implementation involves all activities that occur between making an adoption commitment and the time that an innovation either becomes part of the organizational routine, ceases to be new, or is abandoned (…) [and the] behavior of organizational members over time evolves from avoidance or non-use, through unenthusiastic or compliant use, to skilled or consistent use. (p 65)’

There is a vast literature on implementation in service organisations [2], however efforts at implementing new technologies and practices remain problematic. The gap between research evidence and practice remains wide [3], and concerns about the large numbers of ‘pilot’ studies of new interventions that never lead to sustainable services are repeatedly expressed [4]. This is particularly the case for ‘e-health’ technologies - defined as practicing and delivering health care using information and communication technology [5] - despite significant promises for improving health care quality and efficiency [6].

In attempting to address such problems of implementation, the application of theory to designing health care interventions [7], planning and evaluating them [8][9,10], and developing effective strategies for their implementation [11] offers much potential.

However, obstacles to the use of theory for such purposes are numerous, and include the identification of relevant and useful theoretical perspectives from the huge body of literature on implementation that spans diverse academic disciplines (for example, psychology, sociology, business, healthcare management). Such theoretical diversity includes approaches that emphasise attitudes and behaviours [8,12,13]; diffusion and adoption of innovations through social networks [14]; and Science and Technology Studies (STS) approaches [15,16] that emphasise technology design and its relations with human actors. Reviews such as those of Greenhalgh and colleagues [2] (of literature relevant to the diffusion of innovations in service organisations) and Grol and colleagues [8] (of theories useful for planning and studying initiatives for improving patient care) begin to address this difficulty by mapping the terrain of implementation theories that may be useful for guiding both intervention development and approaches to implementation, and summarising their key processes and emphases.

Advances in theory-based intervention development and implementation have been made particularly with regard to changing healthcare professionals’ behaviour and practice to facilitate the uptake of evidence-based-practice strategies [7,17]. Drawing on psychological theories of behaviour, Michie and colleagues [17] explicitly set out to develop theory-based explanations of factors that affect professional practice in a format that would be accessible to non-academic users, and associated work has included guidance for designing questionnaires based on the Theory of Planned Behavior [18]. Models focused on psychological theory however, tend to over-emphasise the personal agency of individuals and underplay the importance of context. For example, implementation failures are often attributed to slow behaviour change by professionals, when there are likely to be other good and predictable socio-organisational reasons for such failure [19]. Nonetheless, such approaches show promise in facilitating the uptake of new interventions and/or ways of working, particularly where the roles and actions of individuals in making an implementation ‘effective’ are an appropriate focus for implementation efforts.

We would argue, however, that in practice, many interventions being implemented in healthcare settings are subject to more complex influences than those known to directly affect the behaviour of individuals. New practices get taken up and become ‘workable’ due to a complex interplay between features of the intervention/practice itself, the actions of individuals involved in the process, and aspects of the physical and social environment in which implementation activities are undertaken. Normalization Process Theory (NPT) [20,21] approaches the problem of implementation with a view to understanding such dynamics. It emphasises the processes by which new technologies and practices become normalised, focusing on the work that this requires of people working both individually and collaboratively. What really matters here is the extent to which new technologies and practices can – and do – become embedded in both the contexts in which they are to be used, and in the everyday practices of the individuals whose work is affected by these innovations. NPT is concerned with the generative processes [22] that underpin three core problems: implementation (bringing a practice or practices into action); embedding, (when a practice or practices may be routinely incorporated into the everyday work of individuals and groups); and integration (when a practice or practices are reproduced and sustained in the social matrices of an organization or institution). In NPT it is postulated that practices become routinely embedded in social contexts as the result of people working, individually and collectively, to enact them, and that the production and reproduction of a practice requires continuous investment by individuals to carry action forward in time and space. There are four sets of processes that characterise different kinds of ‘normalisation work’, and which require particular kinds of investments from individuals and organisations [20,21]:

Coherence: the process of sense-making and understanding that individuals and organisations have to go through in order to promote or inhibit the routine embedding of a practice to its users. These processes are energized by investments of meaning made by participants.

Cognitive participation: the process that individuals and organisations have to go through in order to enrol individuals to engage with the new practice. These processes are energized by investments of commitment made by participants.

Collective action: the work that individuals and organisations have to do to enact the new practice. These processes are energized by investments of effort made by participants.

Reflexive monitoring: the informal and formal appraisal of a new practice once it is in use, in order to assess its advantages and disadvantages and which develops users’ comprehension of the effects of a practice. These processes are energized by investments in appraisal made by participants.

A considerable body of research now supports NPT as an adequate and useful theory for explaining processes of the normalization of practices associated with complex interventions. This evidence spans diverse settings in which new technologies and practices have been the focus of its application, such as telecare [23], e-health [24,25], clinical decision support systems [26], teledermatology [27], infertility management [28], maternity services [10]and the management and treatment of depression [29,30].

The development of structured tools for assessing implementation processes, which take account of this complex interplay between interventions, individual actions, and context, would represent an advance in applying theory to understand and address implementation problems in practice. Existing assessment tools that focus on organisational factors relevant to ‘readiness’ for interventions in healthcare [31-33] do not adequately reflect the complexity of normalisation processes as characterised by the NPT – for example, the dynamic and iterative relationships between the types of work involved in making sense of a new practice, enacting it (collectively) and appraising its outcome and value. They are therefore limited in the extent to which they offer practical ways of facilitating implementation processes in ways that lead to the embedding of new practices within contexts of use.

A further challenge for the development of theory-based measures that capture the complexity of implementation activities concerns the various ways in which outcomes of such activities may be defined. In contrast to psychological theories of implementation behaviour, which focus on explaining and/or quantifying individuals’ uptake of a new practice, NPT focuses on more subtle – and gradual – processes, such as ‘embedding’, ‘integrating’ and ‘normalisation’. NPT does not offer a ‘definition’ of the term ‘normalisation’, for it can be appropriately used to refer to a process or a ‘state’, depending on the context and the frame of reference – that is, for the most part ‘normalisation’ is considered to be an ongoing cycle of activity aimed at making a new practice ‘fit in’ with the work of individuals and their context of practice, but when a practice ceases to be ‘new’ or no longer requires additional effort, it may be framed as having become ‘normalised’. Further work needs to be done to develop ways of defining and measuring outcomes of efforts to implement new practices, that reflect the complexity and context-dependent nature of what it means to have ‘successfully’ or ‘effectively’ implemented a new practice.

Thus the development of structured assessment tools for understanding the complex processes involved in integrating complex interventions, including e-health [34], into practice remains a priority. Recently, theory-based tools for assisting implementers in planning and ‘thinking through’ particular interventions with reference to the social and organisational contexts in which they are to be implemented have been offered [35,36]. Although promising however, such tools do not provide measurements to be used during implementations to assess progress towards successful implementation (however defined by stakeholders). Such tools would offer the potential to identify (and quantify) problems with an implementation during the process, but so far work in this area remains limited.

The objective of this study then was to advance work on translating theory into structured assessment instruments for research and practical purposes in these contexts, by drawing on the findings of a study [24] that undertook the development and preliminary testing of a Technology Adoption Readiness Scale (TARS) for measuring normalisation processes in the context of e-health service interventions. This paper therefore aims to (1) describe the process and outcome of a project to develop a theory-based instrument for measuring processes involved in the implementation of e-health interventions; and (2) identify key issues and methodological challenges for further advancing work in this field. First however, a fuller explanation of the theoretical development of NPT is required.

Preliminary descriptive analysis was undertaken to make decisions about excluding or combining existing items, analysing each item in terms of (i) the mean rating of importance for that item, and (ii) any correlations between the item and other items in the set (correlations of r>0.5). The results of this decision analysis are presented in Table ​Table3.3. Items that were highly correlated with other items were either discarded or re-written into a single item, particularly where importance ratings were relatively low. This process reduced the 27 items to 21.

Participants in the Expert survey were invited to suggest (using free-text) any factors they felt to be particularly important and which they believed had not been covered in the item set. Analysis of these free-text comments made by (n=31) survey participants resulted in the eventual inclusion of five new items about contextual integration issues (Q.5-9 in Table ​Table5).5). Peer review (amongst the project team) resulted in further revisions, notably the addition of three items to reflect the NPT’s constructs of coherence, cognitive participation, resulting in a final set of 30 generic TARS items ready for adapting for use in specific contexts.

At Site 2, nine TARS items indicated significant differences in responses between participants perceiving different levels of routinisation (Table ​(Table9).9). These results suggested that compared with those who feel that e-health has already become ‘completely routine’, those for whom it hasn’t become routine were less likely to agree that sufficient organisational effort has gone into supporting the system; and less likely to show strong agreement (rather than being neutral or some agreement) that: e-health is a different way of working; that the organisational culture is supportive of change; that they understand their own accountability and liability; and that there are ongoing mechanisms for monitoring and appraising how e-health is used. The group for whom e-health was not yet a completely routine part of their practice were also more likely to disagree that there is good evidence of clinical effectiveness of the e-health system, and that there is a shared understanding of what the system is for and how it is to be used. Here, the strongest differences between groups were evident on items relating to liability, accountability and appropriateness of skills.

The practical output of this study was the development of the TARS instrument, which was intended to enable researchers and practitioners to quantify a range of processes proposed by the NPT to contribute to the successful normalisation of e-health, either as a ‘diagnostic’ tool or for evaluation purposes. Developing TARS required considerable ‘translation work’, both in terms of the methodological implications of the theory’s underlying theoretical assumptions (Table ​(Table2),2), and from theoretical constructs into specific questions. To develop a set of assessment items with good face validity, multiple sources of information were collected and utilised including theoretical specifications of NPT (and its underlying empirical basis), the perspectives of academic experts in e-health implementation, and primary qualitative data concerning professionals’ views of implementation and integration of e-health in the NHS [24]. Whilst the expert survey (Phase 1) endorsed the proposed items as reflecting important factors affecting the potential for e-health to become a routine part of working practices (and suggested further items about contextual integration), health professionals themselves indicated greater emphasis on practice-based issues concerning benefits, particularly to patients, and workload management. Representation of different kinds of ‘expertise’ thus ensures that research instruments being developed for use in practice contexts are ‘fit for purpose’. In this study, we were focused primarily on health professionals using e-health in their day to day work, but even within this focus there were important differences between the roles and experiences of staff in relation to the e-health systems we studied, that affected their capacity to answer all of the questions. Although the questions included in the instrument were developed drawing on multiple sources of stakeholder input in general, this finding does raise concerns about the level of face validity achieved for the specific groups within our samples. We suggest that in using an instrument such as TARS, continued work on ensuring face validity of questions at the level of the participants within the local setting of use is required. We must also acknowledge that the results presented in this study are limited by focusing primarily on nurses as a professional group. In other studies, for example, it will also be important to consider assessments from the perspectives of a more diverse range of medical and healthcare professionals, managers and/or implementers [25], or indeed patients [45] . This study thus highlights the collaborative nature of health care work, and the importance of ensuring that multiple-stakeholders’ perspectives [46] are incorporated into the development of tools to assess implementation processes in these contexts.

As one of the first studies to use the NPT in quantitative research, this study aimed to progress work on NPT towards statistical investigation of relationships between implementation processes and outcomes in terms of ‘normalisation’. Although only tests of associations (rather than causality) between normalisation processes and outcomes were possible in this study, ratings of normalisation processes differed between groups holding different perceptions of whether or not the e-Health systems in the respective study sites had become part of routine practice. The two study sites themselves differed – both in terms of the technology being implemented (mobile electronic devices to facilitate community nursing versus computerised decision support services) and the level of progress towards the technology being considered ‘normal’, so differences between them in terms of which items related to perceptions of normalisation would be expected. Although preliminary, however, these finding lend support to assessing the potential predictive value of the TARS in prospective, longitudinal studies. Furthering work on the predictive utility of TARS – and NPT more generally – will however require flexible approaches to identifying and specifying ‘outcome’ measures. The process undertaken in this study demonstrates that ‘normalisation’ is highly context-dependent, relating to the practice itself, the environment in which it is operating, and the different groups of individuals that relate to it. As such, NPT does not provide any particular definition of ‘normalisation’ for the purpose of measurement as an outcome variable for quantitative studies, and designers of studies based on NPT to assess outcomes will need to develop study-specific measures based on what outcomes are relevant, and which are likely to be multiple and include both subjective (self-report) and objective (eg. usage data) measures. For example, just some normalisation ‘outcomes’ that might be considered are: level of use; increasing use over time; amount of shift from one practice to another; disappearance of a previous practice; reported acceptability of a practice; or measures of quality of work stemming from use of the practice. The development of approaches to measuring such outcomes will require not only developing and testing quantitative measures, but also further qualitative investigation about how people make judgements about whether or not a new practice can be considered ‘normalised’, and how that may or may not have happened.

This project aimed to develop a simple structured research instrument that could be used in other contexts. However, the process of considering the many possible ways to frame questions about processes involving change, demonstrated that use of tools such as TARS in other research context will require highly flexible and adaptive approaches to ensure that questions are framed appropriately to reflect the stage of implementation/use of the new technology or practice being studied. Here, we chose to frame questions as likert type statements about the e-health technology of interest and elicit respondents’ agreement with those statements, but in other situations it might be preferable to frame questions in a multitude of ways, such as: eliciting expectations of a technology planned but not yet used, inviting direct comparisons between key aspects of one type of technology/practice against another (eg. ‘X is a better way of working than Y’), or even assessing the perceived impact of the technology/practice over time (eg. ‘The impact of X on [practice] has been....). Although not intended at the outset of this study, the set of TARS items framed as ‘factors’ in the format in which they were presented for eliciting ratings of perceived relative importance (ie without reference to any direction of effect, as presented in Table ​Table3)3) could be used for the development of research instruments that include questions framed according the specific objective of the study. This consideration may prove challenging for further validation of the TARS items as ‘an instrument’, but also offers a range of opportunities for practical use of the tool in assessing staff perceptions of issues that this study has shown to be important for the normalisation of e-health.

In relation to NPT, the study described in this paper has also contributed to theory development. It has successfully achieved the development of a set of quantitative questions that can be used to assess staff perceptions of processes relevant to the normalisation of e-health with reference to underlying aspects of the constructs within the Collective Action component of NPT, along with single items for assessing perceptions relating to the NPT constructs of coherence, cognitive participation, and reflexive monitoring. This development process, which included gathering and incorporating views from diverse sets of academic and professional stakeholders, challenged our thinking about the constructs, and the multiple interpretations that could be made about their meaning. In part, the processes described here contributed directly to the expansion of the theory from the NPM to the NPT as currently presented (see elsewhere [20,42] for detailed description). This process has continued beyond this study [36], and is likely to continue as the theory is used, tested and challenged for a variety of purposes.

Despite such limitations, the study offers preliminary support for the conceptual distinctions between and within the constructs of the NPT (particularly with respect to the Collective Action construct), and for the potential predictive potential of items in the instrument with respect to normalisation outcomes (as demonstrated by associations between NPT processes as represented by the TARS items and perceived normalisation of e-health in the contexts of study). Although the TARS instrument does not represent balanced coverage of NPT in its entirety, the key underlying assumptions of the theory as a whole – such as the focus on the collaborative nature of work required of a practice-based intervention – remain constant across the developmental shift from NPM to NPT and thus the methodological challenges and issues described in this paper are of enduring relevance. In relation to the TARS study, the emphasis on the ‘collective action’ component for framing data collection was appropriate, as we were undertaking assessments focused on the ‘enactment’ stage of e-health implementations. However, to further develop the TARS instrument – and to develop measures of NPT that more comprehensively cover the wider frame of implementation activity that spans stages of conceptualising (coherence), engagement of individuals (cognitive participation), and reflection/evaluation (reflexive monitoring) – more longitudinal research will be needed.

This study was focused primarily on instrument development rather than formal validation, however key limitations are worth noting. Despite considerable effort by the research team to maximise response rates, achieved rates were lower than expected. It is difficult to consider the implications of the response rates achieved, as the rates themselves are a ‘worst estimate’, as true response rates (ie in terms of percentages participating out of those who received and read the invitation) could not be calculated due to limited access to information. Reliance on key contacts at survey sites (who were helpful but already working under pressure) also limited the timing and frequency of reminders that could be achieved, and thus the need for greater researcher control over access to research participants must be emphasised. Also, selection of sites for data collection in this study was necessarily pragmatic, and access was negotiated well in advance of the instrument being developed and ready for data collection (as is often the case with applied research). Given that the study sites already had at least some level of adoption of e-Health technology, the scope for prospectively assessing the predictive value of the instrument items in terms of normalisation outcomes was not possible in this study, but should be the objective of further studies where assessment of perceptions can be undertaken prior to, during and after the implementation of a new practice-based initiative. In relation to health technology in general, the challenges of assessing new technologies in practice contexts are recognised [47] but worth emphasising here.