Search tips
Search criteria

Results 1-25 (1327377)

Clipboard (0)

Related Articles

1.  Understanding integrated care pathways in palliative care using realist evaluation: a mixed methods study protocol 
BMJ Open  2012;2(4):e001533.
Policy- and evidence-based guidelines have highlighted the need for improved palliative and end-of-life care. However, there is still evidence of individuals dying undignified deaths with little pain control, therefore inflicting unnecessary suffering. New commissioning powers have enabled a 2-year pilot of an innovative integrated care pathway (ICP) designed to improve arrangements for individuals with life-limiting illnesses requiring palliative care. A novel feature of the ICP is its focus on palliative care over the last 6 months of life, aiming to intervene early to prepare for and ensure a good death. What is not known is if this pathway works, how it works and who it works for.
Methods and analysis
A realist evaluation and a complex analytical framework will investigate and discover context, mechanism and outcome conjectures and configurations of the ICP and thus facilitate exploration of how it works and who it works for. A mixed methods approach will be used with small sample sizes to capture the breadth of the ICP. Phase 1 will identify if the pathway works through analysis of NHS Morbidity Information Query and Export Syntax data, locality Death Audit data and the Quality of Dying and Death Questionnaire. Phase 2 employs soft systems methodology with data from focus groups with health professionals to identify how the pathway works. Phase 3 uses the Miller Behavioural Style Scale and interviews with palliative care patients and bereaved relatives to analyse communication in palliative care.
Ethics and dissemination
Ethical approval has been granted from the NHS local ethics committee (REC reference number: 11/NE/0318). Research & Development approval has been gained from four different trusts, and relevant voluntary organisations and the local council have been informed about the research. This protocol illustrates the complexity inherent in evaluating a palliative care ICP. Identification of whether the pathway works, how it works and who it works for will be beneficial to all practices and other care providers involved as it will give objective data on the impact of the ICP. Results will be disseminated throughout the study for continuous quality improvement of the ICP. Outcomes from each data collection phase will be disseminated separately if analysis warrants it; all data collection will be utilised in the realist evaluation. The research provides a potential for the dissemination of the pathway to other localities through the transferable knowledge it will generate, from its focus on the contexts that are crucial for successful implementation, the mechanisms that facilitate implementation and the outcomes achieved.
Article summary
Article focus
This article is a protocol of a realist evaluation of a palliative care ICP, which was developed in Primary Care by health practitioners. The ICP itself uses elements of long-term chronic illness care in order to provide holistic, supportive, high-quality palliative care. The focus of the article is to detail how the ICP will be evaluated, using a variety of data collection tools, which will identify contexts and mechanisms that lead to improved outcomes, thus taking the main focus away from just the outcomes alone. The identification of contexts and mechanisms for improved outcomes is known as realist evaluation and will provide a better knowledge of the essential conditions of effectiveness when the ICP is implemented in other localities.
Key messages
The key aim of this article is to detail the creation of a complex realist evaluation, which utilises a unique and varied methodological framework. It is hoped that through this article, others will understand the groundwork needed to set up and execute a realist evaluation.
Strengths and limitations of this study
The protocol details a complex evaluation of a unique palliative care ICP using a new and innovative methodology: realist evaluation.
Some may perceive the small sample sizes in the qualitative sections of the study as a weakness. However, the aim of the study is not to find a robust causal mechanism; this would be premature with an ICP in its infancy. The aim is to unpack the contexts and mechanisms that work in certain circumstances, from this conditions crucial for effectiveness can be highlighted, which are essential for implementation of the ICP in other localities.
The ICP involves 15 general practitioner practices, which collectively care for 80 300 patients. The study described will use Morbidity Information Query and Export Syntax and Death Audit data from all 15 practices and will conduct the other sections of research within selected practices, both rural and suburban.
Finally, palliative care is commonly misunderstood in the literature and in the field. This paper addresses this confusion and fills a gap in the literature.
PMCID: PMC3391371  PMID: 22761292
2.  Conditions for production of interdisciplinary teamwork outcomes in oncology teams: protocol for a realist evaluation 
Interdisciplinary teamwork (ITW) is designed to promote the active participation of several disciplines in delivering comprehensive cancer care to patients. ITW provides mechanisms to support continuous communication among care providers, optimize professionals’ participation in clinical decision-making within and across disciplines, and foster care coordination along the cancer trajectory. However, ITW mechanisms are not activated optimally by all teams, resulting in a gap between desired outcomes of ITW and actual outcomes observed. The aim of the present study is to identify the conditions underlying outcome production by ITW in local oncology teams.
This retrospective multiple case study will draw upon realist evaluation principles to explore associations among context, mechanisms and outcomes (CMO). The cases are nine interdisciplinary cancer teams that participated in a previous study evaluating ITW outcomes. Qualitative data sources will be used to construct a picture of CMO associations in each case. For data collection, reflexive focus groups will be held to capture patients’ and professionals’ perspectives on ITW, using the guiding question, ‘What works, for whom, and under what circumstances?’ Intra-case analysis will be used to trace associations between context, ITW mechanisms, and patient outcomes. Inter-case analysis will be used to compare the different cases’ CMO associations for a better understanding of the phenomenon under study.
This multiple case study will use realist evaluation principles to draw lessons about how certain contexts are more or less likely to produce particular outcomes. The results will make it possible to target more specifically the actions required to optimize structures and to activate the best mechanisms to meet the needs of cancer patients. This project could also contribute significantly to the development of improved research methods for conducting realist evaluations of complex healthcare interventions. To our knowledge, this study is the first to use CMO associations to improved empirical and theoretical understanding of interdisciplinary teamwork in oncology, and its results could foster more effective implementation in clinical practice.
PMCID: PMC4074333  PMID: 24938443
Interdisciplinarity; Professional practices; Realist evaluation; Patient outcomes; Case study; Cancer
3.  How does capacity building of health managers work? A realist evaluation study protocol 
BMJ Open  2012;2(2):e000882.
There has been a lot of attention on the role of human resource management interventions to improve delivery of health services in low- and middle-income countries. However, studies on this subject are few due to limited research on implementation of programmes and methodological difficulties in conducting experimental studies on human resource interventions. The authors present the protocol of an evaluation of a district-level capacity-building intervention to identify the determinants of performance of health workers in managerial positions and to understand how changes (if any) are brought about.
Methods and analysis
The aim of this study is to understand how capacity building works. The authors will use realist evaluation to evaluate an intervention in Karnataka, India. The intervention is a capacity-building programme that seeks to improve management capacities of health managers at district and subdistrict levels through periodic classroom-based teaching and mentoring support at the workplace. The authors conducted interviews and reviewed literature on capacity building in health to draw out the programme theory of the intervention. Based on this, the authors formulated hypothetical pathways connecting the expected outcomes of the intervention (planning and supervision) to the inputs (contact classes and mentoring). The authors prepared a questionnaire to assess elements of the programme theory—organisational culture, self-efficacy and supervision. The authors shall conduct a survey among health managers as well as collect qualitative data through interviews with participants and non-participants selected purposively based on their planning and supervision performance. The authors will construct explanations in the form of context–mechanism–outcome configurations from the results. This will be iterative and the authors will use a realist evaluation framework to refine the explanatory theories that are based on the findings to explain and validate an improved theory on ‘what works for whom and under what conditions’.
The scope for applying realist evaluation to study human resource management interventions in health are discussed.
Article summary
Article focus
Despite a lot of focus on capacity building of health workforce to improve health outcomes in developing countries, there are very few studies on how capacity building brings about better performance.
Methodological difficulties and complexity of health systems impose restrictions on evaluating human resource management interventions.
Key messages
Health worker practices are complex behaviours that are determined by various individual, institutional and systemic factors.
It is possible to use a theory-driven evaluation approach such as realist evaluation to understand the mechanisms through which capacity-building programmes improve (or do not improve) performance of health workers.
Strengths and limitations of this study
This is the first application of realist evaluation to a local health system intervention (capacity building).
Instead of ‘does it work or not?’, the study seeks to understand ‘what works for whom and under what conditions’.
PMCID: PMC3330260  PMID: 22466036
4.  Exploring why quality circles work in primary health care: a realist review protocol 
Systematic Reviews  2013;2:110.
Quality circles (QCs) are commonly used in primary health care in Europe to consider and improve standard practice over time. They represent a complex social intervention that occurs within the fast-changing system of primary health care. Numerous controlled trials, reviews, and studies have shown small but unpredictable positive effect sizes on behavior change. Although QCs seem to be effective, stakeholders have difficulty understanding how the results are achieved and in generalizing the results with confidence. They also lack understanding of the active components of QCs which result in changes in the behavior of health care professionals. This protocol for a realist synthesis will examine how configurations of components and the contextual features of QCs influence their performance.
Stakeholder interviews and a scoping search revealed the processes of QCs and helped to describe their core components and underlying theories. After clarifying their historical and geographical distribution, a purposive and systematic search was developed to identify relevant papers to answer the research questions, which are: understanding why, how, and when QCs work, over what time frame, and in what circumstances. After selecting and abstracting appropriate data, configurations of contexts and mechanisms which influence the outcome of QCs within each study will be identified. Studies will be grouped by similar propositional statements in order to identify patterns and validation from stakeholders sought. Finally, theories will be explored in order to explain these patterns and to help stakeholders maintain and improve QC performance.
Analyzing context-mechanism-outcome (CMO) patterns will reveal how QCs work and how contextual factors interact to influence their outcome. The aim is to investigate unique configurations that enable them to improve the performance of health care professionals. Using a standardized reporting system, this realist review will allow the research questions to be answered to the satisfaction of key stakeholders and enable on-going critical examination and dissemination of the findings.
Study registration
PROSPERO registration number: CRD42013004826.
PMCID: PMC4029275  PMID: 24321626
Realist synthesis; Realist review; Quality circle; Peer review group; Small group work; Quality improvement
5.  Embedding mental health interventions in early childhood education systems for at-risk preschoolers: an evidence to policy realist review 
Systematic Reviews  2014;3:84.
Current early childhood systems of care are not geared to respond to the complex needs of preschoolers at risk for mental health problems in a timely, coordinated, multidisciplinary, and comprehensive fashion. Evidence-informed policy represents an opportunity for implementing prevention, promotion, and early intervention at the population or at-risk level. Exposure to risk factors as well as the presence of clinical disorders can derail the developmental trajectories of preschoolers, and problems may persist if left untreated. One way to address these multiple research-to-policy gaps are systematic reviews sensitive to context and knowledge user needs, such as the realist review. The realist review is an iterative process between research teams and knowledge users to build mid-level program theories in order to understand which interventions work best for whom and under what context.
The realist review employs five ‘iterative’ steps: (1) clarify scope, (2) search for evidence, (3) appraise primary studies and extract data, (4) synthesize the evidence, and (5) disseminate, implement, and evaluate evidence, to answer two research questions: What interventions improve mental health outcomes for preschoolers at risk for socio-emotional difficulties and under what circumstances do they work? and what are the best models of care for integrating mental health interventions within pre-existing early childhood education (ECE) services for at-risk children? Knowledge users and researchers will work together through each stage of the review starting with refining the questions through to decisions regarding program theory building, data extraction, analysis, and design of a policy dissemination plan. The initial questions will guide preliminary literature reviews, but subsequent more focused searches will be informed by knowledge users familiar with local needs and further building of explanatory program theories.
Policy makers want to know what works best for whom, but are faced with a wide and disparate intervention literature for at-risk children. Applying evidence-based standards is a good start, but the chain of implementation between research results and how to match interventions sensitive to local context are ongoing challenges.
Trial registration
Prospero registration number: CRD42014007301.
PMCID: PMC4127568  PMID: 25073533
Early childhood; Preschoolers; Mental health interventions; At-risk children; Realist review
6.  Interventions targeted at primary care practitioners to improve the identification and referral of patients with co-morbid obesity: a realist review protocol 
Systematic Reviews  2015;4:61.
Obesity is one of the most significant public health challenges in the developed world. Recent policy has suggested that more can be done in primary care to support adults with obesity. In particular, general practitioners (GPs) and practice nurses (PNs) could improve the identification and referral of adults with obesity to appropriate weight management services. Previous interventions targeted at primary care practitioners in this area have had mixed results, suggesting a more complex interplay between patients, practitioners, and systems. The objectives of this review are (i) to identify the underlying ‘programme theory’ of interventions targeted at primary care practitioners to improve the identification and referral of adults with obesity and (ii) to explore how and why GPs and PNs identify and refer individuals with obesity, particularly in the context of weight-related co-morbidity. This protocol will explain the rationale for using a realist review approach and outline the key steps in this process.
Realist review is a theory-led approach to knowledge synthesis that provides an explanatory analysis aimed at discerning what works, for whom, in what circumstances, how, and why. In this review, scoping interviews with key stakeholders involved in the planning and delivery of adult weight management services in Scotland helped to inform the identification of formal theories - from psychology, sociology, and implementation science - that will be tested as the review progresses. A comprehensive search strategy is described, including scope for iterative searching. Data analysis is outlined in three stages (describing context-mechanism-outcome configurations, exploring patterns in these configurations, and developing and testing middle-range theories, informed by the formal theories previously identified), culminating in the production of explanatory programme theory that considers individual, interpersonal, and institutional/systems-level components.
This is the first realist review that we are aware of looking at interventions targeted at primary care practitioners to improve the weight management of adults with obesity. Engagement with stakeholders at an early stage is a unique feature of realist review. This shapes the scope of the review, identification of candidate theories and dissemination strategies. The findings of this review will inform policy and future interventions.
Systematic review registration
PROSPERO CRD42014009391
Electronic supplementary material
The online version of this article (doi:10.1186/s13643-015-0046-y) contains supplementary material, which is available to authorized users.
PMCID: PMC4426175  PMID: 25927993
Realist review; Realist synthesis; Obesity; Primary care; General practitioners; Practice nurses; Referral; Weight management; Effectiveness
7.  Understanding how appraisal of doctors produces its effects: a realist review protocol 
BMJ Open  2014;4(6):e005466.
UK doctors are now required to participate in revalidation to maintain their licence to practise. Appraisal is a fundamental component of revalidation. However, objective evidence of appraisal changing doctors’ behaviour and directly resulting in improved patient care is limited. In particular, it is not clear how the process of appraisal is supposed to change doctors’ behaviour and improve clinical performance. The aim of this research is to understand how and why appraisal of doctors is supposed to produce its effect.
Methods and analysis
Realist review is a theory-driven interpretive approach to evidence synthesis. It applies realist logic of inquiry to produce an explanatory analysis of an intervention that is, what works, for whom, in what circumstances, in what respects. Using a realist review approach, an initial programme theory of appraisal will be developed by consulting with key stakeholders in doctors’ appraisal in expert panels (ethical approval is not required), and by searching the literature to identify relevant existing theories. The search strategy will have a number of phases including a combination of: (1) electronic database searching, for example, EMBASE, MEDLINE, the Cochrane Library, ASSIA, (2) ‘cited by’ articles search, (3) citation searching, (4) contacting authors and (5) grey literature searching. The search for evidence will be iteratively extended and refocused as the review progresses. Studies will be included based on their ability to provide data that enable testing of the programme theory. Data extraction will be conducted, for example, by note taking and annotation at different review stages as is consistent with the realist approach. The evidence will be synthesised using realist logic to interrogate the final programme theory of the impact of appraisal on doctors’ performance. The synthesis results will be written up according to RAMESES guidelines and disseminated through peer-reviewed publication and presentations.
Trial registration number
The protocol is registered with PROSPERO 2014:CRD42014007092.
PMCID: PMC4067866  PMID: 24958211
8.  The effectiveness and cost-effectiveness of shared care: protocol for a realist review 
Systematic Reviews  2013;2:12.
Shared care (an enhanced information exchange over and above routine outpatient letters) is commonly used to improve care coordination and communication between a specialist and primary care services for people with long-term conditions. Evidence of the effectiveness and cost-effectiveness of shared care is mixed. Informed decision-making for targeting shared care requires a greater understanding of how it works, for whom it works, in what contexts and why. This protocol outlines how realist review methods can be used to synthesise evidence on shared care for long-term conditions.
A further aim of the review is to explore economic evaluations of shared care. Economic evaluations are difficult to synthesise due to problems in accounting for contextual differences that impact on resource use and opportunity costs. Realist review methods have been suggested as a way to overcome some of these issues, so this review will also assess whether realist review methods are amenable to synthesising economic evidence.
Database and web searching will be carried out in order to find relevant evidence to develop and test programme theories about how shared care works. The review will have two phases. Phase 1 will concentrate on the contextual conditions and mechanisms that influence how shared care works, in order to develop programme theories, which partially explain how it works. Phase 2 will focus on testing these programme theories. A Project Reference Group made up of health service professionals and people with actual experience of long-term conditions will be used to ground the study in real-life experience. Review findings will be disseminated through local and sub-national networks for integrated care and long-term conditions.
This realist review will explore why and for whom shared care works, in order to support decision-makers working to improve the effectiveness of care for people outside hospital. The development of realist review methods to take into account cost and cost-effectiveness evidence is particularly innovative and challenging, and if successful will offer a new approach to synthesising economic evidence. This systematic review protocol is registered on the PROSPERO database (registration number: CRD42012002842).
PMCID: PMC3599640  PMID: 23402391
Shared care; Long-term conditions; Cost-effectiveness; Economic evaluation; Realist review; Integrated care; Primary care
9.  Implementing health research through academic and clinical partnerships: a realistic evaluation of the Collaborations for Leadership in Applied Health Research and Care (CLAHRC) 
The English National Health Service has made a major investment in nine partnerships between higher education institutions and local health services called Collaborations for Leadership in Applied Health Research and Care (CLAHRC). They have been funded to increase capacity and capability to produce and implement research through sustained interactions between academics and health services. CLAHRCs provide a natural 'test bed' for exploring questions about research implementation within a partnership model of delivery. This protocol describes an externally funded evaluation that focuses on implementation mechanisms and processes within three CLAHRCs. It seeks to uncover what works, for whom, how, and in what circumstances.
Design and methods
This study is a longitudinal three-phase, multi-method realistic evaluation, which deliberately aims to explore the boundaries around knowledge use in context. The evaluation funder wishes to see it conducted for the process of learning, not for judging performance. The study is underpinned by a conceptual framework that combines the Promoting Action on Research Implementation in Health Services and Knowledge to Action frameworks to reflect the complexities of implementation. Three participating CLARHCS will provide in-depth comparative case studies of research implementation using multiple data collection methods including interviews, observation, documents, and publicly available data to test and refine hypotheses over four rounds of data collection. We will test the wider applicability of emerging findings with a wider community using an interpretative forum.
The idea that collaboration between academics and services might lead to more applicable health research that is actually used in practice is theoretically and intuitively appealing; however the evidence for it is limited. Our evaluation is designed to capture the processes and impacts of collaborative approaches for implementing research, and therefore should contribute to the evidence base about an increasingly popular (e.g., Mode two, integrated knowledge transfer, interactive research), but poorly understood approach to knowledge translation. Additionally we hope to develop approaches for evaluating implementation processes and impacts particularly with respect to integrated stakeholder involvement.
PMCID: PMC3168414  PMID: 21771329
10.  Effective health care for older people resident in care homes: the optimal study protocol for realist review 
Systematic Reviews  2014;3:49.
Care homes in the UK rely on general practice for access to specialist medical and nursing care as well as referral to therapists and secondary care. Service delivery to care homes is highly variable in both quantity and quality. This variability is also evident in the commissioning and organisation of care home-specific services that range from the payment of incentives to general practitioners (GPs) to visit care homes, to the creation of care home specialist teams and outreach services run by geriatricians. No primary studies or systematic reviews have robustly evaluated the impact of these different approaches on organisation and resident-level outcomes. Our aim is to identify factors which may explain the perceived or demonstrated effectiveness of programmes to improve health-related outcomes in older people living in care homes.
A realist review approach will be used to develop a theoretical understanding of what works when, why and in what circumstances. Elements of service models of interest include those that focus on assessment and management of residents’ health, those that use strategies to encourage closer working between visiting health care providers and care home staff, and those that address system-wide issues about access to assessment and treatment. These will include studies on continence, dignity, and speech and language assessment as well as interventions to promote person centred dementia care, improve strength and mobility, and nutrition. The impact of these interventions and their different mechanisms will be considered in relation to five key outcomes: residents’ medication use, use of out of hours’ services, hospital admissions (including use of Accident and Emergency) and length of hospital stay, costs and user satisfaction. An iterative three-stage approach will be undertaken that is stakeholder-driven and optimises the knowledge and networks of the research team.
This realist review will explore why and for whom different approaches to providing health care to residents in care homes improves access to health care in the five areas of interest. It will inform commissioning decisions and be the basis for further research. This systematic review protocol is registered on the PROSPERO database reference number: CRD42014009112.
PMCID: PMC4037277  PMID: 24887325
Residential facilities; Primary care; Older people; Health services; Realist review
11.  Using realist evaluation to open the black box of knowledge translation: a state-of-the-art review 
In knowledge translation, complex interventions may be implemented in the attempt to improve uptake of research-based knowledge in practice. Traditional evaluation efforts that focus on aggregate effectiveness represent an oversimplification of both the environment and the interventions themselves. However, theory-based approaches to evaluation, such as realist evaluation (RE), may be better-suited to examination of complex knowledge translation interventions with a view to understanding what works, for whom, and under what conditions. It is the aim of the present state-of-the-art review to examine current literature with regard to the use of RE in the assessment of knowledge translation interventions implemented within healthcare environments.
Multiple online databases were searched from 1997 through June 2013. Primary studies examining the application or implementation of knowledge translation interventions within healthcare settings and using RE were selected for inclusion. Varying applications of RE across studies were examined in terms of a) reporting of core elements of RE, and b) potential feasibility of this evaluation method.
A total of 14 studies (6 study protocols), published between 2007 and 2013, were identified for inclusion. Projects were initiated in a variety of healthcare settings and represented a range of interventions. While a majority of authors mentioned context (C), mechanism (M) and outcome (O), a minority reported the development of C-M-O configurations or testable hypotheses based on these configurations. Four completed studies reported results that included refinement of proposed C-M-O configurations and offered explanations within the RE framework. In the few studies offering insight regarding challenges associated with the use of RE, difficulties were expressed regarding the definition of both mechanisms and contextual factors. Overall, RE was perceived as time-consuming and resource intensive.
The use of RE in knowledge translation is relatively new; however, theory-building approaches to the examination of complex interventions in this area may be increasing as researchers attempt to identify what works, for whom and under what circumstances. Completion of the RE cycle may be challenging, particularly in the development of C-M-O configurations; however, as researchers approach challenges and explore innovations in its application, rich and detailed accounts may improve feasibility.
PMCID: PMC4172789  PMID: 25190100
Realist evaluation; Knowledge translation; State-of-the-art review
12.  Protocol: a realist review of user fee exemption policies for health services in Africa 
BMJ Open  2012;2(1):e000706.
Four years prior to the Millenium Development Goals (MDGs) deadline, low- and middle-income countries and international stakeholders are looking for evidence-based policies to improve access to healthcare for the most vulnerable populations. User fee exemption policies are one of the potential solutions. However, the evidence is disparate, and systematic reviews have failed to provide valuable lessons. The authors propose to produce an innovative synthesis of the available evidence on user fee exemption policies in Africa to feed the policy-making process.
The authors will carry out a realist review to answer the following research question: what are the outcomes of user fee exemption policies implemented in Africa? why do they produce such outcomes? and what contextual elements come into play? This type of review aims to understand how contextual elements influence the production of outcomes through the activation of specific mechanisms, in the form of context–mechanism–outcome configurations. The review will be conducted in five steps: (1) identifying with key stakeholders the mechanisms underlying user fee exemption policies to develop the analytical framework, (2) searching for and selecting primary data, (3) assessing the quality of evidence using the Mixed-Method Appraisal Tool, (4) extracting the data using the analytical framework and (5) synthesising the data in the form of context–mechanism–outcomes configurations. The output will be a middle-range theory specifying how user fee exemption policies work, for what populations and under what circumstances.
Ethics and dissemination
The two main target audiences are researchers who are looking for examples to implement a realist review, and policy-makers and international stakeholders looking for lessons learnt on user fee exemption. For the latter, a knowledge-sharing strategy involving local scientific and policy networks will be implemented. The study has been approved by the ethics committee of the CHUM Research Centre (CR-CHUM). It received funding from the Canadian Institutes of Health Research. The funders will not have any role in study design; collection, management, analysis, and interpretation of data; writing of the report and the decision to submit the report for publication, including who will have ultimate authority over each of these activities.
Article summary
Article focus
Demonstrate how a realist review can provide new insights on user fee exemption policies in Africa.
Highlight the foundations of the realist approach and explicit its key principles.
Key messages
Traditional systematic reviews fail to provide valuable lessons when it comes to understand how the context and the mechanisms of a policy interact to produce outcomes.
The realist approach is a promising way for synthesising scientific and grey literature to understand the black boxes of complex social interventions, such as health policies.
Strengths and limitations of this study
Innovative way to systematically review literature on user fee exemption policies.
Inclusion of grey literature in the review.
Geographic area limited to Africa.
PMCID: PMC3269046  PMID: 22275427
13.  Employment of ex-prisoners with mental health problems, a realistic evaluation protocol 
BMC Psychiatry  2015;15:185.
Offenders with a mental illness are routinely excluded from vocational services due to their mental health. Employment has shown to be very important in improving mental health, reducing recidivism, and connecting people to society. This study examines the effectiveness of an established intervention which is relatively untested in this population, Individual Placement and Support (IPS), to help offenders with mental health problems into competitive employment. The overall research question is whether IPS is effective in gaining and sustaining competitive employment for offenders with a Severe Mental Illness (SMI). The context is an English criminal justice setting across different populations. The study will also measure non-vocational outcomes such as recidivism, mental health and social stability.
A Realistic Evaluation (RE) design will address the questions “What works, for whom, and in what circumstances?” This study includes pre and post comparisons for a cohort of approximately 20 people taking part in IPS, and a similar number of controls, over a one year period. The RE also consists of interviews with practitioners and offenders in order to understand how IPS works and develops within the criminal justice system (CJS). By applying this framework the research can go from discovering whether IPS works, to how and why (or why not) IPS works. This is achieved by examining where the intervention is occurring (Context (C)), the mechanisms (M) that create particular behaviours, and how the outcomes (O) from the intervention all come together (CMOs). Employment outcomes will also be examined for all participants.
By applying RE the research will permit inferences to be drawn about how and why (or why not) IPS works, by examining context, mechanisms and outcomes. IPS has never been implemented within the CJS in the United Kingdom. As a result, this evaluative research will not only provide a novel insight into the core research areas, but also how the intervention can be improved for others in the future.
PMCID: PMC4522126  PMID: 26231657
Individual placement and support; IPS; Mental health; Mentally disordered offenders; Employment; Realistic evaluation; Recidivism; Social network; Social stability
14.  What’s in a mechanism? Development of a key concept in realist evaluation 
The idea that underlying, generative mechanisms give rise to causal regularities has become a guiding principle across many social and natural science disciplines. A specific form of this enquiry, realist evaluation is gaining momentum in the evaluation of complex social interventions. It focuses on ‘what works, how, in which conditions and for whom’ using context, mechanism and outcome configurations as opposed to asking whether an intervention ‘works’. Realist evaluation can be difficult to codify and requires considerable researcher reflection and creativity. As such there is often confusion when operationalising the method in practice. This article aims to clarify and further develop the concept of mechanism in realist evaluation and in doing so aid the learning of those operationalising the methodology.
Using a social science illustration, we argue that disaggregating the concept of mechanism into its constituent parts helps to understand the difference between the resources offered by the intervention and the ways in which this changes the reasoning of participants. This in turn helps to distinguish between a context and mechanism. The notion of mechanisms ‘firing’ in social science research is explored, with discussions surrounding how this may stifle researchers’ realist thinking. We underline the importance of conceptualising mechanisms as operating on a continuum, rather than as an ‘on/off’ switch.
The discussions in this article will hopefully progress and operationalise realist methods. This development is likely to occur due to the infancy of the methodology and its recent increased profile and use in social science research. The arguments we present have been tested and are explained throughout the article using a social science illustration, evidencing their usability and value.
PMCID: PMC4408605  PMID: 25885787
Realist; Methodology; Palliative care; Realist evaluation; Realist synthesis
15.  Understanding success and failure in multimorbidity: protocol for using realist synthesis to identify how social learning and workplace practices can be optimised 
Systematic Reviews  2013;2:87.
Multimorbidity is increasingly prevalent but, aside from epidemiological work, the impact on associated provision of healthcare and/or education is little understood. For example, it is unclear how or why healthcare interventions meet (or do not meet) people’s multiple needs. Professionals working in primary care training sites must reconcile two goals: provision of appropriate individualised healthcare and provision of constructive workplace-based learning for future professionals. Given that professionals, learners and patients may have differing priorities and conceptualisations of success and failure in the absence of cure, achievement of both goals depends on social and cultural mechanisms. This review aims to make sense of how healthcare delivery for, and education about, multimorbidity can be concurrently delivered in primary care through identification of relevant theoretical frameworks.
Realist synthesis identifies and makes sense of variable outcomes caused by interaction between mechanisms and contexts. This review will produce a synthesis of social science, education and primary care literature. Our objective is to understand interactivity between models of workplace-based education and models of patient-centred/integrated care with a focus on perceptions of 'success’ and 'failure’ in multimorbidity. We intend to build a conceptual map and a realist programme theory, populated with evidence from the literature, as the first step towards answering our review question: what is known about how and why concurrent health service delivery and professional medical education interact together to generate outcomes valued by professionals, learners and patients for patients with multimorbidity in primary care? To answer this we are focusing on relationship-based negotiation of needs-based learning and needs-based care as our primary outcome of interest. In this protocol we outline our search strategy and proposed methods of analysis and synthesis of credible and trustworthy data judged to be relevant to our research question.
Findings will be submitted for peer-reviewed publication. Identification of how mechanisms of social learning and workplace practices could be optimised to improve quality and utility of patient care in multimorbidity is important. This can inform the future research regarding interventions that will produce a sustainable medical workforce equipped to provide healthcare, when the possibility of cure is absent.
PMCID: PMC3849769  PMID: 24066719
Realist synthesis; Medical education; Health service delivery; Primary care; Socio-cultural theories
16.  How outcomes are achieved through patient portals: a realist review 
To examine how patient portals contribute to health service delivery and patient outcomes. The specific aims were to examine how outcomes are produced, and how variations in outcomes can be explained.
We used a realist review method, which aims to describe how ‘an intervention works, for whom, and in what circumstances’ by analyzing patterns between context, mechanism, and outcomes. We reviewed 32 evaluation studies of patient portals published since 2003.
The reviewed evaluations indicate that as a complement to existing health services, patient portals can lead to improvements in clinical outcomes, patient behavior, and experiences. Four different mechanisms are reported to yield the reported outcome improvements. These are patient insight into personal health information, activation of information, interpersonal continuity of care, and service convenience. The vast majority of evaluations were conducted in integrated health service networks in the USA, and we detected no substantial variation in outcomes across these networks.
Discussion and conclusions
Patient portals may impact clinical outcomes and health service delivery through multiple mechanisms. Given the relative uniformity of evaluation contexts, we were not able to detect patterns in how patient portals work in different contexts. Nonetheless, it appears from the overwhelming proportion of patient portal evaluations coming from integrated health service networks, that these networks provide more fertile contexts for patient portals to be effective. To improve the understanding of how patient portals work, future evaluations of patient portals should capture information about mechanisms and context that influence their outcomes.
PMCID: PMC4078283  PMID: 24503882
17.  Realist synthesis: illustrating the method for implementation research 
Realist synthesis is an increasingly popular approach to the review and synthesis of evidence, which focuses on understanding the mechanisms by which an intervention works (or not). There are few published examples of realist synthesis. This paper therefore fills a gap by describing, in detail, the process used for a realist review and synthesis to answer the question ‘what interventions and strategies are effective in enabling evidence-informed healthcare?’ The strengths and challenges of conducting realist review are also considered.
The realist approach involves identifying underlying causal mechanisms and exploring how they work under what conditions. The stages of this review included: defining the scope of the review (concept mining and framework formulation); searching for and scrutinising the evidence; extracting and synthesising the evidence; and developing the narrative, including hypotheses.
Based on key terms and concepts related to various interventions to promote evidence-informed healthcare, we developed an outcome-focused theoretical framework. Questions were tailored for each of four theory/intervention areas within the theoretical framework and were used to guide development of a review and data extraction process. The search for literature within our first theory area, change agency, was executed and the screening procedure resulted in inclusion of 52 papers. Using the questions relevant to this theory area, data were extracted by one reviewer and validated by a second reviewer. Synthesis involved organisation of extracted data into evidence tables, theming and formulation of chains of inference, linking between the chains of inference, and hypothesis formulation. The narrative was developed around the hypotheses generated within the change agency theory area.
Realist synthesis lends itself to the review of complex interventions because it accounts for context as well as outcomes in the process of systematically and transparently synthesising relevant literature. While realist synthesis demands flexible thinking and the ability to deal with complexity, the rewards include the potential for more pragmatic conclusions than alternative approaches to systematic reviewing. A separate publication will report the findings of the review.
PMCID: PMC3514310  PMID: 22515663
18.  Early years interventions to improve child health and wellbeing: what works, for whom and in what circumstances? Protocol for a realist review 
Systematic Reviews  2015;4:79.
Child health and wellbeing is influenced by multiple factors, all of which can impact on early childhood development. Adverse early life experiences can have lasting effects across the life course, sustaining inequalities and resulting in negative consequences for the health and wellbeing of individuals and society. The potential to influence future outcomes via early intervention is widely accepted; there are numerous policy initiatives, programmes and interventions clustered around the early years theme, resulting in a broad and disparate evidence base. Existing reviews have addressed the effectiveness of early years interventions, yet there is a knowledge gap regarding the mechanisms underlying why interventions work in given contexts.
This realist review seeks to address the question ‘what works, for whom and in what circumstances?’ in terms of early years interventions to improve child health and wellbeing. The review will be conducted following Pawson’s five-stage iterative realist methodology: (1) clarify scope, (2) search for evidence, (3) appraise primary studies and extract data, (4) synthesise evidence and draw conclusions and (5) disseminate findings. The reviewers will work with stakeholders in the early stages to refine the focus of the review, create a review framework and build programme theory. Searches for primary evidence will be conducted iteratively. Data will be extracted and tested against the programme theory. A review collaboration group will oversee the review process.
The review will demonstrate how early years interventions do or do not work in different contexts and with what outcomes and effects. Review findings will be written up following the RAMESES guidelines and will be disseminated via a report, presentations and peer-reviewed publications.
Systematic review registration
PROSPERO CRD42015017832
PMCID: PMC4464136  PMID: 26047950
Realist review; Realist synthesis; Early years; Child health; Maternal health; Child development; Wellbeing; Health inequalities; Early intervention
19.  Research protocol: a realist synthesis of contestability in community-based mental health markets 
Systematic Reviews  2015;4:32.
In most developed nations, there has been a shift from public services to a marketisation of public goods and services - representing a significant reform process aiming to transform the way in which community-based human services, such as health, are delivered and consumed. For services, this means developing the capacity to adapt and innovate in response to changing circumstances to achieve quality. The availability of rigorous research to demonstrate whether a market approach and contestability, in particular, is a coherent reform process is largely absent. Contestability operates on the premise that better procurement processes allow more providers to enter the market and compete for contracts. This is expected to create stimulus for greater efficiencies, innovation and improved service delivery to consumers. There is limited understanding, however, about how community-based providers morph and re-configure in response to the opportunities posed by contestability. This study focuses on the effect of a contestability policy on the community-managed mental health sector.
A realist review will be undertaken to understand how and why the introduction of contestability into a previously incontestable market influences the ways in which community-based mental health providers respond to contestability. The review will investigate those circumstances that shape organisational response and generate outcomes through activating mechanisms. An early scoping has helped to formulate the initial program theory. A realist synthesis will be undertaken to identify relevant journal articles and grey literature. Data will be extracted in relation to the emerging contextual factors, mechanisms and outcomes and their configurations. The analysis will seek patterns and regularities in these configurations across the extracted data and will focus on addressing our theory-based questions.
Increasingly, community-based mental health markets are moving to contestability models. Rigorous research is needed to understand how such markets work and in what contexts. The knowledge gained from this study in community-based mental health will provide valuable lessons in how contestability works, in what circumstances and who benefits when. The results of the proposed research will be useful to policy-makers and may be applicable in other contexts beyond the community-based mental health sector.
Systematic review registration
PROSPERO CRD42015016808
PMCID: PMC4377008  PMID: 25875110
Realist synthesis; Realist review; Contestable markets; Contestability; Community-based organisations; Mental health; Community mental health
20.  A realist review of educational interventions to improve the delivery of nutrition care by doctors and future doctors 
Systematic Reviews  2014;3:148.
Dietary interventions are considered an important aspect of clinical practice, more so in the face of the rising prevalence of obesity, diabetes and cardiovascular diseases globally. Routinely, most doctors do not provide such intervention to their patients, and several barriers, present during both training and clinical practice, have been identified. Educational interventions to improve nutrition care competencies and delivery have been implemented but with variable success, probably, due to the complex nature of such interventions. Using traditional methods only to investigate whether interventions are effective or not could not provide appropriate lessons. It is therefore pertinent to conduct a realist review that investigates how the interventions work. This realist review aims at determining what sort of educational interventions work, how, for whom, and in what circumstances, to improve the delivery of nutrition care by doctors and future doctors.
This realist review will be conducted according to Pawson’s five practical steps for conducting a realist review: (1) clarifying the scope of the review, (2) determining the search strategy, including adopting broad inclusion/exclusion criteria and purposive snowballing techniques, (3) ensuring proper article selection and study quality assessment using multiple methods, (4) extracting and organising data through the process of note taking, annotation and conceptualization and (5) synthesising the evidence and drawing conclusions through a process of reasoning. This realist review protocol has not been registered in any database before now.
Findings will be reported according to the publication criteria outlined by the realist and meta-narrative evidence synthesis (RAMESES) group.
PMCID: PMC4290450  PMID: 25528058
Realist review; Realist synthesis; Educational interventions; Nutrition care; Future doctors; Improve; Delivery
21.  A realist review of brief interventions for alcohol misuse delivered in emergency departments 
Systematic Reviews  2015;4:45.
Brief interventions (BIs) involve screening for alcohol misuse and providing feedback to patients about their use, with the aim of reducing alcohol consumption and related consequences. BIs have been implemented in various healthcare settings, including emergency departments (ED), where they have been found to contribute mixed results in their ability to address alcohol misuse among adults. Mechanisms through which BIs work and contextual factors impacting BI effectiveness are not clear. The purpose of this review was to understand how, for whom, and under what circumstances BIs work for adults misusing alcohol and who have been admitted to an ED. A realist review was chosen to answer these questions as realist reviews create context-mechanism-outcome configurations, leading to the development of comprehensive and detailed theories; in this case explaining how and for whom BIs work.
Databases including PsycINFO, Healthstar, CINAHL, Medline, and Nursing and Allied Health were searched for articles published until December 2013. The search strategy focused on studies examining BIs that targeted alcohol misuse among adults admitted into the ED. The search identified 145 relevant abstracts, of which 36 were included in the review. The literature was synthesized qualitatively (immersion/crystallization).
Four mechanisms were found within reviewed studies, including engagement in/retention of BI materials, resolving ambivalence, increased awareness/insight into consequences of drinking, and increased self-efficacy/empowerment to use skills for change. The following contexts were found to impact mechanisms: emotional state, injury attributed to alcohol use, severity of alcohol use, and baseline stage of change.
This realist review provides advances in theories regarding which mechanisms to target during a BI and which contexts create the most favorable conditions for these mechanisms to occur, ultimately leading to optimal BI outcomes. These results can inform future clinical decision-making when delivering BIs in ED settings. Future research should conduct quantitative examination to confirm these findings.
Systematic review registration
PROSPERO CRD42013006549.
Electronic supplementary material
The online version of this article (doi:10.1186/s13643-015-0024-4) contains supplementary material, which is available to authorized users.
PMCID: PMC4428000  PMID: 25875021
Brief intervention; Realist review; Alcohol; Emergency department
22.  Improving psychosocial health and employment outcomes for individuals receiving methadone treatment: a realist synthesis of what makes interventions work 
BMC psychology  2014;2(1):26.
For over 50 years, methadone has been prescribed to opioid-dependent individuals as a pharmacological approach for alleviating the symptoms of opioid withdrawal. However, individuals prescribed methadone sometimes require additional interventions (e.g., counseling) to further improve their health. This study undertook a realist synthesis of evaluations of interventions aimed at improving the psychosocial and employment outcomes of individuals on methadone treatment, to determine what interventions work (or not) and why.
The realist synthesis method was utilized because it uncovers the processes (or mechanisms) that lead to particular outcomes, and the contexts within which this occurs. A comprehensive search process resulted in 31 articles for review. Data were extracted from the articles, and placed in four templates to assist with analysis. Data analysis was an iterative process and involved comparing and contrasting data within and across each template, and cross checking with original articles to determine key patterns in the data.
For individuals on methadone, engagement with an intervention appears to be important for improved psychosocial and/or employment outcomes. The engagement process involves attendance at interventions as well as an investment in what is offered. Three intervention contexts (often in some combination) support the engagement process: a) client-centered contexts (or those where clients’ psychosocial and/or employment needs/issues/skills are recognized and/or addressed); b) contexts which address clients’ socio-economic conditions and needs; and, c) contexts where there are positive client-counselor and/or peer relationships. There is some evidence that sometimes ongoing engagement is necessary to maintain positive outcomes. There is also some evidence that complete abstinence from drugs (e.g., cocaine, heroin) is not necessary for engagement.
It is important to consider how the contexts of interventions might elicit and/or support clients’ engagement. Further research is needed to explore how an individual’s background (e.g., involvement with different interventions over an extended period) may influence engagement. Long-term engagement may be necessary to sustain some positive outcomes although how long is unclear and requires further research. Engagement can occur without complete abstinence from such drugs as cocaine or heroin, but additional research is required as engagement may be influenced by the extent and type of drug use.
Electronic supplementary material
The online version of this article (doi:10.1186/s40359-014-0026-3) contains supplementary material, which is available to authorized users.
PMCID: PMC4269989  PMID: 25566385
Methadone treatment; Engagement; Realist synthesis; Opioids; Opiates; Employment outcomes; Psychosocial health; Client-centered; Socio-economic conditions; Positive relationships
23.  A realist review of interventions and strategies to promote evidence-informed healthcare: a focus on change agency 
Change agency in its various forms is one intervention aimed at improving the effectiveness of the uptake of evidence. Facilitators, knowledge brokers and opinion leaders are examples of change agency strategies used to promote knowledge utilization. This review adopts a realist approach and addresses the following question: What change agency characteristics work, for whom do they work, in what circumstances and why?
The literature reviewed spanned the period 1997-2007. Change agency was operationalized as roles that are aimed at effecting successful change in individuals and organizations. A theoretical framework, developed through stakeholder consultation formed the basis for a search for relevant literature. Team members, working in sub groups, independently themed the data and developed chains of inference to form a series of hypotheses regarding change agency and the role of change agency in knowledge use.
24, 478 electronic references were initially returned from search strategies. Preliminary screening of the article titles reduced the list of potentially relevant papers to 196. A review of full document versions of potentially relevant papers resulted in a final list of 52 papers. The findings add to the knowledge of change agency as they raise issues pertaining to how change agents’ function, how individual change agent characteristics effect evidence-informed health care, the influence of interaction between the change agent and the setting and the overall effect of change agency on knowledge utilization. Particular issues are raised such as how accessibility of the change agent, their cultural compatibility and their attitude mediate overall effectiveness. Findings also indicate the importance of promoting reflection on practice and role modeling. The findings of this study are limited by the complexity and diversity of the change agency literature, poor indexing of literature and a lack of theory-driven approaches.
This is the first realist review of change agency. Though effectiveness evidence is weak, change agent roles are evolving, as is the literature, which requires more detailed description of interventions, outcomes measures, the context, intensity, and levels at which interventions are implemented in order to understand how change agent interventions effect evidence-informed health care.
PMCID: PMC3848622  PMID: 24010732
Realist synthesis; Evidence-informed health care; Change agency; Facilitators; Opinion leaders; Knowledge brokers’ knowledge utilization
24.  Evaluating the effectiveness of a multifaceted, multilevel continuous quality improvement program in primary health care: developing a realist theory of change 
Variation in effectiveness of continuous quality improvement (CQI) interventions between services is commonly reported, but with little explanation of how contextual and other factors may interact to produce this variation. Therefore, there is scant information available on which policy makers can draw to inform effective implementation in different settings. In this paper, we explore how patterns of change in delivery of services may have been achieved in a diverse range of health centers participating in a wide-scale program to achieve improvements in quality of care for Indigenous Australians.
We elicited key informants’ interpretations of factors explaining patterns of change in delivery of guideline-scheduled services over three or more years of a wide-scale CQI project, and inductively analyzed these interpretations to propose fine-grained realist hypotheses about what works for whom and in what circumstances. Data were derived from annual clinical audits from 36 health centers operating in diverse settings, quarterly project monitoring reports, and workshops with 12 key informants who had key roles in project implementation. We abstracted potential context-mechanism-outcome configurations from the data, and based on these, identified potential program-strengthening strategies.
Several context-specific, mechanism-based explanations for effectiveness of this CQI project were identified. These were collective valuing of clinical data for improvement purposes; collective efficacy; and organizational change towards a population health orientation. Health centers with strong central management of CQI, and those in which CQI efforts were more dependent on local health center initiative and were adapted to resonate with local priorities were both favorable contexts for collective valuing of clinical data. Where health centers had prior positive experiences of collaboration, effects appeared to be achieved at least partly through the mechanism of collective efficacy. Strong community linkages, staff ability to identify with patients, and staff having the skills and support to take broad ranging action, were favorable contexts for the mechanism of increased population health orientation.
Our study provides evidence to support strategies for program strengthening described in the literature, and extends the understanding of mechanisms through which strategies may be effective in achieving particular outcomes in different contexts.
PMCID: PMC4124892  PMID: 24098940
Quality improvement; Evaluation science; Realist evaluation; Primary health care; Program theory
25.  Methods for Specifying the Target Difference in a Randomised Controlled Trial: The Difference ELicitation in TriAls (DELTA) Systematic Review 
PLoS Medicine  2014;11(5):e1001645.
Jonathan Cook and colleagues systematically reviewed the literature for methods of determining the target difference for use in calculating the necessary sample size for clinical trials, and discuss which methods are best for various types of trials.
Please see later in the article for the Editors' Summary
Randomised controlled trials (RCTs) are widely accepted as the preferred study design for evaluating healthcare interventions. When the sample size is determined, a (target) difference is typically specified that the RCT is designed to detect. This provides reassurance that the study will be informative, i.e., should such a difference exist, it is likely to be detected with the required statistical precision. The aim of this review was to identify potential methods for specifying the target difference in an RCT sample size calculation.
Methods and Findings
A comprehensive systematic review of medical and non-medical literature was carried out for methods that could be used to specify the target difference for an RCT sample size calculation. The databases searched were MEDLINE, MEDLINE In-Process, EMBASE, the Cochrane Central Register of Controlled Trials, the Cochrane Methodology Register, PsycINFO, Science Citation Index, EconLit, the Education Resources Information Center (ERIC), and Scopus (for in-press publications); the search period was from 1966 or the earliest date covered, to between November 2010 and January 2011. Additionally, textbooks addressing the methodology of clinical trials and International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) tripartite guidelines for clinical trials were also consulted. A narrative synthesis of methods was produced. Studies that described a method that could be used for specifying an important and/or realistic difference were included. The search identified 11,485 potentially relevant articles from the databases searched. Of these, 1,434 were selected for full-text assessment, and a further nine were identified from other sources. Fifteen clinical trial textbooks and the ICH tripartite guidelines were also reviewed. In total, 777 studies were included, and within them, seven methods were identified—anchor, distribution, health economic, opinion-seeking, pilot study, review of the evidence base, and standardised effect size.
A variety of methods are available that researchers can use for specifying the target difference in an RCT sample size calculation. Appropriate methods may vary depending on the aim (e.g., specifying an important difference versus a realistic difference), context (e.g., research question and availability of data), and underlying framework adopted (e.g., Bayesian versus conventional statistical approach). Guidance on the use of each method is given. No single method provides a perfect solution for all contexts.
Please see later in the article for the Editors' Summary
Editors' Summary
A clinical trial is a research study in which human volunteers are randomized to receive a given intervention or not, and outcomes are measured in both groups to determine the effect of the intervention. Randomized controlled trials (RCTs) are widely accepted as the preferred study design because by randomly assigning participants to groups, any differences between the two groups, other than the intervention under study, are due to chance. To conduct a RCT, investigators calculate how many patients they need to enroll to determine whether the intervention is effective. The number of patients they need to enroll depends on how effective the intervention is expected to be, or would need to be in order to be clinically important. The assumed difference between the two groups is the target difference. A larger target difference generally means that fewer patients need to be enrolled, relative to a smaller target difference. The target difference and number of patients enrolled contribute to the study's statistical precision, and the ability of the study to determine whether the intervention is effective. Selecting an appropriate target difference is important from both a scientific and ethical standpoint.
Why Was This Study Done?
There are several ways to determine an appropriate target difference. The authors wanted to determine what methods for specifying the target difference are available and when they can be used.
What Did the Researchers Do and Find?
To identify studies that used a method for determining an important and/or realistic difference, the investigators systematically surveyed the research literature. Two reviewers screened each of the abstracts chosen, and a third reviewer was consulted if necessary. The authors identified seven methods to determine target differences. They evaluated the studies to establish similarities and differences of each application. Points about the strengths and limitations of the method and how frequently the method was chosen were also noted.
What Do these Findings Mean?
The study draws attention to an understudied but important part of designing a clinical trial. Enrolling the right number of patients is very important—too few patients and the study may not be able to answer the study question; too many and the study will be more expensive and more difficult to conduct, and will unnecessarily expose more patients to any study risks. The target difference may also be helpful in interpreting the results of the trial. The authors discuss the pros and cons of different ways to calculate target differences and which methods are best for which types of studies, to help inform researchers designing such studies.
Additional Information
Please access these websites via the online version of this summary at
Wikipedia has an entry on sample size determination that discusses the factors that influence sample size calculation, including the target difference and the statistical power of a study (statistical power is the ability of a study to find a difference between treatments when a true difference exists). (Note: Wikipedia is a free online encyclopedia that anyone can edit; available in several languages.)
The University of Ottawa has an article that explains how different factors influence the power of a study
PMCID: PMC4019477  PMID: 24824338

Results 1-25 (1327377)