PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of bmjoBMJ OpenVisit this articleSubmit a manuscriptReceive email alertsContact usBMJ
 
BMJ Open. 2017; 7(3): e014497.
Published online 2017 March 10. doi:  10.1136/bmjopen-2016-014497
PMCID: PMC5353324

Paediatric early warning systems for detecting and responding to clinical deterioration in children: a systematic review

Abstract

Objective

To systematically review the available evidence on paediatric early warning systems (PEWS) for use in acute paediatric healthcare settings for the detection of, and timely response to, clinical deterioration in children.

Method

The electronic databases PubMed, MEDLINE, CINAHL, EMBASE and Cochrane were searched systematically from inception up to August 2016. Eligible studies had to refer to PEWS, inclusive of rapid response systems and teams. Outcomes had to be specific to the identification of and/or response to clinical deterioration in children (including neonates) in paediatric hospital settings (including emergency departments). 2 review authors independently completed the screening and selection process, the quality appraisal of the retrieved evidence and data extraction; with a third reviewer resolving any discrepancies, as required. Results were narratively synthesised.

Results

From a total screening of 2742 papers, 90 papers, of varied designs, were identified as eligible for inclusion in the review. Findings revealed that PEWS are extensively used internationally in paediatric inpatient hospital settings. However, robust empirical evidence on which PEWS is most effective was limited. The studies examined did however highlight some evidence of positive directional trends in improving clinical and process-based outcomes for clinically deteriorating children. Favourable outcomes were also identified for enhanced multidisciplinary team work, communication and confidence in recognising, reporting and making decisions about child clinical deterioration.

Conclusions

Despite many studies reporting on the complexity and multifaceted nature of PEWS, no evidence was sourced which examined PEWS as a complex healthcare intervention. Future research needs to investigate PEWS as a complex multifaceted sociotechnical system that is embedded in a wider safety culture influenced by many organisational and human factors. PEWS should be embraced as a part of a larger multifaceted safety framework that will develop and grow over time with strong governance and leadership, targeted training, ongoing support and continuous improvement.

Keywords: PEWS, Paediatric early warning system, clinical deterioration, children, systematic review

Strengths and limitations of this study

  • This review systematically and collectively synthesises the available evidence on the multiple components of paediatric early warning systems (PEWS).
  • The review highlights that PEWS should be embraced as a part of a larger multifaceted safety framework.
  • Future research needs to investigate PEWS as a complex multifaceted sociotechnical system embedded in a wider safety culture.
  • Owing to heterogeneous research designs, assessing quality across eligible studies was limited.
  • While no strong evidence underpinning any one PEWS was available, emerging work should contribute to this evidence base.

Background

It is known that children who die or deteriorate unexpectedly in the hospital setting will often have observable features in the period before the seriousness of their condition is recognised. A seminal study of paediatric mortality in the UK estimated that approximately one in five children who die in hospital have avoidable factors leading to death and up to half of children have potentially avoidable factors.1 The report concluded that ‘there should be ways of telling if something is wrong with a child as early as possible, for example, an early warning scoring system’.1 Other studies have examined the signs (physiological and behavioural) of deterioration that may be present in the period preceding a cardiopulmonary arrest,2 3 and the fact that these features are often not recognised or acted on in a timely fashion by hospital staff.4 5 Recent years have also witnessed an increased risk of paediatric cardiopulmonary arrest, and its associated mortality, in acute healthcare settings largely as a consequence of increased acuity of care and higher dependency on technology.2 Although the percentage of paediatric cardiopulmonary arrests for inpatient admissions has been reported as low (eg, 0.7–3%),6 7 survival to discharge for children that experience inhospital cardiopulmonary arrest is poor (11–37%).3 6

Early warning scores are generally defined as bedside ‘track and trigger’ tools to help alert staff to clinically deteriorating children by periodic observation of physiological parameters, generation of a numeric score and predetermined criteria for escalating urgent assistance with a clear framework for communication. In using these physiological track and trigger systems, the goal is to ensure timely recognition of patients with potential or established critical illness and to ensure a timely and appropriate response from skilled staff. Critical to early warning scores are four integrated components which work together to provide a comprehensive safety system for clinically deteriorating patients and those that are most likely to identify and manage patients at highest risk for cardiac or respiratory arrest; (1) the afferent component which detects clinical deterioration and triggers an appropriate response; (2) the efferent component which consists of the personnel and resources providing the response (eg, medical emergency team (MET)); (3) the process improvement component containing elements such as auditing/monitoring/evaluation to enhance patient care and safety and (4) the governance/administrative component focusing on the organisational leadership, safety culture, education and processes required to implement and sustain the system.8 This highlights the need to view early warning tools as more than just a ‘score’, rather, they are part of a multifaceted ‘system’ approach based on the implementation of several complementary safety interventions to improve child patient safety and clinical outcomes.

In Ireland, a 2013 patient safety review by the Health Information and Quality Authority (HIQA) into the unexpected death of a young woman in a maternity setting identified several care failures.9 These included a lack of provision of basic fundamental care, failure to recognise risk of clinical deterioration, failure to act or escalate concerns about deterioration to appropriately qualified clinicians and lack of detail in medical record documentation about clinical status and potential risk of clinical deterioration. This led to a request from the Minister for Health that the Department of Health's National Clinical Effectiveness Committee commission and quality assure a number of National Clinical Guidelines; including early warning scores for adult, maternity and paediatric healthcare settings.

For paediatrics, this request presented several design challenges, including the need for an observation tool that would work in all paediatric care settings (secondary and specialist care) and a requirement to align with the Adult and Maternity scores. Additionally, the application of early warning scores to paediatric patients is more complex than in adults. There are several reasons for this: variation in age-specific thresholds for normal and abnormal physiology; children's inability or difficulty in articulating how or what they feel; children's physiological compensatory mechanisms; staff training issues and the need for more focused attention on respiratory deterioration.10 Finally, although many paediatric early warning systems (PEWS) have been developed and tested, uncertainty remains as to which system, or system feature, is most useful for paediatric patients. Even the concept of PEWS as a system (ie, the application of all four components in parallel as described above) is poorly developed.

The aim of this review was to systematically identify and synthesise available evidence on PEWS in acute paediatric healthcare settings for the detection of, and timely response to, clinical deterioration in children. The review questions were set by the Irish Department of Health who commissioned this review:

  1. What is the available evidence on the effectiveness of different PEW detection systems?
  2. What evidence exists on the effectiveness of PEW response mechanisms, and what interventions are used?
  3. What evidence exists on PEWS implementation strategies/interventions?

Methods

Design

This review was conducted and reported in accordance with the Centre for Reviews and Dissemination guidance for undertaking systematic reviews in healthcare,11 the National Clinical Effectiveness Committee Guideline Development Manual12 and the Preferred Reporting in Systematic Reviews and Meta-Analysis (PRISMA) criteria.13

Data sources and search strategy

The following electronic databases PubMed, MEDLINE, CINAHL, EMBASE and Cochrane (inclusive of Cochrane Database of Systematic Review; Database of Abstracts of Review Effects and CENTRAL—Cochrane Central Register of Controlled Trials) were systematically searched from database inception up to August 2016 using various combinations of controlled vocabulary (eg, MeSH) and free text words guided by our PICOS parameters (see online supplementary appendix 1). The search was limited by language (English). For unpublished research reports, grey literature databases, trial registers and national/international professional organisations and association websites were searched. To retrieve evidence-based clinical guidelines, electronic guideline clearinghouses were searched, scoping searches of Google and Bing were performed and a consultation process was conducted with key paediatric experts and paediatric hospitals internationally. Additional literature was sourced by contacting reference study authors and experts in the field and scanning bibliographies of all included papers.

10.1136/bmjopen-2016-014497.supp1

supplementary appendices

bmjopen-2016-014497supp_appendices.pdf

Screening and selection process

Eligible papers had to refer to PEWS, inclusive of rapid response systems (RRS) and rapid response teams (RRT). Outcomes had to be specific to the identification of and/or response to clinical deterioration in child patients (including neonates) in paediatric hospital settings (including emergency departments). No study design restrictions were applied. We excluded papers that focused on paediatric community health settings; PEWS specific to intrahospital and/or interhospital transfer and/or transportation of critically ill children; trigger tools for identification of adverse events and/or harm caused by medical interventions; severity of illness scales and patient classification systems specifically for identifying illness acuity and mortality (except in cases where such studies included PEWS as comparative interventions) and studies which included child and adult populations when child-specific data could not be exclusively extracted.

For stage 1 screening, two reviewers independently assessed each title and abstract retrieved from the electronic searches for relevance. Any discrepancies were resolved by discussion and consensus with a third reviewer. If no abstract was available, the full-text paper was sourced and assessed. For studies deemed to meet the inclusion criteria, full texts of the studies were obtained. Full-text papers were independently assessed by two reviewers against the inclusion criteria before a final decision regarding inclusion/exclusion was confirmed. Any discrepancies were resolved by discussion and consensus with a third reviewer. Reasons for excluding studies from the review were noted (see figure 1).

Figure 1
Flow chart of search strategy output and screening process.

Appraisal of the level of evidence

In an attempt to conduct a comprehensive review, all studies which met the inclusion criteria were included regardless of quality. Two reviewers appraised and classified the level of evidence of the included studies in accordance with the Scottish Intercollegiate Guidelines Network (SIGN) criteria for assessment of studies based on the type of study design. Assessing comparative quality across eligible studies proved difficult due to the heterogeneous nature of the research methodologies employed; including disparate research designs, different ranges for collecting data over time periods (from months to years), localised small case and comparative group selections, and diverse clinical contexts ranging from general medical and surgical units to specialised settings such as oncology, cardiac, endocrine and rehabilitation units.

Data extraction and synthesis

Two reviewers independently extracted and managed data from the included studies. Any discrepancies were resolved through consultation with a third reviewer. A data extraction table was developed to retrieve information pertaining to each study setting, aim, design, sample, intervention and main outcomes/findings. In line with the review research questions, the studies were segregrated by PEW detection systems, response mechanisms and implementation processes. All data were narratively synthesised as it was not possible to conduct a meta-analysis and/or a metasynthesis because of the heterogeneity of evidence retrieved including non-comparative research designs and diversity of systems, approaches and methods adopted in developing and implementing PEWS in paediatric contexts.

Results

Overall search and selection results

A total of 2742 papers were identified as potentially eligible for inclusion in the review. Following first screening of titles and abstracts, 2616 papers were excluded because they were adult-focused, discussion papers, commentaries, conference abstracts and/or duplicate papers. Full texts of the remaining 126 papers were obtained. On second screening of these 126 full-text papers, a further 57 papers were excluded because they were adult-focused, both child and adult-focused in which it was not possible to segregate child and adult data, not specifically focused on the outcome of clinical deterioration, wrong setting (ie, not inpatient), concentrated on clinical deterioration at point of transportation, examined illness severity or acuity or were discussion papers, commentaries or conference abstracts. This left 69 papers that met the inclusion criteria. An additional 21 papers were sourced through secondary citations, personal communications with reference authors/experts in the field and web-resources. Subsequently, 90 papers fulfilled the eligibility criteria. Figure 1, an adapted PRISMA flow chart, visually displays the search and selection process.

Characteristics of included studies

The studies emanated from the USA (n=46), the UK (n=19), Canada (n=10), Canada and the UK (n=1), Australia (n=5), the Netherlands (n=2), Ireland (n=2), Norway (n=1), Pakistan (n=1), Sweden (n=1), Thailand (n=1) and South America (n=1). The majority of the studies were observational in design, and included 13 cohort studies, 11 case–control, 8 before and after and 6 cross-sectional surveys. There were eight review papers and three interrupted time series quasi-experimental studies. The remainder were chart/database reviews (n=23), quality improvement initiatives (n=9), qualitative studies (n=4) or case reports (n=1). There was one feasibility and reliability testing study, one cost-analysis exercise, one protocol and one course evaluation survey. Of the 90 included papers, 45 focused on PEW detection systems,2 3 6 7 10 14–53 29 examined PEW response mechanisms8 54–81 and 16 reported on PEW implementation strategies82–97 (see online supplementary appendix 2 for a summary of these studies including the level of evidence and rationale for judgement).

Review question 1: What is the available evidence on the effectiveness of different PEW detection systems?

Thirty-eight primary studies reported on original3 10 23 25 27 31 35 39 44 46 (see online supplementary appendix 3) and/or adopted/modified2 6 14–21 24 26 28–30 33–38 40–43 47 48 50–53 PEW detection systems for use in paediatric inpatient settings. Twenty-three of these 38 studies reported on the effectiveness of PEW detection systems using the performance criteria of sensitivity, specificity, receiver operating characteristic curve, positive predictive value and/or negative predictive value.2 3 6 10 14 15 18 20 21 23 25–27 29–31 33 37 39 40 48 50 53 Diversity in PEW physiological (and other) parameters and differences in age-dependent vital sign reference ranges made it difficult to compare and contrast performance criteria. To enable some comparisons to be made, further studies were excluded if they; were from specialist units if only one study was published, only reported on inter-rater and intrarater agreement, had <100 cases and did not report data on sensitivity and specificity. Figure 2 shows the diagnostic predictive accuracy of PEW detection systems from 11 studies.6 10 14 18 20 21 25 26 30 40 50 This illustrates that the effectiveness of PEW detection systems demonstrated wide-ranging sensitivity and specificity largely as a consequence of different settings adopting and self-regulating varying end point or surrogate markers for clinical deterioration (ie, cardiopulmonary arrest, PICU admission, mortality and interventions) and different standards for cut-off/threshold scores.

Figure 2
Diagnostic predictive accuracy of paediatric early warning detection systems.

Review question 2: What evidence exists on the effectiveness of PEW response mechanisms, and what interventions are used?

Table 1 provides an overview of the evidence on PEW response interventions. Across 18 primary studies, the main PEW response intervention in use was health professional-activated RRS incorporating paediatric RRTs or METs.54–56 60 61 64–68 70 73–76 78 79 81 Where reported, RRS were available to be activated by any staff member 24 hours/day, 7 days a week. The staffing composition of the majority of RRT/METs included a critical care nurse, a physician and a respiratory therapist. The most common RRT/MET activation criteria were cardiovascular, respiratory and neurological status, alongside staff and family concern. Studies examining the effectiveness of RRSs reported on a number of clinical and process outcome data, for example, cardio/respiratory arrest (CPA) rates, mortality rates unplanned PICU transfers/admissions interventions required (ie, intubation, mechanical ventilation, inotropes) and MET/code blue activations. Collectively, findings revealed mixed evidence on the effectiveness of RRSs. For instance, although four studies reported a significant reduction in CPA rates and five studies found a significant reduction in mortality, there were an equal number of studies reporting non-significant findings.

Table 1
Overview of evidence on paediatric early warning response mechanisms

Five papers reported on quality improvement initiatives for families to activate the RRS.59 62 63 69 71 Findings revealed that families infrequently activate the RRS, but when they do, the reason is largely as a consequence of communication failures rather than critical care deterioration. While physicians value family input and depend on families to explain their child's baseline condition and identify subtle changes in their child, physicians are apprehensive towards family-activated RRS because of potential misuse of resources, undermining of the clinician–family therapeutic relationship, increased family anxiety/burden and a need to provide knowledge/training to families.

Review question 3: What evidence exists on PEWS implementation strategies/interventions?

Table 2 provides an overview of evidence from 16 studies reporting on PEW implementation strategies/interventions. The evidence was diverse in approach, ranging from the adoption of social marketing principles to quality/performance improvement initiatives to chart reviews, qualitative studies and pre–post implementation surveys. Comparative evaluations were therefore difficult and no conclusions were drawn on an optimal implementation strategy to influence change in clinical/process outcomes (or indeed what the best clinical/process outcomes are to measure). Despite the limited evidence, valuable insights were gleaned into cultural, sociotechnical, education/training and organisational issues impacting, either positively or negatively, on the effective implementation of PEWS. For example, a number of qualitative and quality improvement studies highlighted the importance of creating an empowering culture that fosters trusting relationships, opens communication and supportive teamwork.83 85 87 90 96 Working through real-life cases and using a multiprofessional approach to PEWS education/training were positively evaluated for improving doctor–nurse communication, enhanced team-work and better use of the SBAR (Situation, Background, Assessment, Recommendations) communication technique.97 Significant improvements were also found in documented vital signs, communication episodes and intern hand-offs after ABC-SBAR (communication technique) training.92 93 The integration of situation awareness interventions into EWS was also recommended to recognise experienced clinicians tacit knowledge (ie, watcher/clinician gut feeling) and the incorporation of structures, such as huddles, to proactively identify risk and communicate concerns at bedside, unit and organisational level.85 86

Table 2
Overview of evidence on paediatric early warning implementation strategies/interventions

No published evidence for the resource implications of complete PEWS (detection, response and implementation) was found. Bonafide et al84 prepared the cost of a MET component of PEWS and found three clinical deterioration events would offset MET costs (compared with pre-MET). After this, any clinical deterioration events averted (by MET) would represent cost savings. These findings relate to one element of PEWS and may not translate directly to PEW scoring systems or additional safety structures that enhance PEWS implementation.

Discussion

This review systematically examined and synthesised evidence on PEWS as a comprehensive system comprised of detection, response and implementation components. For all three review questions, no conclusive answers on the effectiveness and impact of PEWS on clinical practice were identified. The review revealed the absence of a standard PEW scoring system across paediatric inpatient settings internationally, limited standardisation of outcomes to enable comparison of published PEWS studies and uncertainty regarding PEWS education and implementation processes at different institutions. This highlights the need for more organised multisite coordination and study around PEW scoring, systems usage, implementation and outcome measures. While the review revealed mixed outcomes, it is promising to see evidence suggesting positive directional trends in clinical outcomes, for example, reduced cardiopulmonary arrests, earlier intervention and transition to PICU with accompanying potential improvements in patient safety culture through enhanced multidisciplinary communication and team-work, for example.

The review draws attention to the fact that multiple distinct PEWS scoring systems are in use internationally, yet empirical evidence on which system is most effective is limited. Perhaps this is due to the heterogeneity in how the detection tools were developed, modified and investigated across included studies. Diversity in the composition of PEW detection systems (ie, physiological parameters, reference range values, trigger threshold points and clinical deterioration outcome markers) makes it difficult to compare and contrast performance criteria. It was rare, however, for any PEW detection system to have a high specificity and sensitivity. While some systems showed promising performance criteria, many were unable to be fully validated due to low sensitivity. Many contexts chose simplicity and clinical utility as a priority in deciding which PEW detection system to implement. The variety of PEW parameters used by local units is perhaps reflective of the desire to have locally derived systems.45 This presents difficulty for development of a national, and/or international, standard to guide clinical practice. Challenges exist in standardising a common scoring tool and in establishing a common language among healthcare professionals for recognising and responding to clinically deteriorating children. Indeed, the majority of PEW detection systems were evaluated at one point in time, and in single-site paediatric hospital settings, limiting the transferability of results. One multicentre case–control study40 was identified which validated the Canadian Bedside PEWS across inpatient units in four children's hospitals. Results are eagerly awaited from the first multicentre cluster randomised controlled trial evaluating the impact of Bedside PEWS across 22 hospitals internationally.42

The review identified that the main PEW response intervention in use internationally was health professional-activated RRSs, incorporating RRTs and METs. It was difficult to make comparisons, however, because of variations in how RRT/METs were operationalised in terms of team membership, activation criteria and determination of effectiveness. With limited uniformity on how clinical and process outcomes were defined and measured across studies, uncertainty remains around the impact of RRS on the timely intervention for children with clinical deterioration. Further evidence is also needed on family-activated response mechanisms to demonstrate improved patient outcomes.

Despite many anecdotal accounts emphasising the importance of the process of PEWS implementation, a dearth of published literature was sourced in this area. The review did identify, however, the need for cognisance to be given to the multifaceted nature of PEWS (ie, communication, multidisciplinary team-work and education, parent involvement), including the healthcare cultural context in which PEWS would be implemented. There is a need to move beyond reactive responses to include proactive assessment of children at risk of clinical deterioration (eg, concepts such as the watcher, huddles, roving teams).85 86 98 Healthcare professionals can benefit from improved situational awareness to proactively assess all relevant context around the child, family, tasks required, staff/team and environment.99 100

Despite its limitations, this review contributes important learning because no evidence was sourced that collectively examined the multiple components of PEWS as a complex healthcare intervention in a single study. Rather, the evidence examined PEWS in a piece-meal manner, focusing on one particular aspect (eg, detection, response or implementation) each time. The findings support Chapman et al's22 recently updated review which revealed low evidence to support paediatric track and trigger system (PTTS) implementation as a single intervention. There was, however, some moderate evidence to support the delivery of PTTS as part of a package of interventions or ‘care bundles’. Chapman et al22 contended that this may be reflective of the complexities of healthcare delivery. The multiple challenges inherent in the delivery of effective high-quality safe healthcare are increasingly recognised with the call for more proactive defence layers that focus on system, rather than human, resilience.100 One avenue to potentially assist with addressing the complexity of PEWS, and advancing this field of knowledge, is the integration of quality improvement, science and human factors. This is important because human factors are not independent issues that can be tackled in isolation or on a piece-meal basis but need to be integrated into the life cycle of the systems development.100 This could potentially lead to improvements in better outcomes and experiences for children and their families and also better system performance (ie, care) and professional development (ie, learning).101

Strengths and limitations

This manuscript systematically collated and synthesised evidence on the multiple components (detection, response and implementation) of PEWS collectively in one review. While a comprehensive search strategy was employed, and the recommended practices for the conduct and reporting of systematic reviews were adhered to, it is possible that some relevant papers may have been missed. Additionally, with the exclusion of non-English papers, there is the potential risk of publication bias. Although beyond the scope of this review, there is potentially other literature likely to be of relevance to informing the effectiveness of PEWS; most specifically to examine sociocontextual factors (eg, situation awareness and human factor) that may, or may not, work as active ingredients in the successful implementation of PEWS. There is some work emerging in this area.102

Recommendations for clinical practice

Clinicians working in inpatient paediatric units, and management at unit and organisational levels, need to recognise that the early detection of a deteriorating child is much more than identifying and responding to a score. Instead, through creation of a common language, PEWS should stimulate a heightened sense of situation awareness and open communication among clinicians about children at risk of clinical deterioration, thereby supporting, not replacing, clinical judgement. PEWS should be embraced as a part of a larger multifaceted safety framework that will develop and grow over time with strong governance and leadership, targeted training, ongoing support and continuous improvement.

Directions for future research

Future research needs to investigate PEWS as a complex multifaceted sociotechnical system that is embedded in a wider safety culture influenced by many organisational and human factors such as, but not limited to, clinician knowledge, experience and confidence; effective multidisciplinary communication and team-work; family engagement; situation awareness; decision-making; unit and hospital management and leadership; working conditions and the environment; and stress and fatigue. There is evidence of some potential emerging work in this area in the UK.103

Conclusion

This review identified that PEWS are widely used internationally. However, empirical evidence revealed a lack of consensus on which PEWS is most effective or useful. Notwithstanding the limited consensual evidence, positive trends in improved clinical outcomes, such as reduced cardiopulmonary arrest or earlier intervention and transfer to PICU, were reported. Additionally, the implementation of PEWS as one part of a wider safety culture has the potential to enhance multidisciplinary team working, communication and confidence in recognising and making clinical decisions about clinically deteriorating children. The lack of multicentre studies, no national guidelines, no research evaluating PEWS as a complex healthcare intervention and limited development of any underlying theory all impact on the consistency with which PEWS are defined, implemented and measured for effectiveness. Consequently, further research is required to establish what the true ‘active ingredients’ of PEWS interventions are in contributing to the detection and/or timely identification of, and response to, deterioration in improving clinical outcomes for children in inpatient hospital settings.

Acknowledgments

This work originated from a Department of Health commissioned review lead by the corresponding author in 2014. In August 2016, the authors revised and updated this review. The authors are grateful to the other project team members (Marie T O'Shea, Catherine Walshe, Melissa Corbally, Donal O'Mathuna, Anthony Staines, Caroline O'Connor) for their help in the original review process. The authors would also like to acknowledge members of our national paediatric expert advisory group for the original review, including Cormac Breatnach, Sharon Condon, Orla Franklin, Amanda Halpin, Una McAree and Ann Moran.

Footnotes

Contributors: VL was the lead investigator for the original and revised review. VL and AM designed the review protocol, developed and ran the updated search searches, selected and appraised the papers, extracted data and drafted the initial manuscript. RM and JF commented on the protocol, searches, evidence appraisal and revised the manuscript for important intellectual content. All authors approved the final manuscript.

Funding: This work was supported by the Irish Department of Health and overseen by the National Clinical Effectiveness Committee and Health Service Executive PEWS Guideline Development Group.

Competing interests: None declared.

Provenance and peer review: Not commissioned; externally peer reviewed.

Data sharing statement: No additional data are available.

References

1. Confidential Enquiry into Maternal and Child Health (CEMACH). Why children die—a pilot study 2006. London: CEMACH, 2008.
2. Robson MA, Cooper CL, Medicus LA et al. Comparison of three acute care pediatric early warning scoring tools. J Pediatr Nurs 2013;28:e33–41. doi:10.1016/j.pedn.2012.12.002 [PubMed]
3. McLellan MC, Gauvreau K, Connor JA Validation of the cardiac children's hospital early warning score: an early warning scoring tool to prevent cardiopulmonary arrests in children with heart disease. Congenit Heart Dis 2014;9:194–202. doi:10.1111/chd.12132 [PubMed]
4. Endacott R, Kidd T, Chaboyer W et al. Recognition and communication of patient deterioration in a regional hospital: a multi-methods study. Aust Crit Care 2007;20:100–5. doi:10.1016/j.aucc.2007.05.002 [PubMed]
5. Van Leuvan C, Mitchell I Missed opportunities? An observational study of vital sign measurements. Crit Care Resusc 2008;10:111–5. [PubMed]
6. Tucker KM, Brewer TL, Baker RB et al. Prospective evaluation of a pediatric inpatient early warning scoring system. J Spec Pediatr Nurs 2009;14:79–85. doi:10.1111/j.1744-6155.2008.00178.x [PubMed]
7. Chapman SM, Grocott MPW, Franck LS Systematic review of paediatric alert criteria for identifying hospitalised children at risk of clinical deterioration. Intensive Care Med 2010;36:600–11. doi:10.1007/s00134-009-1715-x [PubMed]
8. Jagt EW. Improving pediatric survival from resuscitation events: the role and organization of hospital-based rapid response systems and code teams. Cur Pediatr Rev 2013;9:158–74. doi:10.2174/1573396311309020009 [PubMed]
9. Health Information and Quality Authority (HIQA). Investigation into the safety, quality and standards of services provided by the Health Service Executive to patients, including pregnant women, at risk of clinical deterioration, including those provided in University Hospital Galway, and as reflected in the care and treatment provided to Savita Halappanavar. Dublin: Health Information and Quality Authority, 2013.
10. Haines C, Perrott M, Weir P Promoting care for acutely ill children—development and evaluation of a paediatric early warning tool. Intensive Crit Care Nurs 2006;22:73–81. doi:10.1016/j.iccn.2005.09.003 [PubMed]
11. Centre for Reviews and Dissemination. Systematic Reviews: CRD's guidance for undertaking reviews in health care. University of York: Centre for Reviews and Dissemination, 2008.
12. National Clinical Effectiveness Committee. Guideline Developers Manual 2013. http://health.gov.ie/wp-content/uploads/2015/01/ncec_guideline_development_manual_january13.pdf (accessed 20 Sep 2016).
13. Moher D, Liberati A, Tetzlaff J et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann Intern Med 2009;151:264–9. doi:10.7326/0003-4819-151-4-200908180-00135 [PubMed]
14. Agulnik A, Forbes PW, Stenquist N et al. Validation of a Pediatric Early Warning Score in hospitalised pediatric oncology and hematopoietic stem cell transplant patients. Pediatr Crit Care Med 2016;17:e146–53. doi:10.1097/PCC.0000000000000662 [PubMed]
15. Akre M, Finkelstein M, Erickson M et al. Sensitivity of the pediatric early warning score to identify patient deterioration. Pediatrics 2010;215:2763–9. doi:10.1542/peds.2009-0338 [PubMed]
16. Bell D, Mac A, Ochoa Y et al. The Texas Children's Hospital Pediatric Advanced Warning Score as a predictor of clinical deterioration in hospitalized infants and children: a modification of the PEWS tool. J Pediatr Nurs 2013;28:e2–9. doi:10.1016/j.pedn.2013.04.005 [PubMed]
17. Bolger T, Clarke N, Crowe S et al. Paediatric early warning trigger: a cry for help. Ir Med J 2015;108:294–6. [PubMed]
18. Bradman K, Maconochie I Can paediatric early warning score be used as a triage tool in paediatric accident and emergency? Eur J Emerg Med 2008;15:359–60. doi:10.1097/MEJ.0b013e3283026208 [PubMed]
19. Bradman K, Borland M, Pascoe E Predicting patient disposition in a paediatric emergency department. J Paediatr Child Health 2014;50:39–44. doi:10.1111/jpc.12011 [PubMed]
20. Breslin K, Marx J, Hoffman H et al. Pediatric early warning score at time if emergency department disposition is associated with level of care. Pediatr Emerg Care 2014;30:97–103. doi:10.1097/PEC.0000000000000063 [PubMed]
21. Chaiyakulsil C, Pandee U Validation of Pediatric Early Warning Score in pediatric emergency departments. Pediatr Int 2015;57:694–8. doi:10.1111/ped.12595 [PubMed]
22. Chapman SM, Wray J, Oulton K et al. Systematic review of paediatric track and trigger systems for hospitalised children. Resuscitation 2016;109:87–109. doi:10.1016/j.resuscitation.2016.07.230 [PubMed]
23. Duncan H, Hutchison J, Parshuram CS The pediatric early warning system score: a severity of illness score to predict urgent medical need in hospitalized children. J Crit Care 2006;21:271–9. [PubMed]
24. Ennis L. Paediatric early warning scores on a children's ward: a quality improvement initiative. Nurs Child Young People 2014;26:25–31. doi:10.7748/ncyp.26.7.25.e478 [PubMed]
25. Edwards ED, Powell CVE, Mason BW et al. Prospective cohort study to test the predictability of the Cardiff and Vale paediatric early warning system. Arch Dis Child 2009;94:602–6. doi:10.1136/adc.2008.142026 [PubMed]
26. Edwards ED, Mason BW, Oliver A et al. Cohort study to test the predictability of the Melbourne criteria for activation of the medical emergency team. Arch Dis Child 2011;96:174–9. doi:10.1136/adc.2010.187617 [PubMed]
27. Egdell P, Finlay L, Pedley DK The PAWS score: validation of an early warning scoring system for the initial assessment of children in the emergency department. Emerg Med J 2008;25:745–9. doi:10.1136/emj.2007.054965 [PubMed]
28. Fenix JB, Gillespie CW, Levin A et al. Comparison of Pediatric Early Warning Score to physician opinion for deteriorating patients. Hosp Pediatr 2015;5:474–9. doi:10.1542/hpeds.2014-0199 [PubMed]
29. Fuijkschot J, Vernhout B, Lemson J et al. Validation of a paediatric early warning score: first results and implications for usage. Eur J Pediatr 2015;174:15–21. doi:10.1007/s00431-014-2357-8 [PubMed]
30. Gold DL, Mihalov LK, Cohen DM Evaluating the Pediatric Early Warning Score (PEWS) System for admitted patients in the pediatric emergency department. Acad Emerg Med 2014;21:1249–56. doi:10.1111/acem.12514 [PMC free article] [PubMed]
31. Holme H, Bhatt R, Koumettou M et al. Retrospective evaluation of a new neonatal trigger score. Pediatrics 2013;131:e837–42. doi:10.1542/peds.2012-0640 [PubMed]
32. Kaul M, Snethen J, Kelber ST et al. Implementation of the Bedside Paediatric Early Warning System (Bedside PEWS) for nurse identification of deteriorating patients. J Spec Pediatr Nurs 2014;19:339–49. doi:10.1111/jspn.12092 [PubMed]
33. Mandell IM, Bynum F, Marshall L et al. Pediatric Early Warning Score and unplanned readmission to the pediatric intensive care unit. J Crit Care 2015;30:1090–5. doi:10.1016/j.jcrc.2015.06.019 [PubMed]
34. Miranda JOF, Camargo CL de, et al. Clinical deterioration in hospitalised children: integrative review of a Paediatric Early Warning Score. J Nurs UFPE Online 2016;10:1128–36.
35. Monaghan A. Detecting and managing deterioration in children. Paediatr Nurs 2005;17:32–5. [PubMed]
36. Murray JS, Williams LA, Pignataro S et al. An integrative review of Paediatric Early Warning System Scores. Pediatr Nurs 2015;41:165–74. [PubMed]
37. Nielsen KR, Migita R, Batra M et al. Identifying high risk children in the emergency department. J Intensive Care Med 2015. doi:10.1177/0885066615571893 [Epub ahead of print: 10 Feb 2015] doi:10.1177/0885066615571893 [PubMed]
38. Niu X, Tilford B, Duffy E et al. Feasibility and reliability of Pediatric Early Warning Score in the emergency department. J Nurs Care Qual 2016;31:161–6. doi:10.1097/NCQ.0000000000000162 [PubMed]
39. Parshuram CS, Hutchinson J, Middaugh K Development and initial validation of the Bedside Paediatric Early Warning System score. Crit Care 2009;13:R135 doi:10.1186/cc7998 [PMC free article] [PubMed]
40. Parshuram CS, Duncan HP, Joffe AR et al. Multicentre validation of the bedside paediatric early warning system score: a severity of illness score to detect evolving critical illness in hospitalised children. Crit Care 2011;15:R184 doi:10.1186/cc10337 [PMC free article] [PubMed]
41. Parshuram CS, Bayliss A, Reiner J et al. Implementing the Bedside Paediatric Early Warning System in a community hospital: a prospective observational study. Paediatr Child Health 2011;16:e18–22. doi:10.1093/pch/16.3.e18 [PMC free article] [PubMed]
42. Parshuram CS, Dryden-Palmer K, Farrell C et al. Evaluating processes of care and outcomes of children in hospital (EPOCH): study of protocol for a randomised controlled trial. Trials 2015;16:245 doi:10.1186/s13063-015-0712-3 [PMC free article] [PubMed]
43. Rahman ZH, Leahy NE, Sessler K et al. A pilot study to validate the Burn Centre Pediatric Early Warning Score tool in clinical practice. J Burn Care Res 2016;37:160–5. doi:10.1097/BCR.0000000000000306 [PubMed]
44. Roland D, Madar J, Connolly G The Newborn Early Warning (NEW) system: development of an at-risk infant intervention system. Infant 2010;6:116–20.
45. Roland D, Oliver A, Edwards ED et al. Use of paediatric early warning systems in Great Britain: has there been a change of practice in the last 7 years? Arch Dis Child 2014;99:26–9. doi:10.1136/archdischild-2012-302783 [PubMed]
46. Roland D, Lewis G, Fielding P et al. The paediatric observation priority score: a system to aid detection of serious illness and assist in safe discharge. Open J Emerg Med 2016;4:38–44. doi:10.4236/ojem.2016.42006
47. Sefton G, McGrath C, Tume L et al. What impact did a Paediatric Early Warning system have on emergency admissions to the paediatric intensive care unit? An observational cohort study. Intensive Crit Care Nurs 2015;31:91–9. doi:10.1016/j.iccn.2014.01.001 [PubMed]
48. Seiger N, Maconochie I, Oostenbrink R et al. Validity of different pediatric early warning scores in the emergency department. Pediatrics 2013;132:e841–9. doi:10.1542/peds.2012-3594 [PubMed]
49. Sinitsky L, Reece A Question 2: Can PEWS predict serious clinical deterioration in paediatric inpatients? Arch Dis Child 2016;101:109–13. doi:10.1136/archdischild-2015-309304 [PubMed]
50. Skaletzky SM, Raszynski A, Totapally BR, Validation of a modified pediatric early warning system score: a retrospective case-control study. Clin Pediatr 2012;51:431–5. doi:10.1177/0009922811430342 [PubMed]
51. Solevag AL, Eggen EH, Schroder J et al. Use of a modified pediatric early warning score in a department of pediatric and adolescent medicine. PLoS ONE 2013;8:e72534 doi:10.1371/journal.pone.0072534 [PMC free article] [PubMed]
52. Tume L. The deterioration of children in ward areas in a specialist children's hospital. Nurs Crit Care 2007;12:12–9. [PubMed]
53. Zhai H, Brady P, Li Q et al. Developing and evaluating a machine learning based algorithm to predict the need of pediatric intensive care unit transfer for newly hospitalized children. Resuscitation 2014;85:1065–71. doi:10.1016/j.resuscitation.2014.04.009 [PMC free article] [PubMed]
54. Bonafide CP, Localio R, Roberts KE et al. Impact of rapid response system implementation on critical deterioration events in children. JAMA Pediatr 2014;168:25–33. doi:10.1001/jamapediatrics.2013.3266 [PubMed]
55. Bonafide CP, Roberts KE, Priestley MA et al. Development of a pragmatic measure for evaluating and optimizing rapid response systems. Pediatrics 2012;129:e874–81. doi:10.1542/peds.2011-2784 [PubMed]
56. Brilli RJ, Gibson R, Luria JW et al. Implementation of a medical emergency team in a large pediatric teaching hospital prevents respiratory and cardiopulmonary arrests outside the intensive care unit. Pediatr Crit Care Med 2007;8:236–46. doi:10.1097/01.PCC.0000262947.72442.EA [PubMed]
57. Chan PS, Jain R, Nallmothu BK et al. Rapid response teams: a systematic review and meta-analysis. Arch Intern Med 2010;170:18–26. doi:10.1001/archinternmed.2009.424 [PubMed]
58. Chen JG, Kemper AR, Odetola F et al. Prevalence, characteristics, and opinions of pediatric rapid response teams in the United States. Hosp Pediatr 2012;2:133–40. doi:10.1542/hpeds.2011-0030 [PubMed]
59. Dean BS, Decker MJ, Hupp D et al. Condition HELP: a pediatric rapid response team triggered by patients and parents. J Healthc Qual 2008;30:28–31. doi:10.1111/j.1945-1474.2008.tb01139.x [PubMed]
60. Hanson CC, Randolph GD, Erickson JA et al. A reduction in cardiac arrests and duration of clinical instability after implementation of a paediatric rapid response system. Postgrad Med J 2010;86:314–8. doi:10.1136/qshc.2007.026054 [PubMed]
61. Haque A, Saleem AF, Zaidi S et al. Experience of pediatric rapid response team in a tertiary care hospital in Pakistan. Indian J Pediatr 2010;77:273–6. doi:10.1007/s12098-010-0032-2 [PubMed]
62. Heath G, Montgomery H, Eyre C et al. Developing a tool to support communication of parental concerns when a child is in hospital. Healthcare 2016;4:pii: E9 doi:10.3390/healthcare4010009 [PMC free article] [PubMed]
63. Hueckel RM, Mericle JM, Martin PL et al. Implementation of condition help family teaching and evaluation of family understanding. J Nurs Care Qual 2012;27:176–81. doi:10.1097/NCQ.0b013e318235bdec [PubMed]
64. Humphreys S, Totapally BR Rapid Response Team calls and unplanned transfers to the pediatric intensive care unit in a pediatric hospital. Am J Crit Care 2016;25:e9–13. doi:10.4037/ajcc2016329 [PubMed]
65. Hunt EA, Zimmer KP, Rinke ML et al. Transition from a traditional code team to a medical emergency team and categorization of cardiopulmonary arrests in a children's center. Arch Pediatr Adolesc Med 2008;162:117–22. doi:10.1001/archpediatrics.2007.33 [PubMed]
66. Kotsakis A, Lobos AT, Parshuram C et al. Implementation of a multicenter rapid response system in pediatric academic hospitals is effective. Pediatrics 2011;128:72–8. doi:10.1542/peds.2010-0756 [PubMed]
67. Lobos AT, Fernandes R, Ramsay T et al. Team (MET) activation: disposition depends on who activates the team patient characteristics and disposition after pediatric medical emergency. Hosp Pediatr 2014;4:99–105. doi:10.1542/hpeds.2013-0032 [PubMed]
68. Lobos AT, Fernandes R, Willams K et al. Routine Medical Emergency Team assessments of patients discharged from PICU: description of a Medical Emergency Team follow up program. Pediatr Crit Care Med 2015;16:359–65. doi:10.1097/PCC.0000000000000354 [PubMed]
69. Paciotti B, Roberts KE, Tibbetts KM et al. Physician attitudes toward family-activated medical emergency teams for hospitalised children. Jt Comm J Qual Patient Saf 2014;40:187–92. doi:10.1016/S1553-7250(14)40024-2 [PubMed]
70. Panesar R, Polikoff LA, Harris D et al. Characteristics and outcomes of pediatric rapid response teams before and after mandatory triggering by an elevated pediatric early warning system (PEWS) score. Hosp Pediatr 2014;4:135–40. doi:10.1542/hpeds.2013-0062 [PubMed]
71. Ray EM, Smith R, Massie S et al. Family alert: implementing direct family activation of a pediatric rapid response team. Jt Comm J Qual Patient Saf 2009;35:575–80. [PubMed]
72. Sen AI, Morgan RW, Morris MC Variability in the implementation of rapid response teams at academic American pediatric hospitals. J Pediatr 2013;163:1772–4. doi:10.1016/j.jpeds.2013.07.018 [PubMed]
73. Sharek PJ, Parast LM, Leong K et al. Effect of a rapid response team on hospital-wide mortality and code rates outside the ICU in a children's hospital. JAMA 2007;298:2267–74. doi:10.1001/jama.298.19.2267 [PubMed]
74. Theilen U, Leonard P, Jones P et al. Regular in situ simulation training of paediatric Medical Emergency Team improves hospital response to deteriorating patients. Resuscitation 2013;84:218–22. doi:10.1016/j.resuscitation.2012.06.027 [PubMed]
75. Tibballs J, Kinney S, Duke T et al. Reduction of paediatric in-patient cardiac arrest and death with a medical emergency team: preliminary results. Arch Dis Child 2005;90:1148–52. doi:10.1136/adc.2004.069401 [PMC free article] [PubMed]
76. Tibballs J, Kinney S Reduction of hospital mortality and of preventable cardiac arrest and death on introduction of a pediatric medical emergency team. Pediatr Crit Care Med 2009;10:306–12. doi:10.1097/PCC.0b013e318198b02c [PubMed]
77. VandenBerg SD, Hutchison JS, Parshuram CS A cross-sectional survey of levels of care and response mechanisms for evolving critical illness in hospitalized children. Pediatrics 2007;119:940–6. doi:10.1542/peds.2006-0852 [PubMed]
78. Van Voorhis KT, Willis TS Implementing a pediatric rapid response system to improve quality and patient safety. Pediatric Clin North Am 2009;56:919–33. doi:10.1016/j.pcl.2009.05.017 [PubMed]
79. Wang GS, Erwin N, Zuk J et al. Retrospective review of emergency response activations during a 13-year period at a tertiary care children's hospital. J Hosp Med 2011;6:131–5. doi:10.1002/jhm.832 [PubMed]
80. Winberg H, Nilsson K, Aneman A Paediatric rapid response systems: a literature review. Acta Anaesthesiol Scand 2008;52:890–6. doi:10.1111/j.1399-6576.2008.01672.x [PubMed]
81. Zenker P, Schlesinger A, Hauck M et al. Implementation and impact of a rapid response team in a children's hospital. Jt Comm J Qual Patient Saf 2007;33:418–25. doi:10.1016/S1553-7250(07)33048-1 [PubMed]
82. Azzopardi P, Kinney S, Moulden A et al. Attitudes and barriers to a Medical Emergency Team system at a tertiary paediatric hospital. Resuscitation 2011;82:167–74. doi:10.1016/j.resuscitation.2010.10.013 [PubMed]
83. Bonafide PC, Roberts KE, Weirich CM et al. Beyond statistical prediction: qualitative evaluation of the mechanisms by which pediatric early warning scores impact patient safety. J HospMed 2013;8:248–53. [PubMed]
84. Bonafide CP, Localio AR, Song L et al. Cost-benefit analysis of a medical emergency team in a children's hospital. Pediatrics 2014;134:235–41. doi:10.1542/peds.2014-0140 [PubMed]
85. Brady PW, Goldenhar LM A qualitative study examining the influences on situation awareness and the identification, mitigation and escalation of recognised patient risk. BMJ Qual Saf 2014;23:153–61. doi:10.1136/bmjqs-2012-001747 [PubMed]
86. Brady PW, Muething S, Kotagal U et al. Improving situation awareness to reduce unrecognised clinical deterioration and serious safety events. Pediatrics 2013;131:e298–308. doi:10.1542/peds.2012-1364 [PMC free article] [PubMed]
87. Demmel KM, Williams L, Flesch L Implementation of the pediatric early warning scoring system on a pediatric Hematology/Oncology Unit. J Pediatr Oncol Nurs 2010;27:229–40. doi:10.1177/1043454209358410 [PubMed]
88. Duncan HP, Frew E Short-term health system costs of paediatric in-hospital acute life-threatening events including cardiac arrest. Resuscitation 2009;80:529–34. doi:10.1016/j.resuscitation.2009.02.018 [PubMed]
89. Hayes LW, Dobyns EL, DiGiovine B et al. A multicenter collaborative approach to reducing pediatric codes outside the ICU. Pediatrics 2012;129:785–91. doi:10.1542/peds.2011-0227 [PubMed]
90. Kukreti V, Gaiteiro R, Mohseni-Bod H Implementation of a pediatric rapid response team: experience of the hospital for sick children in Toronto. Indian Pediatr 2014;51:11–15. [PubMed]
91. Lobos AT, Costello J, Gilleland J et al. An implementation strategy for a multicentre pediatric rapid response system in Ontario. Jt Comm J Qual Patient Saf 2010;36:271–80. doi:10.1016/S1553-7250(10)36043-0 [PubMed]
92. McCrory MC, Aboumatar H, Custer JW et al. ‘ABC-SBAR’ training improves simulated critical patient hand-off by pediatric interns. Pediatr Emerg Care 2012;28:538–43. doi:10.1097/PEC.0b013e3182587f6e [PubMed]
93. McKay H, Mitchell IA, Sinn K et al. Effect of a multifaceted intervention on documentation of vital signs and staff communication regarding deteriorating paediatric patients. J Paediatr Child Health 2013;49:48–56. doi:10.1111/jpc.12019 [PubMed]
94. McLellan M, Connors JA The cardiac children's hospital early warning score (C-CHEWS). J Pediatr Nurs 2013;28:171–8. doi:10.1016/j.pedn.2012.07.009 [PubMed]
95. Randhawa S, Roberts-Turner R, Woronick K et al. Implementing and sustaining evidence-based nursing practice to reduce pediatric cardiopulmonary arrest. West J Nurs Res 2011;33:443–56. doi:10.1177/0193945910379585 [PubMed]
96. Roberts KE, Bonafide CP, Paine CW et al. Barriers to calling for urgent assistance despite a comprehensive pediatric rapid response system. Am J Crit Care 2014;23:223–9. doi:10.4037/ajcc2014594 [PubMed]
97. Tume LN, Sefton G, Arrowsmith P Teaching paediatric ward teams to recognise and manage the deteriorating child. Nurs Crit Care 2014;19:196–203. doi:10.1111/nicc.12050 [PubMed]
98. Hueckel RM, Turi JL, Cheoifetz IM et al. Beyond Rapid Response Teams: Instituting a ‘Rover Team’ Improves the Management of At-Risk Patients, Facilitates Proactive Interventions, and Improves Outcomes. http://www.ahrq.gov/professionals/quality-patient-safety/patient-safety-resources/resources/advances-in-patient-safety-2/vol3/advances-hueckel_27.pdf (accessed 20 Sep 2016).
99. Deighton J, Edbrooke-Childs J, Stapley E et al. Realistic evaluation of situation awareness for everyone (SAFE) on paediatric wards: study protocol. BMJ Open 2016;6:e014014 doi:10.1136/bmjopen-2016-014014 [PMC free article] [PubMed]
100. Hignett S, Lang A, Pickup L et al. More holes than cheese: what prevents the delivery of effective, high quality and safe healthcare in England? Ergonomics 2016: doi:10.1080/00140139.2016.1245446 [Epub ahead of print: 20 Oct 2016] doi:10.1080/00140139.2016.1245446 [PubMed]
101. Hignett S, Jones EL, Miller D Human factors and ergonomics and quality improvement science: integrating approaches for safety in healthcare. BMJ Qual Saf 2015;24:250–4. doi:10.1136/bmjqs-2014-003623 [PMC free article] [PubMed]
102. Hood K, Powell C, Allen D et al. Review of Paediatric Early Warning Systems (PEWS) and scores for clinical deterioration of children in hospital: their development and validation, effectiveness and factors associated with implementation and generative mechanisms. PROSPERO 2015:CRD42015015326. http://www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42015015326 (accessed 3 Jan 2017).
103. PUMA HS&DR—12/178/17: PUMA—Paediatric early warning system (PEWS): Utilisation and Mortality Avoidance. A prospective, mixed methods, before and after study identifying the evidence base for the core components of an effective PEWS and the development of an implementation package for implementation and use in the UK. http://www.nets.nihr.ac.uk/projects/hsdr/1217817 (accessed 10 Sep 2016).

Articles from BMJ Open are provided here courtesy of BMJ Publishing Group