|Home | About | Journals | Submit | Contact Us | Français|
A long and rich research legacy shows that under the right conditions, simulation-based medical education (SBME) is a powerful intervention to increase medical learner competence. SBME translational science demonstrates that results achieved in the educational laboratory (T1) transfer to improved downstream patient care practices (T2) and improved patient and public health (T3).
This is a qualitative synthesis of SBME translational science research that employs a critical review approach to literature aggregation.
Evidence from SBME and health services research programs that are thematic, sustained, and cumulative shows that measured outcomes can be achieved at T1, T2, and T3 levels. There is also evidence that SBME translational science research can yield a favorable return on financial investment and contributes to long-term retention of acquired clinical skills. The review identifies best practices in SBME translational science research, presents challenges and critical gaps in the field, and sets forth a translational science research agenda for SBME.
Rigorous SBME translational science research can contribute to better patient care and improved patient safety. Consensus conference outcomes and recommendations should be presented and used judiciously.
At least forty years of empirical research shows that simulation-based medical education (SBME) promotes learner acquisition and maintenance of clinical knowledge, attitudes, and skills. Thousands of individual research reports synthesized in five comprehensive reviews1-5 reveal that SBME is a powerful educational intervention to increase medical learner competence measured in the learning laboratory, during patient care delivery, and improves patient health outcomes measured quantitatively. The scientific legacy leaves no doubt that SBME technology works to achieve educational goals under the right conditions: (a) coupled with mastery learning and deliberate practice, (b) involving skillful faculty, (c) with curriculum integration and institutional endorsement, and (d) receiving healthcare system acceptance.
Translational science is usually defined as biomedical or biomedical engineering research designed to accelerate movement of results from the laboratory bench to the patient bedside. Translational science advances from bench to bedside in at least three seamless phases. T1 science aims to move basic laboratory discoveries to clinical research. T2 science aims to produce evidence of clinical effectiveness at the level of the patient; compare the success of different treatments to identify “the right treatment for the right patient in the right way at the right time;” and translate these results into practice guidelines for patients, clinicians, and policy-makers.6 T3 science addresses health care delivery, community engagement, and preventive services that yield measureable improvements in the health of individuals and society.6
Simulation-based medical education research qualifies as translational science when it stretches the outcome measurement endpoint just like its biomedical counterparts. SBME translational science demonstrates that results achieved in the educational laboratory (T1) transfer to improved downstream patient care practices (T2) and improved patient and public health (T3).7
This is a qualitative synthesis of SBME translational science research (TSR). The review is deliberately selective and critical, rather than exhaustive. It relies on Norman and Eva’s “critical review” approach to literature synthesis.8,9 Eva asserts, “A good educational literature review … is one that presents a critical synthesis of a variety of literatures, identifies knowledge that is well established, highlights gaps in understanding, and provides some guidance regarding what remains to be understood. The result should give a new perspective of an old problem … The author … should feel bound by a moral code to try to represent the literature (and the various perspectives therein) fairly, but need not adopt a guise of absolute systematicity.”9
This article covers five SBME translational science research issues that address distal outcomes involving better healthcare delivery (T2) and improved patient and public health (T3). The five translational science research issues are: (a) its logic and progression, (b) evidence that TSR can yield T2 and T3 outcomes, (c) best practices in translational science research, (d) challenges and critical gaps in TSR, and (e) the translational science research agenda. The report concludes with a summary and a brief discussion about the context of translational science research in health professions education and healthcare.
Translational science in health professions education progresses from the simulation laboratory to patient outcomes in three seamless phases. Health professions education translational science research at the T1 level, the most common form of such research, involves the design and delivery of education protocols and measurement of educational outcomes in controlled laboratory settings. Translational science research at the T2 level stretches the endpoint beyond the T1 laboratory setting. Better patient care delivery practices (e.g., ACLS, complicated obstetrical delivery) are the goal of T2 translational science. Translational science research at the T3 level advances the measurement endpoint further downstream. The target of T3 translational science is improved patient or public health outcomes (e.g., faster surgical recovery) directly linked to educational interventions (e.g., skillful laparoscopic surgery).
Most of health related simulation education and research has measured T1 learning outcomes. There are many good reasons for this limited focus. Simultaneous pressures of high volume training demands, work overload, insufficient staff, tight budgets, and research inexperience hinder researchers from reporting results beyond the laboratory exit door. The body of reliable T2 and T3 knowledge will no doubt expand in breadth and depth as the scientific field matures, research resources increase, and as staff research competence grows from training and experience.
The key point about the logic and progression of translational science research is that it stems from a thematic, sustained, and cumulative research program. Effective translational science is not a product of stand alone, one-shot studies. Instead, productive translational science requires thoughtful, continuous research planning that addresses education and health services research outcomes. It is usually performed best by multi-disciplinary, functionally diverse research teams composed of clinicians, research methodologists, reference librarians, and other professionals.
An example of a translational science research program is seen in the work of a team headed by Northwestern University internist Jeffrey Barsuk. This team developed a SBME program featuring mastery learning and deliberate practice to increase internal medicine and emergency medicine residents’ skills of central venous catheter (CVC) insertion. T1 outcomes were evaluated in a cohort study that compared internal jugular and subclavian CVC insertion skills of simulation-trained versus traditionally trained residents (historical controls) in a laboratory setting. SBME produced much more skillful residents than traditional bedside training.10 In a subsequent T2 study, Barsuk et al. showed that residents who received internal jugular and subclavian SBME inserted CVCs in the medical intensive care unit with significantly fewer needle passes, catheter adjustments, arterial punctures, and with higher success rates than traditionally trained residents (historical controls).11 Barsuk et al. then conducted a before/after observational study in the medical intensive care unit on the incidence of catheter-related bloodstream infections over 32 months. This team reported a 85% reduction in catheter related bloodstream infections after the SBME trained residents entered the intervention intensive care unit (0.50 infections per 1000 catheter days) compared to both the same unit before the intervention (3.20 infections per 1000 catheter days, p = .001) and to a comparison intensive care unit in the same hospital throughout the study period (5.03 infections per 1000 catheter days, p = .001). This is a powerful T3 translational science research outcome.12
A subsequent cost-effectiveness study by the Barsuk team demonstrates the SBME intervention resulted in significant medical care cost savings, a 7-1 return on investment, an index of treatment value.13 The Barsuk research group has also reported that CVC insertion skills acquired in the simulation laboratory under mastery learning conditions are largely retained and are robust to decay measured 6 months and 12 months downstream.14 However, residents whose performance did not meet mastery standards on follow-up measurement could not be predicted from prior assessments. This suggests that periodic testing and refresher training for invasive procedures like CVC insertion are needed.
The Barsuk et al. T1 to T3 and treatment value and retention translational science results link improved patient outcomes, health care cost return on investment, and skill retention directly to the SBME intervention featuring mastery learning and deliberate practice.
Other illustrations of translational science research programs that lead to T3 outcomes can also be cited although they are few in number. Obstetrician Tim Draycott and colleagues have reported a series of simulation-based translational science studies in the United Kingdom that demonstrate individual and team training in obstetric emergencies produce statistically and clinically significant reductions in birth complications due to shoulder dystocia (brachial plexus injury), low Apgar scores (< 6 on a 1-10 scale) 5 min. after birth, and neonatal hypoxic-ischemic encephalopathy (HIE), a brain injury caused by lack of oxygen.15-17 These are all T3 outcomes.
Opthalmologists at the University of Iowa have developed and implemented an intense, structured surgical curriculum for resident trainees that includes “[a] structured wet lab and simulator training during the first year, [b] backing into cases of senior residents during the first year, [c] formative feedback during the second year, and [d] deliberate practice of capsulorhexis during the second year. In a before/after study these investigators compared sentinel events—“a posterior capsule tear (with or without vitreous loss) or vitreous loss (from any cause)”—from surgical cases of residents trained before the surgical curriculum change versus cases of residents trained with the enhanced curriculum. The results show, “There was a statistically [and clinically] significant reduction in the sentinel complication rate, from 7.17% before the curriculum changes to 3.77% with the enhanced curriculum (P = .008, unpaired 2-tailed t test).” 18 Better surgical skills acquired by the residents from the enhanced curriculum (T2 outcomes) produced a measureable reduction in sentinel complications among patients during cataract surgery (T3 outcomes).
A selected set of 15 research reports that address SBME translational science research outcomes at the downstream T2 level is summarized in Table 1. The reports are representative, not exhaustive. They are presented here to demonstrate that medical education research outcomes can have a direct impact on medical care practices. A skillful healthcare workforce is a key variable in the patient safety and healthcare outcomes equation.
Quality translational science research is grounded in a set of basic principles that shape best practices in the enterprise. The basic principles include the previously stated need to establish thematic, sustained, and cumulative research programs; an emphasis on methodological and measurement rigor; unification of educational and health services research; and a constant search for opportunities to link SBME interventions with better patient care practices and patient and public health outcomes.
These principles do not dictate a specific set of research practices that define the scope and boundaries of acceptable translational science. Instead, we endorse the idea that decisions about research methods and measurement procedures must be fitted to specific research questions. Quality translational science does not imply methodological “gold standards.” It seeks instead to use the best available research approach to each research question in its clinical and professional context.
A wide variety of experimental and quasi-experimental designs are available to conduct rigorous translational science research.34-35 The designs vary in strength on grounds of internal, external, statistical conclusion, and construct validity;35 require investigators to make educated research design decisions; and often involve tradeoffs among rigor and the practical realities of the research situation.
Shadish, Cook, and Campbell teach that quantitative research designs involve decisions about at least five research variables that form the UTOST model.35 Investigators need to decide on the unit (U) of analysis [e.g., individual, team, class]; treatment (T) imposed [e.g., SBME intervention]; observations (O) made [e.g., measurements]; setting (S) where the research is conducted [e.g., laboratory, clinic, OR]; and time (T) [e.g., duration of the study, follow-up, etc.]. UTOST decisions should be made with thought and care to increase the likelihood of informative, useful translational science research results.
Kirlik reminds TSR investigators to be mindful about the ecological validity of research. Ecological validity aims to insure that clinical problems and contexts, not just research participants, are sampled systematically to capture the variety and richness of clinical experience.36 Ecological validity is achieved when the methods, materials, and setting of a study closely approximate the situation that is under investigation.
Rigorous measurement of translational science research processes and outcomes is essential for research success. At a minimum research measures must yield reliable data at each translational science endpoint to establish solid connections for every link of the cascaded research chain.37 The reliability of data should be estimated and reported in the context of every translational science research study, not assumed from earlier research and publications. Scholars remind translational science investigators that reliable measurement is a necessary but not a sufficient condition to allow valid inferences or decisions from research data.38-40
The granularity versus breadth of research data exerts a strong influence on their reliability and usefulness. This concerns the persistent “fidelity versus bandwidth” or breadth of measurement issue that has vexed scientists for at least 50 years.41-42 Most performance measures that are used for SBME research are task specific, unambiguous, and aim to produce data having high reliability (fidelity). However, there are research situations where more general performance measures having greater bandwidth are needed. Translational scientists are urged to make tradeoff decisions about the fidelity versus bandwidth of research measurements to boost data reliability within practical limits so that valid judgments and decisions can be reached.
There are many challenges involved in conducting translational science, especially when studies aim to affect T3 outcomes. The challenges highlight critical gaps in our thinking about translational science research and its methodological drivers. We choose to name five key challenges.
This short report does not permit detailed discussion of the broad and deep translational science research agenda. Other writing provides a modest start toward this goal.47 The corpus of papers and vigorous discussion presented at the 2011 IMSH Research Consensus Summit reported in this issue of Simulation in Healthcare begins to set forth such an agenda on behalf of the Society for Simulation in Healthcare. Consequently, in addition to the translational science research ideas and challenges presented earlier in this report we choose to add two items to the research agenda.
We conclude from this brief, critical review that SBME interventions, especially those that feature deliberate practice toward mastery learning goals, can achieve translational science research outcomes at the T2 and T3 levels. Such translational science research outcomes are more likely when SBME interventions are embedded in rigorous educational and health services research programs that are thematic, sustained, and cumulative. We are also aware that T3 health services research outcomes can be achieved without obvious educational interventions.52-55 Such work complements translational science research featuring SBME because both research models contribute to better patient care and improved patient safety.
This report, and other contributions to the SSH Research Consensus Summit, are intended to describe and evaluate the state of the science regarding the use of educational simulation in the healthcare professions. These are important and valuable goals. However we must be mindful about blind spots that can distort consensus conference proceedings and their conclusions. In particular, scholars warn that the assembly of consensus conference participants from like-minded people can be a source of selection bias.56 Selection bias, in turn, can contribute to confirmation bias, i.e., reaching conclusions that affirm a priori convictions or interests. This is not to discount the potential value of a consensus conference or its results. We only issue a word of caution about the certainty of the Research Consensus Summit conclusions.
Dr. McGaghie’s contribution was supported in part by the Jacob R. Suker, MD, professorship in medical education at Northwestern University and by grant UL 1 RR 025741 from the National Center for Research Resources, National Institutes of Health. The National Institutes of Health had no role in the preparation, review, or approval of the article.
The authors have no conflicts of interest to disclose.
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
William C. McGaghie, Center for Education in Medicine, Northwestern University Feinberg School of Medicine, Chicago, IL.
Timothy J. Draycott, Southmead Hospital, Bristol, UK.
William F. Dunn, College of Medicine, Mayo Clinic, Rochester, MN.
Connie M. Lopez, Kaiser Permanente Program Offices, Oakland, CA.
Dimitrios Stefanidis, Department of Surgery, Carolinas Healthcare System, Charlotte, NC.