Logic and progression
Translational science in health professions education progresses from the simulation laboratory to patient outcomes in three seamless phases. Health professions education translational science research at the T1 level, the most common form of such research, involves the design and delivery of education protocols and measurement of educational outcomes in controlled laboratory settings. Translational science research at the T2 level stretches the endpoint beyond the T1 laboratory setting. Better patient care delivery practices (e.g., ACLS, complicated obstetrical delivery) are the goal of T2 translational science. Translational science research at the T3 level advances the measurement endpoint further downstream. The target of T3 translational science is improved patient or public health outcomes (e.g., faster surgical recovery) directly linked to educational interventions (e.g., skillful laparoscopic surgery).
Most of health related simulation education and research has measured T1 learning outcomes. There are many good reasons for this limited focus. Simultaneous pressures of high volume training demands, work overload, insufficient staff, tight budgets, and research inexperience hinder researchers from reporting results beyond the laboratory exit door. The body of reliable T2 and T3 knowledge will no doubt expand in breadth and depth as the scientific field matures, research resources increase, and as staff research competence grows from training and experience.
The key point about the logic and progression of translational science research is that it stems from a thematic, sustained, and cumulative research program. Effective translational science is not a product of stand alone, one-shot studies. Instead, productive translational science requires thoughtful, continuous research planning that addresses education and health services research outcomes. It is usually performed best by multi-disciplinary, functionally diverse research teams composed of clinicians, research methodologists, reference librarians, and other professionals.
Evidence that TSR can yield T2 and T3 outcomes and beyond
An example of a translational science research program
is seen in the work of a team headed by Northwestern University internist Jeffrey Barsuk. This team developed a SBME program featuring mastery learning and deliberate practice to increase internal medicine and emergency medicine residents’ skills of central venous catheter (CVC) insertion. T1 outcomes were evaluated in a cohort study that compared internal jugular and subclavian CVC insertion skills of simulation-trained versus traditionally trained residents (historical controls) in a laboratory setting. SBME produced much more skillful residents than traditional bedside training.10
In a subsequent T2 study, Barsuk et al. showed that residents who received internal jugular and subclavian SBME inserted CVCs in the medical intensive care unit with significantly fewer needle passes, catheter adjustments, arterial punctures, and with higher success rates than traditionally trained residents (historical controls).11
Barsuk et al. then conducted a before/after observational study in the medical intensive care unit on the incidence of catheter-related bloodstream infections over 32 months. This team reported a 85% reduction in catheter related bloodstream infections after the SBME trained residents entered the intervention intensive care unit (0.50 infections per 1000 catheter days) compared to both the same unit before the intervention (3.20 infections per 1000 catheter days, p
= .001) and to a comparison intensive care unit in the same hospital throughout the study period (5.03 infections per 1000 catheter days, p
= .001). This is a powerful T3 translational science research outcome.12
A subsequent cost-effectiveness study by the Barsuk team demonstrates the SBME intervention resulted in significant medical care cost savings, a 7-1 return on investment, an index of treatment value.13
The Barsuk research group has also reported that CVC insertion skills acquired in the simulation laboratory under mastery learning conditions are largely retained and are robust to decay measured 6 months and 12 months downstream.14
However, residents whose performance did not meet mastery standards on follow-up measurement could not be predicted from prior assessments. This suggests that periodic testing and refresher training for invasive procedures like CVC insertion are needed.
The Barsuk et al. T1 to T3 and treatment value and retention translational science results link improved patient outcomes, health care cost return on investment, and skill retention directly to the SBME intervention featuring mastery learning and deliberate practice.
Other illustrations of translational science research programs that lead to T3 outcomes can also be cited although they are few in number. Obstetrician Tim Draycott and colleagues have reported a series of simulation-based translational science studies in the United Kingdom that demonstrate individual and team training in obstetric emergencies produce statistically and clinically significant reductions in birth complications due to shoulder dystocia (brachial plexus injury), low Apgar scores (< 6 on a 1-10 scale) 5 min. after birth, and neonatal hypoxic-ischemic encephalopathy (HIE), a brain injury caused by lack of oxygen.15-17
These are all T3 outcomes.
Opthalmologists at the University of Iowa have developed and implemented an intense, structured surgical curriculum for resident trainees that includes “[a] structured wet lab and simulator training during the first year, [b] backing into cases of senior residents during the first year, [c] formative feedback during the second year, and [d] deliberate practice of capsulorhexis during the second year. In a before/after study these investigators compared sentinel events—“a posterior capsule tear (with or without vitreous loss) or vitreous loss (from any cause)”—from surgical cases of residents trained before the surgical curriculum change versus cases of residents trained with the enhanced curriculum. The results show, “There was a statistically [and clinically] significant reduction in the sentinel complication rate, from 7.17% before the curriculum changes to 3.77% with the enhanced curriculum (P = .008, unpaired 2-tailed t
Better surgical skills acquired by the residents from the enhanced curriculum (T2 outcomes) produced a measureable reduction in sentinel complications among patients during cataract surgery (T3 outcomes).
A selected set of 15 research reports that address SBME translational science research outcomes at the downstream T2 level is summarized in . The reports are representative, not exhaustive. They are presented here to demonstrate that medical education research outcomes can have a direct impact on medical care practices. A skillful healthcare workforce is a key variable in the patient safety and healthcare outcomes equation.
SBME Transfer to Patient Care Practices (T2)
Best practices in translational science research
Quality translational science research is grounded in a set of basic principles that shape best practices in the enterprise. The basic principles include the previously stated need to establish thematic, sustained, and cumulative research programs; an emphasis on methodological and measurement rigor; unification of educational and health services research; and a constant search for opportunities to link SBME interventions with better patient care practices and patient and public health outcomes.
These principles do not dictate a specific set of research practices that define the scope and boundaries of acceptable translational science. Instead, we endorse the idea that decisions about research methods and measurement procedures must be fitted to specific research questions. Quality translational science does not imply methodological “gold standards.” It seeks instead to use the best available research approach to each research question in its clinical and professional context.
A wide variety of experimental and quasi-experimental designs are available to conduct rigorous translational science research.34-35
The designs vary in strength on grounds of internal, external, statistical conclusion, and construct validity;35
require investigators to make educated research design decisions; and often involve tradeoffs among rigor and the practical realities of the research situation.
Shadish, Cook, and Campbell teach that quantitative research designs involve decisions about at least five research variables that form the UTOST model.35
Investigators need to decide on the unit (U)
of analysis [e.g., individual, team, class]; treatment (T)
imposed [e.g., SBME intervention]; observations (O)
made [e.g., measurements]; setting (S)
where the research is conducted [e.g., laboratory, clinic, OR]; and time (T)
[e.g., duration of the study, follow-up, etc.]. UTOST
decisions should be made with thought and care to increase the likelihood of informative, useful translational science research results.
Kirlik reminds TSR investigators to be mindful about the ecological validity of research. Ecological validity aims to insure that clinical problems and contexts
, not just research participants, are sampled systematically to capture the variety and richness of clinical experience.36
Ecological validity is achieved when the methods, materials, and setting of a study closely approximate the situation that is under investigation.
Rigorous measurement of translational science research processes and outcomes is essential for research success. At a minimum research measures must yield reliable data at each translational science endpoint to establish solid connections for every link of the cascaded research chain.37
The reliability of data should be estimated and reported in the context of every translational science research study, not assumed from earlier research and publications. Scholars remind translational science investigators that reliable measurement is a necessary but not a sufficient condition to allow valid inferences or decisions from research data.38-40
The granularity versus breadth of research data exerts a strong influence on their reliability and usefulness. This concerns the persistent “fidelity versus bandwidth” or breadth of measurement issue that has vexed scientists for at least 50 years.41-42
Most performance measures that are used for SBME research are task specific, unambiguous, and aim to produce data having high reliability (fidelity). However, there are research situations where more general performance measures having greater bandwidth are needed. Translational scientists are urged to make tradeoff decisions about the fidelity versus bandwidth of research measurements to boost data reliability within practical limits so that valid judgments and decisions can be reached.
Challenges and critical gaps in TSR
There are many challenges involved in conducting translational science, especially when studies aim to affect T3 outcomes. The challenges highlight critical gaps in our thinking about translational science research and its methodological drivers. We choose to name five key challenges.
- Translational scientists need to identify clinical problems (e.g., CRBSI, neonatal HIE) whose solutions depend on the decision-making and skillful behavior of health professionals—individuals and teams. SBME interventions featuring mastery learning and deliberate practice can then be designed and implemented to sharpen clinical decision-making and boost clinical skills. Embedding the educational interventions within rigorous research protocols using outcome measures that yield reliable data allows translational scientists to address the clinical problems directly and improve patient or public health.
- Measuring and capturing clinical data that reliably represent distal (T3) outcomes is a persistent translational science challenge. Clinical data contained in hospital and clinic records are notoriously unreliable. Thus the research utility of most clinical data collected and stored routinely is suspect. Recent advances in developing sophisticated data storage facilities at academic medical centers are attempts to solve the clinical data reliability problem that deserve increased effort. For example, the Northwestern University Enterprise Data Warehouse (EDW), created under auspices of the Northwestern University Clinical and Translational Sciences (NUCATS) Institute “… is a single, integrated database of all clinical and research data from all patients receiving treatment throughout Northwestern healthcare affiliates. Consolidating this wealth of [reliable] data into a single database maximizes efficiency and centralizes security, making data available yet controlling access to assure consistency with consents and regulatory requirements.”43
- Translational scientists should acknowledge that some important clinical problems cannot be studied beyond the T2 level. Such clinical problems as end-of-life cancer care, ALS, and refractory addictions test the limits of translational science research that seeks T3 outcomes.
- The biomedical and educational translational science research community needs to educate health policy and research thought leaders that health professions education can have a measureable impact on patient and public health outcomes. In the U.S., for example, health research policies recently published by the Institute of Medicine (IOM) under the title, Knowing What Works in Health Care: A Roadmap for the Nation44 and research priorities published by the Agency for Healthcare Research and Quality (AHRQ)45 are silent about the contribution of health professions education to healthcare delivery. By contrast, we assert that human capital, embodied in competent physicians and other healthcare professionals, is an essential feature of quality healthcare that should also be a translational science research priority.5
- The biomedical and educational translational science research community should exercise leadership in healthcare by arguing that clinical performance competency standards for healthcare professionals are needed, but remarkably absent. Improved standards of care will be realized when SBME measures that yield reliable data are used to reach increasingly valid competency decisions about healthcare professionals. Translational science research that approximates T3 goals contributes to creation of such safety-based metrics.46
Translational science research agenda
This short report does not permit detailed discussion of the broad and deep translational science research agenda. Other writing provides a modest start toward this goal.47
The corpus of papers and vigorous discussion presented at the 2011 IMSH Research Consensus Summit reported in this issue of Simulation in Healthcare
begins to set forth such an agenda on behalf of the Society for Simulation in Healthcare. Consequently, in addition to the translational science research ideas and challenges presented earlier in this report we choose to add two items to the research agenda.
- Translational science research in the healthcare professions should not only address the acquisition and maintenance of procedural skills but also such thorny research targets as clinical judgment, decision-making, mental workload,48 comparative and reflective analysis,49 and other cognitive and affective outcomes. The ability to engage a family in a difficult conversation about end-of-life issues is a clinical skill amenable to SBME just like inserting a chest tube. The translational science research agenda should make room for investigations about a broad array of clinical and educational outcomes.
- Rigorous quantitative research, which accounts for the majority of translational science in healthcare, is complemented by qualitative research50 that is also judged using rigorous standards.51 Research based on words rather than numbers can inform and enrich our understanding of complex clinical events.