We offer here a framework that provides distinct advantages for evaluation of translational research and can either be used by itself or can be applied to current or prospective multi-phase models to help translate between and among them, regardless of the number or type of “T’ phases in use. The process marker approach advocated here is both an operationally precise way to structure a systematic evaluation of translational interventions and a complementary methodology that can enhance research and scholarship on the nature of various models of translational research processes that exist, perhaps in phases, at different times and places.
This framework might be termed a process marker model, characterized by the two components that constitute its name. First, it views translational research as a continuous process that moves from basic research through clinical, post-clinical and practice-based research and ultimately to health policies, outcomes and impacts. It assumes that this process may be bidirectional, variable and complex and that any particular discovery may follow a unique pathway through the process. Second, it assumes that there are many different potential markers along this process. The focus in this model is on identifying a set of observable points in the process that can be operationally defined and measured, in order to enable to evaluation of the duration of segments of the research-practice continuum.
For instance, in the three-phase Westfall et al. (2007)
model described earlier, they describe T1 research as “translation to humans” and use as examples both case series and phase 1 and 2 clinical trials. In the model offered here, we might operationally define a process marker such as “date of first accrual of a human subject into a research protocol investigating the target treatment”. Note that the operational process marker is a specific time point in the presumed process of translation, in this case expressed as a date. It is clear what the marker refers to – the accrual of the first human subject in the first research study that employs the treatment in humans. The central point is that the operational marker is one that can be readily understood and measured since it is based on an action or behavior.
The process marker model assumes that one would define a number of such operational markers along a presumed process continuum. Assuming that all of the markers use a common measurement scale (e.g., dates) it is then relatively easy to operationally define the difference between any two markers as the duration of time between their dates. For instance, later in their model Westfall et al. (2007)
define T2 as the “translation to patients” and give as examples “guideline development, meta-analyses or systematic reviews”. In our framework we might set up a marker such as “date of inclusion of the results of a research study of the target treatment in a meta-analysis.” Or, even more specifically we could define it as “date of publication of the results of a research study of the target treatment in a Cochrane Collaboration meta-analysis.” Then, to determine the amount of time it took to translate the research in question from the first marker to the second one would need only to subtract the two dates.
One of the obvious questions raised by this approach is what is the “correct” operational marker to use? A simple answer is that there isn’t a single correct marker – different markers simply represent different reference points in an assumed continuous process. That said, some markers may be better than others for different purposes and in different contexts. For example, some potential markers may be more feasible to measure. We could define an alternative marker for the introduction of a discovery in humans as “the date the first research protocol for use of the target treatment in humans was submitted for human subjects review.” It may or may not be easier to measure this than our earlier suggested marker of date of first accrual. Records of IRB submissions may or may not be easier to obtain than dates of subject accrual, and the marker that is more feasibly measured will generally be more useful for evaluation.
Another consideration is that some markers are likely to be encountered by more protocols than others. Since protocols can take different pathways in the process of translation one would generally want to select markers that are more likely to be commonly passed. For example, it may be that for some discoveries they typically will have research publications (date of first publication of results of a human trial) but may not generate a patentable intervention (date of submission of first patent application). In general using markers that are more likely to be encountered by more protocols will enhance our ability to explore empirically what factors affect durations for that part of the process.
A third consideration is that there are likely to be subprocesses that get repeated throughout the overall translational process. For instance, the subprocess of conducting, replicating and using a research study (shown in ) follows the same basic steps regardless of whether it is a basic, clinical or post-clinical study. In some process analyses it might be useful to look at the durations between two steps in this subprocess, say from application to funding, across all instances, regardless of where in the translational process the study is done. In other analyses it is may be necessary to separate the results based on whether the studies are basic, clinical or practice-based. In this critical repeating subprocess it’s important to recognize that each step has the potential to be influenced by strategies embodied in the CTSAs and other organizations involved in biomedical research.
Generic subprocess of a research study.
How does the operational process marker model deal with the issue of directionality? It would be possible to define operational markers that can be encountered moving in either direction (“date a basic research study that is based on a clinical finding is initiated”). Furthermore, if between two markers the durations have great variability this might suggest that there are sub-processes and perhaps even bi-directional loops that occur and might warrant further study.
There are a number of characteristics that commend the process marker framework over multi-phase models:
- It is pragmatic in that it avoids theoretical presumptions and undefined abstractions. It should help the field focus greater attention on how we can practically measure and improve translation.
- This model is an objective one in that it emphasizes observable measureable phenomena, allowing anyone to readily see how any marker is defined.
- The model is conceptually clear. It avoids the debates about how many phases there are in translational research, while enabling evaluators to use phased-based approaches (and even translate between multiple models) as long as they operationally define what they mean. For instance, if one wants to classify a portfolio of research for where it is on a general multi-phase model you could do so better using an operationally-defined process marker approach than with any of the existing multiphase models.
- Because key markers are expected to be defined operationally, this model encourages replicability. It encourages the community of evaluators to look at and adopt others’ markers once the have been demonstrated to be feasible.
- The model is robust and forgiving. If a particular discovery does not pass a particular marker one can simply find a subsequent marker that it does hit and again pick up the trail. In other words, missing data and variable protocol pathways can be accommodated.
- The process marker model will encourage development of new hypotheses that involve more precise operational definitions. For example, in current CTSA pilot work on IRB processes the first marker defined in the IRB process was the date of submission of a complete human subjects application and the end marker was the date of final approval without any contingencies. The results indicated that different centers had markedly different median durations. This led to a more refined subsequent hypothesis that some centers may do considerable work with researchers prior to submitting their IRB application. This in turn suggested that to measure this segment of the process well in subsequent evaluations we need to set up markers prior to the submission of the IRB application.
- The model avoids debates about the scope of translational research. For instance, there is disagreement about when translation starts. Does it begin with the genesis of the idea in basic research, with the first basic studies that anticipate human applicability, with the first study involving humans, and so on? This model is silent on such questions. The scope of translation being examined in any given process marker evaluation is simply the process that is encompassed between the first and last marker measured.
- The process marker model can be applied prospectively or retrospectively. For instance, we could use it to conduct historical analyses of the durations involved to translate research to practice in the same way that others have done (Balas & Boren, 2000; Contopoulos-Ioannidis, et al., 2008; Westfall, et al., 2007). And it can be used prospectively when setting up evaluation monitoring systems of translation in progress.
The process marker model is firmly rooted in a process modeling research tradition in biomedical research (Balas & Boren, 2000
; Dilts et al., 2009
) as well as in other fields like quality control and assurance. These traditions have begun to be applied in the context of translational research generally and in the CTSAs in particular. For instance, the authors are aware of process studies for segments of the translational process that include the time it takes to: apply for a pilot grant; accomplish an IRB review; complete contract negotiations for a research protocol; accrue subjects into existing protocols; and even the time it takes for a research publication to be included in a research synthesis. As more such studies are completed across the research-practice continuum we should be able to get a clearer sense of how the overall translational process occurs, where the major barriers are, and how effective different interventions are in addressing these barriers.
To help concretize the idea of a process marker model, we present an example of how it might look in . Before discussing it, some caveats are in order. We don’t pretend to offer this as the process marker model as if there can only be one correct one. Any such model is simply an imperfect representation of some presumed underlying process. And because such a process is assumed to be continuous, it would always be possible to detail more precise sub-processes. The articulation of good process models will be an ongoing endeavor that will need to extend beyond the scope of this paper and involve a broad range of stakeholders and subject matter experts across disciplines and “translational phases”. The common value of such models is that they ultimately depend on operationally definable marker variables. If the model is wrong about the underlying process, other researchers will be able to pose alternative models while still utilizing the findings of the model they are criticizing.
Examples of process marker models at three levels of scale.
shows an example of a multi-level process marker model. In the bottom box of the figure is a very high-level process model of the research-practice translational continuum. On the left is the general region of basic research. The center shows the region typically associated with clinical trials research. And the right side depicts applied clinical research, translation to practice and policy, and ultimately use in populations and the health of the public. There are a multitude of operational markers that might be constructed across this continuum, and the ones included in are depicted primarily for illustrative purposes. Attempts to study the entire duration of translation could utilize early and late-stage markers and follow studies throughout this course, but because of the length of time involved, this is only likely to be feasible for retrospective historical studies of the type mentioned earlier. Prospectively the more feasible course would be to follow protocols over a comparatively short segment of the model such as illustrated in the middle of the figure in the breakout of a Phase II clinical trial process. The durations for the segment described there – typical of the process sequence for almost any basic or clinical research study – could be readily tracked and estimated. By assembling multiple such pieces across the continuum we can begin to gather contemporary data on translation that would be critical for evaluating interventions designed to reduce translational time. The Phase II clinical trial breakout in turn is further specified in the top box of the figure in a breakout detailing the segment of the process that involves IRB approval. Again, this sequence is a general one that would apply here and in all clinical trials.
Each vertical “pin” in the figure represents an operationally-definable marker. Each of these markers can be operationalized as a specific date. In the translational research continuum at the bottom we identify a specific marker at the date when a Phase II clinical trial began. In the more detailed breakout of this segment of the model in the middle box we see that this is in turn operationalized as the date of first accrual of a subject into the trial. We also see that in the process segment for the Phase II trial we have a marker for IRB approval. This in turn is operationalized in the top topmost IRB approval process segment as the date the IRB proposal was approved.
The hypothetical process marker model offered here illustrates several important features of this approach. It shows how we can evaluate translational research at any level of scale from the overarching basic research to health impacts scope to the assessment of the length of any segment or sub-segment. It suggests that such studies should be a major feature of the evaluation of translational research and that it would be possible to integrate the results of many such studies at a macro or meta level, essentially stitching together duration estimates of segments to arrive at a more integrated understanding of how long the process takes.
Perhaps the most important feature of this model is that it provides a foundation for the evaluation of interventions designed to improve translational research and the integration of these findings into a field of translational studies. Our expectation is that the CTSAs and many others there will be experimenting with a wide range of interventions to enhance translation: improved efficiency of IRB and contract review processes, better clinical research management, new methods for interdisciplinary team research, and so on. The process marker model provides a common framework that can link these many and varied studies together, a common basis for assessing whether such interventions contribute to reducing time to translation.
There are some important challenges that need to be addressed with respect to the operational process marker framework. Perhaps most obvious is that it will lead to more complex and difficult to communicate models of translational research. It is much easier to describe translational research in broader terms that are not operationally defined. The more precisely stated operational definitions are cumbersome. Nevertheless, they provide distinct advantages from an evaluation and research perspective and they help to address the conceptual confusions present in the current literature.
Another major challenge is analytic in nature. Much of the process modeling literature relies upon descriptive statistics – such as median durations – as the heart of the results. But an inferential statistical analysis framework would both enable one to ask whether process changes lead to statistically significant improvements in translation and whether there are statistically significant variables that predict translational duration. Several possible analytic frameworks seem worth investigating. For instance, many of the analyses are likely to be hierarchical in nature. Research protocols may be nested within CTSA centers. Or research publications may be nested within fields or disciplines. Consequently, a general framework like hierarchical linear modeling (Raudenbush & Bryk, 2002
) may be applicable for testing hierarchical hypotheses using process data. Alternatively we can view process data on durations using a survival analysis framework (Cox, 1972
), sometimes referred to as a Cox proportional hazard model that would enable us to provide statistical inferences regarding how long protocols “survive” or stay in various process intervals.
Increased application of an operational process marker approach to the study of translational research is likely to lead to considerable evolution and adaption over time. The field will be able to determine empirically the degree to which various markers are feasible to measure and yield results that have value for our understanding of translation. Over time it is likely that a set of markers will emerge across the research-practice continuum that have survived the test of repeated application. Such a set of measures would be critical to establishing a basis for the emerging field of translational research.