In the initial assessment of the protocol development process, the project team determined that the total processing time per protocol (actual physical time needed to do the tasks) was 4 to 25 days, the total waiting time (for responses or paper documents) was 174 days, and the total lead time (the time it takes to deal with one protocol, start to finish, including all steps) was 177 to 199 days. The value-added ratio (percentage of time that is actually productive) was 2% to 12.6%, and “first-time quality” was 0%, meaning that 100% of the protocols required rework.
Analyze Phase Outcomes
In the first step of the value stream map, it was determined that the protocol development coordinator spent significant effort reviewing the protocol received from the PI. Because of the lack of a standardized template, the coordinator spent 1 to 9 hours ensuring consistency and accuracy within the document itself. Beyond this, an average of 15 days of wait time accumulated during communication exchanges between the coordinator and PI.
Next, the coordinator or assistant spent a highly variable 1.5 to 24 hours developing three documents and proceeded to forward those on to three different groups for parallel processing: consent form sent to PI, budget proposal sent to Office of Sponsored Projects, and contract checklist sent to Legal Contracts Administration.
After 10 days waiting for responses, the coordinator or assistant proceeded to the third processing step, at which time the protocol was sent along with the consent form and budget for peer review, departmental review, and funding agency review. In addition, the processor prepared other regulatory documents within this step, including the financial disclosure form and conflict of interest form. In all, this step amounted to 3.5 to 8 hours of processing time.
After 30 days of waiting for responses from review groups, the coordinator or assistant spent 6 to 24 hours incorporating changes into the protocol. The final protocol was then sent to the funding agency, requiring 0.05 hours of processing time. At this point, the coordinator arranged a meeting with Data Management Services to develop the appropriate data capture mechanisms for the study. This meeting occurred, on average, 28 days later and required an additional 18 to 127 hours of processing time to prepare, discuss, and refine the data plan. Finally, in the final step of the value stream map, the coordinator completed the electronic IRB application within 1.5 to 8 hours of processing time.
Of the 199 days required to complete the process, 174 of these days are considered non–value-added. These days are spent with tasks waiting in a queue or undergoing rework. The first-time quality was 0%, meaning that every protocol experienced rework at some point in the process. In addition, the processing time varied widely because of the lack of standardization.
During this phase, a future state value stream map was created (). In the future state, there are two parallel processes: the upper lane belongs to the protocol development coordinator, whereas the lower lane belongs to the data management systems analyst, who begins concurrently rather than sequentially because of the multiple changes in processing each protocol. In the upper lane, the first box (subprocess) includes the tasks that can take a place concurrently and are not dependent on another; processing time for those steps was estimated to be 12 to 18 hours. Also, errors from upstream are targeted to be reduced to 0 through standardization and training.
Fig 2. Future state value stream map for the protocol development process. The figure shows all the activities the protocol development process should go through, with the time to process each activity. The stars indicate “kaizens,” tasks that (more ...)
In terms of batch size, this process deals with one protocol at a time, and this map kept track of the staff involved to make sure the PDC has enough time to process the volume. The second box represents several approvals that have to take a place for the protocol to continue. Because any involved committee meets at least once every 2 weeks, all approvals should happen concurrently within those 2 weeks. Although processing time will not exceed 2 hours, there will be 2 weeks of unavoidable non–value-added time as a result of meeting logistics. As protocols become approved by those committees, the protocol development coordinator is able to finalize the protocol. It is expected that this will take up to 6 hours.
The map defines steps and time frames to take a protocol from an approved concept to a document ready for submission to the approval body. Calculations indicate that this process should take 3 to 4 weeks, with a processing time of no more than 1 week for externally authored protocols and only 10 weeks for internally authored protocols. This decrease in processing time would increase the value-added ratio to 30% for all protocols facilitated, including cooperative groups.
Implementation of the new design began in August 2007. A measurement system was developed using Microsoft Access to report valid and reliable data. The system is currently used by processors to record the completion of each step to enable ongoing monitoring of process efficacy. In addition, teams continue to contribute to the kaizens and are pursuing completion of those projects. Communication and education tools were developed and shared with key stakeholders.
Outcome of Redesign
Preimprovement and postimprovement data were collected to measure mean turnaround time (TAT), which is the time from the start of protocol development until initial IRB/NCI submission. Data were collected on Microsoft Excel spreadsheets and imported into SAS version 9 (SAS Institute, Cary, NC) for analysis. P values were generated using two-sample t tests (variances unequal).
Preimprovement data were collected between July 2003 and March 2007, and postimprovement data were collected between August 2007 and August 2008. Mean TAT for internally authored protocols improved (P < .001) from 25.00 weeks (n = 41) to 10.15 weeks (n = 14; ). In addition, a significant difference was detected in variance (P = .009), with preimprovement protocols ranging from 3.43 to 94.14 weeks and postimprovement protocols ranging from 4.00 to 22.14 weeks ().
Mean turnaround time (TAT) for internally authored studies. A significant difference in the mean TAT was observed between the two groups (P < .001).
Fig 4. Boxplot of the turnaround time for each internally authored study in the analysis. Although distributions for both groups were normal, a significant difference was observed in the variances (P = .009), suggesting a more controlled postimprovement process. (more ...)
Mean TAT for externally authored protocols improved (P < .001) from 20.61 weeks (n = 85) to 7.79 weeks (n = 50; ). As in internally authored protocols, the variance in TAT for protocol development improved considerably (P = .0002). Preimprovement TAT ranged from 3.29 to 108.57 weeks, whereas postimprovement TAT ranged from 2.00 to 20.86 weeks (). Improvements in TAT for both internally authored and externally authored protocols were achieved without increasing staffing levels within the PDU.
Mean turnaround time (TAT) for externally authored studies. A significant difference in the mean TAT was observed between the two groups (P < .001).
Boxplot of the turnaround time for each externally authored study in the analysis. Again, distributions for both groups were normal but a significant difference was observed in the variances (P = .0002).
The data are notable both as evidence of successes in bringing novel therapies to patients more quickly as well as of creating a process that is controlled and predictable. The improvements shown in variances of TAT allow the investigator as well as the myriad of involved parties, committees, and agencies the ability to predict more precisely the amount of time required for protocol development. This affords them the ability to begin planning and scheduling for the other aspects of protocol activation (approvals, logistics, enrollment, and so on).