|Home | About | Journals | Submit | Contact Us | Français|
Inconsistencies and errors resulting from nonstandard processes, together with redundancies, rework, and excess workload, lead to extended time frames for clinical trial protocol development. This results in dissatisfaction among sponsors, investigators, and staff and restricts the availability of novel treatment options for patients.
A team of experts from Mayo Clinic formed, including Protocol Development Unit staff and management from the three Mayo Clinic campuses (Florida, Minnesota, and Arizona), a systems and procedures analyst, a quality office analyst, and two physician members to address the identified deficiencies. The current-state process was intensively reviewed, and improvement steps were taken to accelerate the development and approval of cancer-related clinical trials. The primary goal was to decrease the time from receipt of a new protocol through submission to an approving authority, such as the National Cancer Institute or institutional review board.
Using the Define, Measure, Analyze, Improve, Control (DMAIC) framework infused with Lean waste-reduction methodologies, areas were identified for improvement, including enhancing first-time quality and processing new studies on a first-in/first-out basis. The project was successful in improving the mean turnaround time for internally authored protocols (P < .001) from 25.00 weeks (n = 41; range, 3.43 to 94.14 weeks) to 10.15 weeks (n = 14; range, 4.00 to 22.14 weeks). The mean turnaround time for externally authored protocols was improved (P < .001) from 20.61 weeks (n = 85; range, 3.29 to 108.57 weeks) to 7.79 weeks (n = 50; range, 2.00 to 20.86 weeks).
DMAIC framework combined with Lean methodologies is an effective tool to structure the definition, planning, analysis, and implementation of significant process changes.
Nearly 1.5 million new cancer cases are diagnosed each year in the United States, excluding most noninvasive cancers and approximately 1 million basal and squamous cell skin cancers.1 Although improvements in detection and treatments have significantly improved 5-year survival rates over the past three decades, more than half a million people die of cancer each year in America.1 Development and activation of clinical trials remains a central component in efforts to improve treatment options and survival rates for patients with cancer.
Bringing new and innovative clinical trials to patients is a major goal of academic centers. Most cancers still are not curable, and patients are often best served by enrolling onto clinical trials that investigate new therapeutic options. Academic cancer centers must be efficient with protocol development processes to be cost-responsible and to attract and develop the most important clinical trials for our patients.
Protocol development, especially for cancer clinical trials, is an increasingly complex process, with multiple approval and regulatory issues. Newly written protocols require approval from physician specialists in a particular disease area who are familiar with the underlying science and rationale. Then, they require review and approval by a protocol review committee that includes unbiased, expert cancer researchers from other disease groups and, ultimately, the institutional review board (IRB). These reviews are required to fulfill regulatory obligations and requirements of funding institutions, especially the National Cancer Institute (NCI). In addition, the protocol requires review by peer reviewers and allied health personnel with expertise in informed consent development, nursing, adverse event reporting, clinical pharmacology, and budgetary and legal issues.
During a trial's development and after its activation, modifications are often made. Delays during this process are frequent, because modifications typically occur in a sequential rather than in a parallel process. Delays also occur as a result of issues such as collaborations with multiple industry sponsors (with sometimes conflicting views) on a single clinical trial, detailed or highly technical laboratory logistics (intake, accessioning, and tracking of biospecimens), or facilitating protocol development when the principal investigator (PI) operates from a separate physical location. As a result, it can take more than 1 year for a protocol to reach the IRB for approval from the time it is first submitted by the PI to a protocol development office. The processing time can be shortened substantially if the precise steps required for protocol development are carefully analyzed and redundancies and inefficiencies addressed.
To address these obstacles and the current challenges of protocol development, a team of experts from Mayo Clinic formed to begin a detailed review and revision of the process using Six Sigma. The Six Sigma improvement process is well established and has been used in a variety of health care settings.2–5 It has also been used to evaluate institutional processes.6
The Protocol Development Unit (PDU) at Mayo Clinic is a division of the Mayo Clinic Cancer Center Clinical Research Office and provides support through the enterprise network of Mayo within Arizona, Florida, and Minnesota. The PDU facilitates the development and approval processes for all Mayo Clinic cancer research clinical trials from the point of concept approval through obtaining approval from regulatory entities (including, but not limited to, the Mayo Clinic IRB) to trial activation and patient accrual. The PDU is responsible for ensuring that study content, regulatory submissions (eg, to the United States Food and Drug Administration, IRB, or NCI), and contractual and budget information are complete and accurate. This involves approximately 200 new trials annually and the management of more than 1,800 trials in progress, open to enrollment, and active to follow-up.
The Quality Management Services department at Mayo Clinic provided resources, including a quality improvement advisor to guide the project through the DMAIC methodology. DMAIC is an acronym for five interconnected phases of a Six Sigma improvement process: Define, Measure, Analyze, Improve, and Control.7 This data-driven method was chosen by Quality Management Services as the central methodology for Mayo Clinic's quality improvement initiative because of its proven capability to improve business processes in several industries (ie, automotive, banking, and so on).
The Define phase yielded a comprehensive project charter that defined the problem, tied the project to institutional directives, and clearly stated the project goal. Within the charter, the team documented its focus on accelerating the translation of discovery to patient care through standardizing, streamlining, and eliminating rework in the process of clinical trial protocol development. The goal of the initiative was to reduce the protocol development time from delivery of the clinical trial draft to a protocol development coordinator until submission for approval to the institutional review board (IRB) from the existing average of 25 weeks to 10 weeks or less for internally authored protocols and from an existing average of 21 weeks to 4 weeks or less for externally authored protocols. The charter was vetted and approved by the project sponsor (chair of Mayo Foundation Research Administration) and the relevant physician advisors and operations administrators.
On completion of the Define phase, the project progressed to the Measure phase, in which a 2-day workshop was conducted to describe and time individual tasks of the process in its current state. The team members and other content experts timed each step of the process to gather metrics for this phase of the DMAIC project. The main goal was to identify opportunities for improvement by comparing the time needed to do the work versus the time the process actually took.
On transitioning from the Measure phase into the Analyze phase, the team constructed the current state value stream map (Fig 1), which showed great opportunity for improvement. The team used the descriptive value stream mapping tool as a method to define areas of waste in the process. A “value stream” is the set of all activities, both value-added and non–value-added, required to bring the service from the start of the process to delivery to the customer.8
During the Improve phase, the team held a planning retreat and defined potential changes to the existing process. These ideas were categorized as “kaizens,” a Japanese term commonly used within the Six Sigma methodology meaning “to take apart and make good.”9 Particular examples included (1) creating standardized templates for protocols to be used by PIs, statisticians, and others involved in the process; (2) standardizing coding process tools; (3) creating a phased work plan for the PDU with designated time frames of completion; (4) enhancing training programs; and (5) expediting IRB review, with minimal requirements for externally authored protocols that have already received a central IRB approval. Each of the kaizens was assigned to a content expert to develop, refine, and implement the specific improvement. An additional activity during the Improve phase was the creation of the future state value stream map. The future state is what the team would be working to achieve.
The project is currently in the Control phase, and the team is collecting data to validate the improvements. Our data suggest that the future state timelines are achievable, as illustrated in the results below.
In the initial assessment of the protocol development process, the project team determined that the total processing time per protocol (actual physical time needed to do the tasks) was 4 to 25 days, the total waiting time (for responses or paper documents) was 174 days, and the total lead time (the time it takes to deal with one protocol, start to finish, including all steps) was 177 to 199 days. The value-added ratio (percentage of time that is actually productive) was 2% to 12.6%, and “first-time quality” was 0%, meaning that 100% of the protocols required rework.
In the first step of the value stream map, it was determined that the protocol development coordinator spent significant effort reviewing the protocol received from the PI. Because of the lack of a standardized template, the coordinator spent 1 to 9 hours ensuring consistency and accuracy within the document itself. Beyond this, an average of 15 days of wait time accumulated during communication exchanges between the coordinator and PI.
Next, the coordinator or assistant spent a highly variable 1.5 to 24 hours developing three documents and proceeded to forward those on to three different groups for parallel processing: consent form sent to PI, budget proposal sent to Office of Sponsored Projects, and contract checklist sent to Legal Contracts Administration.
After 10 days waiting for responses, the coordinator or assistant proceeded to the third processing step, at which time the protocol was sent along with the consent form and budget for peer review, departmental review, and funding agency review. In addition, the processor prepared other regulatory documents within this step, including the financial disclosure form and conflict of interest form. In all, this step amounted to 3.5 to 8 hours of processing time.
After 30 days of waiting for responses from review groups, the coordinator or assistant spent 6 to 24 hours incorporating changes into the protocol. The final protocol was then sent to the funding agency, requiring 0.05 hours of processing time. At this point, the coordinator arranged a meeting with Data Management Services to develop the appropriate data capture mechanisms for the study. This meeting occurred, on average, 28 days later and required an additional 18 to 127 hours of processing time to prepare, discuss, and refine the data plan. Finally, in the final step of the value stream map, the coordinator completed the electronic IRB application within 1.5 to 8 hours of processing time.
Of the 199 days required to complete the process, 174 of these days are considered non–value-added. These days are spent with tasks waiting in a queue or undergoing rework. The first-time quality was 0%, meaning that every protocol experienced rework at some point in the process. In addition, the processing time varied widely because of the lack of standardization.
During this phase, a future state value stream map was created (Fig 2). In the future state, there are two parallel processes: the upper lane belongs to the protocol development coordinator, whereas the lower lane belongs to the data management systems analyst, who begins concurrently rather than sequentially because of the multiple changes in processing each protocol. In the upper lane, the first box (subprocess) includes the tasks that can take a place concurrently and are not dependent on another; processing time for those steps was estimated to be 12 to 18 hours. Also, errors from upstream are targeted to be reduced to 0 through standardization and training.
In terms of batch size, this process deals with one protocol at a time, and this map kept track of the staff involved to make sure the PDC has enough time to process the volume. The second box represents several approvals that have to take a place for the protocol to continue. Because any involved committee meets at least once every 2 weeks, all approvals should happen concurrently within those 2 weeks. Although processing time will not exceed 2 hours, there will be 2 weeks of unavoidable non–value-added time as a result of meeting logistics. As protocols become approved by those committees, the protocol development coordinator is able to finalize the protocol. It is expected that this will take up to 6 hours.
The map defines steps and time frames to take a protocol from an approved concept to a document ready for submission to the approval body. Calculations indicate that this process should take 3 to 4 weeks, with a processing time of no more than 1 week for externally authored protocols and only 10 weeks for internally authored protocols. This decrease in processing time would increase the value-added ratio to 30% for all protocols facilitated, including cooperative groups.
Implementation of the new design began in August 2007. A measurement system was developed using Microsoft Access to report valid and reliable data. The system is currently used by processors to record the completion of each step to enable ongoing monitoring of process efficacy. In addition, teams continue to contribute to the kaizens and are pursuing completion of those projects. Communication and education tools were developed and shared with key stakeholders.
Preimprovement and postimprovement data were collected to measure mean turnaround time (TAT), which is the time from the start of protocol development until initial IRB/NCI submission. Data were collected on Microsoft Excel spreadsheets and imported into SAS version 9 (SAS Institute, Cary, NC) for analysis. P values were generated using two-sample t tests (variances unequal).
Preimprovement data were collected between July 2003 and March 2007, and postimprovement data were collected between August 2007 and August 2008. Mean TAT for internally authored protocols improved (P < .001) from 25.00 weeks (n = 41) to 10.15 weeks (n = 14; Fig 3). In addition, a significant difference was detected in variance (P = .009), with preimprovement protocols ranging from 3.43 to 94.14 weeks and postimprovement protocols ranging from 4.00 to 22.14 weeks (Fig 4).
Mean TAT for externally authored protocols improved (P < .001) from 20.61 weeks (n = 85) to 7.79 weeks (n = 50; Fig 5). As in internally authored protocols, the variance in TAT for protocol development improved considerably (P = .0002). Preimprovement TAT ranged from 3.29 to 108.57 weeks, whereas postimprovement TAT ranged from 2.00 to 20.86 weeks (Fig 6). Improvements in TAT for both internally authored and externally authored protocols were achieved without increasing staffing levels within the PDU.
The data are notable both as evidence of successes in bringing novel therapies to patients more quickly as well as of creating a process that is controlled and predictable. The improvements shown in variances of TAT allow the investigator as well as the myriad of involved parties, committees, and agencies the ability to predict more precisely the amount of time required for protocol development. This affords them the ability to begin planning and scheduling for the other aspects of protocol activation (approvals, logistics, enrollment, and so on).
Mayo Clinic serves as the research base for several NCI-funded multisite research groups (Chemoprevention Network, North Central Cancer Treatment Group, Phase 2 Consortium) and single-institutional activities (U01-supported Novel Therapeutics program, Specialized Programs of Research Excellence–sponsored clinical trials). It also serves as the research base for non–NCI-sponsored clinical research activities through the Mayo Clinic Cancer Research Consortium. Mayo thus offers a unique setting in which to conduct clinical trials. However, these clinical trial venues have been hindered by slow processes, particularly as viewed by pharmaceutical companies who indicate “time is money.” It is reported to cost approximately $1 billion dollars for pharmaceutical companies to bring an idea to market for a new drug.10 A 1-day delay in the development of a clinical trial for a drug that ultimately becomes successful costs the pharmaceutical company between $800 thousand and $5.4 million.11
Additionally, the budget allocated to the NCI has remained relatively flat since 2004 and is expected to remain so at least through 2009.12 This translates to a decrease in buying power and an effective decrease in federal funding for research when adjusted for the effects of inflation. In the current economic environment and increasingly global competition for clinical trials, increasing expenses limits the ability of NCI-sponsored trials to compete.
The development of clinical trials in the NCI-sponsored cooperative group system is complex and time consuming. In reviews of the development of phase III trials in the Eastern Cooperative Oncology Group and Cancer and Leukemia Group B, the number of distinct steps needed from concept approval to active trial ranged from 370 to 481.13,14 Given this complex process, it takes on average approximately 800 days to open a phase III trial.
Dilts and Sandler15 identify the problem of inefficient development of clinical trials owing to many process steps, the involvement of many groups, and an extremely low ratio of value-added steps. The Mayo Clinic Cancer Center recognized the problem outlined by Dilts and Sandler in its processes and embarked on an aggressive process improvement project to apply Lean/Six Sigma methodologies to protocol development, thereby mitigating a portion of current cost pressures.
The Mayo Clinic Cancer Center's process improvement shows that the development of clinical trial protocols can be greatly accelerated. Using the DMAIC framework, inefficiencies in the current system were identified and corrected, and rework was avoided through the development of a standard protocol template, which became required for all submissions to the PDU. The future state map estimates that the process should take no more than 10 weeks from the receipt of a protocol through official submission to an approving authority for internally written protocols and 4 weeks for externally written protocols. Indeed, in the initial review of this streamlined system, the process time decreased from 25 weeks to 10 weeks for internally authored protocols and from 21 weeks to 8 weeks for externally authored protocols.
Protocols using the new system are now approaching the goal of 4-week and 10-week time frame for areas within the control of the PDU. Using concurrent operations, defining a strategic plan for the staff with deadlines, deploying an electronic measurement and template system, working to equalize work distribution, and eliminating redundancies have been beneficial. However, continuous monitoring is required after any process improvement to ensure the gains are sustained.
On a broader basis, the Mayo Clinic Cancer Center process improvement has shown that significant progress can be shown in decreasing protocol development TATs through focused process engineering. Such improvements can be realized within an extremely short timeframe and with little to no incremental investment in administrative resources.
Recognizing the need to expand their improvement efforts beyond the timeline to IRB submission, the group commenced a second phase of the project at the beginning of 2009. This project, still using the DMAIC method, expands the scope of work to include the time frame from study concept through patient enrollment. The work will include measurements of each component of the process (ie, concept development, scientific review, IRB review, patient recruitment, and so on) and will examine the impact of process improvements on patient accrual, study completion, and publication.
Supported in part by grants from the Mayo Clinic Cancer Center (CA015083), Rochester, MN, and the North Central Cancer Treatment Group (CA25224).
Authors' disclosures of potential conflicts of interest and author contributions are found at the end of this article.
The author(s) indicated no potential conflicts of interest.
Conception and design: Terre A. McJoynt, Muhanad A. Hirzallah, Daniel V. Satele, Jason H. Pitzen, Steven R. Alberts, S. Vincent Rajkumar
Collection and assembly of data: Terre A. McJoynt, Daniel V. Satele, Jason H. Pitzen
Data analysis and interpretation: Terre A. McJoynt, Muhanad A. Hirzallah, Daniel V. Satele, Jason H. Pitzen, Steven R. Alberts, S. Vincent Rajkumar
Manuscript writing: Terre A. McJoynt, Muhanad A. Hirzallah, Daniel V. Satele, Jason H. Pitzen, Steven R. Alberts, S. Vincent Rajkumar
Final approval of manuscript: Terre A. McJoynt, Muhanad A. Hirzallah, Daniel V. Satele, Jason H. Pitzen, Steven R. Alberts, S. Vincent Rajkumar