|Home | About | Journals | Submit | Contact Us | Français|
Noncompliance with radiotherapy (RT) protocol guidelines has been linked to inferior clinical outcomes. We performed a meta-analysis of cooperative group trials to examine the association between RT quality assurance (QA) deviations and disease control and overall survival (OS).
We searched MEDLINE and the Cochrane Central Register of Controlled Trials for multi-institutional trials that reported clinical outcomes in relation to RT QA results. Hazard ratios (HRs) describing the association between RT protocol noncompliance and patient outcomes were extracted directly from the original studies or calculated from survival curves. Inverse variance meta-analyses were performed to assess the association between RT QA deviations and OS. A second meta-analysis tested the association between RT QA deviations and secondary outcomes, including local or locoregional control, event-free survival, and relapse. Random-effects models were used in cases of statistically significant (P < .10) effect heterogeneity. The Egger test was used to detect publication bias. All statistical tests were two-sided.
Eight studies (four pediatric, four adult) met all inclusion criteria and were incorporated into this analysis. The frequency of RT QA deviations ranged from 8% to 71% (median = 32%). In a random-effects model, RT deviations were associated with a statistically significant decrease in OS (HR of death = 1.74, 95% confidence interval [CI] = 1.28 to 2.35; P < .001). A similar effect was seen for secondary outcomes (HR of treatment failure = 1.79, 95% CI = 1.15 to 2.78; P = .009). No evidence of publication bias was detected.
In clinical trials, RT protocol deviations are associated with increased risks of treatment failure and overall mortality.
Quality improvement is an essential process in the field of medicine. On an individual level, adherence to quality assurance (QA) measures can help ensure the delivery of safe and effective care for each patient. Quality control is also critical for measuring the effectiveness of medical advances: QA measures are used to standardize therapy in clinical trials and ensure that the intended study questions are addressed (1).
In the field of radiation oncology, the reliable delivery of high-quality therapy and verification that such treatment has been delivered poses unique challenges (2,3). Cooperative research groups have progressively incorporated QA procedures into clinical radiotherapy (RT) protocols over the past 40 years (4). Some modern trials, for example, require that RT plans pass centralized review before study therapy begins. These measures have been shown to minimize treatment deviations (5,6) and may increase the likelihood that the study question can be answered (7). Although it seems intuitive that rigorous RT QA would improve patient outcomes, the benefits of such measures have not been established (8).
Several recent secondary analyses of clinical trials have demonstrated that RT QA deviations may be independent predictors of poor outcomes (9,10). We performed a meta-analysis of cooperative group trials to examine the association between RT QA deviations and clinical outcomes and estimate the magnitude of these effects.
We searched MEDLINE citations on February 7, 2012, for the terms “radiotherapy,” “quality assurance,” and “survival.” We did not apply any filter for publication date. We also searched the Cochrane Central Register of Controlled Trials for the terms “radiotherapy” and “quality assurance.” Publications that described clinical outcomes in multi-institutional clinical trials in relation to RT QA results were included in this analysis. Articles that detailed QA techniques but did not relate QA results to time-to-event outcome data were excluded. When more than one publication was identified from the same clinical trial, we used the most recent or most complete report of that trial.
Data abstraction was conducted by the lead investigator (N. Ohri) according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement (11). For each study, we extracted the following information: name of the first author, year of publication, name of the clinical trial, study question, number of enrolled patients, number of patients included in QA analysis, definition of RT protocol deviation, and numbers of patients with and without RT deviations. The primary clinical outcome of interest for this analysis was overall survival (OS). Data for secondary outcomes, such as local control or event-free survival, were also recorded.
Hazard ratios (HRs) for time-to-event data (eg, OS, event-free survival) were extracted directly from the original studies or were estimated indirectly from the survival curves, as suggested by Parmar et al. (12). Data extraction from survival curves and estimations of effect sizes were performed using customized scripts in Matlab version 7.13 (The Mathworks, Natick, MA). The 95% confidence interval (CI) for each hazard ratio was extracted directly from the original report if available or estimated as described by Parmar et al. (12). All calculations were performed using study-level data because individual patient data were not available for this analysis. Because only two studies reported multivariable analyses examining the association between RT deviations and clinical outcomes, we used unadjusted hazard ratios.
Meta-analyses were performed using the inverse variance method. Separate analyses were performed for OS and for the secondary outcomes, which were grouped together. For each meta-analysis, we calculated Cochran’s Q, which is a classical measure of heterogeneity of effect sizes across trials (13). The assumption of homogeneity was considered invalid for P values less than .10 (a conservative cutoff commonly used in meta-analyses). This prompted the use of the random-effects model to derive summary estimates for hazard ratios and 95% confidence intervals. Publication bias was evaluated visually with funnel plots and statistically as described by Egger et al. (14). A two-tailed P value of less than .10 was considered statistically significant.
Our initial searches yielded 600 results (523 from MEDLINE and 77 from the Cochrane Central Register of Controlled Trials). A total of 59 publications were found in both searches; thus there were 541 unique publications. After reviewing each abstract, 12 candidates for meeting our eligibility criteria were identified. After reading those 12 full-text articles, one paper (15) was excluded because updated results of the same trial were reported in another publication (16). A secondary analysis of data from five clinical trials was excluded because it did not report the impact of RT compliance on clinical outcomes (17). A third study was excluded because it only reported QA results for patients who experienced disease recurrence (18). A fourth was excluded because it did not report the association between RT QA results and clinical outcomes using time-to-event data (19). Trial selection results are depicted in Figure 1.
Eight studies met all inclusion criteria and were incorporated into this analysis (Table 1). These included two lung cancer trials (20,21), three trials for medulloblastoma or supratentorial primitive neuroectodermal tumor (16,22,23), and one trial each for Ewing sarcoma (24), pancreatic cancer (9), and head and neck cancer (10). In all but one trial (10), RT QA reviews were performed in a post hoc fashion. Six studies (9,10,16,20,21,23) reported the influence of RT QA deviations on OS. Six studies (9,10, 16,22–24) reported the effects of RT QA deviations on secondary endpoints, including local or locoregional control (10,24), event-free survival (16,23), and relapse (9,22). Various definitions of RT deviations were employed by the included studies and are summarized in Table 1. The frequency of RT deviations ranged from 8% to 71% (median = 32%).
Meta-analysis results describing the association between RT deviations and OS are displayed in Figure 2. The Cochran Q statistic using a fixed-effects model was 19.7 (P = .001), prompting the use of a random-effects model. Using the random-effects model, RT deviations were associated with a statistically significant decrease in OS (HR = 1.74, 95% CI = 1.28 to 2.35). No evidence of publication bias was detected using the Egger test (P = .36).
Figure 3 depicts the association between RT deviations and secondary outcomes. When a fixed-effects model was used, the Cochran Q statistic was 16.8 (P = .005). We consequently performed a random-effects meta-analysis (Figure 2). Using that model, RT deviations were associated with statistically significantly worse secondary outcomes (HR = 1.79, 95% CI = 1.15 to 2.78). There was no evidence of publication bias using the Egger test (P = .47).
This analysis of published reports relating RT protocol deviations to patient outcomes in cooperative group clinical trials revealed that in the majority of cases, failure to meet RT QA measures was associated with inferior outcomes. Our meta-analysis demonstrated that RT protocol deviations occurred in 8% to 71% of cases and were associated with an approximately 75% increase in the risk of treatment failure and overall mortality. The magnitude of these effect sizes suggests that the delivery of high-quality RT is critical for the successful execution of clinical trials and for effective treatment of cancer patients.
Adherence to RT protocol guidelines is an important practical concern in the conduct of clinical trials because RT quality may influence the interpretability of study results. For example, many clinicians believe that identification of the optimal treatment approach following surgery for pancreatic cancer has been hindered by poor RT quality, outdated RT schedules, and a lack of RT quality control in landmark randomized trials (25). Data from Radiation Therapy Oncology Group (RTOG) 97-04 (a study included in this analysis) link adherence to RT guidelines in the adjuvant treatment of pancreatic cancer with improved disease control and OS (9). This finding is particularly striking because it has been difficult to prove that adjuvant RT itself is beneficial in the adjuvant setting.
The importance of incorporating QA measures into multi-institutional trials has been recognized for some time. Findings from a 2010 National Cancer Institute conference entitled “Methods and Issues for Redesigning Clinical Trial Quality Assurance in Radiotherapy” were recently published (8). Conference participants made four recommendations: 1) develop a tiered system and tailor intensity of QA to clinical trial objectives; 2) establish a case QA repository; 3) develop an evidence base for clinical trial QA; and 4) explore the feasibility of consolidating clinical trial QA in the United States. We agree with these recommendations, and we believe that our findings support the practice of incorporating real-time QA measures into clinical RT trials. As more advanced RT technologies are incorporated into clinical trials, the resources required to maintain a rigorous QA program will increase substantially (26). Consequently, judicious implementation of QA measures that are both effective and practical poses a challenge to current and future trialists.
Our findings may have implications outside the clinical trials arena. The effect sizes reported in this study surpass those typically seen in successful trials of novel treatment strategies. Although we acknowledge that the association between RT QA results and clinical outcomes may not be entirely causal, our findings suggest that the implementation of rigorous QA procedures may provide clinically significant benefits for all RT patients, including the vast majority of cancer patients who are not enrolled in clinical trials. This may be a worthy subject for future prospective study.
The definition, frequency, and clinical significance of RT QA deviations may all vary with treatment technique. The vast majority of the studies included in this analysis utilized two-dimensional RT. Three-dimensional RT planning was allowed in the most recent studies (9,10), but none included patients treated with intensity-modulated RT. Although the applicability of our findings to newer RT techniques is unclear, we believe that rigorous RT QA becomes even more critical as treatment complexity increases.
The delivery of RT is a complicated, multistep process. This analysis, however, focused solely on deviations that occurred during treatment planning. In the studies we examined, RT deviations were scored based on treatment fields and/or RT plans that were generated before the initiation of therapy. Subsequent changes in patient set-up or modifications to the treatment delivery were not accounted for. This practice is consistent with the QA procedures typically employed in modern RT trials. In certain situations, specialized imaging tools (27) or implantable dosimeters (28) may improve our ability to verify that the intended RT plan is actually delivered.
This study has several limitations that should be considered when interpreting our results. The QA data incorporated into this meta-analysis were generally collected retrospectively and were usually available for only a subset of patients from each study. Multivariable analyses examining the statistical significance of RT deviations along with other prognostic factors were reported in only a few papers. The definitions of RT deviations varied across studies, and bias could clearly have led authors to choose definitions that were statistically significantly correlated with outcomes. In addition, publication bias may have influenced our findings; “negative” studies that failed to link RT protocol noncompliance with inferior outcomes may have been performed within cooperative groups but never published. Although we did not find any statistical evidence of publication bias, our sensitivity to detect such an effect was limited by our sample size. Finally, the strong association between RT deviations and clinical outcomes may not truly represent causation; digression from protocol guidelines may be related to unfavorable features within the patient (eg, poor health or large tumor size) or the treatment team (eg, lack of experience).
In conclusion, in clinical trials, RT protocol deviations are associated with increased risk of treatment failure and overall mortality. The magnitude of these effect sizes indicates that rigorous RT QA is an important component of the clinical trial process. Additionally, these findings raise the possibility that strict adherence to RT planning guidelines might benefit all cancer patients who are treated with RT.
This research was supported in part by a grant from the National Cancer Institute Cancer Center Support (P30 CA56036).
The study sponsor did not play any role in the design of the study, the collection, analysis, or interpretation of the data, the writing of the study, or the decision to submit the study for publication. None of the authors have any conflicts of interest that are relevant to this publication.