|Home | About | Journals | Submit | Contact Us | Français|
The National Cancer Institute’s (NCI) Community Clinical Oncology Program (CCOP) contributes one third of NCI treatment trial enrollment (“accrual”) and most cancer prevention and control (CP/C) trial enrollment. Prior research indicated that the local clinical environment influenced CCOP accrual performance during the 1990s. As the NCI seeks to improve the operations of the clinical trials system following critical reports by the Institute of Medicine and the NCI Operational Efficiency Working Group, the current relevance of the local environmental context on accrual performance is unknown.
This longitudinal quasi-experimental study used panel data on 45 CCOPs nationally for years 2000–2007. Multivariable models examine organizational, research network, and environmental factors associated with accrual to treatment trials, CP/C trials, and trials overall.
For total trial accrual and treatment trial accrual, the number of active CCOP physicians and the number of trials were associated with CCOP performance. Factors differ for CP/C trials. CCOPs in areas with fewer medical school-affiliated hospitals had greater treatment trial accrual.
Findings suggest a shift in the relevance of the clinical environment since the 1990s, as well as changes in CCOP structure associated with accrual performance. Rather than a limited number of physicians being responsible for the preponderance of trial accrual, there is a trend toward accrual among a larger number of physicians each accruing relatively fewer patients to trial. Understanding this dynamic in the context of CCOP efficiency may inform and strengthen CCOP organization and physician practice.
We stand on the edge of a new era in cancer clinical research. Rapid advances in proteomics and genomics are reshaping not only the practice of clinical cancer care, but also clinical research itself . Simultaneously, there is a new imperative on improving the efficiency and productivity of the national cancer clinical trials system. The Institute of Medicine (IOM) and the National Cancer Institute’s (NCI’s) Operational Efficiency Working Group (OEWG) have recently issued stark reports advocating for substantial change in the cancer clinical trials system, which has reached “a state of crisis [2–4].” Cancer research leaders recently reiterated this urgency at the NCI-American Society of Clinical Oncology (ASCO) symposium, and specifically emphasized better understanding of the drivers of research efficiency and productivity .
To date, the NCI Cooperative groups that develop and manage NCI clinical trials have been a focal point of emphasis, and the IOM and OEWG reports have catalyzed their massive reorganization, consolidating from 9 groups to 4 with substantial corresponding system changes . However, 30% of NCI treatment trial accrual and nearly all of its cancer prevention and control (CP/C) trial accrual (over 28,000 accruals in 2007) comes from the NCI Community Clinical Oncology Program (CCOP). The CCOP is tightly connected to the Cooperative groups; however, it is still independent of them, and while efforts to restructure the cooperative groups will affect CCOPs, research shows that organizational characteristics and local market conditions significantly influence CCOP productivity. For example, studies of CCOP performance in the 1990s demonstrated that managed care penetration and hospital competition were relevant factors affecting trial accrual, and that accrual to treatment trials and CP/C trials were differently affected by not only these factors, but also CCOP organizational characteristics [7–9] (Fig. 1).
Much has changed since the 1990s. The clinical practice dynamic surrounding managed care is no longer what it was, and other factors have arisen and changed the clinical environment in which CCOPs operate. Among them, the Medicare Modernization Act of 2003 changed the reimbursement landscape substantially for cancer care, especially for chemotherapy, which is a dominant focus of investigation in NCI clinical trials [10,11]. These changes have spawned concerns regarding consequent changes in patient access and referral, which may also affect clinical trials participation . As the nation engages in historic health care reform accompanied by substantial uncertainty [13,14], the current effect of the healthcare environment on NCI clinical trials is unknown.
This study reexamines the NCI CCOPs to inform and update our understanding of factors associated with trial enrollment and program sustainability. It employs a multilevel approach to examine factors associated with the CCOP, NCI research network, and local clinical care environment to understand factors associated with treatment trial accrual, CP/C trial accrual, and total trial accrual in the past 10 years. In doing so, this study addresses the calls from the IOM and NCI/ASCO to develop a better understanding of the drivers of clinical trial productivity and efficiency in the context of the clinical provider community in which cancer care is delivered.
This study focuses on the NCI CCOP program, which has been described extensively elsewhere [1,15]. Briefly, the CCOP is a nationwide provider-based research network (PBRN) enabling community physicians to participate in NCI clinical research not only by enrolling patients in trials, but also by engaging them with academic researchers to contribute practice-based practical insight to collaboratively develop practical studies. The CCOP goals include not only enrolling patients onto trials, but also accelerating the use of evidence-based medicine in the community. Through first-hand participation in the clinical trials, community physicians practice the state-of-the-art science being investigated through the trial, and develop a sense of ownership and acceptance of study findings. This strengthens their likelihood of acting on the research findings and incorporating the evidence-based medicine into practice. Recent studies have validated this NCI goal, by showing that organizational participation in NCI clinical research is positively associated with adoption of state-of-the-art care [16–18] (Fig. 2).
This single-group, longitudinal quasi-experimental study used yearly panel data from 2000 to 2007 collected from CCOP progress reports and the NCI’s CCOP, MBCCOP, and Research Base Management System. The unit of analysis was the CCOP. Because healthcare environmental data were measured at the metropolitan statistical area (MSA), we included all CCOPs that served at least one MSA and were active in the year 2000. This resulted in 45 CCOPs in the year 2000 decreasing to 41 CCOPs in 2007, for a total of 355 CCOP-year observations in the analytic sample.
Three dependent variables were separately examined as markers of CCOP performance: treatment trial accrual, CP/C trial accrual, and total trial accrual. Independent variables included CCOP characteristics, CCOP-Research Base (RB)1 network characteristics, and environmental characteristics. Among CCOP characteristics, we examined the number of active and enrolling CCOP physicians, the number of CCOP components, the number of active institutional review boards (IRBs), and the number of CCOP or RB meetings attended by CCOP physicians or staff. A gini index was developed to enumerate each CCOP site’s accrual as being concentrated among a few CCOP physicians, or more broadly and evenly distributed among multiple physicians.  This index was used to create a categorical variable indicating low, medium, and high physician accrual equity. The CCOP-RB network characteristics included the number of affiliated RBs and the number of treatment trials, CP/C trials, and total trials for which the CCOP had at least one accrual during the year.
To measure environmental characteristics, we used managed care penetration, hospital competition, and clinical trials competition. Using the InterStudy Competitive Edge Regional Market Analysis dataset, managed care penetration was measured as health maintenance organization (HMO) penetration for each CCOP’s MSA . HMO penetration was calculated by dividing the number of HMO enrollees in the MSA by its total population. For hospital competition, we used the American Hospital Association (AHA) Survey Database to calculate the Herfindahl–Hirschman Index (HHI) for each MSA, based on each short-term acute care hospital’s total number of admissions divided by the total of all admissions in the MSA, then squared and summed . The final measure was calculated as one minus the HHI. Data from the Area Resource File were used to develop a measure of clinical trials availability or competition in the MSA, and measured as the proportion of hospitals in the MSA that were unaffiliated with medical schools. Additional environmental control variables included the proportion of the MSA that was insured, and whether or not the state had a legislative mandate that regular care be covered by insurance for those enrolled in clinical trials .
Although the study captured data on nearly all CCOPs, the total sample size called for using a limited set of variables to conserve degrees of freedom in the multivariable analysis. Accordingly, informed by annual accrual trends and a review of preliminary regression analysis, categorical variables indicating the year were collapsed from eight variables to four reflecting the years 2000–2001, 2002–2003, 2004–2005, 2006–2007. CCOP-level data were linked to MSA-level data based on the MSA in which the CCOP headquarter was located, as done previously .
The dependent variable was available for all 355 observations, as was the case for nearly all observations on independent variables. In 2006, HHI was missing for 7 observations at the MSA level. Because of the high correlation between MSA-level and county-level measures of HHI (0.829, p>0.001), county-level HHI replaced missing MSA-level values for this year. For 2007, because of the very high inertia in this measure, observations for HHI were imputed by linear interpolation/extrapolation based on immediately prior years.
To select the functional form for the dependent variable, we used the Wooldridge test examining if logged annual accrual was superior to un-logged annual accrual in terms of overall model fit . To select the functional forms of the independent variables in the model, we used adjusted R-squared values . We determined that the non-logged annual accrual model and non-quadratic forms of HMO penetration and hospital competition best fit the data. Additionally, in order to allow the effect of the number of IRBs on trial accrual to vary for CCOPs with a higher or lower number of CCOP components, we examined an interaction term of number of IRBs number of components.
Using a single-group, longitudinal panel design, we analyzed the three dependent variables separately in three models. We compared three different types of analytic methods: pooled-OLS with dummy variables for time, fixed effects, and random effects estimation. A Hausman test and a Breusch–Pagan Lagrange Multiplier test suggested that fixed effects estimation for total accrual and treatment accrual, and random effects estimation for the CP/C accrual model were most appropriate [23,24]. T-tests were used to test the significance of single variables and F-tests to examine the joint significance categorical constructs in each of the three models described above. All analyses were performed using Stata 11, and tests were conducted using a minimum significance level of 0.05 (StataCorp. 2009. Stata Statistical Software: Release 11. College Station, TX: StataCorp LP).
During the period of 2000–2007, the average accrual per CCOP for total trials, treatment trials and CP/C trials was 240.3, 123.2, and 117.1 patients per year, respectively (Table 1). The average CCOP had approximately 21 physicians accruing patients to a trial, was affiliated with slightly less than five cooperative groups, and had accrual to 31 treatment trials and 13 CP/C trials. Accrual was evenly distributed among the time periods. The majority of CCOPs (52%) were located in the North Central region of the United States. Bivariate correlations are presented in Table 2.
In the total trial accrual model, only the number of active CCOP physicians (p<.009) and the number of total trials (p<.001) were significant. An increase in either of these factors was associated with an increase in total clinical trial accrual. The categorical variable indicating low, medium, and high physician accrual equity was not jointly significant [F(2, 292)=1.0, Prob>F=0.37]. The marginal effect of Number of IRBs and Number of components (using the interaction) was not significant in this or any of the other models. The only significant environmental factor was the categorical variable for proportion of hospitals in MSA with medical school affiliations [F(2, 292)= 3.3, Prob>F=0.04]; CCOPs in MSAs with fewer medical school affiliated hospitals were associated with higher total trial accrual.
In the treatment accrual model, the number of active CCOP physicians (p<.001) and the number of treatment trials (p<.001) were significantly associated with greater treatment trial accrual. The categorical variable indicating low, medium, and high physician accrual equity was jointly significant [F(2, 292)=3.1, Prob>F=0.047]. In this model, lower accrual equity among physicians (i.e., the majority of accrual concentrated among few physicians) was generally associated with higher treatment trial accrual, though this association was not consistently statistically significant. Similar to the total accrual model, the CCOPs in MSAs with fewer medical school affiliated hospitals were significantly associated with greater treatment trial accrual [F(2, 292)=3.1, Prob>F=0. 047].
In the CP/C model, the number of CP/C trials (p<.001) and RB affiliations (p<.028) was again significantly associated with greater total trial accrual. Unlike the treatment trial accrual model, higher accrual equity among physicians was generally associated with higher CP/C trial accrual; however, this was not statistically significant. Hospital competition was significantly associated with a decrease in CP/C trial accrual (p<0.039). All other variables and constructs were statistically insignificant (Table 3).
This study examined the organizational and environmental characteristics associated with accrual in the NCI’s CCOP program. One of the most striking findings in the analysis may be that, compared with analyses of the prior decade , very few traditionally-measured CCOP organizational or environmental characteristics remain associated with CCOP accrual performance. A hallmark of the CCOP has been its long history of performance data collection and a philosophy of systematic self-examination to inform continuous improvement [1,7–9,25,26]. As the CCOP continues to embrace this philosophy and engage in strategic planning efforts, also incorporating guidance from the recent IOM and OEWG [3,4], one of its most important priorities may be to examine new measures associated with organizational performance and implement them to augment the set of measures it currently collects.
Interpreting the results of this analysis in the context of prior research, we see current CCOPs experiencing higher per-CCOP accrual in current years, suggesting greater efficiencies in operation than before. The CCOPs had a comparable number of CP/C trials open, though substantially fewer treatment trials open compared to the 1990s. Given this trend, it may be the case that funding previously allocated to administrative staff for opening and processing trials may have been reallocated to fund more staff that enrolled or helped manage patients on trials, and thus be a key driver of the apparent accrual efficiencies. CCOPs in the 2000s tended to be larger and have a slightly greater number of accruing physicians than those in the 1990s, further suggesting that achieving a “critical mass” may be associated with CCOP success. This may be through greater programmatic stability (i.e., better able to absorb individual physicians coming and going) or broader access to additional funding from affiliated hospitals and clinical programs. Alternatively, it may offer enhanced access to a greater number of patients or a more diverse patient population. This may also be reflected in CCOPs recently being affiliated with a greater number of Research Bases than before, with associated access to a more diverse set of clinical trials from which to choose to meet the needs of the more diverse population.
The non-significance of HMO penetration and hospital competition suggests that CCOPs able to adapt to the disruptive changes in healthcare financing seen in the 1980s and 1990s were more likely to survive and thrive in the 2000s. It may also indicate that the once-volatile market characteristics are in fact less disruptive than they once were.
For CP/C accrual, surprisingly few factors are strongly predictive of performance compared to those associated with either CP/C in the 1990s or with treatment accrual in the 2000s. Another notable contrast is the physician accrual profile, in which treatment trial performance appears to have favored few physicians who were responsible for most accrual, compared to a broader set of physicians with more equal accrual among CP/C trials. Comparing CP/C and treatment models further, statistical specification tests suggested that a different model (Random Effects estimation) had a better fit for CP/C compared to treatment (Fixed Effects estimation). Together, these points jointly suggest that factors associated with CP/C accrual performance are fundamentally different from those associated with treatment accrual. In practical terms, this observation reflects the reality of CCOP operations and is quite logical, as treatment accrual tends to take place in oncology practices while CP/C accrual commonly takes place in a much broader set of clinical as well as non-clinical settings. Therefore, it makes sense that organizational and environmental factors associated with oncology treatment may differently affect accrual to treatment trials compared to accrual to CP/C trials, for example, in primary care practices, follow-up clinics, or the broader non-clinical community.
Study findings suggest that, more so than clinical competition, there may be academic competition. Specifically, there tends to be less accrual among CCOPs in communities with a stronger academic presence vis-à-vis the proportion of hospitals affiliated with a medical school. It may be that patients who would contemplate enrollment in a clinical trial would be likely to accept referral or self-refer to an academic center, rather than refer to a CCOP practice. On one hand, this speaks to the CCOP’s strategic priority focus outside of the academic medical centers, and the establishment of local CCOPs in communities that do not have a medical school. Conversely, it may be that CCOPs in communities with a strong academic presence are functionally different from other CCOPs either in terms of their structure, the trials they offer, the strength or consistency of referral relationships, or the different practice focus and priorities of CCOP practitioners in these communities. Again, we look at the trend toward greater treatment trial accrual among CCOPs with low accrual equity physicians (i.e., a small proportion of physicians have a preponderance of accrual). There are several possible causes of this phenomenon. CCOP physicians in communities where there is an academic presence may feel greater pressure to more broadly serve the practice organization’s many missions, rather than “specialize” in clinical research. This may contrast to communities without academic centers, where the CCOP may be the only source of clinical trial participation, and thus functionally serve as “the academic center” for that community, in turn promoting cultural norms that encourage specialization in different areas among professionals — clinical care, research, and others. Additionally, this finding may indicate that a dynamic, motivated physician may provide a bolus in trial enrollment. It may also suggest that small practices with a strong focus on trial enrollment may outperform their larger peer practices. Further research should examine these issues, as understanding them may inform CCOP organization and physician practice focus, and strengthen those CCOPs operating in both academic and non-academic communities.
Among study limitations, our study involves a sample of only 45 CCOPs followed for eight years. Though this represents nearly all CCOPs, it is a small sample to examine with fixed effects and random effects estimation, and close attention must be paid to balance sufficient model specification with a need for erring on the side of parsimony in terms of degrees of freedom. A tradeoff of being parsimonious is that this minimally-specified model may suffer from excluding relevant variables. This study examined the variables collected by CCOPs that were most relevant and incorporated additional measures. However, there may be other environmental and organizational factors or patient population, physician, hospital and clinical trial specific characteristics that may have had an effect on clinical trial accrual for which our model did not control. If these factors truly belonged in the model, omitting them will result in biased estimates. As the CCOP goes forward to identify additional relevant variables, understanding the relative strength of relevance for each will be important.
Among other issues, many of our environmental variables are linked to the CCOPs by the MSA where the CCOP headquarters is located, which is suboptimal because some CCOPs are active in more than one MSA. In these instances, variables such as hospital competition or HMO penetration might not comprehensively reflect the environment in which the CCOP actually operates. However, most CCOPs operate in a limited region, and for this study sample, there was limited variation with these regions, and thus limited risk of bias due to this measurement method. In this study, hospital competition was based on the number of hospital admissions; given the outpatient nature of the majority of cancer care, it may be more relevant to measure hospital competition differently in the future. Finally, physician competition was measured using the number of practicing physicians in the MSA. Given that this study focuses on clinical trials for cancer patients, it may have been more accurate to reflect physician competition in terms of practicing oncologists in the MSA or CCOP service area. This data was not available for this study.
More so than any one organizational or environmental characteristic, the flexibility of CCOPs and the motivation of their research community have been central aspects enabling a great number of CCOPs to succeed ; CCOPs that stopped operations may have done so in part because they were unable to adapt on the local level to the changing environments around them. The CCOP’s consistent history of self-examination has fueled both its flexibility and success. Moving forward, as the CCOP continues its strategic planning, it will be important to understand factors that drive performance and to systematically identify measures allowing these factors to be monitored over time, to provide ongoing feedback and inform the performance and success of local CCOPs. The successes of CCOP and PBRNs like it that reach broadly into the community are of increasing importance as development of effective treatments for all populations increasingly demands access to heterogeneous populations for research. Accordingly, understanding factors relevant to PBRN success will continue to be of great importance. This knowledge will thus inform the National Institutes of Health Roadmap and be responsive to the IOM, NCI, and NCI/ASCO as the CCOP pursues its goals of accelerating the pace of scientific discovery, while facilitating the translation of evidence-based research into practice.
This analysis was funded in part by contract HHSN-261200800726P: CCOP Sustainability: Assessing the Interplay Between Organizational, environmental, and Network Factors that Predict long-term Performance and Survival; and 5R01CA124402: Implementing Systemic Interventions to Close the Discovery Delivery Gap; and 5R25CA116339: NCI Cancer Care Quality Training Program. We thank Lori Minasian, Cynthia Whitman, and colleagues at the NCI Division of Cancer Prevention for their gracious assistance with access to key information and insight and relevant to this examination.
1NCI Research Bases include the NCI Cooperative Groups and four other NCI affiliated organizations that develop and manage clinical trials conducted through the NCI research networks.