|Home | About | Journals | Submit | Contact Us | Français|
To compute net cancer-specific survival rates using population data sources (e.g., Surveillance, Epidemiology and End Results (SEER)), two approaches are primarily used: relative survival (observed survival adjusted for life expectancy) and cause-specific survival based on death certificates. We evaluate the performance of these estimates relative to a third approach based on detailed clinical follow-up history.
Using data from Cancer Cooperative Group clinical trials in breast cancer, we estimate 1) relative survival, 2) breast cancer specific survival (BCSS) determined from death certificates, and 3) BCSS obtained by attributing cause according to clinical events after diagnosis, which for this analysis we consider the benchmark “true” estimate. We also compare non-cancer life expectancy between trial participants, SEER registry patients, and the general population.
Among trial patients, relative survival overestimated true BCSS in lymph node- negative breast cancer, while in node-positive cases, the two estimates were similar. For higher risk patients (younger age, larger tumors), relative survival accurately estimated true BCSS. In lower risk patients, death certificate BCSS was more accurate than relative survival. Non-cancer life expectancy was more favorable among trial participants than the population and SEER patients. Tumor size at diagnosis, a potential surrogate for screening utilization, partially accounted for this difference.
In the clinical trials, relative survival accurately estimated BCSS in higher risk cases, despite more favorable other-cause mortality than the population at large. In lower risk patients, estimates using death certificate information may be preferred in SEER and other data sources where detailed post-diagnosis clinical history is unavailable.
For individuals with a cancer diagnosis, one would ideally like to have reliable disease-specific mortality information, for example, an estimate of the median survival time from diagnosis until death due to cancer or the proportion of individuals who will likely die of the cancer by a certain time landmark such as five years. These measures are considered to better reflect the impact of the disease than all-cause survival, which varies widely by demographic factors such as age and race, and may exhibit geographic and temporal trends. Thus, methods have been developed to estimate quantities representing the net influence of the cancer on the lifetime, and these are frequently used for reporting of cancer survival trends. For example, the Surveillance, Epidemiology and End Results (SEER) program, a geographically based cancer registry cohort of the U.S. National Cancer Institute, reports net cancer survival obtained by relative survival estimates, an established method (described below) that does not depend on patient-specific cause of death information [1,2]. This method, as well as alternatives that rely on attributed cause of death, all suffer from shortcomings that may limit the degree to which true cancer-specific survival metrics are obtained.
Once the specific cause of death (COD) is determined, then (under certain assumptions) calculating cause-specific mortality is relatively straightforward . However, COD can be surprisingly difficult to accurately ascertain in population data. For example, in the SEER program and similar registries, COD may be unreported, erroneous due to a tendency towards over-attribution to the cancer once diagnosed, or simply too ambiguous with respect to attributing a single cause, requiring some subjective rule for resolution [4–6]. In multi-center clinical trials, similar problems arise in establishing a COD [7,8]. Specifically, COD information can be incomplete or inaccurate, and is often not rigorously reviewed or coded by nosologists. For this reason and others, clinical trials commonly report all-cause survival as the primary endpoint.
The relative survival estimate is a frequently used alternative that does not require cause of death information and accounts for demographic variations in expected mortality [1,2]. Relative survival estimates equal observed all-cause survival divided by expected survival estimated from the general population in a comparable geographic area, age, sex, race, and calendar year. Relative survival is typically reported by SEER for cancer survival estimates [9,10], and has been widely used to examine temporal and geographical trends in cancer survival rates [11,12]. However, this approach relies on population expected survival data, which may lack sufficient representativeness of the cancer patients at hand. For example, it is well known that patients diagnosed with early stage colorectal or prostate cancer via a screening exam may tend to have lower other-cause mortality than the general population. The consequence of then using a population adjustment for expected survival in the cancer patient cohort is an overestimate of cancer-specific survival. For clinical trials, relative survival is not typically used, because while aiming to be representative, participant selection is not population-based in any strict sense, and so the appropriate expected survival adjustment is unknown.
In this study, we address several questions related to estimation of cancer-specific survival, using data from multi-center cancer clinical trials conducted over the past 20 years, as well as the SEER Program data from comparable year periods. First, in the clinical trial cohort we compare relative survival to cancer-specific survival as determined from clinical history (i.e., disease progression events after initial diagnosis), which for purposes of this study we will consider the true cancer-specific survival. We also compare survival estimates based on reported COD from death certificates to relative survival estimates. Second, we examine relative survival and cancer-specific survival as determined by death certificate information in comparable SEER cohorts. Finally, to evaluate to what extent both clinical trial cohorts and patients included in SEER may reflect the population at large with respect to non-cancer mortality, we estimate non-cancer life expectancy in these respective groups.
The National Surgical Adjuvant Breast and Bowel Project (NSABP) is a National Cancer Institute sponsored cancer Cooperative Group with over 500 participating clinical centers throughout North America . Institutions enrolling patients include university hospitals, cancer research centers, and community-based hospitals and practices. For this study, we included randomized trials for early invasive breast cancer (i.e., no metastatic disease) that were open to patient accrual between 1982 and 1998. These trials investigated adjuvant systemic therapy regimens (tamoxifen, multi-drug cytotoxic chemotherapy) that could feasibly be delivered in a variety of care settings. Trial eligibility criteria included operable unilateral breast cancer (with or without axillary lymph node involvement depending on the specific trial), no prior history cancer, health status consistent with ability to undergo adjuvant therapy, and specific hormone receptor criteria for some trials. Follow-up continued for at least 15 years on earlier trials, with more recent studies remaining in active follow-up.
From the SEER 9 registries, we selected patients from diagnosis years 1989–2003 with follow-up through 2004, exclusive of intraductal (DCIS), locally advanced (stage III), and metastatic (stage IV) cases. Patients were grouped by axillary lymph node status (denoted node-negative or node-positive) for comparability to the NSABP trials, which used node status as principal determinant of eligibility for individual trials. In the SEER program, lymph node status is available beginning in 1989, and at the time of this analysis, 2004 was the most recent follow-up year available. Consequently, the maximum follow-up length for the cohort is 15 years.
The clinical trial cohort consisted of 20,737 patients from 11 clinical trials (Table 1). Among these patients, there have been 6,676 deaths; among node-negative cases, 47% of deaths were attributed to breast cancer (according to death certificates) and 22% had missing cause, compared to 71% breast cancer and 18% missing among node-positive patients. Patients from SEER tend to be older at diagnosis, have smaller tumors, and among node-positive patients, have a smaller number of positive nodes (Table 1). The proportion of deaths attributable to breast cancer (based on death certificates) is lower for the SEER cohort, particularly for node-negative patients.
Three methods used to estimate breast cancer specific survival are described here and summarized in Table 2.
Patients in NSABP trials have regular protocol specified follow-up, with breast cancer recurrence events at these visits or in intervening intervals documented and reported. The protocol calls for reporting of the first recurrence at any anatomic site (local, regional, or distant) and the first distant metastatic recurrence, although typically all events that occur (i.e., multiple distant site failures) are reported. Second primary cancers are also reported. By examining this clinical history, deaths can be classified as either being a) subsequent to a breast cancer event (recurrence or second primary (contralateral) breast cancer) or b) without documentation of any preceding breast cancer events. For this analysis, we considered all deaths of the former type as breast cancer related and those of the latter type as due to causes other than breast cancer. This estimate (denoted recurrence algorithm-based survival) was used as the benchmark estimate for breast cancer specific survival to which relative survival and death certificate based estimates were compared in the trial cohort.
Using International Classification of Disease (ICD) codes provided on death certificates, deaths were assigned as due to either breast cancer (ICD = 174.0) or to other causes. In the NSABP trials, COD codes are missing in 18–22% of cases (Table 1). In the trials, whether the COD was missing was unrelated to the clinical history (recurrence events, second cancers), as the proportion of these patients with recurrence was similar to those with reported COD (not shown). For an estimate based on COD, we adopted the conservative approach of treating deaths with unknown COD as breast cancer related, providing a lower bound estimate of breast cancer specific survival (assuming COD assignment is otherwise correct).
For the SEER cohort, COD codes are unknown in approximately 3% of cases (Table 1). In an analysis of earlier SEER data where the proportion missing was greater, Gamel evaluated different ways to allocate unknown cause deaths and recommended a proportional allocation between causes . Because the proportion missing in the current SEER data is small, estimates do not differ by more than two percentage points through 15 years regardless of whether the missing COD is considered due to breast cancer death or due to other causes. Thus, as in the trials, we attributed deaths with missing COD to breast cancer for the death certificate based estimates.
For both the recurrence algorithm and death certificate method of assigning cause of death, breast cancer specific survival was computed via life table (actuarial) methods . Specifically, we treat other-cause deaths as censored observations, and, assuming independent competing risks, we can interpret this estimate as a net survival function .
Relative survival estimates were computed for both the NSABP and SEER patients. For relative survival analysis, observed all-cause survival estimates are divided by age, race, sex, and calendar year specific life expectancy (from U.S. life tables ), to produce estimates that are then commonly interpreted as (approximately) breast cancer-specific survival, accounting for strata-specific differences in overall life expectancy. Interval-specific estimates were restricted to be ≤ 1.0 (i.e., 100%), and thus all relative survival curves are non-increasing. The SEER*STAT software package (http://seer.cancer.gov/seerstat/) was used to produce these estimates.
In order to investigate the representation of the non-cancer (other cause) life expectancy of the cancer patients in the trials and SEER to the general population, other cause life expectancy for cancer patients was estimated by treating patient age as the time axis and estimating a left-truncated Kaplan-Meier survival function, with other-cause deaths treated as the event, breast cancer deaths treated as censored, and the age at diagnosis as the left truncation point. The resulting survival curve represents the probability of non-cancer survival, conditional on survival to the earliest age at diagnosis (e.g., youngest patient). For comparison, overall life expectancy curves for the U.S. population in years 1990 and 2000, obtained from U.S. life tables (NCHS) were averaged at each age and plotted. Differences in life expectancy curves between the cancer patient cohorts and the general population indicate whether the former tend to have more or less favorable other-cause mortality relative to the general population. Note that because the actual proportion of cancer deaths in the population is small, the all-cause life expectancy for general population provides a suitable estimate to non-cancer life expectancy .
We computed recurrence algorithm-based breast cancer survival estimates (breast cancer deaths based on prior clinical history), relative survival estimates, and estimates based on reported COD from death certificates for the clinical trial cohort trial data, separately by lymph node status.
Among women with node-negative breast cancer, relative survival estimates are greater than the algorithm-based estimate, with absolute differences of 2.0 percentage points at five years and 2.9 percentage points at ten years (Fig. 1A). In women with node-positive disease, where a greater proportion of deaths are likely to be due to breast cancer, relative survival and the algorithm-based survival estimates are more similar, with an absolute difference of 1.0 percentage points at 5 years and 1.4 percentage points at ten years (Fig. 1B).
For estimates based on death certificates, the resulting survival estimate slightly underestimates the algorithm-based breast cancer survival in node negative patients (Fig. 1A) and more closely resembles the relative survival and algorithm based estimates in node positive cases (Fig. 1B).
In the SEER cohort, only death certificate-based survival and relative survival are estimable, since recurrence history is not available. In node-negative cases, the difference between death certificate based and relative survival rates resemble those of the trial data, in that the relative survival estimate is higher (Fig. 1C). For node-positive cases, these estimates are nearly identical, similar to results seen for the trial cohort (Fig. 1D).
Discrepancies between relative survival and true disease-specific survival are often attributable to the “adjustment” population being dissimilar from the actual non-cancer survival experience of the cohort. Specifically, greater health resource access can result in both earlier stage cancer diagnosis with more favorable features (i.e., small tumors) and more favorable other-cause mortality. We explored this possible relationship of breast cancer characteristics to overall mortality risk by examining estimates within patient and tumor characteristic strata for the clinical trial cohort. In addition to node status, the strata are defined based on commonly used breast cancer prognostic groupings for age at diagnosis (<50, 50–64, ≥ 65) and tumor size (≤ 2.0 cm, 2.1–4.9 cm, ≥ 5.0 cm).
Figure 2 shows algorithm-based and relative survival estimates within age and tumor size groups. For younger patients (< 50), where any deaths occurring are more likely to be due to breast cancer than in older patients (both because young patients are expected to have lower other-cause mortality and their disease tends to be more aggressive), relative survival closely estimates the algorithm-based estimate. In patients with large tumors (≥ 5.0 cm), where screening is less likely the sole detection method, the two estimates are also very similar. For patients with disease characteristics indicative of high mortality risk, such as more than ten positive lymph nodes, the relative and algorithm-based survival were nearly identical (not shown). In contrast, for older patients and those with smaller tumors, the two estimates deviate more widely, with relative survival overestimating algorithm-based survival. For some subsets, such as node-negative patients aged 65 and over, the relative survival estimate equals 100%, indicating that the overall survival in the trial cohort (including cancer deaths) is greater than the expected survival based on the age comparable general population. This problematic behavior is sometimes observed in very early screen detected cancers, such as DCIS.
For both node status groups, the death certificate based estimate treating missing cause as breast cancer deviates more widely from the algorithm-based estimate as prognosis improves and/or the patient’s likelihood of other cause death increases (i.e., older age at diagnosis, smaller tumors, Figs. 2C and 2D), because a greater proportion of these missing cause cases did not have prior recurrence, which the algorithm based approach takes into account.
Because overall mortality differs by race/ethnicity (as does cancer-specific mortality in population-based studies), we examined differences between relative survival and breast cancer specific survival separately for blacks and whites. The approaches do not appear to perform differently by race/ethnicity (not shown).
A critical issue in the use of relative survival and the interpretation of clinical trials results in general is the health status of participants aside from their cancer diagnosis. For example, if other cause life expectancy for trial participants is greater than that of the reference population, because for example they have better healthcare utilization, then relative survival estimates, which incorporate expected survival from this reference population, are likely to be inflated.
We computed non-cancer life expectancy for the NSABP trials and SEER data, and compared these estimates to the life expectancy of the United States female population (separately by race) obtained from life tables. In node-negative breast cancer, non-cancer life expectancy among trial participants is greater than that of the population (Figs. 3A and 3C). The SEER patients had greater but somewhat more similar non-cancer life expectancy relative to the population, approximately coinciding with the population until about age 70 for whites and 75 for blacks, after which it was higher but not as favorable as that of the trial participants. In node-positive disease, the non-cancer life expectancy of trial participants is again greater than that of the population, but the non-cancer life expectancy of the SEER patients is lower (Figs. 3B and 3D). We next examined estimates separately by tumor size at diagnosis, which may serve as a surrogate for breast screening activity and consequently greater health resource utilization. This analysis was restricted to whites only because race and tumor size are associated. Among node-negative patients, trial participants with small tumors had superior life expectancy to the population, whereas for large tumors, the trial and SEER cohort patients both had similar life expectancy to the population (Figs. 4A-4C). When node-positive patients were stratified by tumor size, the SEER patients showed a trend towards less favorable life expectancy relative to the population with increasing tumor size (Figs. 4D-4F), which may lead to underestimation of the cause-specific survival. Among trial participants, life expectancy remained greater than the population for all tumor size groups.
Relative survival estimates are frequently used to report net cancer survival statistics, despite some anticipated shortcomings for certain disease sites and types of patients. In this study, we found that relative survival over-estimates breast cancer specific survival in lower risk (node-negative) breast cancer clinical trial patients, particularly among those who were older and those with small tumors (Fig 2C, 2D). In contrast, among trial participants with less favorable prognosis (node-positive, node negative with large tumors) or where breast cancer is the predominant cause of death, such as in those under age 50, relative survival and breast cancer specific survival were remarkably comparable (Fig 2A, 2F). Similar results were observed in the SEER patient cohort when we compared breast cancer specific survival (based on death certificate information) to relative survival. As putative risk of breast cancer death increased, the two approaches yielded similar estimates, while for lower risk patients, relative survival produced over-estimates. These observations support the use of relative survival in trials and in data sources such as SEER in instances with high cancer-specific mortality, but suggest that caution is warranted in cases where cancer mortality risk may be lower.
In order to extrapolate observations from either observational cohorts or clinical trials to the cancer patient population, the cohorts must be suitably representative. In the case of clinical trials, participants often differ in important ways from the population at large, and while within-trial treatment comparisons are nonetheless valid, treatment efficacy realized in the trial may be diminished in the population if trial participants are highly non-representative. In this study, trials participants had more favorable non-cancer life expectancy than the general population (Fig 3A, 3B), reflecting a likely ‘healthy participant’ effect with respect to morbidities. Interestingly, when comparisons were stratified by tumor size at diagnosis, the greatest discrepancy is seen for those women with small tumors (Fig 4A, 4D), while for large tumors, the estimates are similar (Fig 4C, 4F). For node-positive patients with large tumors from SEER, life expectancy was less favorable than the population, possibly reflecting the fact that women with large tumors may have postponed responding to symptoms or using screening, and concurrently may also have other untreated health problems (Fig 4F). This situation may be less likely to occur in clinical trials, due to eligibility requirements with respect to health status. In general, a greater degree of similarity of the SEER cohort to the population (Fig 3C, 3D) supports using relative survival as an estimate of cancer specific survival from SEER.
Even under ideal conditions, assignment of a single definitive cause of death can be difficult, and is necessarily subjective to some degree. In large geographically dispersed registries such as SEER, and in multi-center clinical trials, additional limitations result from incomplete records, or inconsistent rules with respect to cause attribution on death certificates. While an independent review panel could apply well-defined rules and attribute COD in an internally consistent manner, the process is labor intensive and limited by the information centrally available. Relative survival rates provide an alternative that does not require cause of death assignment, but may be inaccurate when the cancer patient cohort is not comparable to the reference population. For cancers that can be detected by screening before any symptomatic evidence, such as breast, prostate, and early stage colon cancer, those patients diagnosed at a very early stage tend to have better health status than the general population (because they have access to and make use of preventative medical services at a greater frequency), and thus their non-cancer morbidity and mortality are lower than the general population. When a standard reference population is then used to adjust observed all-cause survival, the resulting cancer-specific survival curve is overly high. This phenomenon most likely explains the overestimates seen among individuals with small tumors and those aged 65 or older at diagnosis, and also resembles the association between DCIS diagnosis and more favorable cardiovascular disease risk . To address the effects of heterogeneity in life expectancy when computing relative survival, numerous stratification and modeling approaches have been proposed [18–21].
There are limitations to the approach used to determine true breast cancer survival, considered here the reference standard to which relative survival is compared. Our tautological definition of breast cancer death (death after a recurrence event) will result in some misclassification, and the approach has not been validated against a more comprehensive COD determination based on, for example, panel review of all the relevant information. Also, we included all recurrence types, including local (e.g., breast) and regional (e.g., axilla) sites, which alone may not increase risk of death. However, among patients with local/ regional failure from these trials, there was a substantially increased distant recurrence (and consequently breast cancer death) risk [22,23]. Also, to the extent that reported cause of death is accurate, we note that among patients reported as having died of breast cancer by death certificate ICD code, over 90% had prior breast cancer recurrence events. In general, recurrence has not proven to be an acceptable surrogate for survival in early stage breast cancer, although it is an important and widely used endpoint . This is unlike the case of colon cancer, where the disease-free survival (time without recurrence, second primary cancer, or death) has recently been advocated as a primary endpoint in clinical trials [25,26]. Here, we use recurrence not as a surrogate endpoint in the usual sense, but rather to infer whether the eventual death was due to breast cancer. Incorporating anatomic site of recurrence (i.e., distant metastatic vs. local recurrence), time of recurrence in relation to death, and information on second primary cancer incidence after recurrence could improve cause of death classification.
A broader concern is ambiguity in the meaning of “cause of death” determined by any method. In therapy development, all-cause survival has been appropriately adopted as an endpoint because it implicitly accounts for both beneficial and adverse treatment effects, and indeed many cancer treatments are associated with high risk for morbidity and mortality. Assigning cause of death solely according to evidence of disease progression ignores increased risk of other chronic diseases that may be attributable to cancer treatment. Thus, while identifying cancer-specific survival is useful for biologic considerations and studying trends in populations, all-cause survival will remain primary as an endpoint in cancer treatment development.
With respect to limitations in extending findings on performance of relative survival from the trials to the population-based (SEER) data, SEER patients tended to be older and have more favorable disease characteristics. The clinical trial cohort was more favorable than SEER with respect to other-cause life expectancy. Frequency of missing COD differed markedly between the data sources, and it is unknown how accuracy of deaths certificate COD compares between clinical trials and SEER. Interestingly, the relative survival and death certificate based estimates are remarkably similar between SEER and trial participants, particularly among node positive patients (Fig. 1).
In summary, this evaluation of the performance of relative survival confirms the concern that, when the cancer patient cohort has more favorable non-cancer survival than the population at large, the method overestimates cancer-specific survival. In these instances, and where information on intervening clinical events is not available (e.g., in SEER), estimation of cancer-specific survival using COD information and treating those with unknown cause as due to cancer provides a reasonable alternative, provided that missing COD is infrequent and effectively random. When the cohort has population typical other-cause mortality or when the dominant mortality cause is expected to be cancer, as seen here in node-positive patients and specifically in younger patients and those with larger tumors, then relative survival accurately estimates breast cancer specific survival. In clinical trials, while overall survival will remain an endpoint of primary interest, cancer specific survival derived from clinical event histories is an attractive option that requires further validation. Relative survival appears to be a reasonable alternative in clinical trials when the aforementioned conditions hold. With progress towards targeted therapies and more complex multi-agent regimens, accurate estimation of disease-specific survival will be critical to assessing both benefits and risks of new treatments.
Work (JJD) supported by contract # 263-MQ-611334 from the Division of Cancer Control and Population Sciences/Surveillance Research Program/Statistical Research and Applications Branch of the U.S. National Cancer Institute and a research grant from the Susan G. Komen for the Cure Foundation. The clinical trials were supported by Public Health Service grants NCI-U10-CA-69651 and NCI-U10-CA-12027 from the U.S. National Cancer Institute.