|Home | About | Journals | Submit | Contact Us | Français|
We characterized variation in adherence to quality measures of external beam radiotherapy (EBRT) for localized prostate cancer and its relation to patient and provider characteristics in a population-based, representative sample of US men.
We evaluated EBRT quality measures proposed by a RAND expert panel of physicians among men age 65 or older diagnosed from 2000 to 2002 with localized prostate cancer and treated with primary EBRT using data from the linked Surveillance, Epidemiology, and End Results (SEER) Medicare program. We assessed adherence to five EBRT quality measures that were amenable to analysis using SEER-Medicare data: 1) use of conformal radiotherapy treatment planning; 2) use of high-energy (>10MV) photons; 3) use of custom immobilization; 4) completion of two follow-up visits with a radiation oncologist in the year following therapy; and 5) radiation oncologist board certification.
Of the 11,674 patients, 85% received conformal radiotherapy treatment planning, 75% received high-energy photons, and 97% received custom immobilization. One-third of patients completed two follow-up visits with a radiation oncologist, though 91% had at least one visit with a urologist or a radiation oncologist. The majority of patients (85%) were treated by a board certified radiation oncologist.
Overall high adherence to EBRT quality measures masked substantial variation by geography, socioeconomic status in area of residence, and teaching affiliation of the radiotherapy facility. Future research should examine reasons for variation in these measures and whether variation is associated with important clinical outcomes.
External beam radiotherapy (EBRT) is a potentially curative therapy for patients with clinically localized prostate cancer. The delivery of EBRT is highly technical, and, as with other technical fields in medicine, the need for quality assessment and improvement is essential. Previous studies have identified variation in radiotherapy quality in clinical trials1, 2, prompting the establishment of rigorous quality assurance procedures within the confines of the research setting.3, 4 Yet, measuring and improving cancer care quality for the 97% of patients who are cared for outside of the clinical trial context has become a national priority5, 6, and several groups have initiated programs to examine quality across the spectrum of cancer care.7
Given substantial healthcare and economic burdens of treatment for localized prostate cancer8, attention has been directed toward characterizing variation in the quality of definitive therapies for this disease.9 In 2000, the RAND Corporation, a nonprofit institution that undertakes health policy research, assembled an expert panel of physicians that proposed an inventory of quality measures for the treatment of localized prostate cancer.10, 11 The project endorsed 25 structure and process measures that were germane to the evaluation of prostate cancer EBRT quality. Of these 25 measures, 9 obtained agreement among panel members, reflecting a high level of consensus as to their appropriateness for quality assessment using medical records.11
However, initial efforts relying on medical record review to evaluate the highly-technical processes of EBRT have confronted challenges stemming from the inaccessibility of radiotherapy records. One study of 186 men at a single academic institution found that documentation of EBRT planning was not accessible in clinical data available to investigators.12 Often, radiotherapy documentation is managed and stored within outpatient radiotherapy facilities, separate from inpatient hospital documentation and not consistently included in patient charts. Therefore, a knowledge gap has emerged in the quality assessment of EBRT, despite its substantial utilization as definitive treatment for patients with localized prostate cancer.
Because radiation oncologists and radiotherapy facilities must report specific, technical information on radiotherapy delivery for reimbursement, we anticipated that details concerning adherence to RAND-proposed measures of radiotherapy quality might be discernable in billing claims data. Accordingly, we undertook this study to characterize variation in adherence to RAND-proposed EBRT quality measures using the linked Surveillance, Epidemiology and End Results (SEER)-Medicare database, a population-based source of information on patients 65 years and older with cancer; and, secondarily, to examine the relation between variation in adherence and patient and provider characteristics.
The study cohort was comprised of patients from the Surveillance, Epidemiology and End Results (SEER)-Medicare database, which links patient demographic and tumor-specific data collected by SEER cancer registries to health care claims for Medicare enrollees. Information on incident cancer cases was available from 16 cancer registries from 1994 through 2002, covering 26% of the US population.13 Greater California, Kentucky, Louisiana, and New Jersey case contributions began in 2000. SEER registries collect data on each patient’s cancer site, extent of disease, histology, date of diagnosis, and initial treatment. We staged patients according to the American Joint Committee on Cancer Staging Manual, Sixth Edition.14
The Medicare program provides health care benefits to 97% of the US population 65 years old or older. SEER data have been linked to Medicare claims for inpatient and outpatient care, which provide information about initial treatment and allow patients to be followed longitudinally. Approximately 94% of patients in SEER aged 65 years or older have been successfully linked with their Medicare claims.15
Figure 1 shows criteria used to create the cohort of 23,018 patients with localized prostate cancer diagnosed between 1994 and 2002 who received primary EBRT. These patients were further stratified into three cohorts by year of diagnosis: 1994-96, 1997-99, and 2000-02. To characterize adherence to EBRT quality measures (our primary aim) in a contemporary cohort, we focus on the 11,674 men diagnosed from 2000 to 2002 (Figure 1). To report secular trends in adherence, we consider men in all three cohorts by year of diagnosis.
We classified initial therapy as definitive surgery, definitive radiotherapy (defined as EBRT, brachytherapy (BRT), or combination BRT with EBRT), primary androgen deprivation therapy (PADT), or expectant management occurring within 9 months after diagnosis. This time frame was selected to ensure sufficient time span from diagnosis for multidisciplinary consultation and initiation of adjuvant androgen deprivation therapy. We identified corresponding SEER variables and Medicare diagnosis and procedure billing codes for each therapy using International Classification of Diseases-9 (ICD-9)16 and Current Procedural Terminology (CPT)17 codes and searched for these codes in SEER data and Medicare claims. To enhance specificity, we required stringent evidence of radiotherapy delivery in ICD-9 or CPT codes. Codes used for classification of treatment are available from the authors by request.
Our objective was to evaluate the extent to which observed practice adheres to RAND-proposed EBRT structure and process quality measures.18 We chose a priori to evaluate only those EBRT structure and process measures that attained RAND expert panel agreement on both feasibility and validity and were amenable for analysis with SEER-Medicare data. In the RAND report, a measure was considered valid if adequate scientific evidence or professional consensus supported the measure as one that would differentiate lower-quality from higher-quality care. A measure was considered feasible if the information necessary to assess the measure could be obtained from a medical record, cancer registry, or other systematically collected data source. In addition, the RAND report characterized each measure by whether there was agreement or disagreement among the panel as to its validity or feasibility.11
Nine EBRT measures (two structure and seven process measures) attained RAND panel agreement on both feasibility and validity. Of these, one structure measure and four process measures were suitable for SEER-Medicare analysis (Table 1, grey). Therefore, a total of five quality measures met our selection criteria: 1) use of conformal radiotherapy treatment planning; 2) use of high-energy (>10MV) photons; 3) use of custom immobilization; 4) completion of two follow-up visits with a radiation oncologist in the year following therapy; and 5) radiation oncologist board certification.
The primary endpoints were adherence to the five EBRT quality measures proposed by the RAND expert panel that were amenable to analysis using SEER-Medicare data. We identified conformal radiotherapy treatment planning, high-energy (>10MV) photons, and custom immobilization using the strategy outlined in Table 1. CPT codes for conformal treatment planning are specific as they require three-dimensional beam’s eye view volume reconstruction and the calculation of dose-volume histograms as part of the radiotherapy simulation procedure.17, 19 CPT codes for each technical measure were in use throughout the study period, except for IMRT treatment planning codes, which were introduced in 2001.
We identified the number of follow-up visits with a radiation oncologist in the 12 months following completion of radiotherapy using Medicare claims for radiation oncologist visits (Table 1). We used physician specialty codes from Medicare claims and the American Medical Association (AMA) Physician Masterfile to identify radiation oncologists.20 Linkage between Medicare claims and the AMA Masterfile was achieved using Unique Physician Identifier Numbers (UPINs). We dichotomized follow-up visits as ≥2 or less than 2 according to the RAND quality measure.10, 11 For comparison, we also identified follow-up visits with urologists in the year following radiotherapy through similar methods using Medicare claims and AMA data. We obtained the board certification of the treating radiation oncologist from the AMA Masterfile.
For each patient, “adherence” to a radiotherapy quality measure was defined as having Medicare claims indicating measure performance. We calculated an “adherence rate” for each measure in which the numerator was the number of patients with adherence and the denominator was the total number of patients in the primary analytic cohort diagnosed from 2000 to 2002. We also calculated adherence to a composite technical measure representing the proportion of patients with adherence to all three technical measures (conformal radiotherapy, high-energy photons, and custom immobilization).
Demographic characteristics included age at diagnosis, race and ethnicity according to SEER designation, and marital status. Clinical characteristics included clinical tumor stage, according to the AJCC 6th edition staging system, Gleason’s sum, pretreatment PSA level, and use of adjuvant androgen deprivation therapy. We calculated a modified Charlson comorbidity index21 from Part A and Part B Medicare claims during a prediagnosis interval from 13 months to 1 month. Socioeconomic and geographic characteristics included diagnosis year, SEER registry, population of patients’ county or metropolitan area, and proxies for patient income and educational attainment. Because individual-level socioeconomic data are not available in either SEER or Medicare records, proxy measures were obtained from the 2000 US Census. The SEER-Medicare program links patient census tract of residence reported to SEER to census tract socioeconomic variables collected by the US Bureau of the Census. Income and educational attainment were estimated by census tract median household income and the percentage of adults in the census tract with less than high-school education. For approximately 6% of cases, the census tract of residence was unavailable, and zip code-level data were used.
We classified hospital-based radiotherapy facilities as teaching affiliated or community. Hospital teaching affiliation was self-reported in the Healthcare Cost Report and Provider of Service surveys available from the Center for Medicare and Medicaid Services for 2000, 2001, and 2002. Linkage between Medicare claims and hospital surveys was achieved using hospital provider identification numbers (PINs). We classified facilities whose radiotherapy claims were reported only through non-hospital-based Medicare National Claims History (NCH) records as stand-alone facilities.
For each quality measure and for the combined technical measure, we performed univariate and multivariate logistic regression to evaluate the relation between patient characteristics and measure adherence. We choose a priori to include all covariates in adjusted analysis. Lastly, we evaluated secular trends in the adherence rate for each measure for the 23,018 patients diagnosed between 1994 and 2002. All statistical analyses were conducted using SAS version 9.1 (SAS Institute, Cary, NC). Reported analyses were two-sided and considered statistically significant when P ≤ 0.05.
Demographic, clinical, socioeconomic, and provider characteristics are shown in Table 2. Of the 11,674 patients diagnosed from 2000 to 2002, 9,944 (85%) were white and 1,052 (9%) were black. Comorbidity was absent in 7,895 (68%), mild in 2,229 (19%), and moderate to severe in 902 (8%) patients. There were 5,219 (45%) patients treated at teaching hospitals, 2,900 (25%) at community hospitals, and 3,555 (30%) at stand-alone radiotherapy facilities.
Table 3 demonstrates adherence to each of the five RAND-proposed EBRT quality measures. The adherence rate for each technical measure was high: 85% received conformal radiotherapy treatment planning, 75% received high-energy photons, and 97% received custom immobilization. The overall adherence rate to the composite technical measure was lower (64%), due to discordance between receipt of conformal radiotherapy and high-energy photons. While 7,610 (65%) patients received both conformal radiotherapy and high-energy photons, 2,301 patients received conformal radiotherapy without high-energy photons and 1,095 patients received high-energy photons without conformal radiotherapy. Only 34% of patients adhered to the RAND quality metric for two follow-up visits with a radiation oncologist within one year of the completion of radiotherapy. The majority of patients (85%) were treated by a board certified radiation oncologist.
Because follow-up adherence with radiation oncologists was uniformly low, we also analyzed the proportion of patients who had follow-up visits with either urologists or radiation oncologists in the year following completion of radiotherapy. In the year following completion of radiotherapy, 80% of patients had at least 2 follow-up visits with either urologists or radiation oncologists. Furthermore, 68% of patients had at least 1 (rather than 2, as proposed by RAND) follow-up visit with a radiation oncologist and 91% had at least 1 follow-up visit with either a urologist or a radiation oncologist.
In Table 3, characteristics significantly associated (P ≤0.05) with measure adherence on both unadjusted and adjusted analysis are displayed in bold type font. No significant difference was noted among white and black patients in the use of conformal radiotherapy (85% vs. 86%, respectively). Older age was significantly associated with declining adherence to the follow-up visit measure. Notably, older age was not associated with less follow-up among patients seen by both a urologist and a radiation oncologist (data not shown).
We did not find any meaningful trends to suggest that adherence varied substantially by clinical characteristics such as tumor stage, Gleason’s sum, pretreatment PSA, adjuvant androgen deprivation, or comorbidity index (Table 3).
Among all of the measures, there was substantial variation in adherence on adjusted analysis by SEER registry and county or metropolitan population. For example, adherence to the conformal radiotherapy measure varied from 65% in Kentucky to 95% in Detroit and 78% in less populated areas (<250,000 persons) to 88% in highly populated areas (≥1,000,000 persons). Higher census tract education and income levels were also associated with higher adherence (Table 3).
Radiotherapy facilities affiliated with teaching hospitals had significantly higher adherence to each quality measure compared to community hospital affiliated facilities and stand-alone facilities (Table 3). However, teaching hospitals and stand-alone facilities both had high adherence to conformal radiotherapy (89% and 86%, respectively).
Table 3 also identifies patient characteristics that were associated with adherence to the composite technical measure (conformal radiotherapy, high-energy photons, and custom-immobilization). On adjusted analysis, adherence did not vary meaningfully by demographic or clinical characteristics. However, there was significant variation in adherence by geographic area of SEER registry, population, radiotherapy facility teaching affiliation, census tract education level, and year of diagnosis. For example, adjusted analysis demonstrated lower adherence to the composite technical measure among community hospitals and stand-alone facilities when compared to teaching hospitals (51% vs. 61% vs. 75% for stand-alone facilities, community hospitals, and teaching hospitals, respectively). Unadjusted and adjusted regression models for each measure are available from the authors by request.
Figure 2 illustrates secular trends in adherence to the five radiotherapy quality measures for the 23,018 men diagnosed from 1994 to 2002. Conformal radiotherapy adherence increased from 6% (122 of 1,997 patients) in 1994 to 91% (3,730/4,086) in 2002. Adherence to the use of high-energy photons and custom immobilization also increased, though less dramatically. The proportion of patients who had consistent follow-up with radiation oncologists plateaued at approximately one-third while self-reported board certification increased slightly during the study period.
We conducted this study to characterize variation in adherence to RAND-proposed EBRT quality measures for the treatment of localized prostate cancer using the linked SEER-Medicare database. Analyzing administrative claims data from 11,674 elderly men diagnosed between 2000 and 2002, we observed a high level of adherence to each of the three EBRT technical measures. However, overall high technical measure adherence masked variation in adherence by geography, socioeconomic status, and radiotherapy facility teaching affiliation.
Our findings are consistent with and extend previous studies describing cancer care quality. Results of the National Initiative for Cancer Care Quality documented overall high adherence to evidence-based quality measures among patients with colorectal (86% adherence) and breast cancer (78% adherence), but noted variability in adherence across metropolitan areas.7 Other studies of breast cancer care quality have found that geography and higher socioeconomic status are associated with higher adherence to quality measures and variation in treatment.22, 23 While previous work has noted the importance of geographic and socioeconomic factors in initial therapy selection for men with prostate cancer24, 25, our work highlights these factors as potentially contributing to variation in the quality of radiotherapy treatment delivery. Lastly, our work extends findings from patterns of care surveys that have documented increasing use of conformal radiotherapy for the treatment of localized prostate cancer.26
The SEER-Medicare data lack sufficient detail to determine the reasons underlying the observed variation in adherence. The principle advantage of using SEER-Medicare data is that it portrays, in broad brush strokes, an overview of radiation oncology treatment delivery to older Americans. Follow-up studies using more detailed sources of data may drill down to determine reasons for underlying variation. The variation we observed has several plausible explanations. Patients may receive care in settings which may differ in their capacity to deliver high quality care.27 This may relate to the geographic distribution of facilities with advanced technologies and patients’ ability to access to them.28 Because all patients in our cohort had Medicare insurance, financial access barriers are a less likely source of variation. Nonetheless, financial hurdles above and beyond insurance (such as co-payments, transportation costs, or time spent away from work) may present challenges for some patients.
We observed widely different adherence to the follow-up measure. The RAND report specifies that at least 2 follow-up visits be completed by the treating physician in the first post treatment year.10, 11 However, while only one-third of patients consistently follow-up with their radiation oncologists, 80% of patients follow-up with either their radiation oncologists or urologists, suggesting that urologists are playing an active role in caring for patients who receive radiotherapy. We also found that two-thirds of patients had follow-up visits with their radiation oncologists at least once in the year following therapy. Perhaps, following radiotherapy, radiation oncologists triage patients back to referring urologists. It is also possible that patients prefer to maintain their care relationships with both referring urologists and treating radiation oncologists. In addition, urologists and treating radiation oncologists may have a cooperative approach to follow-up visits that incorporates both specialties.
Multidisciplinary care is important and likely beneficial to men after radiotherapy for prostate cancer. Yet, patients may also gain from maintaining follow-up with their radiation oncologists, who have particular training in evaluating and managing the potential long-term toxicities of radiotherapy. Whether care quality or outcomes differ among patients followed by referring urologists compared to treating radiation oncologists is uncertain and cannot be ascertained from our data. It is clear that observed practice patterns deviate from the RAND metric specifying two follow-up visits by treating radiation oncologists. As a result, greater awareness of follow-up in the radiation oncology community may be required.29 Furthermore, in view of the prevalence of “shared care” models between urologists and radiation oncologists, the quality metric itself merits clarification.
This work augments the emerging literature examining the RAND-proposed quality measures for localized prostate cancer.12, 30 One study assessed RAND measures for 168 men at a single academic institution and found generally high adherence.12 However, the investigators were unable to measure adherence to the technical measures of radiotherapy despite extensive review of electronic data and medical charts. Often, radiotherapy documentation is stored within outpatient radiotherapy departments or stand-alone facilities, separate from inpatient hospital documentation and not consistently included in patient charts. We found that important, specific technical measures of radiotherapy quality can be assessed for large numbers of patients using administrative claims data. Nonetheless, challenges in accessing radiotherapy documentation could be alleviated by developing standardized treatment summaries that include appropriate quality metrics and become part of patients’ permanent medical records.
Recently, Miller et al assessed adherence to RAND radiotherapy quality metrics among a sample of 1,385 men diagnosed with localized prostate cancer between 2000 and 2001 who received EBRT.31 The great majority (93%) of the cohort was 60 years or older. Using explicit chart review to assess quality measure adherence, the investigators found similarly high adherence to measures of computed tomography planning (88% adherence), high-energy photons (82%), and board certification (94%), compared to 85%, 75% and 85%, respectively, observed in the current claims-based analysis. However, they found lower adherence to immobilization (66%) and higher adherence to follow-up visits (66%) than we observed (97% and 34% in the current study, respectively). While the similarities lend external validity to our findings, the differences raise questions about the extent to which quality measure performance is accurately reflected in medical chart documentation versus Medicare claims.32
Differences in claims reimbursement among quality measures may be one reason for the discordance between the findings of Miller et al and the current study. Claims for conformal radiotherapy and beam energy represent the largest portion of EBRT reimbursement.19 Chart documentation and Medicare claims are likely to be similar for these measures, as omitting chart documentation or claims for these highly reimbursed processes (Table 1) would be akin to not noting or billing for surgeries. There may be less financial incentive to report claims for follow-up visits, leading to differences between what is noted in patient charts and what is reported in claims. Nonetheless, the high proportion of patients (91%) who had at least 1 visit with radiation oncologists and urologists suggests that unbilled visits may not be a logical explanation for the lower follow-up observed in our study. Furthermore, we observed higher adherence to the immobilization measure in claims data compared to chart data, despite its lower reimbursement, suggesting that immobilization may have been provided but not documented in the chart. Therefore, the definition of what constitutes follow-up and with whom and the extent of immobilization may differ between claims data and chart data. These differences highlight the need for collaborative, complimentary methods of care quality assessment, combining the broad-based view of administrative databases with the nuanced, detail-rich information offered by direct medical chart abstraction.33
The use of administrative claims data for quality assessment has both advantages and limitations. While claims data can be efficiently analyzed to monitor population-based adherence to quality measures, such data are not collected for research purposes, can lack important, clinically relevant information, and may under-report treatment or processes of care that may be critical to quality ascertainment.34-37 Our research did not examine a comprehensive set of quality measures for EBRT. For example, we were not able to assess radiotherapy dose, a RAND-proposed quality measure, because it is not captured in SEER data or Medicare claims. Furthermore, while we were able to assess claims for conformal radiotherapy, we could not measure the quality of conformal planning itself (eg protection of the rectal mucosa, another RAND-proposed measure). Another limitation is that measures of socioeconomic status were captured at the census tract rather than the individual level. While previous studies have validated the use of proxy census indicators for individual socioeconomic status in health services research, such estimates may not provide precise measures of individual socioeconomic status.38 Lastly, our conclusions may reflect variation in coding practices rather than variation in care quality. While measure adherence differed among SEER registries, perhaps this reflects systematic, less detailed claims reporting in certain geographic areas rather than variation in care quality.35
In a population-based cohort of men with localized prostate cancer, we found that high adherence to care quality measures proposed by a RAND expert panel masked substantial variation in care quality by geography, socioeconomic status, and radiotherapy facility teaching affiliation. Future research should examine reasons for variation in these measures and whether variation is associated with important clinical outcomes.
This study used the linked Surveillance, Epidemiology, and End Results (SEER)–Medicare database. The interpretation and reporting of these data are the sole responsibility of the authors. The authors acknowledge the efforts of the Applied Research Program, National Cancer Institute, the Office of Research, Development and Information, Centers for Medicare & Medicaid Services, Information Management Services, Inc., and the SEER Program tumor registries in the creation of the SEER–Medicare database. We thank Yihai Liu, MS for assistance with data manipulation in the early stages of this project.
Funding/Support: Supported by an individual postdoctoral fellowship Ruth L. Kirschstein National Research Service Award (National Cancer Institute 1F32 CA 123964-01, Dr. Bekelman).
Meeting Presentation: This work was selected to be presented in part at the American Society of Clinical Oncology 43rd Annual Meeting, Chicago, IL, June 1 - 4, 2007 and to receive a 2007 ASCO Foundation Merit Award and a C. Julian Rosenthal Fellowship Award.
Institutional Review Board Approval: SEER-Medicare data are submitted electronically to the National Cancer Institute without personal identifiers (ie, are de-identified) and are available to the public for research purposes. As portions of this study required access to restricted variables identifying radiation oncologists and urologists, this research was approved by the review boards of the Memorial Sloan-Kettering Cancer Center and each of the individual 16 SEER registries from which data was obtained.
Conflicts of Interest Notification The authors report no actual or potential conflicts of interest.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.