|Home | About | Journals | Submit | Contact Us | Français|
Appropriate care is often not defined and when defined, is often not uniformly provided across institutions or demographic populations in the current American healthcare system.
We sought to investigate receipt of appropriate surgical care in Medicare beneficiaries with cancer.
Retrospective cohort study using national Surveillance Epidemiology and End Results (SEER) registry linked to Medicare claims data.
Fee-for-service Medicare patients aged 65 years or older who underwent a definitive surgical resection for breast, colon, gastric, rectal, or thyroid cancers that were diagnosed between 2000 and 2005. Claims data were available from 1999 through 2007.
Receipt of care concordant with established practice guidelines in surgical oncology in the aggregate and by hospital.
Concordance with guidelines was > 90% for seven of eleven measures. All guidelines regarding adjuvant therapy had concordance rates >90%. Only two of five measures for nodal management had concordance rates >90%. At least 50% of hospitals provided guideline-concordant care to 100% of their patients for 6 of 11 guidelines. Patients receiving appropriate care tended to be younger, healthier, white, more affluent, have less advanced disease, and live in the midwest.
We found a high level of concordance with guidelines in some domains of surgical oncology care, but far less so in others, particularly for gastric and colon nodal management. Given the current national focus on improving the quality of healthcare, surgeons must focus on generating data to define appropriate care and translating that data into everyday practice.
There is currently a major focus on improving the quality of healthcare in America. Quality healthcare means delivering the right care to the right patient at the right time. In order to ensure such care is provided, we must first know what “right” or appropriate care is and then be able to determine whether that care was provided. Currently, it is well-documented that the practice of healthcare in America varies widely across both institutions and demographic populations; this is true across specialties and disease sites.1-2 This variation may reflect two things: (1) a lack of knowledge about the optimal approach to care or (2) a lack of acceptance regarding currently defined standards of care.
We sought to use practice guidelines to determine whether appropriate surgical care was provided to Medicare beneficiaries with a new diagnosis of cancer. We investigated factors associated with likelihood of receiving guideline-concordant care at a patient-level. We then analyzed guideline concordance at an aggregate patient level to determine the degree of variation in hospital-level performance to study whether each guideline has been accepted into routine practice.
We organized our study around the three areas in which a surgeon treating cancer must be proficient: 1) surgical management of the primary tumor; 2) evaluation and treatment of regional nodal basins; and 3) appropriate referral for multidisciplinary adjuvant therapy. We first identified existing disease-specific guidelines in each of these areas for five common cancers in which surgery plays an important role , including breast, colon, gastric, rectal, and thyroid cancers, based on the following criteria:
In order to classify the strength of a given guideline we used the classification system published by the National Comprehensive Cancer Network.4 The NCCN clinical practice guidelines are readily available, widely used, span multiple cancer sites and classify the strength of both the evidence and panel members’ consensus supporting a recommendation. The NCCN identifies four guideline categories:
The Surveillance Epidemiology and End Results (SEER) program of the National Cancer Institute collects detailed data including stage at diagnosis, tumor morphology, first course of treatment and demographic variables for persons with cancer living in a SEER region.5 SEER data has been linked to Medicare enrollment and claims data in order to facilitate evaluation of treatment and outcomes of care across the cancer continuum.6
Patients diagnosed with breast, colon, gastric, rectal, or thyroid cancers as their first or only cancer between the years 2000 and 2005 were identified and linked to Medicare claims data from 1999 through 2007. We required that all patients have at least one cancer-specific surgical procedure between 30 days before through 365 days after diagnosis and be associated with a non-missing hospital identifier. Patients were also required to be continuously enrolled in Medicare and have no HMO enrollment during this time period.
Between 2000 and 2005, there were a total of 181,534 patients with breast cancer, 116,383 with colon cancer, 22,625 with gastric cancer, 41,857 with rectal cancer, and 10,453 with thyroid cancer identified. Criteria used to specify the cohort are shown in Figure 1. We excluded patients under 65 years of age at the time of diagnosis and those with end stage renal disease or disability as their qualifying condition. Patients with Stage IV or unknown stage (except for thyroid cancer) or a discrepancy in the death date reported in SEER and Medicare of greater than 3 months were also excluded.
Additional specific inclusion and exclusion criteria for the denominator and numerator for each guideline were applied and are available from the authors upon request. For situations where there was a discrepancy or ambiguity between identified guidelines regarding appropriate criteria, we performed sensitivity analyses varying the inclusion criteria. Tumor size criteria for receipt of total thyroidectomy were varied between 1 and 2cm. As conclusions did not change, results are only presented for a threshold of 1.5cm. Because this study was focused specifically on the surgical practitioner, guidelines that related to the receipt of adjuvant therapy were expanded to include a visit with a medical oncologist or radiation oncologist regardless of whether the patient received treatment. Medical oncologists were identified by an associated Medicare specialty code for medical oncology or hematology/oncology or having submitted a claim for chemotherapy.7-9 Radiation oncologists were defined by a Medicare specialty code for radiation oncology or having submitted a claim for radiation planning or administration.
Concordance is measured as the proportion of patients who met the numerator criteria among all patients who met the denominator criteria for each guideline. The concordance rate for each guideline was determined at the aggregate patient level, by considering all patients with the relevant condition and at the hospital level, by considering all hospitals that treated at least 5 patients with the relevant condition. This was done to ensure stability in the estimate of the hospital concordance rate. While patients treated at hospitals with fewer than 5 patients were excluded from the hospital-level analysis, they were included in the aggregate patient-level analysis. We report aggregate patient-level concordance rates with 95% confidence intervals. Generalized estimating equations were used to account for clustering at the hospital level when calculating 95% confidence intervals.10 For the hospital level analysis, we determined the proportion of hospitals for which 100% of treated patients received guideline-concordant care, as well as exact binomial 95% confidence intervals. For each of the guidelines meeting our eligibility criteria, logistic regression models were built to investigate covariates associated with receipt of concordant care. Generalized estimating equations were used to estimate the logistic regression parameters, which allowed us to investigate both patient and known institutional factors while accounting for clustering at the institutional level. Because the logistic regression analyses were exploratory, we did not adjust the type I error rate to account for multiple comparisons; thus, p values should be interpreted cautiously. The number of significant associations (p <0.05) in these analyses is greater than expected by chance if there were no associations in the data. All tests were two-tailed. All data management and statistical analyses were performed using SAS v. 9.2 (Cary, NC).
Eleven guidelines were identified and are listed in Table 1, showing the original source, year it was proposed, and whether the National Comprehensive Cancer Network grades them as based on a high level of evidence (Grade 1) or uniform consensus but a lower level of evidence (Grade 2A).4, 11-18 Given that we are using guidelines to represent appropriate care, we did not include any guidelines for which there was non-uniform consensus or major disagreement. The 11 guidelines include five for breast cancer, two for colon cancer, one for gastric cancer, one for rectal cancer, and two for thyroid cancer. They represent the following domains of surgical oncology care: 1 measure for surgery directed at the primary cancer, 5 measures for nodal management and 5 measures related to adjuvant therapy.
Table 2 provides a description of the cohort for each disease site included in this analysis.
Concordance with guidelines was greater than 90% for seven of the eleven measures (see Table 3). All guidelines regarding evaluation for or receipt of adjuvant therapy had concordance rates >90%. The measures that evaluated breast cancer management had the highest concordance rates. These included radiation therapy or evaluation following breast conserving surgery (BCS) (99.2%), chemotherapy or medical oncology evaluation for estrogen receptor (ER) negative breast cancer (98.1%), axillary dissection for node positive breast cancer (96.7%), post-mastectomy radiation therapy (PMRT) or evaluation for >4 positive nodes, stage III, node positive and T>5cm (94.9%) cancers and nodal evaluation for invasive breast cancer (91.8%). The lowest concordance rates were seen for central neck dissection for node-positive thyroid papillary cancer (72.7%), colon cancer nodal evaluation of at least 12 nodes (48.5%), and gastric nodal evaluation of at least 15 nodes (32.5%).
As shown in Figure 2, there was wide variation in the proportion of hospitals providing uniformly guideline-concordant care, i.e., treating 100% of their patients according to the guideline recommendation. Fewer than 1% of institutions met this standard for colon cancer nodal evaluation or central neck dissection while 93% provided either evaluation for or treatment with radiation for all patients following BCS. More institutions had uniformly concordant care for guidelines concerning adjuvant therapy than for guidelines dealing with surgery directed at the primary cancer or nodal management.
Table 4 summarizes the factors associated with increased or decreased likelihood of receiving concordant care by guideline. For a number of the guidelines, younger age, less aggressive disease, white race, higher income level, being married, and care at a hospital that participates in an oncology group were independently associated with higher guideline concordance. Patients treated in the Midwest were more likely than patients treated in the Northeast to receive guideline-concordant care for all measures; the direction of the effect for the South and West (as compared to the referent Northeast) varied by measure.
Using practice guidelines to define appropriate care, we found that over 90% of all patients received recommended care for 7 of the 11 guidelines examined at an aggregate patient level. We also identified 6 measures for which at least half the institutions were concordant with the guidelines 100% of the time. These high concordance rates suggest that the factors influencing clinical decision-making are adequately captured in the current guidelines and surgeons recognize the importance of these therapies. As a result, we were able to demonstrate that Medicare beneficiaries are highly likely to receive appropriate care and that this finding is consistent across hospitals.
For several of the guidelines related to nodal management, however, concordance rates were low and few hospitals provided appropriate care to all patients. Specifically, we found that published guidelines recommending examination of a minimum number of lymph nodes in colon and gastric cancer have not resulted in routine adoption of this practice in elderly Americans despite the inclusion of these recommendations in the AJCC staging manual . It is possible that despite apparent expert consensus regarding the importance of evaluation of a minimum number of nodes, the lack of definitive evidence supporting a particular threshold has left many practitioners (surgeons and/or pathologists) unconvinced. They may not believe the potential benefits of more extensive lymph node harvests outweigh the added operative risks, perhaps particularly in older patients. The limitations of the evidence base supporting nodal evaluation and the difficulty in defining an appropriate nodal threshold for use in guidelines or quality measurement are well-described in two recent reviews, a report from the Cochrane Collaboration regarding extent of lymph node dissection for gastric cancer and a meta-analysis on colon cancer nodal evaluation. 19-20
We observed a somewhat different pattern of concordance for central neck dissection for node positive papillary thyroid cancer, with a higher aggregate concordance rate of 71.6%. In the hospital-level analysis, but no institutions at all performing neck dissections in all of their patients. Instead, most institutions performed them in approximately 80% of patients. One possible explanation for these findings is that there is agreement that a central neck dissection constitutes appropriate care for most but not all elderly patients with node positive papillary thyroid cancer, and that the factors relevant to selecting patients for the procedure are not adequately captured by the current guideline inclusion criteria, at least in this population of elderly Americans. Such factors might include the presence of a macro-rather than micrometastasis, or nodes detectable pre-operatively or at the time of surgery, rather than only on post-operative pathology review.
Several other explanations could account for the patterns observed in this study. Prior research on the Hospital Quality Alliance measures has suggested an association between the length of time guidelines have been in place and concordance rates.21 We found high concordance rates and minimal variation for the breast measures, guidelines that were among the first to be developed in cancer care.
It's also possible that because referral for adjuvant therapy is a dichotomous decision, which facilitates both compliance and measurement, it's easier for institutions to achieve consistent and high concordance rates on these measures than on guidelines measuring a continuous outcome such as number of lymph nodes examined. Alternatively or in addition, the fact that referrals result in reimbursement for the provider or the health system while lymph node harvests do not may contribute to the higher concordance we observed for guidelines related to adjuvant therapy.
Finally, there is a suggestion in our data that the level of evidence on which a guideline is based is associated with the level of concordance. Six of the 8 guidelines with concordance rates > 90% were based on high level evidence and all 3 of the guidelines with concordance rates <90% were based on lower level evidence (Table 3). Similarly, 5 of the 6 guidelines where more than 50% of institutions provided concordant care to 100% of their patients were graded as 1, while 4 of the 5 guidelines for which less than half of the institutions provided fully concordant care were graded 2 (Figure 2). The role that level of evidence may play in acceptance of guidelines deserves further investigation.
This study has the usual limitations associated with analyses of large national databases, including incomplete capture of cases, loss of follow up, and missing data as well as the limitations of claims data such as variation in billing practices and coding inaccuracies. Additionally, because Medicare provides the only consistent and comprehensive national data source on medical services delivered, our analysis was limited to patients 65 and older. A focus on older patients did allow us to study a particularly vulnerable population and one that makes up the majority of patients with gastric and colorectal cancer. However, breast and thyroid cancer are common in younger Americans, so our reliance on Medicare data represents a more significant limitation for these diagnoses.
We found a high level of concordance with guidelines in some domains of surgical oncology care, but far less so in others, especially those that are associated with nodal management. Five of the six measures with wide acceptance into practice relate to appropriate referral for or receipt of adjuvant therapy. Given the current national focus on quality in healthcare, there is increasing pressure to develop measures to determine whether patients are getting appropriate care; however, within the surgical disciplines, there is a paucity of data to support what constitutes appropriate care. It is critical that surgeons focus on the generation of the data necessary to inform clinical decision-making and promote quality surgical care.
This study used the linked SEER-Medicare database. The interpretation and reporting of these data are the sole responsibility of the authors. The authors acknowledge the efforts of the Applied Research Program, NCI; the Office of Research, Development and Information, CMS; Information Management Services (IMS), Inc.; and the Surveillance, Epidemiology, and End Results (SEER) Program tumor registries in the creation of the SEER-Medicare database.
This work was supported in part by a grant from the American Surgical Association Foundation. The sponsor did not have any role in design or conduct of the study or manuscript preparation or review.
Authors’ contributions: conception and study design (Greenberg, Lipsitz, Jha, Gawande, J. Weeks), data acquisition (Greenberg, Neville, Hevelone), data analysis (Greenberg, Lipsitz, Neville, Hevelone, Porter, In, C. Weeks, Schrag), data interpretation (all authors), drafting of manuscript (Greenberg, Porter, C. Weeks, In), critical review of manuscript (Lipsitz, Neville, Hevelone, Jha, Gawande, Schrag, J.Weeks), final approval of published version (all authors). Dr. Greenberg had full access to all of the data in the study and takes personal responsibility for the integrity of the data and the accuracy of the data analysis. The authors report no financial conflicts of interest.
The authors would like to acknowledge the contribution of Scott Regenbogen, MD, MPH to study conception and design and Katherine Corso MPH for her assistance in manuscript preparation.