PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
Med Care. Author manuscript; available in PMC 2014 April 1.
Published in final edited form as:
PMCID: PMC3662234
NIHMSID: NIHMS446791

Intended Versus Inferred Management after PET For Cancer Restaging: Analysis of Medicare Claims Linked to a Coverage With Evidence Development Registry

Abstract

Background

The National Oncologic PET Registry (NOPR) ascertained changes in the intended management of cancer patients using questionnaire data obtained before and after PET under Medicare’s coverage with evidence development (CED) policy.

Objective

To assess the concordance between intended care plans and care received as ascertained through administrative claims data.

Research Design

Analysis of linked data of NOPR participants from 2006–2008 and their corresponding Medicare claims.

Subjects

Consenting patients age >65 years having their first PET for restaging of bladder, kidney, ovarian, pancreas, prostate, small cell lung or stomach cancer.

Measures

Agreement [positive predictive values and kappas] between NOPR post-PET intended management plans for treatment (systemic-therapy, radiotherapy, surgery or combinations), biopsy, or watching as compared to claims-inferred care 30 days after PET.

Results

8,460 patients with linked data were assessed. 43.5% had metastatic disease. 45.3% had treatment planned (predominantly systemic therapy only), 11.1% biopsy and 43.5% watching. Claims confirmed intended plans (PPV) for single-mode systemic therapy in 62.0%, radiation in 66.0%, surgery in 45.6%, and biopsy in 55.7%. 25.7% of patients with a plan of watching had treatment claims. By cancer type, kappa ranged for systemic-therapy only from 0.17–0.40 and for watching from 0.21–0.41. Agreement rates varied by cancer types but were minimally associated with patient age, performance status, comorbidity or stage.

Conclusion

Among elderly cancer patients undergoing PET for restaging, there was moderate concordance between their physicians’ planned management and claims-inferred actions within a narrow time window. When higher accuracy levels are required in future CED studies, alternative designs will be needed.

Keywords: positron emission tomography, cohort studies, Medicare, medical record linkage, health services research, neoplasm staging

INTRODUCTION

Positron emission tomography (PET) using 18F-fluorodeoxyglucose represents an imaging paradigm based on characterizing metabolic processes. Since 2001, the Centers for Medicare & Medicaid Services (CMS) has covered PET for many common cancers.(1) However, about one-fourth of beneficiaries developing cancer have a non-covered cancer type.(2) For these cancers, it was recognized that developing sufficient evidence to inform coverage decisions would be unlikely. Therefore, CMS initiated a novel mechanism under its coverage with evidence development (CED) policy.(25) In response to the CED requirements, the National Oncologic PET Registry (NOPR) opened with a primary objective of measuring PET’s impact on intended patient management by collecting prospective questionnaire data before and after PET.

Two criticisms of NOPR have been that changes in planned management are only a surrogate for actual health outcomes and that the dataset does not document the care actually delivered.(11, 12) Concordance studies, between recommended actions and actual care, are rare and are important for informing the design and implementation of future CED programs using health outcome surrogates. We measured confirmation of intended plans with inferred management using a linked dataset of NOPR participants’ CMS claims following PET for restaging.

PATIENTS and METHODS

NOPR is a prospective data registry (ClinicalTrials.gov #NCT00868582); its operational details, human subject protection procedures, and findings of PET’s impact on intended management were previously reported.(710) In brief, the PET facility collects referring physician responses on pre-PET and post-PET forms. The pre-PET form collects the testing indication, the cancer type, working stage, performance status, and the referring physician’s plan if PET were not available. After PET, the referring physician records an estimate of the patient’s stage and management plan in light of the PET findings.

Claims Linkage

We linked NOPR data from December 2006 thorough 2008 for consenting participants to their CMS claim files by matching individual identifiers for the seven most frequent cancer types (Appendix 1, supplemental digital content 1). We assessed the first PET for restaging or suspected recurrence (hereafter labeled as restaging) and excluded patients who were age <65 years; were HMO participants; or for whom we were unsuccessful in linking identifiers, the registry and claim dates for PET differed by >7 days, or the post-PET plan was “other treatment(s) or additional imaging”.

Management Categories

The post-PET categories assessed were watching, biopsy, and treatment. Treatment categories were systemic-therapy (chemotherapy, hormonal, or immunotherapy), radiotherapy, or surgery alone or in combination.

Claims Definitions

Appendix 2, supplemental digital content 2 lists the Current Procedural Terminology (CPT©) codes, the Healthcare Common Procedure Coding System (HCPCS) codes and International Classification of Diseases (ICD-9) codes for outpatient care used to classify claims into inferred-care categories.

Given the numerous cancer types assessed, a list of possible biopsy or surgery CPT codes would likely be incomplete. Instead, the claims-inferred definition of “biopsy” was primarily based on surgical-pathology or cytology CPT coding.

Our preliminary analysis found surgical procedures for complications (e.g., chest tube insertion) rather than directed at the patient’s cancer. Therefore, we used anticipated surgical CPT codes and anesthesia claims after excluding eye, central vascular access, gastrointestinal endoscopy and conscious sedation. Chemotherapy, hormonal therapy and immunotherapy were inferred from professional claims, HCPCS codes for hospital-based chemotherapy and drug J-codes. Radiotherapy codes included all common techniques. These systemic-therapy and radiotherapy definitions were similar to those used in SEER-Medicare analyses.(1315)

The NOPR forms did not specify an action timeframe. Given the indication of restaging, we used 30 days as the default and explored extending it to 60 days.

Comorbidity

We determined the Klabunde comorbidity index(16), derived from inpatient and physician claims in the preceding 12 months, using a publically available SAS algorithm(17).

Specialties

The CMS provider part-B taxonomy codes for physician specialty designation were used to categorize referring providers. If no specialty or non-physician coding was found, then specialty was coded as “other”.

Statistical Analysis

The initial analyses treated the claims-inferred care as the reference standard to calculate measures of agreement between treatment plans and claims-inferred actual management, including positive predictive value (PPV), raw agreement, and kappa (chance-adjusted agreement).18 For the treatments (systemic-therapy, radiotherapy or surgery), agreement was defined as claims for that type action within the interval without considering other treatments; for biopsy, agreement included any procedures with surgical pathology or cytology claims; and for watching, it was the absence of treatment claims. The measures of agreement were computed separately by cancer type and compared with chi-square tests.

To assess the effect of patient, cancer, and provider factors in predicting agreement, separate logistic regression models were fit for patients with plans for systemic-therapy only, radiation only, surgery only, and watching. The outcome was the indicator of agreement with the plan. Calculations were done with PROC LOGIST in Linux SAS version 9.2.

RESULTS

Clinical characteristics

Table 1 summarizes the cancer type, age, performance status, comorbidity scores, referring specialty and pre-PET plan for our 8,460 patient cohort.

Table 1
NOPR cohort clinical characteristics

Overall, the stage distribution showed 43.0% with no evidence or low probability of recurrence (range 36.9%–51.4%), 43.5% with metastatic disease (range 35.9%–52.0%) and 13.5% with local or nodal recurrences (range 8.3%–17%).

The referrers were medical oncologists in one-half of all patients, gynecologists/gynecologic oncologists in one-third of ovarian cancer patients, and urologists in 9%–16% of bladder, kidney and prostate cancer patients. Radiation oncologists and surgeons were infrequent referrers except in prostate and small cell lung (SLC) cancer (radiation oncologists) and stomach cancer (surgeons).

In the 30 days following PET, 1.6% had died (range 0.4%–3.6%).

NOPR Plan

Table 2 summarizes the post-PET intended plans. Modest distribution variations were found by cancer type. Overall, treatment was planned in 45.3% (range 35.9%–52.8%), biopsy in 11.1%, and watching in 43.5% (range 38.7%–50.6%). For all types, the most common treatment plan was systemic-therapy only (range 20.7%–39.1%). Combination therapy, radiotherapy or surgery only were planned in <11% in all cancer types. Combination therapy components were 34.8% systemic-therapy, 12.4% radiotherapy, and 6.4% surgery.

Table 2
NOPR post-PET intended management plan

30-day Agreement

Table 3 shows by cancer type, the agreement between NOPR intended plan and claims-inferred actions.

Table 3
Agreement of post-PET plan and claims-inferred actions at 30 days by cancer type

For systemic-therapy alone, NOPR plans had a PPV of 65–67% for bladder, ovarian, pancreas, prostate, and SLC cancers and 51.5% for stomach cancer; the PPV for kidney cancer was lowest (40.1%), as anticipated. A Part D analysis of 40% of the cohort found 3% additional kidney cancer chemotherapies (see footer Table 3). Raw agreements ranged from 65.7% to 73.7% and kappas ranged from 0.17–0.40, in the slight to fair range. Among the radiotherapy alone cohorts, the PPVs were slighter higher (56.1%–78.4%), as were their kappas of 0.23 to 0.48, in the fair to moderate range.

Surgery only was planned in over 50 patients in kidney, bladder and ovarian cancer. For this group, the PPV ranged from 44.0%–57.5%, with only modest kappas (0.14–0.27).

Among planned combination therapy patients, claims for at least one of the planned therapies were found in 69.0% (data not shown).

A biopsy plan was confirmed by claims in 55.7% (range 48.0%–64.7%). Non-treatment (watching) plans were confirmed in only 76.3% of patients and were lowest in SCL and stomach cancer patients.

Timeframe

Extending the post-PET window to 60 days increased the PPV by 6%–13% for systemic-therapy, 0–6% for radiotherapy, and 12–18% for surgery. However, there was also an overall decline in kappas (Appendix 3, supplemental digital content 3).

Predictors

Table 4 shows the impact of patient age, performance status, comorbidity, cancer type, stage, referring specialty, and pre-PET plan in predicting post-PET plan and claims agreement.

Table 4
Factors associated with agreement of treatment plan and claims for treatment

With the exception of cancer type, the other factors had little impact in predicting PPV (62.0%) for systemic-therapies. If the referrer was a medical oncologist or if the pre-PET plan was also treatment, then the PPV increased minimally (3%–4%’s).

Radiotherapy’s PPV was slightly greater at 66%. Patient factors or cancer stage were non-predictive. Radiotherapy was most commonly planned in prostate cancer, yet it had the second lowest PPV (56%). Not surprisingly, when the referrer was a radiation oncologist, the PPV was greatest (81%).

Surgery alone was an infrequent plan and had a PPV 46%. Age over 75 years or a non-surgeon referrer was associated with even lower PPVs.

The absence of treatment claims in the intended watching cohort was unrelated to age and was more common if patients had a good performance status, low co-morbidity, non-metastatic stage, had kidney cancer, were not referred by a medical oncologist, and had a pre-PET plan other than treatment.

Discussion

We assessed how often referring physicians’ intended management after PET in the NOPR database, is concordant with actual management inferred from Medicare claims for the indication of restaging of previously treated cancers. We found only moderate agreements across all NOPR plan categories—treatment, biopsy or watching (non-action). The PPV of systemic-therapy plans clustered between 64–67% in five different cancer types. Claims confirming plans for invasive procedures (surgery or biopsy) were particularly infrequent. Moreover, over 25% of patients with an observation plan had treatment claims within 30 days. When longer timeframes were used, the PPV rates increased but the overall kappas declined.

We sought correlations that might explain these differences by assessing patient, cancer, and physician factors. This was largely unsuccessful. Patient age, performance status, and comorbidity had little association with PPV within a treatment category. For the two most frequent actions, medical oncologists versus all other specialties had similar PPVs for claims confirming chemotherapy as well as similar crossover rates from watching plans to treatment.

Information gained from PET is only one factor influencing patient management. Unavailable or unmeasured factors include other diagnostic test results, the extent and type of prior treatment, whether the referrer is the patient’s primary cancer care provider, and the physician’s experience with PET. The NOPR management plan reflects the physician’s intent or recommendation shortly after the PET scan is reported, and not necessarily the patient’s agreement to a plan. Also, the accuracy of NOPR questionnaire responses was not independently confirmed by review of the physician’s office records. Accordingly, one source of discordance may be inaccuracy of the NOPR data.

Our results show agreement rates comparable to those of smaller, less detailed analysis by Henderson that assessed 489 NOPR scans for all indications in kidney and pancreatic cancer patients(18). Their analysis found only fair agreement rates for observation and treatment (kappa=0.39 and 0.36).

Another limitation is the expectation that claims will accurately reflect actual clinical actions performed. Since 2000, numerous reports using SEER-Medicare linked data have assessed the completeness of claims in identifying initial chemotherapy(1924), surgery(25, 26), and radiation(15, 27). Most of these studies used broad time windows (up to one year post-diagnosis). Since our analysis used a narrow 30-day window following PET, inaccurate claims service dates could be a major source of non-matching (false-negatives). Two recent reports partly address this issue. First, Lamont assessed CMS claims against trial data as the reference standard in six CALGB trial cohorts of first-line metastatic chemotherapy.(23) They found that claims correctly identified 78% of the drugs given and the treatment schedule. In a second study, Lund validated SEER initial treatment plans against hospital and outpatient records that were re-abstracted for chemotherapy in four cancer types.(24) They found that the claim sensitivities were very time dependent ranging from 36%–50% at two months to 84%–96% at six months.

Australian investigators have used a design similar to NOPR in that they collected prospective data at three to six centers for a range of cancer types in which PET was used predominantly for initial staging and determined the change in management plans associated with PET.(2832) They subsequently assessed medical records for actual care over the next 3 to 12 months. They found agreement rates ranging from 53% to 75% (average 65%), similar to our claims-inferred actions.

We previously noted that intended management changes might not always be appropriate.(9) Moreover, changes in intended actions have a presumed, but uncertain, relationship to more tangible health outcomes—progression-free or overall survival(33)—that are so heavily dependent on the effectiveness of the clinical actions chosen.

Our analysis highlights that intended management in patients with previously treated cancer does not consistently reflect implemented action. When higher accuracy levels are required, future evaluations of other diagnostic tests under the CED policy likely will require alternative designs.

Supplementary Material

Acknowledgments

Financial Support: National Cancer Institute Grand Opportunity Award RC2CA148259 and the Academy for Molecular Imaging (BEH). Funding for development of the NOPR was provided by the Academy for Molecular Imaging, but the registry is otherwise self-supported by the fees paid by participating PET facilities.

Contributor Information

Bruce E. Hillner, Department of Internal Medicine and the Massey Cancer Center, Virginia Commonwealth University, Richmond, VA.

Tor D. Tosteson, The Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth, Lebanon, NH.

Anna N. A. Tosteson, The Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth, Lebanon, NH.

Qianfei Wang, The Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth, Lebanon, NH.

Yunjie Song, The Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth, Lebanon, NH.

Tracy Onega, The Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth, Lebanon, NH.

Lucy G. Hanna, Center for Statistical Sciences, Brown University, Providence, RI.

Barry A. Siegel, Division of Nuclear Medicine, Mallinckrodt Institute of Radiology andSiteman Cancer Center, Washington University School of Medicine, St. Louis, MO.

References

1. Centers for Medicare and Medicaid Services. AB-01-54: Expanded Coverage of Positron Emission Tomography (PET) Scans and Related Claims Processing Changes. 2000 Dec 15; http://www.cms.gov/transmittals/downloads/R136CIM.pdf.
2. Lindsay MJ, Siegel BA, Tunis SR, Hillner BE, Shields AF, Carey BP, et al. The National Oncologic PET Registry: Expanded Medicare Coverage for PET Under Coverage with Evidence Development. AJR Am J Roentgenol. 2007;188(4):1109–13. [PubMed]
3. Tunis SR, Pearson SD. Coverage Options For Promising Technologies: Medicare’s ‘Coverage With Evidence Development’ Health Aff. 2006;25(5):1218–30. [PubMed]
4. Centers for Medicare & Medicaid Services. National Coverage Determinations with Data Collection as a Condition of Coverage: Coverage with Evidence Development. Document Issued July 12, 2006. http://www.cms.hhs.gov/Transmittals/downloads/R956CP.pdf.
5. Tunis SR, Carino TV, Williams RD, II, Bach PB. Federal Initiatives To Support Rapid Learning About New Technologies. Health Aff. 2007;26(2):w140–9. [PubMed]
6. Hillman BJ, Goldsmith JC. The Sorcerer’s Apprentice: How medical imaging is changing health care. New York: Oxford University Press; 2011.
7. Hillner BE, Siegel BA, Shields AF, Liu D, Gareen IF, Hanna L, et al. The impact of positron emission tomography (PET) on expected management during cancer treatment: findings of the National Oncologic PET Registry. Cancer. 2009;115:410–8. [PubMed]
8. Hillner BE, Siegel BA, Shields AF, Liu D, Gareen IF, Hunt E, et al. Relationship Between Cancer Type and Impact of PET and PET/CT on Intended Management: Findings of the National Oncologic PET Registry. J Nucl Med. 2008;49:1928–35. [PubMed]
9. Hillner BE, Siegel BA, Liu D, Shields AF, Gareen IF, Hanna L, et al. Impact of positron emission tomography/computed tomography and positron emission tomography (PET) alone on expected management of patients with cancer: initial results from the National Oncologic PET Registry. J Clin Oncol. 2008;26(13):2155–61. [PubMed]
10. Hillner BE, Siegel BA, Shields AF, Duan F, Gareen IF, Hanna L, et al. Impact of dedicated brain PET on intended patient management in participants of the national oncologic PET Registry. Mol Imaging Biol. 2011;13(1):161–5. Epub 2010/11/17. [PMC free article] [PubMed]
11. Levine MN, Julian JA. Registries That Show Efficacy: Good, but Not Good Enough. J Clin Oncol. 2008;26(33):5316–9. [PubMed]
12. Tunis S, Whicher D. The National Oncologic PET Registry: Lessons Learned for Coverage With Evidence Development. J Am Coll Radiol. 2009;6(5):360–5. [PubMed]
13. Keating NL, Landrum MB, Lamont EB, Bozeman SR, Krasnow SH, Shulman LN, et al. Quality of Care for Older Patients With Cancer in the Veterans Health Administration Versus the Private Sector. Ann Intern Med. 2011;154(11):727–36. [PubMed]
14. Lamont EB, Lauderdale DS, Schilsky RL, Christakis NA. Construct validity of medicare chemotherapy claims: the case of 5FU. Med Care. 2002;40(3):p201–11. [PubMed]
15. Smith BD, Pan I-W, Shih Y-CT, Smith GL, Harris JR, Punglia R, et al. Adoption of Intensity-Modulated Radiation Therapy for Breast Cancer in the United States. J Natl Cancer Inst. 2011;103(10):798–809. [PubMed]
16. Klabunde CN, Legler JM, Warren JL, Baldwin L-M, Schrag D. A Refined Comorbidity Measurement Algorithm for Claims-Based Studies of Breast, Prostate, Colorectal, and Lung Cancer Patients. Ann Epidemiol. 2007;17(8):584–90. [PubMed]
18. Henderson LM, Reeder-Hayes K, Hinton SP, Carpenter WR, Chen RC. Comparing physician-reported cancer management plans with Medicare services received. Arch Intern Med. 2012;172(8):664–6. Epub 2012/04/25. [PubMed]
19. Lamont EB, Herndon JE, 2nd, Weeks JC, Henderson IC, Lilenbaum R, Schilsky RL, et al. Criterion validity of Medicare chemotherapy claims in Cancer and Leukemia Group B breast and lung cancer trial participants. J Natl Cancer Inst. 2005;97(14):1080–3. [PubMed]
20. Du XL, Key CR, Dickie L, Darling R, Geraci JM, Zhang D. External validation of medicare claims for breast cancer chemotherapy compared with medical chart reviews. Med Care. 2006;44(2):124–31. [PMC free article] [PubMed]
21. Liang SY, Phillips KA, Wang G, Keohane C, Armstrong J, Morris WM, et al. Tradeoffs of using administrative claims and medical records to identify the use of personalized medicine for patients with breast cancer. Med Care. 2011;49(6):e1–8. [PMC free article] [PubMed]
22. Warren JL, Harlan LC, Fahey A, Virnig BA, Freeman JL, Klabunde CN, et al. Utility of the SEER-Medicare data to identify chemotherapy use. Med Care. 2002;40(8 Suppl):IV-55–61. [PubMed]
23. Lamont EB, Lan L. Sensitivity of Medicare Claims Data for Measuring Use of Standard Multiagent Chemotherapy Regimens. Med Care. 2012 Epub 2012/03/14. [PMC free article] [PubMed]
24. Lund JL, Sturmer T, Harlan LC, Sanoff HK, Sandler RS, Brookhart MA, et al. Identifying Specific Chemotherapeutic Agents in Medicare Data: A Validation Study. Med Care. 2011 Epub 2011/11/15. [PMC free article] [PubMed]
25. Miller DC, Saigal CS, Warren JL, Leventhal M, Deapen D, Banerjee M, et al. External validation of a claims-based algorithm for classifying kidney-cancer surgeries. BMC Health Serv Res. 2009;9:92. [PMC free article] [PubMed]
26. Cooper GS, Virnig B, Klabunde CN, Schussler N, Freeman J, Warren JL. Use of SEER-Medicare data for measuring cancer surgery. Med Care. 2002;40(8 Suppl):IV-43–8. [PubMed]
27. Virnig BA, Warren JL, Cooper GS, Klabunde CN, Schussler N, Freeman J. Studying radiation therapy using SEER-Medicare-linked data. Med Care. 2002;40(8 Suppl):IV-49–54. [PubMed]
28. Scott AM, Gunawardana DH, Kelley B, Stuckey JG, Byrne AJ, Ramshaw JE, et al. PET changes management and improves prognostic stratification in patients with recurrent colorectal cancer: results of a multicenter prospective study. J Nuclear Med. 2008;49(9):1451–7. [PubMed]
29. Scott AM, Gunawardana DH, Bartholomeusz D, Ramshaw JE, Lin P. PET changes management and improves prognostic stratification in patients with head and neck cancer: results of a multicenter prospective study. J Nuclear Med. 2008;49(10):1593–600. Epub 2008/09/17. [PubMed]
30. Scott AM, Gunawardana DH, Wong J, Kirkwood I, Hicks RJ, Ho Shon I, et al. Positron emission tomography changes management, improves prognostic stratification and is superior to gallium scintigraphy in patients with low-grade lymphoma: results of a multicentre prospective study. Eur J Nucl Med Mol Imag. 2009;36(3):347–53. [PubMed]
31. Fulham MJ, Carter J, Baldey A, Hicks RJ, Ramshaw JE, Gibson M. The impact of PET-CT in suspected recurrent ovarian cancer: A prospective multi-centre study as part of the Australian PET Data Collection Project. Gyne Oncol. 2009;112(3):462–8. [PubMed]
32. Chatterton BE, Ho Shon I, Baldey A, Lenzo N, Patrikeos A, Kelley B, et al. Positron emission tomography changes management and prognostic stratification in patients with oesophageal cancer: results of a multicentre prospective study. Eur J Nucl Med Mol Imag. 2009;36(3):354–61. [PubMed]
33. Staub LP, Lord SJ, Simes RJ, Dyer S, Houssami N, Chen RY, et al. Using patient management as a surrogate for patient health outcomes in diagnostic test evaluation. BMC Med Res Method. 2012;12:12. [PMC free article] [PubMed]