|Home | About | Journals | Submit | Contact Us | Français|
The National Oncologic PET Registry (NOPR) ascertained changes in the intended management of cancer patients using questionnaire data obtained before and after PET under Medicare’s coverage with evidence development (CED) policy.
To assess the concordance between intended care plans and care received as ascertained through administrative claims data.
Analysis of linked data of NOPR participants from 2006–2008 and their corresponding Medicare claims.
Consenting patients age >65 years having their first PET for restaging of bladder, kidney, ovarian, pancreas, prostate, small cell lung or stomach cancer.
Agreement [positive predictive values and kappas] between NOPR post-PET intended management plans for treatment (systemic-therapy, radiotherapy, surgery or combinations), biopsy, or watching as compared to claims-inferred care 30 days after PET.
8,460 patients with linked data were assessed. 43.5% had metastatic disease. 45.3% had treatment planned (predominantly systemic therapy only), 11.1% biopsy and 43.5% watching. Claims confirmed intended plans (PPV) for single-mode systemic therapy in 62.0%, radiation in 66.0%, surgery in 45.6%, and biopsy in 55.7%. 25.7% of patients with a plan of watching had treatment claims. By cancer type, kappa ranged for systemic-therapy only from 0.17–0.40 and for watching from 0.21–0.41. Agreement rates varied by cancer types but were minimally associated with patient age, performance status, comorbidity or stage.
Among elderly cancer patients undergoing PET for restaging, there was moderate concordance between their physicians’ planned management and claims-inferred actions within a narrow time window. When higher accuracy levels are required in future CED studies, alternative designs will be needed.
Positron emission tomography (PET) using 18F-fluorodeoxyglucose represents an imaging paradigm based on characterizing metabolic processes. Since 2001, the Centers for Medicare & Medicaid Services (CMS) has covered PET for many common cancers.(1) However, about one-fourth of beneficiaries developing cancer have a non-covered cancer type.(2) For these cancers, it was recognized that developing sufficient evidence to inform coverage decisions would be unlikely. Therefore, CMS initiated a novel mechanism under its coverage with evidence development (CED) policy.(2–5) In response to the CED requirements, the National Oncologic PET Registry (NOPR) opened with a primary objective of measuring PET’s impact on intended patient management by collecting prospective questionnaire data before and after PET.
Two criticisms of NOPR have been that changes in planned management are only a surrogate for actual health outcomes and that the dataset does not document the care actually delivered.(11, 12) Concordance studies, between recommended actions and actual care, are rare and are important for informing the design and implementation of future CED programs using health outcome surrogates. We measured confirmation of intended plans with inferred management using a linked dataset of NOPR participants’ CMS claims following PET for restaging.
NOPR is a prospective data registry (ClinicalTrials.gov #NCT00868582); its operational details, human subject protection procedures, and findings of PET’s impact on intended management were previously reported.(7–10) In brief, the PET facility collects referring physician responses on pre-PET and post-PET forms. The pre-PET form collects the testing indication, the cancer type, working stage, performance status, and the referring physician’s plan if PET were not available. After PET, the referring physician records an estimate of the patient’s stage and management plan in light of the PET findings.
We linked NOPR data from December 2006 thorough 2008 for consenting participants to their CMS claim files by matching individual identifiers for the seven most frequent cancer types (Appendix 1, supplemental digital content 1). We assessed the first PET for restaging or suspected recurrence (hereafter labeled as restaging) and excluded patients who were age <65 years; were HMO participants; or for whom we were unsuccessful in linking identifiers, the registry and claim dates for PET differed by >7 days, or the post-PET plan was “other treatment(s) or additional imaging”.
The post-PET categories assessed were watching, biopsy, and treatment. Treatment categories were systemic-therapy (chemotherapy, hormonal, or immunotherapy), radiotherapy, or surgery alone or in combination.
Appendix 2, supplemental digital content 2 lists the Current Procedural Terminology (CPT©) codes, the Healthcare Common Procedure Coding System (HCPCS) codes and International Classification of Diseases (ICD-9) codes for outpatient care used to classify claims into inferred-care categories.
Given the numerous cancer types assessed, a list of possible biopsy or surgery CPT codes would likely be incomplete. Instead, the claims-inferred definition of “biopsy” was primarily based on surgical-pathology or cytology CPT coding.
Our preliminary analysis found surgical procedures for complications (e.g., chest tube insertion) rather than directed at the patient’s cancer. Therefore, we used anticipated surgical CPT codes and anesthesia claims after excluding eye, central vascular access, gastrointestinal endoscopy and conscious sedation. Chemotherapy, hormonal therapy and immunotherapy were inferred from professional claims, HCPCS codes for hospital-based chemotherapy and drug J-codes. Radiotherapy codes included all common techniques. These systemic-therapy and radiotherapy definitions were similar to those used in SEER-Medicare analyses.(13–15)
The NOPR forms did not specify an action timeframe. Given the indication of restaging, we used 30 days as the default and explored extending it to 60 days.
The CMS provider part-B taxonomy codes for physician specialty designation were used to categorize referring providers. If no specialty or non-physician coding was found, then specialty was coded as “other”.
The initial analyses treated the claims-inferred care as the reference standard to calculate measures of agreement between treatment plans and claims-inferred actual management, including positive predictive value (PPV), raw agreement, and kappa (chance-adjusted agreement).18 For the treatments (systemic-therapy, radiotherapy or surgery), agreement was defined as claims for that type action within the interval without considering other treatments; for biopsy, agreement included any procedures with surgical pathology or cytology claims; and for watching, it was the absence of treatment claims. The measures of agreement were computed separately by cancer type and compared with chi-square tests.
To assess the effect of patient, cancer, and provider factors in predicting agreement, separate logistic regression models were fit for patients with plans for systemic-therapy only, radiation only, surgery only, and watching. The outcome was the indicator of agreement with the plan. Calculations were done with PROC LOGIST in Linux SAS version 9.2.
Table 1 summarizes the cancer type, age, performance status, comorbidity scores, referring specialty and pre-PET plan for our 8,460 patient cohort.
Overall, the stage distribution showed 43.0% with no evidence or low probability of recurrence (range 36.9%–51.4%), 43.5% with metastatic disease (range 35.9%–52.0%) and 13.5% with local or nodal recurrences (range 8.3%–17%).
The referrers were medical oncologists in one-half of all patients, gynecologists/gynecologic oncologists in one-third of ovarian cancer patients, and urologists in 9%–16% of bladder, kidney and prostate cancer patients. Radiation oncologists and surgeons were infrequent referrers except in prostate and small cell lung (SLC) cancer (radiation oncologists) and stomach cancer (surgeons).
In the 30 days following PET, 1.6% had died (range 0.4%–3.6%).
Table 2 summarizes the post-PET intended plans. Modest distribution variations were found by cancer type. Overall, treatment was planned in 45.3% (range 35.9%–52.8%), biopsy in 11.1%, and watching in 43.5% (range 38.7%–50.6%). For all types, the most common treatment plan was systemic-therapy only (range 20.7%–39.1%). Combination therapy, radiotherapy or surgery only were planned in <11% in all cancer types. Combination therapy components were 34.8% systemic-therapy, 12.4% radiotherapy, and 6.4% surgery.
Table 3 shows by cancer type, the agreement between NOPR intended plan and claims-inferred actions.
For systemic-therapy alone, NOPR plans had a PPV of 65–67% for bladder, ovarian, pancreas, prostate, and SLC cancers and 51.5% for stomach cancer; the PPV for kidney cancer was lowest (40.1%), as anticipated. A Part D analysis of 40% of the cohort found 3% additional kidney cancer chemotherapies (see footer Table 3). Raw agreements ranged from 65.7% to 73.7% and kappas ranged from 0.17–0.40, in the slight to fair range. Among the radiotherapy alone cohorts, the PPVs were slighter higher (56.1%–78.4%), as were their kappas of 0.23 to 0.48, in the fair to moderate range.
Surgery only was planned in over 50 patients in kidney, bladder and ovarian cancer. For this group, the PPV ranged from 44.0%–57.5%, with only modest kappas (0.14–0.27).
Among planned combination therapy patients, claims for at least one of the planned therapies were found in 69.0% (data not shown).
A biopsy plan was confirmed by claims in 55.7% (range 48.0%–64.7%). Non-treatment (watching) plans were confirmed in only 76.3% of patients and were lowest in SCL and stomach cancer patients.
Extending the post-PET window to 60 days increased the PPV by 6%–13% for systemic-therapy, 0–6% for radiotherapy, and 12–18% for surgery. However, there was also an overall decline in kappas (Appendix 3, supplemental digital content 3).
Table 4 shows the impact of patient age, performance status, comorbidity, cancer type, stage, referring specialty, and pre-PET plan in predicting post-PET plan and claims agreement.
With the exception of cancer type, the other factors had little impact in predicting PPV (62.0%) for systemic-therapies. If the referrer was a medical oncologist or if the pre-PET plan was also treatment, then the PPV increased minimally (3%–4%’s).
Radiotherapy’s PPV was slightly greater at 66%. Patient factors or cancer stage were non-predictive. Radiotherapy was most commonly planned in prostate cancer, yet it had the second lowest PPV (56%). Not surprisingly, when the referrer was a radiation oncologist, the PPV was greatest (81%).
Surgery alone was an infrequent plan and had a PPV 46%. Age over 75 years or a non-surgeon referrer was associated with even lower PPVs.
The absence of treatment claims in the intended watching cohort was unrelated to age and was more common if patients had a good performance status, low co-morbidity, non-metastatic stage, had kidney cancer, were not referred by a medical oncologist, and had a pre-PET plan other than treatment.
We assessed how often referring physicians’ intended management after PET in the NOPR database, is concordant with actual management inferred from Medicare claims for the indication of restaging of previously treated cancers. We found only moderate agreements across all NOPR plan categories—treatment, biopsy or watching (non-action). The PPV of systemic-therapy plans clustered between 64–67% in five different cancer types. Claims confirming plans for invasive procedures (surgery or biopsy) were particularly infrequent. Moreover, over 25% of patients with an observation plan had treatment claims within 30 days. When longer timeframes were used, the PPV rates increased but the overall kappas declined.
We sought correlations that might explain these differences by assessing patient, cancer, and physician factors. This was largely unsuccessful. Patient age, performance status, and comorbidity had little association with PPV within a treatment category. For the two most frequent actions, medical oncologists versus all other specialties had similar PPVs for claims confirming chemotherapy as well as similar crossover rates from watching plans to treatment.
Information gained from PET is only one factor influencing patient management. Unavailable or unmeasured factors include other diagnostic test results, the extent and type of prior treatment, whether the referrer is the patient’s primary cancer care provider, and the physician’s experience with PET. The NOPR management plan reflects the physician’s intent or recommendation shortly after the PET scan is reported, and not necessarily the patient’s agreement to a plan. Also, the accuracy of NOPR questionnaire responses was not independently confirmed by review of the physician’s office records. Accordingly, one source of discordance may be inaccuracy of the NOPR data.
Our results show agreement rates comparable to those of smaller, less detailed analysis by Henderson that assessed 489 NOPR scans for all indications in kidney and pancreatic cancer patients(18). Their analysis found only fair agreement rates for observation and treatment (kappa=0.39 and 0.36).
Another limitation is the expectation that claims will accurately reflect actual clinical actions performed. Since 2000, numerous reports using SEER-Medicare linked data have assessed the completeness of claims in identifying initial chemotherapy(19–24), surgery(25, 26), and radiation(15, 27). Most of these studies used broad time windows (up to one year post-diagnosis). Since our analysis used a narrow 30-day window following PET, inaccurate claims service dates could be a major source of non-matching (false-negatives). Two recent reports partly address this issue. First, Lamont assessed CMS claims against trial data as the reference standard in six CALGB trial cohorts of first-line metastatic chemotherapy.(23) They found that claims correctly identified 78% of the drugs given and the treatment schedule. In a second study, Lund validated SEER initial treatment plans against hospital and outpatient records that were re-abstracted for chemotherapy in four cancer types.(24) They found that the claim sensitivities were very time dependent ranging from 36%–50% at two months to 84%–96% at six months.
Australian investigators have used a design similar to NOPR in that they collected prospective data at three to six centers for a range of cancer types in which PET was used predominantly for initial staging and determined the change in management plans associated with PET.(28–32) They subsequently assessed medical records for actual care over the next 3 to 12 months. They found agreement rates ranging from 53% to 75% (average 65%), similar to our claims-inferred actions.
We previously noted that intended management changes might not always be appropriate.(9) Moreover, changes in intended actions have a presumed, but uncertain, relationship to more tangible health outcomes—progression-free or overall survival(33)—that are so heavily dependent on the effectiveness of the clinical actions chosen.
Our analysis highlights that intended management in patients with previously treated cancer does not consistently reflect implemented action. When higher accuracy levels are required, future evaluations of other diagnostic tests under the CED policy likely will require alternative designs.
Financial Support: National Cancer Institute Grand Opportunity Award RC2CA148259 and the Academy for Molecular Imaging (BEH). Funding for development of the NOPR was provided by the Academy for Molecular Imaging, but the registry is otherwise self-supported by the fees paid by participating PET facilities.
Bruce E. Hillner, Department of Internal Medicine and the Massey Cancer Center, Virginia Commonwealth University, Richmond, VA.
Tor D. Tosteson, The Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth, Lebanon, NH.
Anna N. A. Tosteson, The Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth, Lebanon, NH.
Qianfei Wang, The Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth, Lebanon, NH.
Yunjie Song, The Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth, Lebanon, NH.
Tracy Onega, The Dartmouth Institute for Health Policy and Clinical Practice, Geisel School of Medicine at Dartmouth, Lebanon, NH.
Lucy G. Hanna, Center for Statistical Sciences, Brown University, Providence, RI.
Barry A. Siegel, Division of Nuclear Medicine, Mallinckrodt Institute of Radiology andSiteman Cancer Center, Washington University School of Medicine, St. Louis, MO.