PMCCPMCCPMCC

Search tips
Search criteria 

Advanced

 
Logo of nihpaAbout Author manuscriptsSubmit a manuscriptHHS Public Access; Author Manuscript; Accepted for publication in peer reviewed journal;
 
J Thorac Oncol. Author manuscript; available in PMC 2017 March 27.
Published in final edited form as:
PMCID: PMC5367043
NIHMSID: NIHMS849425

Autoantibody Signature Enhances the Positive Predictive Power of Computed Tomography and Nodule-Based Risk Models for Detection of Lung Cancer

Abstract

Introduction

The incidence of pulmonary nodules is increasing with the movement toward screening for lung cancer by low-dose computed tomography. Given the large number of benign nodules detected by computed tomography, an adjunctive test capable of distinguishing malignant from benign nodules would benefit practitioners. The ability of the EarlyCDT-Lung blood test (Oncimmune Ltd., Nottingham, United Kingdom) to make this distinction by measuring autoantibodies to seven tumor-associated antigens was evaluated in a prospective registry.

Methods

Of the members of a cohort of 1987 individuals with Health Insurance Portability and Accountability Act authorization, those with pulmonary nodules detected, imaging, and pathology reports were reviewed. All patients for whom a nodule was identified within 6 months of testing by EarlyCDT-Lung were included. The additivity of the test to nodule size and nodule-based risk models was explored.

Results

A total of 451 patients (32%) had at least one nodule, leading to 296 eligible patients after exclusions, with a lung cancer prevalence of 25%. In 4- to 20-mm nodules, a positive test result represented a greater than twofold increased relative risk for development of lung cancer as compared with a negative test result. Also, when the “both-positive rule” for combining binary tests was used, adding EarlyCDT-Lung to risk models improved diagnostic performance with high specificity (>92%) and positive predictive value (>70%).

Conclusions

A positive autoantibody test result reflects a significant increased risk for malignancy in lung nodules 4 to 20 mm in largest diameter. These data confirm that EarlyCDT-Lung may add value to the armamentarium of the practitioner in assessing the risk for malignancy in indeterminate pulmonary nodules.

Keywords: Autoantibodies, Lung cancer, Pulmonary nodules, CT scanning, Risk models

Introduction

More than 1.5 million individuals in the United States have lung nodules identified annually, including 63,000 who receive a new lung cancer diagnosis within 2 years.1,2 This number is growing given the increasing number of computed tomography (CT) scans being performed since the National Lung Screening Trial reported that annual CT screening reduced lung cancer mortality by 20% as compared with chest radiography.3 The problem remains, however, that more than 50% of CT-screened patients have at least one noncalcified nodule, with more than 96% of nodules larger than 4 mm being false-positives.3,4 Furthermore, there is a high false-positive rate and morbidity associated with biopsy or resection of benign nodules.2,3,5 Therefore, a critical need exists for an adjunctive test to help evaluate the malignancy potential and reduce the false-positive rate.

EarlyCDT-Lung (Oncimmune Ltd., Nottingham, United Kingdom) detects the presence of serum autoantibodies (AAbs) to a panel of lung cancer–associated antigens using an indirect enzyme-linked immunosorbent assay method.6,7 A sample is positive if at least one AAb is elevated above a predetermined cutoff. Test specificity was improved in November 2010 by changing from a six-AAb panel to the current seven-AAb panel.8 Test performance in routine clinical practice for approximately 1600 patients with unknown nodule status was assessed previously, showing 87% specificity, 41% sensitivity, and a 5.4-fold relative risk of lung cancer in cases with a positive test result.9 We demonstrate how the test can assist in evaluation of the malignancy of pulmonary nodules in the clinic with or without lung cancer risk models.

Methods

Patients

A cohort of 1987 Health Insurance Portability and Accountability Act–authorized patients was tested in routine practice between May 2009 and December 2012 by using EarlyCDT-Lung as described previously.810 For this study CT imaging reports from within 6 months of the AAb test were assessed for the presence of pulmonary nodules. The size of the largest noncalcified nodule was recorded. Patients who had invalid AAb tests, were lost to follow-up, or had a history of cancer were excluded. PET scan data were generally unavailable. Although most cancer cases were confirmed by pathology reports, some were diagnosed solely on imaging reports. To reflect this, the analysis considered two cohorts defined by their eligibility criteria (Fig. 1).

Figure 1
Patient cohorts in the clinical registry of nodule status. Of 1987 patients enrolled, 585 (29%) were excluded either on the basis of being lost to follow-up (LTFU) for all reasons, including practices closed, patients changing practices, and physicians ...

Inclusive Cohort

The clinician’s diagnosis was accepted even if CT or pathology reports were not available. This corresponds most closely to the commercial setting. The inclusive cohort retains as many cancers as possible to enhance statistical power.

Exclusive Cohort

Starting with the inclusive cohort, only those members with data from actual CTs and pathology reports on file, as opposed to office visits or physician phone interviews, were accepted. These criteria correspond most closely to ideal clinical practice.

Statistical Analysis

A model-free analysis summarizing the association between nodule size (<4 mm, 4–20 mm, and ≥20 mm), cancer diagnosis, and test positivity was performed. Diagnostic performance metrics were calculated for the inclusive, exclusive, and exclusive (7AAb panel only) cohorts. Pretest risk was calculated by applying size thresholds above which a patient would be deemed to have a positive result (nodules-only). Posttest risk (positive predictive value [PPV]) was calculated by adding the AAb test result using a “both-positive rule” (i.e., positive for both size threshold and AAb test result). The curve for PPV versus nodule size threshold was compared with that for the combined test (nodules + AAb). Finally, the observed nodule distribution was compared with the Mayo study4 to confirm similarity.

A risk-model analysis was applied using the inclusive cohort. The pretest cancer risk was estimated using three published nodule-based models: (1) the GOULD model, which is based on Department of Veterans Affairs data11; (2) the BROCK model, which was developed on the basis of pan-Canadian study data12; and (3) the MAYO model, which is based on Mayo study data.13,14 The calibration of these models over nodule size was examined by comparing observed and predicted cancer rates using χ2 goodness-of-fit tests.

The additivity of the AAb test to the models was assessed for 4-mm to 20-mm nodules, which are the most relevant for early detection. The receiver operating characteristic curve for each model was formed by varying the risk threshold above which a patient was deemed positive and calculating specificity and sensitivity at each threshold (model only). Then, at each threshold the data were split by AAb test result and a new curve was formed by applying the both-positive rule (Model + AAb). The sensitivity observed at fixed specificity values was compared across the two curves. Plots of the respective PPVs were also created.

Results

Cohorts

From the original cohort (n = 1987), exclusions led to 296 patients (221 without cancer and 75 with cancer) for the inclusive cohort and 269 (217 without cancer and 52 with cancer) for the exclusive cohort (Table 1 and Supplementary Table 1 for the six-AAb and seven-AAb panels).

Table 1
Summary of Demographics of Patients with Nodules (Inclusive and Exclusive Data Sets)

Model-Free Analysis

The nodule distribution was most similar to that in the Mayo study (Supplementary Fig. 1). For the exclusive cohort, 22 of 68 of patients with a positive test result (32%) and 30 of 201 of patients with a negative test result (15%) had lung cancer, giving a relative risk of 2.2 (95% confidence interval [CI]: 1.3–3.5, PPV = 32%) (Table 2). For the seven-AAb test the relative risk increased to 2.3 (95% CI: 1.3–3.9, PPV = 40%). For the 4-mm to 20-mm group, the relative risk was 2.7-fold (95% CI: 1.3–5.7), equating to an increase in absolute risk from 13% to 24%, and for the ≥20-mm group, the relative risk was 2.1-fold (95% CI: 1.3–3.3), which equates to an increase in absolute risk from 46% to 77% (Supplementary Fig. 2).

Table 2
Comparison of Test Performance Characteristics in the Inclusive and Exclusive Cohorts by Nodule Size

p Values for comparing EarlyCDT-Lung results by malignant and nonmalignant nodules were calculated using Fisher’s exact test. The 95% binomial confidence intervals are shown.

Risk Model Analysis

For the inclusive cohort, the GOULD and BROCK models, respectively, overestimated and underestimated the risk relative to the actual rates observed (p<0.001). The MAYO estimates were closest to those observed (p = 0.54) (Table 3).

Table 3
Calibration of Risk Models versus Observed Cancers in the Registry—Inclusive Cohort (n = 296)

For nodules in the 4- to 20-mm range, the receiver operating characteristic curves for the models show that as specificity surpassed 90%, the sensitivity declined to 30% and less (Fig. 2 for MAYO and Supplementary Figures 3 and 4 for GOULD and BROCK, respectively). When EarlyCDT-Lung was added, the sensitivity was increased at the high-specificity end (left-hand side). More precisely, for a fixed risk threshold, when a biomarker test was added using the both-positives rule, the specificity increased and the sensitivity decreased. For the MAYO model, for example, as the risk threshold moved from 35% to 60%, the specificity for the model-only case increased from 88% to 97% and the sensitivity decreased from 50% to 13% (the PPV stayed at 50%). To control the specificity of the combined test at 88% to 97%, the risk thresholds need to be lowered to 10% to 20%, giving a sensitivity of 40% to 33% (PPV 43%–72%). So for the same false-positive rate, a total of eight extra cancers (a 2.6-fold increase) were declared positive, all (in this case) with a pretest risk between 20% and 60% (Table 4 and Supplementary Table 2). So lower-risk patients are having their cancer detected. This effect was seen only at high specificity levels.

Figure 2
Partial receiver operating characteristic curves for the MAYO model and with EarlyCDT-Lung added using the both-positive rule. Curves are shown for the MAYO model only (black line); the MAYO model plus EarlyCDT-Lung (dashed line); the proportional line, ...
Table 4
Example Using a Specificity of 97%

For fixed risk threshold, for example, a threshold of 30% with the MAYO model, the PPV increased from 48% to 91% when EarlyCDT-Lung was added (Fig. 3 for MAYO and Supplementary Figs. 5 and 6 for GOULD and BROCK, respectively). Virtually all false-positive results have been eliminated (25 down to 1), with approximately half of the cancers being lost (23 down to 11) (Table 5). Also shown is the percentage of subjects whose risk is above the threshold and are thus eligible for a positive combined test.

Figure 3
Plot of positive predictive value (PPV) versus risk threshold (R) for the MAYO model. Curves are shown for the MAYO model only (black line with dots) and the MAYO model plus EarlyCDT-Lung (gray dashed line with triangles). Also shown is the plot of the ...
Table 5
Example Using a Risk Threshold of 30% with the MAYO Model

Discussion

The study showed, under strict eligibility conditions, that the addition of a positive EarlyCDT-Lung AAb test to nodule size alone significantly increased the PPV for malignancy prediction, with relative risks of 2.7-fold for nodules 4 to 20 mm in the largest diameter.

Using nodule-based risk models with positivity thresholds, lowering the risk threshold, and adding the AAb test using a both-positive rule increased sensitivity by reclassifying a proportion of the lower-risk false-negatives to true positives. This was demonstrated in 4- to 20-mm nodules, which is important because malignant nodules smaller than 20 mm are largely stage IA.4,15 Alternatively, the false-positive rate can be reduced while still detecting a reasonable number of cancers by fixing the risk threshold for the combined test.

The testing was not controlled by a formal protocol and may have been subject to biases, particularly with respect to diagnosis not based on a pathology report, hence the two cohorts. However, the study reflects the clinical setting within which a biomarker test might be expected to operate, and so the findings do have validity. Additionally, as indicated previously,9 those with a negative test were also followed up to avoid observer bias and allow unbiased estimates of the relative risk. The risk model analysis is, of course, only approximate. The models themselves were trained on specific data sets with different nodule size distributions and malignancy rates.14 We hope to confirm the results in a prospective study.

Lung cancer tends to be detected at a later stage as a nodule grows.4 A biomarker that suggests an increased probability of malignancy while the nodule is still relatively small could lead to early detection and decreased mortality. We have shown that an AAb test can significantly add to clinicians’ interpretation of pulmonary nodules, both with and without risk models, for nodules smaller than 20 mm in diameter. For larger nodules (e.g., >20 mm) there is less need for a biomarker test because the clinical pathway is better established. For indeterminate nodules and a negative AAb test, the physician should continue to assess patients on the basis of their other risk factors and subsequent scans.

Supplementary Material

Supplemental Figure 1

Supplemental Figure 2

Supplemental Figure 3

Supplemental Figure 4

Supplemental Figure 5

Supplemental Figure 6

Supplemental Table 1

Supplemental Table 2

Acknowledgments

This work was supported by funding from Oncimmune, Ltd., and the University of Nottingham, United Kingdom. Dr. Massion was funded by the Early Detection Research Network from the National Cancer Institute (grant number U01 CA152662). The authors wish to acknowledge and thank the physicians and office staff who were an integral part of this project.

Footnotes

Disclosure: Prof. Massion is a member of an Oncimmune scientific advisory group. Dr. Murray and Mr. Healey are employees of Oncimmune, Ltd. Dr. Peek is an employee of Oncimmune USA LLC, Dr. Fredericks is a consultant to Oncimmune USA LLC, and Prof. Robertson is a previous director and current shareholder of Oncimmune, Ltd. Prof. Sewell is a previous member of an Oncimmune advisory board and a current option holder.

Supplementary Data

Note: To access the supplementary material accompanying this article, visit the online version of the Journal of Thoracic Oncology at www.jto.org and at http://dx.doi.org/10.1016/j.jtho.2016.08.143.

References

1. Berrington de González A, Mahesh M, Kim K-P, et al. Projected cancer risks from computed tomographic scans performed in the United States in 2007. Arch Intern Med. 2009;169:2071–2077. [PubMed]
2. Gould MJ, Tang T, Liu IA, et al. Recent trends in the identification of incidental pulmonary nodules. Am J Respir Crit Care Med. 2015;192:1208–1214. [PubMed]
3. The National Lung Screening Trial Research Team (NLST) Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med. 2011;365:395–409. [PMC free article] [PubMed]
4. Swensen SJ, Jett JR, Hartman TE, et al. Lung cancer screening with CT: Mayo Clinic experience. Radiology. 2003;226:756–761. [PubMed]
5. Jett JR. Limitations of screening for lung cancer with low-dose spiral computed tomography. Clin Cancer Res. 2005;11:4988s–4992s. [PubMed]
6. Murray A, Chapman CJ, Healey G, et al. Technical validation of an autoantibody test for lung cancer. Ann Oncol. 2010;21:1687–1693. [PMC free article] [PubMed]
7. Boyle P, Chapman CJ, Holdenrieder S, et al. Clinical validation of an autoantibody test for lung cancer. Ann Oncol. 2011;22:383–389. [PMC free article] [PubMed]
8. Chapman CJ, Healey GF, Murray A, et al. EarlyCDT®-Lung test: improved clinical utility through additional autoantibody assays. Tumor Biology. 2012;33:1319–1326. [PMC free article] [PubMed]
9. Jett JR, Peek LJ, Fredericks L, et al. Audit of the auto-antibody test, EarlyCDT®-Lung, in 1600 patients: an evaluation of its performance in routine clinical practice. Lung Cancer. 2014;83:51–55. [PubMed]
10. Lam S, Boyle P, Healey G, et al. EarlyCDT-Lung: an immuno-biomarker test as an aid to early detection of lung cancer. Cancer Prev Res. 2011;4:1126–1134. [PubMed]
11. Gould MK, Ananth L, Barnett PG. A clinical model to estimate the pretest probability of lung cancer in patients with solitary pulmonary nodules. Chest. 2007;131:383–388. [PMC free article] [PubMed]
12. McWilliams A, Tammemagi MC, Mayo JR, et al. Probability of cancer in pulmonary nodules detected on first screening CT. N Engl J Med. 2013;369:910–919. [PMC free article] [PubMed]
13. Swensen SJ, Silverstein MD, Ilstrup DM, et al. The probability of malignancy in solitary pulmonary nodules. Application to small radiologically indeterminate nodules. Arch Intern Med. 1997;157:849–855. [PubMed]
14. Schultz EM, Sanders GD, Trotter PR, et al. Validation of two models to estimate the probability of malignancy in patients with solitary pulmonary nodules. Thorax. 2008;63:335–341. [PMC free article] [PubMed]
15. Henschke CI, McCauley DI, Yankelevitz DF, et al. Early Lung Cancer Action Project: overall design and findings from baseline screening. Lancet. 1999;354:99–105. [PubMed]