Accurate classification of incident malignancies is essential for pharmacoepidemiologic studies, as well as to compare rates between different cohort studies and assess the relationship between disease characteristics and treatments with the development of cancer. Patient self-report may be an imperfect method for ascertaining incident malignancies, as the difference between a benign mass and true neoplasm is not always clearly communicated or understood. Even physician report may be unreliable, as the treating rheumatologist may not have the primary information about an incident malignancy. Administrative databases based on claims data are also not ideal, as the methodology used to define malignancy can cause the positive predictive value to vary widely [21
]. We attempted to validate physician report of incident malignancy based on physician questionnaires and adverse event forms using a gold standard of pertinent medical records review in a large cohort of patients with RA. We were able to confirm approximately two-thirds of reported incident malignancies. We found that having a positive report of incident malignancy on the adverse event forms increased the PPV only slightly compared to questionnaires alone. However, the PPV of the physician report alone may not be sufficiently high enough to be relied on for epidemiologic purposes. Hence, use of non-validated reports of incident malignancy might overestimate the true incidence in patients with RA and might bias the results of pharmacoepidemiologic studies assessing causal associations with specific treatments.
Our second notable finding was the exclusion of 14 malignancies (3.0%) because they were either duplicate entries, established cancers that were not incident, or proven by corroborating documents to not be malignant. An additional 43 cases (9.3%) had insufficient data and could not even be deemed possible malignancies. Clearly some of those are incident cases, although it is not clear how many. In addition, 101 cases (21.9%) could at best be classified as possible incident malignancies (either histology report or date of diagnosis missing). The accuracy of classifying cancer in a study assessing the risk of different treatments is important. If the possible malignancies are dismissed as non-cancers, it might artificially create a sense of decreased risk for a given medication, when at least some of those cases are likely to be true malignancies. Alternatively, including all possible malignancies might change the perception of the risk/benefit ratio and prevent patients or physicians from choosing a medication that could have benefit. The first approach would provide greater specificity; however, this is at the sacrifice of sensitivity. Assessing both rates would provide a range within which the true value most likely exists, but also illustrates the degree of uncertainty generated by physician report and even follow-up adverse event forms.
Our third significant finding was that despite follow-up with the primary treating rheumatologist, almost a third of the time, we could not obtain source documents to corroborate the malignancy. In some cases, this may be due to lack of diligence by the site. However, in most cases, this was because records truly could not be obtained. As a result, since 2008 the registry now requests source documents at the time the cancer is initially reported, to avoid requesting records of events from multiple years earlier. This proactive approach has improved the proportion of cases with source documents.
The methodology defines the classification of the malignancies, as it does depend on appropriate documentation submitted from the patient’s rheumatologist. The use of appropriate specialists for specific malignancies, such as a dermatologist for a skin cancer or a urologist for a genitourinary cancer, was deemed appropriate as for those malignancies where that specialist will treat the patient directly, as opposed to having an oncologist manage treatment. All ‘possible’ malignancies were classified as such because of some inadequacy of the requisite data to have confidence that it was truly an incident case.
The type of dataset itself also can impact the rate of malignancy found in a given population. The intensity of evaluation is much higher in a clinical trial, and virtually all cases of malignancy during the follow-up period would be expected to be reported. However, Phase II and III randomized controlled trials only follow patients with a single intervention for a finite period, and thus cannot offer the kind of data that can be possible in a long-term disease registry. It is possible that the intensity of surveillance itself can impact clinical care and outcome (Hawthorne effect), but it is still unclear if it will affect the overall incidence of a specific comorbidity such as malignancy in patients with RA.
The strengths of this analysis include the use of a very large dataset which includes information gathered from both providers and patients in the context of an established infrastructure which permitted the identification and follow-up of all reported cases. This allowed us to procure source documents in most cases. In addition, we used a standardized record review process with multiple adjudicators. All of these steps increased the validity of the results.
This study did have certain weaknesses as well. Although standard questions on the development of new malignancies were included in both the physician and patient questionnaires at each visit, it is still possible that some malignancies were not reported. As previously noted, the absence of a national cancer registry in the US makes independent assessment of new malignancies extremely difficult if not impossible. As such, we cannot calculate a negative predictive value of the default evaluation of “no malignancy.” Individual state cancer registries are available, are sponsored by both the NCI and the CDC, and their reliability continues to improve, though they are not reflective of nationwide rates, as the SEER data is. However, identification of individual patients requires the use of personal health information (PHI), and the CORRONA consent does not permit the use of PHI to cross-link with other registries. In addition, despite repeated attempts to obtain source documents, in many cases none were available. This led to the classification of many malignancies as “possible,” even while reported independently on the primary questionnaire from either the patient or treating physician. Certainly many of these cases likely were true malignancies. We believe that this problem is likely to be endemic to all observational registry studies in the absence of a national cancer registry. The response rate for pertinent records did improve when cases were reported to CORRONA more recently. However, there was no significant difference in rate of excluded cases (either not an incident malignancy or confirmed to be not a malignancy) from earlier versus
more recent cases. A cancer registry where all malignancies are reported and validated, as is frequently done outside the United States, is superior to our methodology. However, information from within the United States is still of great value, as drug utilization patterns in the US are quite different from European registries where penetration of biologic agents is significantly greater than in Europe[23
]. We therefore believe that it is critical to appropriately analyze data on major comorbidites from a US source.