|Home | About | Journals | Submit | Contact Us | Français|
Abbreviations: ACR, American College of Rheumatology, anti-CCP, anti-cyclic citrullinated peptide antibodies; anti-CCP2, anti-cyclic citrullinated peptide antibodies, second-generation assay; DMARD, disease-modifying antirheumatic drug; NPV, negative predictive value; PPV, positive predictive value; RA, rheumatoid arthritis; WHI, Women's Health Initiative.
Rheumatoid arthritis (RA) research using large databases is limited by insufficient case validity. Of 161,808 postmenopausal women in the Women's Health Initiative, 15,691 (10.2%) reported having RA, far higher than the expected 1% population prevalence. Since chart review for confirmation of an RA diagnosis is impractical in large cohort studies, the current study (2009–2011) tested the ability of baseline serum measurements of rheumatoid factor and anti-cyclic citrullinated peptide antibodies, second-generation assay (anti-CCP2), to identify physician-validated RA among the chart-review study participants with self-reported RA (n = 286). Anti-CCP2 positivity had the highest positive predictive value (PPV) (80.0%), and rheumatoid factor positivity the lowest (44.6%). Together, use of disease-modifying antirheumatic drugs and anti-CCP2 positivity increased PPV to 100% but excluded all seronegative cases (approximately 15% of all RA cases). Case definitions inclusive of seronegative cases had PPVs between 59.6% and 63.6%. False-negative results were minimized in these test definitions, as evidenced by negative predictive values of approximately 90%. Serological measurements, particularly measurement of anti-CCP2, improved the test characteristics of RA case definitions in the Women's Health Initiative.
Despite great potential to investigate the causes and consequences of rheumatoid arthritis (RA) in large, population-based longitudinal cohorts, existing cohort studies generally have not collected measurements that would satisfy RA classification criteria (1, 2). Instead, such studies typically rely on self-reporting, which is only 7%–35% accurate (3–5). Previously, we investigated the validity of self-reported RA in the Women's Health Initiative (WHI), which enrolled 161,808 postmenopausal women aged 50–79 years at baseline in either the WHI Observational Study or the overlapping WHI Clinical Trials at 40 clinical centers across the United States (6). Of these, 16,461 women (10.2%) reported having RA at baseline, far exceeding the expected prevalence. According to rheumatologist review of medical records of consenting participants with self-reported RA at 2 WHI clinical centers, only 14.7% of women with self-reported RA had clinical RA (6). This positive predictive value (PPV) of 14.7% increased to 62.2% (negative predictive value (NPV) = 95.4%) when the RA case definition required use of disease-modifying antirheumatic drugs (DMARDs) in addition to self-reported RA. DMARD data were collected by study nurses during medication reviews that occurred at periodic WHI study visits.
Using our definition of self-reported RA plus DMARD use, the Black Women's Health Study had a PPV of 76% (7). However, there is potential selection bias in requiring use of DMARDs in the criteria for RA (8), and our previous study demonstrated that a requirement for DMARD use in addition to self-reporting missed 25%–33% of RA cases (5). Therefore, utilizing other factors to improve the PPV for RA would allow for more robust usage of existing cohorts for RA-related research.
Validation techniques that make use of preexisting data and biological samples are a practical way to investigate RA in past, present, and future data sets of interest. Laboratory testing for the autoantibody rheumatoid factor has been part of the diagnostic criteria for RA since the initial American Rheumatology Association guidelines were published in 1987 (2). However, rheumatoid factor has limitations in its diagnostic accuracy for RA, being positive in only two-thirds of patients with clinical RA (9) and in fewer than 50% of patients with early RA (10). Furthermore, rheumatoid factor has low specificity (69.6%–77.5%) because it is elevated in multiple non-RA conditions, as well as in 3%–5% of healthy adults (11), with increasing positivity in the elderly (10%–30%) (10). Anti-citrullinated protein antibodies have similar sensitivities but higher specificities (87.8%–96.4%) (12), making testing of stored biospecimens for anti-citrullinated protein antibodies a promising method of improving the validation of RA in large cohort studies.
In the current study, we evaluated the effectiveness of combining serological criteria with the RA case definitions in our previously reported WHI physician-validated RA cohort, using serum samples collected at baseline to determine each woman's serological status.
The complete WHI study design, recruitment, screening, randomization, and eligibility criteria have been described in detail elsewhere (6, 13–16). Participants in the current study were recruited from all women who reported RA at 2 of 40 WHI clinical sites throughout the United States (5). Briefly, WHI women with self-reported RA and controls without rheumatic disease were contacted to participate. Of the 643 women with self-reported RA, 305 consented to participate and 286 (44.5% response rate) had sufficient medical records for review. Of the 357 women without sufficient records, 129 declined to participate, 160 did not respond, 36 could not be contacted, 11 had incomplete or absent medical records, and 21 had died prior to study initiation (5). Medical records and physician contact information were obtained for consenting women and were reviewed by a rheumatologist blinded to the self-reported diagnosis, with 10% of charts undergoing a double-review process. RA diagnosis was determined through chart reviews and conversations with treating physicians. For a case to be deemed RA, chart review had to identify a diagnosis of RA by a board-certified rheumatologist or the presence of at least 4 of the seven 1987 American College of Rheumatology (ACR) criteria (2). DMARDs were defined as hydroxychloroquine, sulfasalazine, minocycline, methotrexate, leflunomide, azathioprine, cyclosporine, gold, cyclophosphamide, antirheumatic biological agents, or oral steroids, and their use was based on WHI medication data collected by WHI staff nurses, who reviewed medication bottles at each study visit.
At their initial study visits, WHI participants had serum samples collected according to standard protocols and stored at −70°C. Baseline serum samples for WHI women who participated in the aforementioned validation study (n = 286) were requested from the WHI Coordinating Center, which shipped them via overnight courier in dry ice to the University of Colorado, where they were immediately unpacked and stored at −70°C. The WHI Clinical Coordinating Center labeled samples using deidentified codes to ensure that the laboratory results were performed without knowledge of participant characteristics.
Measurement of rheumatoid factor and anti-cyclic citrullinated peptide antibodies (anti-CCP) was carried out using previously described methods (17–19). Briefly, anti-CCP (immunoglobulin G) antibodies were measured using commercially available second-generation (anti-CCP2) enzyme-linked immunosorbent assay kits (Diastat; Axis-Shield Diagnostics Ltd., Dundee, United Kingdom). Anti-CCP2 antibodies were measured in arbitrary units (U) per mL and were considered positive at a cutoff value ≥5 U/mL, which has been demonstrated to be more than 98% specific for RA (18).
Rheumatoid factor was measured quantitatively by the reactivity of diluted test serum with heterologous immunoglobulin G in solution via nephelometry, which provides continuously variable quantitative results in International Units (Dade Behring, Newark, Delaware). Per the 1987 ACR RA classification criteria (2), the positive cutoff value for this test was set so that 5% of a population of 490 randomly selected healthy anonymous blood donors were positive (20). Quality control was routinely assessed by means of a procedure whereby all autoantibody-positive serum samples (anti-CCP2 and/or rheumatoid factor) were retested in a blinded fashion, along with 5% of the negative sera, with more than 97.5% agreement in repeat testing.
A physician-validated RA case definition, based on clinical review of medical records, discussions with treating physicians, and the reviewing physician's judgment, was the gold standard for comparisons with other case definitions. The PPVs and NPVs were determined for various case definitions. Differences in demographic characteristics were determined using χ2 tests and analysis of variance. The study's sampling frame did not attempt to capture false-negative cases under the assumption that false-negative cases (i.e., women who had clinical RA but never reported it at WHI visits) would be exceedingly rare.
All research activities were approved by the institutional review boards of all involved institutions. Authors followed the principles outlined in the Declaration of Helsinki (21).
Of the 286 self-reported RA cases with chart reviews, 283 (99%) had anti-CCP2 and rheumatoid factor measures, with 42 of the 283 RA cases being validated by a physician (14.8%). There were no substantial differences between validated RA and non-RA cases, except in regards to education (Table 1). Positive serological results are reported in Table 2, stratified by physician validation status. Of women with physician-validated RA (n = 42), 47.6% were anti-CCP2-positive and 59.5% were rheumatoid factor-positive. When data were restricted to women with self-reported RA and self-reported DMARD use (n = 23), 65% were anti-CCP-positive and 65% were rheumatoid factor-positive. Among 19 women with chart-validated RA but no self-reported DMARD use, 5 (26%) were anti-CCP-positive. Among women with self-reported RA (n = 244) who were found not to have RA by chart review, only 5 (2%) were anti-CCP-positive, and among those who reported DMARD use (n = 14) but did not have RA, none were anti-CCP-positive. Thus, 20 of the 25 women with anti-CCP positivity (80%) had physician-validated RA.
Table 3 shows the PPVs and NPVs for physician-validated RA among women with self-reported RA, according to various RA case definitions, compared with chart-review clinical diagnosis. The PPV varied from 15% for a self-report of RA to 80% for a self-report and anti-CCP positivity, to 62% for self-reported RA and DMARD use, to 100% for self-reported RA, DMARD use, and anti-CCP positivity. Of the women with validated RA, 28 of the 42 (67%) were identified through the combination of anti-CCP positivity and reported DMARD use at baseline. The addition of rheumatoid factor positivity to the definition identified 5 more RA cases, or 33 of the 42 (79%). However, of the 227 women who were chart review-negative and had no reported DMARD use, 27 were rheumatoid factor-positive (NPV = 88.1%). Thus, the addition of rheumatoid factor substantially decreased the NPV and the specificity of the testing (Table 3).
Although adding antibody results to patients’ self-reported DMARD use results in improved accuracy in identifying physician-validated RA among women reporting RA in the WHI, it also excludes all seronegative RA cases, which account for 15%–50% of all RA cases (20, 22, 23). Therefore, to capture seronegative RA cases, we also tested the definition of “self-reported RA plus DMARD use or self-reported RA plus anti-CCP2 positivity” (Table 3). This combined definition resulted in a PPV of 59.6%, or 63.6% with the addition of rheumatoid factor positivity to the definition (self-reported RA plus DMARD use or self-reported RA plus anti-CCP2 plus rheumatoid factor).
Accurate epidemiologic methods for identifying RA cases in large population studies can allow for the use of these studies to improve our understanding of RA. Of all single measurements, anti-CCP2 provided the highest PPV (80.0%), and when combined with history of DMARD use, it had 100% PPV for physician-validated RA. Potentially, anti-CCP2 could be used to accurately identify participants with RA in the entire WHI sample, or other existing cohort studies, without requiring expensive and unfeasible clinical chart reviews. Furthermore, because of its high diagnostic accuracy for RA, using anti-CCP2 would allow investigators to be confident in the RA case definition as a probable RA case within a population study. In contrast, rheumatoid factor testing had little value on its own and was inferior to that seen with DMARD use.
Antibody measurements do have an implicit bias because they fail to capture seronegative RA cases. We addressed this problem by using a test definition that classifies participants as RA cases on the basis of self-reported RA and DMARD use or self-reported RA and anti-CCP2 positivity, which had a PPV of 59.6%, with the addition of rheumatoid factor. In future cohort studies that use a combination of serological and DMARD data to define RA, investigators will need to carefully consider the possibility of differences in anti-CCP2-positive and anti-CCP2-negative cases, the potential effects of DMARD use on outcomes, and potential bias in identifying cases of seronegative RA due to non-DMARD use or failure to report DMARD use.
It is important to note that no false-negative RA self-reporting is assumed with these test definitions. Patients with RA who do not report their condition are exceedingly rare. With a population prevalence rate of 0.5%–1% (24, 25) and an estimated false reporting rate of 10%, it would require the review of 1,000 control charts to capture a single nonreported case of RA. As for patients misclassifying themselves as having another form of arthritis, we noted that no RA cases miscoded themselves as having lupus or osteoarthritis when specifically questioned about RA status in our prior study (5).
Our method has a moderate limitation. For practical reasons, our diagnostic classification of RA utilized reviewers’ analyses based on information gleaned from chart reviews and conversations with treating physicians rather than explicitly meeting the ACR's RA diagnostic criteria after clinical examinations by the investigators. While the ACR criteria represent a standard diagnostic tool that can be reproduced by other research groups, they are not validated for use in chart reviews. In future studies, investigators with the ability to define RA by ACR criteria should evaluate these RA case definitions.
Our results demonstrate that it is possible to use a combination of self-reported information (diagnosis and medications) and serological testing (anti-CCP2) to accurately determine whether a study participant with self-reported RA truly has RA. It should be possible to apply the test definitions described above to many existing population studies.
Author affiliations: Section of Rheumatology, Department of Medicine, Washington Hospital Center, Washington, DC (Brian Walitt); Department of Epidemiology, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, Pennsylvania (Rachel Mackey, Lewis Kuller); Department of Neurology, School of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania (Yue-Fang Chang); Department of Medicine, School of Medicine, University of Pittsburgh, Pittsburgh, Pennsylvania (Larry Moreland); Department of Medicine, School of Medicine, Stanford University, Stanford, California (William Robinson); and Department of Medicine, School of Medicine, University of Colorado, Aurora, Colorado (Kevin D. Deane, V. Michael Holers).
This work was funded by Broad Agency Announcement NHLBI-WH-09-01 contract HHSN268200960006C (National Heart, Lung, and Blood Institute). The Women's Health Initiative is funded by the National Heart, Lung, and Blood Institute through contracts N01WH22110, 24152, 32100-2, 32105-6, 32108-9, 32111-13, 32115, 32118-32119, 32122, 42107-26, 42129-32, and 44221.
The Women's Health Initiative investigators—Program Office: Elizabeth Nabel, Jacques Rossouw, Shari Ludlam, Linda Pottern, Joan McGowan, Leslie Ford, and Nancy Geller (National Heart, Lung, and Blood Institute, Bethesda, Maryland); Clinical Coordinating Center: Ross Prentice, Garnet Anderson, Andrea LaCroix, Charles L. Kooperberg, Ruth E. Patterson, and Anne McTiernan (Fred Hutchinson Cancer Research Center, Seattle, Washington); Sally Shumaker (Wake Forest University School of Medicine, Winston-Salem, North Carolina); Evan Stein (Medical Research Laboratories, Highland Heights, Kentucky); and Steven Cummings (University of California, San Francisco, San Francisco, California). Clinical Centers: Sylvia Wassertheil-Smoller (Albert Einstein College of Medicine, Bronx, New York); Jennifer Hays (Baylor College of Medicine, Houston, Texas); JoAnn Manson (Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts); Annlouise R. Assaf (Brown University, Providence, Rhode Island); Lawrence Phillips (Emory University, Atlanta, Georgia); Shirley Beresford (Fred Hutchinson Cancer Research Center, Seattle, Washington); Judith Hsia (George Washington University Medical Center, Washington, DC); Rowan Chlebowski (Los Angeles Biomedical Research Institute at Harbor-UCLA Medical Center, Torrance, California); Evelyn Whitlock (Kaiser Permanente Center for Health Research, Portland, Oregon); Bette Caan (Kaiser Permanente Division of Research, Oakland, California); Jane Morley Kotchen (Medical College of Wisconsin, Milwaukee, Wisconsin); Barbara V. Howard (MedStar Research Institute/Howard University, Washington, DC); Linda Van Horn (Northwestern University, Chicago/Evanston, Illinois); Henry Black (Rush Medical Center, Chicago, Illinois); Marcia L. Stefanick (Stanford Prevention Research Center, Stanford, California); Dorothy Lane (University at Stony Brook, State University of New York, Stony Brook, New York); Rebecca Jackson (Ohio State University, Columbus, Ohio); Cora E. Lewis (University of Alabama at Birmingham, Birmingham, Alabama); Tamsen Bassford (University of Arizona, Tucson, Arizona); Jean Wactawski-Wende (University at Buffalo, State University of New York, Buffalo, New York); John Robbins (University of California at Davis, Sacramento, California); F. Allan Hubbell (University of California, Irvine, Irvine, California); Howard Judd (University of California, Los Angeles, Los Angeles, California); Robert D. Langer (University of California, San Diego, La Jolla, California); Margery Gass (University of Cincinnati, Cincinnati, Ohio); Marian Limacher (University of Florida, Gainesville, Florida); David Curb (University of Hawaii, Honolulu, Hawaii); Robert Wallace (University of Iowa, Iowa City, Iowa); Judith Ockene (University of Massachusetts/Fallon Clinic, Worcester, Massachusetts); Norman Lasser (University of Medicine and Dentistry of New Jersey, Newark, New Jersey); Mary Jo O'Sullivan (University of Miami, Miami, Florida); Karen Margolis (University of Minnesota, Minneapolis, Minnesota); Robert Brunner (University of Nevada, Reno, Nevada); Gerardo Heiss (University of North Carolina, Chapel Hill, North Carolina); Lewis Kuller (University of Pittsburgh, Pittsburgh, Pennsylvania); Karen C. Johnson (University of Tennessee, Memphis, Tennessee); Robert Brzyski (University of Texas Health Science Center, San Antonio, Texas); Gloria E. Sarto (University of Wisconsin, Madison, Wisconsin); Denise Bonds (Wake Forest University School of Medicine, Winston-Salem, North Carolina); and Susan Hendrix (Wayne State University School of Medicine/Hutzel Hospital, Detroit, Michigan).
These data were presented at the American College of Rheumatology annual meeting in Chicago, Illinois, on November 9, 2011 (poster 322).
Conflict of interest: none declared.