|Home | About | Journals | Submit | Contact Us | Français|
The SLICC Damage Index (SDI) is a validated instrument for assessing organ damage in systemic lupus erythematosus (SLE). Trained physicians must complete it, limiting utility where this is impossible.
We developed and pilot-tested a self-assessed organ damage instrument, the Lupus Damage Index Questionnaire (LDIQ), in 37 SLE subjects and 7 physicians. After refinement, 569 English-speaking SLE subjects and 14 rheumatologists from 11 international SLE clinics participated in validation. Subjects and physicians completed instruments separately. We calculated sensitivity, specificity, Spearman correlations and agreement, using the SDI as gold standard. 605 SLE participants in the community-based National Data Bank for Rheumatic Diseases (NDB) study completed the LDIQ and we assessed correlations with outcome and disability measures.
Mean LDIQ score was 3.3 (0-16) and mean SDI score was 1.5 (0-9). LDIQ had a moderately high correlation with SDI (Spearman r=0.50, p<0.001). Specificities of individual LDIQ items were >80%, except for neuropathy. Sensitivities were variable and lowest for damage with <1% prevalence. Agreement between SDI and LDIQ was > 85% for all but neuropathy, reduced renal function, deforming arthritis and alopecia. In the NDB, LDIQ correlated well with comorbidity index (r=0.45), SF-36 physical component scale (0.43), Medical Research Council dyspnea scale (0.40), disability (0.37) and SLE Activity Questionnaire score (0.37).
The LDIQ’s metric properties are good compared to the SDI. It has construct validity and correlations with health assessments similar to the SDI. The LDIQ should allow expansion of SLE research. Its ultimate value will be determined in longitudinal studies.
The overall prognosis in systemic lupus erythematosus (SLE) has improved since the 1950s, with 10 year survival rates now 85% or better1. Unfortunately, however, patients with SLE still suffer from significant clinical manifestations and often irreversible organ damage, including accelerated atherosclerotic vascular disease2, osteoporosis3, and other sequelae of disease activity and immunosuppressive therapies4.
The Systemic Lupus International Collaborating Clinics (SLICC)-Damage Index (SDI) is a 42-item questionnaire concerning long-term damage due to SLE in 12 organ systems for use in clinical research studies5-8. The SDI was developed between 1985 and 1996 by a panel of SLE experts and members of SLICC a multinational collaboration of SLE centers9. The SDI index is completed by the physician at enrollment into clinical studies and assesses major irreversible organ damage and treatment toxicity, including stroke, loss of limb, malignancy, and premature gonadal failure. Damage from SLE is defined as, “an irreversible change in an organ or system that has occurred since the onset of SLE, and present for at least 6 months“5. Each item is assessed as present or absent and scores of two or three are possible for recurrent events (e.g. stroke) or serious damage (e.g. end stage renal disease). Only complications that occur after the onset of SLE are assessed. The widely-used index was reviewed at each of the SLICC participating medical centers for content and face validity6. Inter-observer variability of the index when performed by different members of SLICC was good10. Retrospective completion of the SLICC Damage Index (SDI) by another physician reviewing medical records has also been shown to be valid11 and the SDI has been endorsed by the American College of Rheumatology and validated in multiple international settings6, 12-14. The SDI is predictive of both long-term survival and quality of life for patients with SLE15-17.
Often in clinical research in SLE, however, it is not practical to request that many busy physicians complete time-consuming forms and assessments, in particular in large national cohort studies. We aimed to develop and validate a patient self-administered version of the SDI, The Lupus Damage Index Questionnaire (LDIQ), for use where direct assessment by physicians is not practical. A self-reported instrument would enable assessment of organ damage in community-based SLE patients without requiring participation of trained physicians familiar with the SDI. This would facilitate assessment of SLE-related organ damage in epidemiological cohort studies in which subjects are followed in the community, ultimately allowing the expansion of SLE investigations beyond academic centers.
This study proceeded in 3 stages: 1) initial development, pilot-testing, and revisions, 2) a multicenter clinic-based validation study and 3) a mail-based validation study in a separate longitudinal cohort of community-based US participants with SLE, the National Databank of Rheumatic Diseases (NDB), to assess construct validity and correlates of the LDIQ.
Eligible subjects for all stages had SLE according to the Updated American College of Rheumatology (ACR) criteria18, 19, and were over age 18 . Development and stage 1 piloting took place at the Brigham and Women’s Hospital in Boston, MA and the University of Alabama at Birmingham, in Birmingham, AL. Stage 2 was a multicenter validation performed at 11 different clinical lupus centers in the U.S., Canada and England. Eligible physicians for both stages were trained attending rheumatologists specializing in SLE at these clinical centers and with experience in completing the SDI instrument. In stage 3, community-based SLE participants in the NDB, a longitudinal study of rheumatic disease outcomes, completed the LDIQ. These participants are followed longitudinally with semi-annual, detailed, 28-page questionnaires, as previously described20, 21. SLE participants were initially enrolled largely by rheumatologist referral, but also by self-referral after confirmation of SLE by their rheumatologist.
All SDI items (42 items in 12 organ systems) were translated into lay language questions (55 total questions) to which subjects were asked to respond “yes” or “no”. Several clinical researchers, clinicians and literacy experts reviewed the translation, attempting to target an 8th grade reading level. The questions were reviewed and refined by the developers (KC, GSA, FW) and the performance of this instrument was then assessed in a pilot study involving 37 SLE subjects22. these subjects were asked to complete the three-page pilot LDIQ and their rheumatologists separately completed and returned the SDI, blinded to subject responses. Subjects were also asked to circle words and questions that they did not understand and to make comments in provided space. Subjects who did not read English, who refused to complete the questionnaire, or whose physicians did not wish their patients to participate were not included. While LDIQ overall metric properties were good, a few items on the questionnaire were thought to produce false positive or false negative responses from the subjects. We refined these items changing the wording of problematic questions. Rather than including a “don’t know” response for each item, which may have had a high response rate, the directions stated, “Don’t worry if there are some medical words you don’t understand. This usually means that you don’t have the problem the question is asking about”.
We tested the revised index comparing it to physician-reported SDI as above in a multi-center study. We contacted 11 SLE centers in the US, Canada and England and invited them to participate in this validation study. Consecutive English-speaking SLE patients were recruited to fill out the questionnaire in each clinical setting during the months of July-December 2007. Independently, the rheumatologists seeing the subjects that day completed a SDI for each subject. In a subset of 51 of these subjects at BWH and UAB, the questionnaire was mailed to the participant, rather than completed during an office visit, and the rheumatologist independently completed and returned the SDI.
From September 2007- December 2007, we asked 673 SLE participants in the NDB to complete the LDIQ, at the time of their semi-annual questionnaires for the ongoing cohort study. Participants also completed the SF-36 from which the physical (PCS) and mental (MCS) component summary scores are derived23, 24, the Medical Research Council (MRC) dyspnea index25, the Health Assessment Questionnaire (HAQ)26, VAS scales for pain, global and fatigue, and measures of service utilization, treatments and medical costs. We examined data for the Systemic Lupus Erythematosus Activity Questionnaire (SLAQ), completed 6 months previously. The SLAQ, a patient-reported version of the physician-reported SLAM, is a validated 24-item questionnaire that reduces to 17 items for scoring and has a range of 0-4427-29. Comorbidity was measured by a patient-reported composite comorbidity index (range 0-9) comprised of 11 present or past comorbid conditions including: pulmonary disorders, myocardial infarction, other cardiovascular disorders, stroke, hypertension, diabetes, spine/hip/leg fracture, depression, GI ulcer, other GI disorders, and cancer 30, 31.
All forms were returned to the central processing site, the NDB in Wichita, KS. Questionnaires completed by subjects and physicians in all stages of the study were anonymized and tracked only study numbers. Institutional Review Board approval from all participating institutions was obtained for all aspects of this study.
Data collected from the SDI and LDIQ questionnaires were dichotomous responses to a series of questions to assess the severity of damage associated with the subject’s SLE. For each question on the LDIQ, the subject’s response was recorded as either “true positive” (TP), “true negative” (TN), “false positive” (FP) or “false negative” (FN), using the responses provided by the physicians on the SDI for that patient as the gold standard. The performance of the LDIQ in assessing SLE damage was analyzed by calculating the mean sensitivity, specificity, percent agreement and kappa agreement statistic for subjects and physicians for each item on the LDIQ and for the overall questionnaire, using the SDI as the gold standard. We calculated Spearman correlation coefficients comparing organ damage domains and total scores on the LDIQ and SDI. We examined agreement in SLE subject and physician scores as a function of SLE disease duration employing linear regression analyses.
Associations between the LDIQ, comorbidity index, patient global and other study variables were investigated by Spearman correlation coefficients. Associations between reported work disability and SF-36 PCS and LDIQ and its domains were investigated using Kendall’s tau-a. Kendall’s tau-a is related to the area under the ROC curve, and allows us to understand the degree to which variables are simultaneously associated with LDIQ domains. Kendall’s tau has a simple interpretation: the percent concordance between variables. For example, a value of 0.151 indicates that it is 15.1% more likely that a disabled person will have a high neuropsychiatric domain score than a non-disabled person will have such a score. For PCS, the comparison is a score above and below the median.
Data were analyzed using Stata Version 10.1 (StataCorp, College Station, TX) for all statistical analyses. Statistical significance was set at the 0.05 level and all tests were two-tailed.
The demographic characteristics of the subject populations in stages 1, 2 and 3 are summarized in Table 1. In general, the study subjects were predominantly women, with 10-16 years of SLE. The subjects in the Stage 1 initial pilot were slightly younger and had shorter mean disease duration compared with those in the Stage 2 validation.
Thirty-seven English-speaking SLE subjects and seven rheumatologists at two U.S. SLE centers participated in LDIQ initial development and pilot testing. The overall sensitivity of the pilot LDIQ was 92.4%, specificity 94.9%, and agreement 94.4%. The following items needed further refinement as there were either false positives (end-stage renal disease, cognitive impairment, alopecia, scarring skin and claudication) or false negatives (retinal changes, muscle atrophy and osteomyelitis), when compared to SDI responses. The refined LDIQ questionnaire included 56 questions organized into 12 organ damage domains. (Appendix A)
Five hundred and sixty-nine English-speaking subjects with SLE and 14 trained SLE rheumatologists from 11 academic medical centers in the U.S., U.K. and Canada participated in the multicenter validation. Subject demographics are shown in Table 1. Of the physicians, 56% were female, with a mean age of 49 (±9.6) years, 16.8 (±8.4) mean years in practice and 100% had formal rheumatology and SLE training. The questionnaire took most subjects less than 10 minutes to complete. The overall metric properties of the LDIQ questionnaire compared to the SDI of this study are shown in Table 2. Mean score across the LDIQ items was 3.30 (range 0-16) and mean physician SDI score was 1.53 (range 0-9). The mean sensitivity, specificity and agreement were 53.3, 94.6 and 93.2% respectively, reflecting good overall agreement between the LDIQ questionnaire and the SDI. The LDIQ had a moderately high correlation with the SDI (Spearman r=0.48, p<0.001). When stratified by subjects’ educational level, the LDIQ–SDI correlations were surprisingly lower among subjects with higher educational levels: Spearman r=0.49 for subjects above the mean educational level and r=0.54 among subjects at or below the mean. A comparison of responses to the LDIQ by SLE subjects and the SDI by their rheumatologists by individual organ-system item is shown in Table 3. Specificities of individual LDIQ items were all >80% except for neuropathy (78.7%). Sensitivities were variable, and low for low prevalence damage, such as myocardial infarction, significant tissue loss, gastrointestinal infarction, gastrointestinal stricture or surgery. Agreement between physician and patient was > 85% for all items but neuropathy, reduced renal function, deforming arthritis and alopecia. Kappa agreement statistics were also variable, but above 0.30 for most items. Using linear regression, we found an increase of 0.03 points (95%CI 0.01-0.05) in subject LDIQ-physician SDI disagreement per year of SLE duration. Among the 51 subjects who received and returned the LDIQ by mail, overall agreement with their physicians’ SDI scores was high (ranging from 80% to 100%). The mean subject LDIQ score was 3.3 (range 0-11), the mean physician SDI score was 1.6 (range 0-7) and the Spearman correlation between the two was again high (r=0.71, p < 0.001).
Table 4 displays total and domain scores for the SDI completed by rheumatologists and the LDIQ completed by SLE subjects. Subjects scored more positively than their physicians for every domain of the SDI. Correlation coefficients between the SDI and the LDIQ scores were generally good, and highest for the ocular, cardiovascular, malignancy and diabetes domains.
Six hundred and five NDB SLE participants, mainly Caucasian females, whose demographics are shown in Table 1, (90% of those mailed), returned the LDIQ. Twenty-three percent were currently receiving U.S. Social Security disability benefits, 42.3% had received disability payments at some time in their lives, and 27.6% considered themselves disabled. Agreement between LDIQ and SDI was lower in this community-based part of the study (Table 1, mean agreement 81%) than in the office-based samples.
The LDIQ was skewed to the right. The mean (SD) score was 4.9 (3.5), range (1-22), the median (IQR) was 4.0 (2.0, 6.0), and alpha reliability was 0.72. The major contributions to the LDIQ total score were from the neuropsychiatric and musculoskeletal domains (Figure 1). Neuropathy was the most commonly reported item (35%). The next most common item (34.4%) was arthritis in the hands. Cataracts were noted by 32.4%. As Figure 1 indicated substantial differences in the degree to which individual domains contributed to the LDIQ total scores, we investigated the relation between the degree of contribution of the individual domains and two SLE outcomes. Table 5 shows that the neuropsychiatric and musculoskeletal domains, which contributed most to the LDIQ total score, also were the most strongly associated domains with disabled status and SF-36 PCS.
We studied the relationship between LDIQ, the comorbidity index, and patient global in relation to important SLE outcome and status variables using Spearman correlations (Tables (Tables55 and and6).6). The LDIQ was significantly (p<0.001) associated with important outcome and status variables. As hypothesized, the association between the LDIQ and clinical variables was stronger in all cases than the association between the comorbidity index and these variables (Table 5). The strongest correlates of LDIQ were the comorbidity index (r=0.45), the SF-36 PCS (r=0.43), the MRC dyspnea scale (0.40), disabled status (0.37) and prior SLAQ score (0.37). By contrast, patient’s global status, a measure of current status, was correlated with the LDIQ at 0.17 and comorbidity index at 0.15.
To assess the association between the LDIQ and outcomes quantitatively, we studied the four quartiles of LDIQ scores in relation to direct medical costs, work disability, the comorbidity index, and hospitalization with the last 6 months. As shown in Table 7, increasingly abnormal outcomes were associated with the quartiles of LDIQ score. The p value for linear trend for each variable was < 0.001.
The purpose of this study was to develop, refine and perform the initial validation of a subject self-assessed LDIQ questionnaire in a large English-speaking population of SLE patients. We found good overall agreement between the newly-developed LDIQ questionnaire and the SDI. In the community-based NDB stage 3 of this study, in which participants completed the questionnaire by mail, rather than in the rheumatologist’s office, the mean agreement was slightly lower at 82%, suggesting that physicians and patients may have had some influence on each others’ responses. The specificities of individual LDIQ items were generally greater than 80% compared to the SDI results. Sensitivities were variable and were lower for low prevalence damage items, such as myocardial infarction, mesenteric insufficiency, and gastrointestinal stricture or surgery, and pancreatic insufficiency (all of which had <1% prevalence). The low prevalence of certain types of damage does limit the interpretation of comparisons of the two instruments. Correlations between the SDI and the LDIQ instruments were high for most organ damage domains in both the office-based and mailed validation studies. This suggests that the LDIQ questionnaire may be useful and reliable alternative to the SDI in assessing SLE-related damage in population studies.
In developing this index, we expected a moderate degree of disagreement between physicians and patients regarding specific damage questions. For physicians, over-reporting should not occur, but under-reporting might occur. For example, specialty lupus physicians might be unaware of diagnoses made by other physicians, might have out of date records, or might not adequately examine the eyes for cataracts or retinal damage. Such under-reporting might be major problem when general physicians, not lupus specialists, provide data. More important, however, is misreporting by patients. Although subject under-reporting would be expected for some items, as subjects may not have had or understood all of the data that physicians had, over-reporting was the more common misclassification. Questions concerning end-stage renal disease, cognitive impairment, alopecia, skin scarring, claudication, retinal changes, muscle atrophy and osteomyelitis, which were confusing to subjects or which showed high disagreement with clinicians’ responses in the pilot study were subsequently refined and did show improved performance in the validation study. Overall, we found the patient damage score to exceed the physician score by 1.9 units and the correlation between patients and physician damage score to be 0.65.
The subjects in the first two stages of this study were consecutive, willing English-speaking patients in major SLE centers. They were relatively young (mean age in early 40s), with mean disease duration of over ten years, and SDI scores ranging from 0 to 9, and a wide variety of SLE manifestations (Tables (Tables11 and and3).3). Non-random subject refusal could have introduced a selection bias or limited the generalizability of the results. Stage 3 involved community-based patients, which improves the generalizability of the study. While we targeted an 8th grade reading level, given SLE’s complicated manifestations, many of the challenging words exceeded this level (e.g.pulmonary hypertension, claudication, cardiomyopathy, and pericarditis). We placed these terms in quotes, aiming for patients’ recognition. When stratified by participants’ educational level, correlations with rheumatologists’ SDI scores were actually higher among subjects with fewer years of formal education.
One limitation of this validation study is that the LDIQ was assessed using responses of subjects’ physicians on the SDI as the gold standard. The assumption was that the physicians’ responses were correct, but this may not have always been true. As our analyses suggested, this is particularly likely for subjects who have had SLE for many years and suffered some of their SLE-related damage early in their disease (possibly before they were known to their current physician). The SDI was validated and endorsed by the American College of Rheumatology (ACR) in 19966 and has since been validated in a number of international centers representing different health care systems6, 10, 13. These studies have demonstrated that this instrument can reliably identify changes in damage seen in both active and inactive disease and that, at one point in time, there is no correlation between the SDI score and disease activity, measured by the SLEDAI (a disease-activity scale), or health-related quality of life, measured by the Medical Outcome Survey (MOS) Short-Form 20 (SF-20)6, 10, 32, 33. The SDI therefore functions as intended, measuring accumulated damage in patients with SLE, independent of simultaneous disease activity and or health-related quality of life (although these three facets of disease are inextricably linked as with progression over time15-17). Higher scores on the SDI are predictive of poor outcomes, defined as death or hospitalization34-36. African-Caribbean, African-American, Hispanic and Asian patients accumulate more damage as assessed by the SDI than do Caucasian patients12, 37, suggesting a racial/ethnic influence on the expression of this disease. SDI assessed damage clearly increases with disease duration 38. The SDI therefore provides an important outcome measure in SLE, both for studies of prognosis and in the assessment of the long-term effects of the disease process and its medical therapy. Whether the self-assessed LDIQ will be predictive of long-term clinical outcomes and mortality in the same way as the SDI will be the subject of future investigation. Age, disease duration and race/ethnicity have known influences upon the severity of SLE-related damage and are also likely to influence the subjects’ understanding of their disease and the questionnaire.
If the purpose of the LDIQ is to evaluate an individual patient, the level of concordance that we found would be unacceptable. However, if the purpose is to evaluate groups of patients in outcomes research, particularly in community settings, the degree of concordance may be acceptable if the damage scale is substantially predictive of health status and outcomes. Data from the 3rd stage of this study, the observational cohort, suggests this is the case. In this community-based cohort study, the LDIQ was more useful for identifying health status and outcomes than standard assessment measures. In this setting, the higher LDIQ value of 4.9, compared with 3.4, likely reflects increased age, SLE duration39, and other characteristics of the NDB cohort. It is possible that patients followed in academic lupus centers and are more educated about SLE (regardless of educational level) than are community patients, and therefore more likely to answer the questions correctly, as suggested by the lower agreement between physician and patient scores was lower in this cohort. Regression analyses on age and SLE duration (not shown) indicated that about 0.8 units of the 4.9 LDIQ score in the current study can be accounted for by age and SLE duration. Both variables have been shown to associated with increased damage as measured by the SDI 39, 40. Correlations between SLE disease activity (by SLAQ) and LDIQ were as good as those between SLAQ and patient comorbidity index, and patient global assessment. As shown in table 5, the LDIQ is a “better” comorbidity index than the standard comorbidity index with respect to outcomes that can be considered fixed or relatively fixed. Similarly, LDIQ is less correlated to patient activity variables like fatigue and HAQ than is the patient global. This is not surprising, as the LDIQ measures the domain of damage, which is conceptually different than the disease activity domain. The data of table 5 suggest the LDIQ is performing as expected in these cross-sectional analyses. In one study of 54 SLE patients the SDI was correlated with the SF-36 PCS, MCS and patient global at values of 0.34, 0.02, and 0.14 41. In the current report the LDIQ correlation with these variables was 0.432, 0.141, and 0.166, suggesting similar, or perhaps stronger, associations between LDIQ and self-reported patient variables compared with the SDI data.
We have also found that stepwise increases in the LDIQ were associated with clinically important increases in direct medical costs, work disability, comorbidity, and hospitalization (Table 7). We also noted in Figure 1 that different domains contributed different amounts to the final LDIQ score. We noted that the two domains that contributed most to the LDIQ also were the domains most strongly associated with disability status and health status (Table 6). The associations were weaker with the other domains, reflecting the uncommonness of some of the outcomes, and that many of the outcomes are also common in individuals without SLE, and the weighting design of the SDI authors. However, attempts to reweight the SDI after the use of Rasch analysis have shown the current weighting system to be clinically appropriate42.
The LDIQ has been translated into several languages. In one validation study, a total of 887 SLE patients and 40 rheumatologists with a special interest in SLE participated: 659 of the patients completed the Spanish (611 in Latin America, 30 in the US, and 18 in Spain), 140 the Portuguese (122 in Brazil and 18 in Portugal) and 88 the French (all in Canada) versions of the LDIQ as well as the SDI43. In that study, the SDI total score was 1.9 and the LDIQ total score was 3.7 and the correlation between LDIQ and SDI was 0.61.
Quantifying irreversible organ damage from SLE is important for characterization of SLE populations and for outcomes research. The results of this study are encouraging and suggest that the LDIQ, a self-reported questionnaire completed by patients rather than physicians, may be able to reliably assess SLE-related damage in patients, providing an alternative to the physician-completed SDI for lupus patients who are primarily managed in the community rather than in specialist lupus centers. Physicians caring for SLE patients in the community may be unfamiliar with the use of this assessment tool, nor have the time or motivation to complete it. This currently limits the use of the SDI to large academic specialist centers, thus the generalizability of research findings to these settings. While it may not be possible to use these two instruments interchangeably within the same study, the LDIQ could be employed for future studies where the collection of physician-assessed data is impossible, ultimately allowing the expansion of investigations of the impact of SLE beyond academic medical centers. The LDIQ has preliminary validity, but comparative longitudinal studies will be required to determine its reliability, sensitivity to change over time and real value in assessing organ damage from SLE in cohort studies.
The authors are grateful to Victoria Gall for her help in translating the SDI into lay language, to Amy Yearout for her technical assistance, and to Leigh Fritz for her efforts recruiting subjects.
Supported by: Dr. Costenbader receives research funding from the American College of Rheumatology, the Arthritis Foundation, and the National Institutes of Health (NIH, P60 AR047782) and BIRCWH K12 (HD051959, supported by NIMH, NIAID, NICHD, and OD). Dr. Petri has received research support for the Hopkins Lupus Cohort (AR43727), and a National Center for Research Resources Grant (UL1 RR 025005). Dr. Elliott has research funding from a National Research Science Award (F32AR054278-01A1) and the Arthritis Foundation, and Dr. Manzi’s research is supported by NIH K24 AR002213-05. Dr. Karlson is supported by NIH grants R01 AR49880, P60 AR047782, and K24 AR052401. Dr. Hanly is funded by the Canadian Institutes of Health Research and Dr. Clarke acknowledges support of the Singer Family Fund for Lupus Research. Dr. Paul Fortin is a Distinguished Senior Investigator of The Arthritis Society with additional support from the Arthritis Centre of Excellence, University of Toronto, Lupus Canada, Lupus Ontario, the Lupus Foundation of Ontario and BC Lupus as well as from the Arthritis and Autoimmune Research Centre Foundation. Dr. Alarcón receives support from the NIH P01 AR49084. Administrative, programming, printing, postage, and analysis costs for this study were underwritten by the National Data Bank for Rheumatic Diseases.