|Home | About | Journals | Submit | Contact Us | Français|
To evaluate the measurement properties of a new osteoarthritis (OA) pain measure.
The new tool, comprised of 12 questions on constant vs intermittent pain was administered by phone to 100 subjects aged 40+ years with hip or knee OA, followed by three global hip/knee questions, the Western Ontario and McMaster Universities (WOMAC) pain subscale, the symptom subscales of the Hip Disability and OA Outcome Score (HOOS) or Knee Injury and OA Outcome Score (KOOS), and the limitation dimension of the Late Life Function and Disability Instrument (LLFDI). Test-retest reliability was assessed by re-administration after 48–96 h. Item response distributions, inter-item correlations, item-total correlations and Cronbach's alpha were assessed. Principle component analysis was performed and test-retest reliability was assessed by intra-class correlation coefficient (ICC).
There was good distribution of response options across all items. The mean intensity was higher for intermittent vs constant pain, indicating subjects could distinguish the two concepts. Inter-item correlations ranged from 0.37 to 0.76 indicating no item redundancy. One item, predictability of pain, was removed from subsequent analyses as correlations with other items and item-total correlations were low. The 11-item scale had a corrected inter-item correlation range of 0.54–0.81 with Cronbach's alpha of 0.93 for the combined sample. Principle components analysis demonstrated factorial complexity. As such, scoring was based on the summing of individual items. Test-retest reliability was excellent (ICC 0.85). The measure was significantly correlated with each of the other measures [Spearman correlations −0.60 (KOOS symptoms) to 0.81 (WOMAC pain scale)], except the LLFDI, where correlations were low.
Preliminary psychometric testing suggests this OA pain measure is reliable and valid.
In prior research, we conducted focus groups to examine the pain experience of people with hip/knee osteoarthritis (OA) from early to late disease, including those aspects of the pain experience that were considered most distressing. These focus group discussions identified two distinct types of pain in OA - an aching and fairly constant background pain and a less frequent, but more intense and often unpredictable pain1. Of these, intermittent but intense pain, particularly when not predictable, had a greater impact on quality of life than did background aching pain. These data suggest that the evaluation of pain in people with hip or knee OA needs to include questions that ask about both constant and intermittent pain taking into account both pain intensity and impact on quality of life. To our knowledge, no such measure exists. Hence, using the data from these focus groups to generate items, we developed a new pain measure for hip/knee OA, the Measure of Intermittent and Constant Osteoarthritis Pain (ICOAP), suitable for use by clinicians to document progression or worsening of pain, response to therapy and indication for need for referral to surgery for consideration of joint replacement. Here we report the preliminary psychometric testing of this measure.
Item generation was performed using standardized focus groups described in detail elsewhere1. Briefly, we used a “funnel approach”, starting with broad open-ended questions about the characteristics of the hip/knee pain over time, and then increasingly focused on more specific issues, including those aspects of the pain that participants found most distressing. Following these discussions, a modification of Ruta et al.'s2. Patient Generated Index (PGI) was used to further assess the priorities and concerns of individuals living with hip or knee OA pain. Focus group transcripts were reviewed independently by two or more researchers to identify distinct themes. Content analysis was then performed on the coded transcripts to examine for trends in responses over time from early to late disease. PGI responses were analyzed descriptively. Summaries of our findings were created for hips and knees, separately, and disseminated to participants, investigators and focus group moderators to ensure they reflected what they heard in their particular group.
The 12-item measure, developed based on focus group results, was comprised of two sections, one for `constant pain' and one for `pain that comes and goes'. For each of these pain types, single items assessed pain intensity, affect on sleep, impact on quality of life, extent to which the pain `frustrates or annoys', and the extent to which it `worries or upsets'. For pain that comes and goes, two additional items asked respondents to report the frequency of pain and the degree to which the pain could be predicted. The time frame used was 1 week, in keeping with other widely used OA pain measures. All items were constructed as rating scales, with five levels of response (0–4) - for questions asking about intensity, response options were `not at all' (0), `mildly', `moderately', `severely' and `extremely' (4), while those that asked about frequency had the following response options: `never' (0), `rarely', `sometimes', `often', and `very often' (4). People with OA reviewed the questionnaire for wording and to ensure correct interpretation of the items prior to testing of the measurement properties.
Preliminary testing of the measurement properties was performed in 100 individuals with hip (n = 18) or knee OA (n = 82) living in Ontario, Canada drawn from investigators' clinical practices, joint replacement wait lists, and from among the members of an existing OA cohort. Those eligible to participate were: English-speaking men and women with hip or knee OA, aged 40+ years, who responded “yes” to the question: “Have you experienced aching, discomfort, pain and/or stiffness in or around a hip or knee on most days of at least 1 month (15 or more days of the month) during the past year?”. In addition, participants had not experienced an injury to the joint area within the last year, or had a total joint replacement (TJR) of the symptomatic joint. Subjects with rheumatoid arthritis or any other type of inflammatory arthritis, fibromyalgia, chronic low back pain, or another chronic pain disorder, such as diabetic neuropathy, were excluded. In addition, hip participants experiencing radicular pain were not eligible.
Questionnaires were administered over the phone. The two components of the tool, constant pain and pain that comes and goes, were administered in random order, followed by three global hip/knee questions (In the past week, how frustrated or annoyed have you been by your hip/knee problems?; In the past week, how upset or worried have you been by your hip/knee problems?; In the past week, how much has your overall quality of life been affected by your hip/knee problems?), and finally five questions to obtain feedback from the participants regarding unclear, difficult or uncomfortable questions, inappropriate response categories and acceptability of interview length. A final section was comprised of questions taken from existing validated OA measures, in order to assess the validity of the new tool. These included the pain subscale of the Western Ontario and McMaster Universities (WOMAC) OA Index3, the symptoms subscale of the Hip Disability and OA Outcome Score (HOOS)4 or Knee Injury and OA Outcome Score (KOOS)5 and the limitation dimension of the Late Life Function and Disability Instrument (LLFDI)6,7. Demographics such as age, sex, education and whether they were on a wait list for or had ever had a TJR were also collected.
Test-retest reliability was assessed by re-administration to subjects who agreed to be contacted again. The same interviewer contacted them again between 48 and 96 h after the first interview and repeated the new instrument only. The questionnaire was not completed if the participant reported that they had experienced any change in their pain since completion of the first interview. If the original interviewer was unable to contact the subject at the prearranged time, another interviewer completed the re-contact (n = 16).
The distribution of item responses was evaluated, including the response option frequencies and mean, standard deviation (SD) and median scores for each item. Item responses were compared for individuals with hip vs knee OA, and by gender, using chi square analysis. Where differences were observed for the hip vs knee, we also assessed for differences by gender within each joint group. Inter-item correlations were determined using Spearman or Kendall's tau, depending on the distribution of responses and potential need to collapse item response categories based on response frequencies. Corrected item-total correlations were also calculated. Correlations in the approximate range of 0.30–0.70 are desirable as lower values would indicate lack of homogeneity and high correlations would indicate item redundancy8.
As most inter-item correlations were in the desired range, Cronbach's alpha was calculated for all items and the constant and intermittent pain items to determine scale homogeneity where Cronbach's alpha of 0.80 was considered a minimum standard for creating a summated score9,10. Principle component analysis with varimax rotation and then with promax rotation was performed to determine whether the constant and intermittent pain functioned as separate domains or whether the questionnaire score was best reflected as a single dimension.
Among those who reported no change in pain, the intra-class correlation coefficient (ICC) (version 2) was calculated as a measure of test–retest reliability11. For the initial stages of test development, an ICC of at least 0.80 is required to minimize measurement error while 0.90 or greater is preferred12,13.
Content and face validity were determined a priori through development of the instrument based on data derived through patient focus groups, outlined above. Thus, the initial focus here was on preliminary assessment of construct validity. We hypothesized the following: the new measure would correlate modestly with the WOMAC pain and the HOOS/KOOS symptom subscales (intra-class correlation of 0.40–0.70); intermittent pain would have a higher correlation with the LLFDI score than constant pain; females would tend to report higher levels of pain than males, consistent with prior research in OA; and, the impact of pain on quality of life would be greater in those with greater pain and greater unpredictability of pain on the new measure.
There was no differential item functioning by hip or knee and the results for reliability and validity testing, although in a small sample of people with hip OA, were similar. As such, all analyses were conducted for the hip sample alone, the knee sample alone and then the combined group.
Ethics approval was obtained from the Sunnybrook and Women's College Health Sciences Centre Research Ethics Board.
The new measure was pilot tested in 18 individuals with hip OA and 82 with knee OA; of these, 12 and 64, respectively, participated in the retest. Characteristics of the participants by index joint (hip or knee) are presented in Table I. The mean age of participants with hip and knee OA was similar, at 72.1 ± 9.0 years and 75.4 ± 9.6 years, respectively (range overall 51–93 years); 78.0% of participants were female. Approximately 68% of participants had a high school education or less, and four (22.2%) hip and three (3.7%) knee participants were on a wait list for joint replacement surgery. The mean WOMAC pain subscale scores were similar for hip and knee participants, at 7.8/20 (3–16) and 8.2/20 (0–19), respectively. Hip participants' HOOS symptom scores and knee participants' KOOS symptom scores were similarly distributed across the range of possible values. LLFDI – limitation scores were 70.0 ± 15.9, 60.3 ± 9.0 and 62.1 ± 11.2 for hips, knees and all subjects, respectively.
Descriptive analyses of the items demonstrated good distribution of response options (i.e., use of the entire scale) across all items (see Table II for distributions for combined hip and knee sample). Of note, given that the mean intensity for intermittent as compared to constant pain was higher (2.0/4 vs 1.6/4, P = 0.003), it appeared that subjects could distinguish the two types of pain and that intermittent pain was of greater intensity as had been indicated in the focus groups and interviews1. One hip participant and two knee participants reported having only constant pain.
The inter-item correlations ranged from 0 to 0.90, 0.29 to 0.81, and 0.30 to 0.75 for each of the hip, knee and combined samples, respectively, indicating no item redundancy. In each case the lowest inter-item correlations were with the same item (predictability of pain that comes and goes), which correlated with each of the other items in the range of 0.03–0.47. This item was problematic in both the hip and knee samples, where the corrected item-total correlations were low (<0.30); Cronbach's alpha increased with the removal of this item. Removing this item in all subsequent analyses, Cronbach's alpha was 0.95, 0.93 and 0.93 for the hip, knee and combined samples, respectively. Cronbach's alpha was above 0.85 for constant pain and intermittent pain for each of the hip, knee and combined samples.
Given the high Cronbach's alpha, we used principle component analysis with varimax rotation to evaluate whether constant and intermittent pain would be best represented by two domains of pain rather than a single construct. This analysis was done only for the total sample as we had insufficient power to perform analyses by hip and knee, separately. In an exploratory analysis using eigenvalue = 1 criteria to consider retaining factors, three factors were extracted that explained 81.7% of the variance (factor 1 = 54.9%, factor 2 = 16.8%, factor 3 = 10.0%). The solution was complex with multiple loadings of >0.40 across factors. Eight (five constant pain and three intermittent pain) of the 11 items loaded on the first factor (loadings from 0.44 to 0.87) but four of these also loaded on the second factor as well. Two items, constant pain affecting sleep and pain that comes and goes affecting sleep, loaded on a third factor but also demonstrated complexity in that they also loaded on the first factor. These two items may reflect sleep difficulty in general as well as pain. Given the complexity of the results from the orthogonal rotation, a solution was also obtained using promax rotation. This solution further confirmed the complexity as all items with the exception of frequency of pain that comes and goes, loaded on multiple factors. In a further attempt to determine if there was reason to scale the constant and intermittent pain separately, we looked at whether either of these two types of pain was more highly correlated with participants' global ratings of (1) frustration, (2) upset/worry or (3) quality of life. However, the correlations were similar for constant and intermittent pain: 0.59 and 0.51, 0.21 and 0.13, and 0.24 and 0.35 (P < 0.01 for all) for each of the global constructs, respectively. Based on these results, our subsequent analyses were based on a total summed score for the 11 items. In 76 individuals who completed the pain measure twice, test–retest reliability was excellent [ICC2 = 0.85, 95% confidence interval (CI): 0.76–0.91 for the combined sample].
The 11-item measure was significantly correlated, and in the directions expected, with the WOMAC pain scale, the KOOS symptoms scale, and self-rated affect of hip/knee problems on quality of life with Spearman correlation coefficients ranging in magnitude from 0.60 (KOOS symptoms) to 0.81 (WOMAC pain scale) (Table III). As predicted, scores were slightly higher in women than men [mean score 17.4 (95% CI: 15.2–19.6) vs 15.0 (95% CI: 10.9–19.1)] although not significantly different. Our hypothesis that intermittent pain would be more highly correlated with LLFDI – limitations subscale was not supported as the correlations were of very small magnitude (i.e., <0.3) and not statistically significantly different than zero for the hip, knee or combined samples.
This study evaluated the preliminary psychometric properties of a new OA pain measure, ICOAP. The measure, developed based on focus groups conducted in people living with painful hip and knee OA residing in four English-speaking countries, includes evaluation of two distinct types of pain – constant pain and pain that comes and goes – that were identified by people with OA as important. Preliminary psychometric testing suggests the measure is reliable and valid.
Content validity of the measure was achieved by our methods of determining items. The relevance of items that ask about the two types of pain is further supported by our findings that the mean intensity for intermittent as compared to constant pain was higher among study participants. This lends credence to the fact that people with OA can indeed distinguish these two types of pain, and that the OA pain experience is characterized by less severe, aching constant pain punctuated by more intense but intermittent pain, as was indicated in the focus groups1. Among our subjects, most of whom had moderate to severe hip or knee OA, only three participants reported having only constant pain and 12 reported having only intermittent pain, consistent with our hypothesis that constant pain comes later in the course of the arthritis pain experience.
Although our focus group findings suggested that the time course for evolution of pain in hip OA is over an abbreviated time period compared with knee OA, aspects of the pain that were deemed distressing were similar for hip and knee OA focus group participants. Nonetheless, we evaluated the new tool separately for subjects with hip and knee OA to evaluate for differential item functioning, and found none. These preliminary findings suggest the tool can be used similarly for patients with hip or knee OA; however, as our sample size of hip pilot test participants was relatively small, we recommend additional evaluation.
From focus group discussions23, it was clear that intermittent but intense pain, particularly when not predictable, had the most significant impact on one's quality of life, often leading to significant curtailing of activities. For this reason, we initially included in the measure an item that asked about the degree to which the pain that comes and goes was predictable. Unfortunately, this item was found to perform poorly; removal of this item improved the psychometric properties of the measure. From pilot test participants' feedback, we believe the main reason for this item's poor performance was the phrasing of this item, which lead to variation in subjects' interpretation of the word “predictable”. Although we have removed this item from the measure at the present time, we encourage clinicians assessing pain in individuals with OA to ask about the predictability of the pain, as well as the strategies they employ, if any, to deal with this unpredictability when present. As we continue the testing of the measure we will test alternative wording for this predictability of intermittent pain concept that people with OA have told us is an extremely bothersome feature of their symptoms.
In conceptualizing this pain measure based on the focus group data, we anticipated that we had an overall construct of pain that was characterized by constant and intermittent pain but it was unclear to us whether these characteristics of the pain would result in a multi-dimensional measure or a single construct. The Cronbach's alpha values for the constant and intermittent pain as well as for the combined 11 items were all high suggesting sufficient homogeneity to create a single summative scale. The complexity of the factor analysis solutions was such that it was not clear that subscale scores, rather than a single score should be created. As such, we recommend that a total summative score based on the 11 items be created and that this be transformed to a 0–100 score where 0 represents no pain. However, depending on the specific questions being asked, investigators or clinicians may wish to also evaluate the constant and intermittent pain separately based on scoring as for the total pain measure. Additionally, two items, those referring to the impact of constant and intermitting pain affecting sleep, loaded on a third factor as well as on the first factor. This finding may indicate that responses to these two items reflect underlying sleep disorders in addition to the direct impact of pain on sleep (e.g., awakening due to pain). Sleep disturbances are well recognized as common among older people with OA13–15. In a large survey of older individuals, complaints of insomnia or un-refreshing sleep were twice as frequent among people with arthritis compared to controls and that pain contributed significantly to these sleep complaints16. Prior work by our group has also documented positive correlations between arthritis pain, fatigue and sleepiness17. For these reasons, as further testing of the measure is conducted we encourage ongoing testing of the dimensionality of the measure.
In general, the new measure was significantly correlated, and in the directions expected, with the WOMAC pain and the KOOS symptom subscales. As was expected, the correlation between the WOMAC pain scale score and the new pain measure was moderate, with a Spearman correlation coefficient of 0.81, indicating that the two measures are evaluating different constructs. This level of correlation is not surprising in light of the fact that the 5-item WOMAC pain scale largely evaluates pain associated with specific activities (correlations between the WOMAC pain and physical function subscales have been consistently shown to be very high, on the order of 0.8–0.918,19) while our new measure evaluates aspects of the pain experience alone, independent of function. A modest correlation between the new pain measure and the KOOS symptom subscale was observed. This lower correlation is most likely attributable to the differences in constructs between the two scales. Our new pain measure assesses pain intensity, related distress and impact of pain on quality of life while the KOOS symptoms subscale captures physical symptoms other than pain associated with the painful joint.
Surprisingly, the correlation between the LLFDI and the new measure was low. There are several potential explanations for this finding. First, our pilot test sample was generally comprised of older individuals, who may have given up activities as a result of their age, as reflected by low LLFDI scores, rather than because of their painful OA. Second, we know from the literature that impairments (like pain measures) correlate with activity in the range of 0.40–0.6020,21. However, relatively little is known about how activity limitations and participation are related, but based on research in other clinical areas, it seems likely that they are probably similarly related, with correlations of about 0.40 as well22. If impairments, activities and participation are linear in relationship, the partial correlation of pain to participation would be 0.25 if pain to activity is 0.5 and activity to participation is 0.5. Greater understanding of the relationships between pain, activity and participation is needed before we can determine if our hypothesis was incorrect.
Although preliminary testing supports the reliability and validity of our new measure, there are some limitations that must be considered, in addition to those already addressed. First, the measure was developed based on the experiences of English-speaking people living with painful OA. Since the development and validation of the pain tool reported here were completed, the ICOAP has undergone cross-cultural translation into multiple languages and further validation in other hip and knee OA samples. The outcome of this additional testing was very positive and reinforced our confidence in the ability of this new measure to capture the aspects of pain that are most relevant to individuals with OA. Based on this further work, minor modifications were made to improve the succinctness of questionnaire section framing. Second, pilot testing was performed mainly in individuals with moderately severe hip or knee OA. Evaluation of the measurement properties of the new tool in individuals with early OA is therefore needed. Finally, this first study did not assess the ability of the new tool to evaluate meaningful changes in pain due to hip or knee OA over time or in response to treatment. While additional studies are needed to evaluate the ability of the ICOAP to measure changes over time and to determine clinical cutpoints, we believe that this tool is generalizable, suitable for use in research and clinical settings, and addresses aspects of truth, discrimination and feasibility as defined by the OMERACT filter23.
In summary, the new 11-item ICOAP comprehensively evaluates the experience of pain among people living with hip and knee OA. Consistent with what we learned from extensive focus groups conducted in people with painful hip/knee OA, the new measure evaluates, both constant and intermittent pain and takes into account not only pain intensity but also related distress and the impact of OA pain on quality of life. Preliminary psychometric testing suggests the measure is reliable and valid. Further studies are required to confirm these preliminary findings, and to evaluate the responsiveness of this measure over time.
The authors would like to thank the following collaborators and staff that made this work possible: Rachael Gooberman-Hill, Annica Barcenilla, Sowmyar Gandham, Marita Cross, Lindsay Stewart, Kerri Moore, Shalini Misir, Sarah Birmingham, Aodhan Wall, Pilar Furio-Hong. We would like to acknowledge the following members of the OMERACT 7 Total Articular Replacement Special Interest Group: Maxime Dougados, Jean-Francis Maillefert, Laure Gossec, Roy D Altman, Karsten Dreinhoefer, Klaus-Peter Geunther, Karel Pavelka, Marc C Hochberg, L Stephan Lomander, Nizar Mahomed, Ewa M Roos, Thasia G Woodworth and Gustavo Zanoli. We would like to thank OMERACT and OARSI for their support for this research through facilitation of meetings of the project team and partial unrestricted funding of the work, and unrestricted support from Pfizer, Novartis, Astra Zeneca, Pharmasciences, Rottapharm and Negma.
Conflict of interest Dr Cibere received a speaker honorarium from Amgen and research funding from Centocor Research and Development, Inc. Dr Katz has received research funding from Novartis. The other authors have not conflict of interest to report.