Search tips
Search criteria 


Logo of springeropenLink to Publisher's site
Quality of Life Research
Qual Life Res. 2010 September; 19(7): 985–994.
Published online 2010 May 8. doi:  10.1007/s11136-010-9665-x
PMCID: PMC2923719

The relationship between change in subjective outcome and change in disease: a potential paradox



Response shift theory suggests that improvements in health lead patients to change their internal standards and re-assess former health states as worse than initially rated when using retrospective ratings via the then-test. The predictions of response shift theory can be illustrated using prospect theory, whereby a change in current health causes a change in reference frame. Therefore, if health deteriorates, the former health state will receive a better rating, whereas if it improves, the former health state will receive a worse rating.


To explore the predictions of response shift and prospect theory by relating subjective change to objective change.


Baseline and 3-month follow-up data from a cohort of rheumatoid arthritis patients (N = 197) starting on TNFα-blocking agents were used. Objective disease change was classified according to a disease-specific clinical outcome measure (DAS28). Visual analogue scales (VAS) for general health (GH) and pain were used as self-reported measures. Three months after starting on anti-TNFα, patients used the then-test to re-rate their baseline health with regard to general health and pain. Differences between then-test value and baseline values were calculated and tested between improved, non-improved and deteriorated patients by the Student t-test.


At 3 months, 51 (25.9%) patients had good improvement in health, 83 (42.1%) had moderate improvement, and 63 (32.0%) had no improvement or deteriorated in health. All patients no matter whether they improved, did not improve, or even became worse rated their health as worse retrospectively. The difference between the then-test rating and the baseline value was similarly sized in all groups.


More positive ratings of retrospective health are independent of disease change. This suggests that patients do not necessarily change their standards in line with their disease change, and therefore it is inappropriate to use the then-test to correct for such a change. If a then-test is used to correct for shifts in internal standards, it might lead to the paradoxical result that patients who do not improve or even deteriorate increase significantly on self-reported health and pain. An alternative explanation for differences in retrospective and prospective ratings of health is the implicit theory of change which is more successful in explaining our results than prospect theory.

Keywords: Response shift, Retrospective pre-test, Then-test, Rheumatoid arthritis, Implicit theory of change


When a physician sees a patient, the most frequently asked question will probably be “How do you feel today concerning your general health (or pain)?”. This question can be formally measured using a Visual Analogue Scale (VAS) for general health (GH) or pain, which is considered a feasible, valid and reliable method [13]. In the field of rheumatology, indices based on patient reported outcomes, among which the VAS GH and VAS pain, are suggested to use for systematically monitoring patients on therapy effectiveness [4]. In general, to compare assessments over time and between individuals, it is a prerequisite of a measurement that the patient’s concept of the subject of assessment is the same in all situations. However, there are many examples of patients reporting their health state as stable over time, despite improvements in their objective health status [512], which challenge the use of self-reporting outcomes. It has been shown in a variety of disorders that patients whose health improved re-assessed their former health state as worse than initially rated [513]. Furthermore, in cancer patients the relative importance of different domains of quality of life varied over time [14]. Finally, there are many studies that demonstrate that patients value a certain health state differently than individuals not in that particular health state [7, 1518], for example that healthy people perceive the general health of patients suffering from end-stage renal disease as worse than the patients themselves [7].

In response shift theory, these findings are interpreted [19] as follows: patients do not rate their own health in reference to an absolute standard but in reference to a relative standard. In health-related subjective measures, this relative standard is tentatively taken as the patient’s own current health state. However, current health, and therefore an individual’s standard, may change over time [1921]. For example, when a patient is asked whether or not he or she has improved, the fact that the patient refers to his/her current health can lead to different valuations of the same objective health state at different moments in time. Some authors suggest that self-reported outcomes should be corrected for changes in relative standards [22, 23] by, for instance, a then-test [22]. By using a then-test, patients are asked to re-rate a former health state from their current perspective. Because the former health state and the follow-up health state are then rated from the same perspective, the change in health (then-minus-post) is corrected for a possible change in internal standards.

Support for the then-test as a correction for changes in relative standards can be found in prospect theory [24, 25]. Prospect theory was originally developed in the context of gambling for money and is a theory about decisions under risk [26]. Prospect theory defines money outcomes as gains and losses relative to a reference point. Different reference frames are assumed to shift the reference point along the outcome dimension, altering the location of an S-shaped value function [25, 27]. In valuing health, the reference point of the S-shaped value functions can be described as the individual’s current health, as described by Treadwell and Lenert [25]. Figure 1 shows two such S-shaped value functions for two different health states in time, A and B. The x-axis represents a person’s health state, and the y-axis represents subjective values attributed to health states. When a patient improves in health (moves from A to B), his value function shifts to B. According to the shifted S-shaped value function, the former state is rated worse (A’) than previously (A). When a patient deteriorates in health (moves from B to A), his value function shifts to the left. According to that function, the former health state will receive a better rating (B’) than previously (B). When a health state remains stable, no shift in the value function is expected. Prospect theory therefore provides a clear illustration of the predictions of response shift theory with respect to recalibration response shifts. Hence, this study explores the predictions by prospect theory to explain response shift.

Fig. 1
Two S-shaped value functions for two different health states in time. Modified figure from the article by Treadwell and Lenert [25]. Response shift: the difference between A’-A or B’ and B. Prospective change: difference between A and ...

However, it should be noted that alternative predictions of outcomes of the then-test can be found in, for example, Norman’s discussion of implicit theory of change [28]. This theory originates in psychological research [29] and states that individuals might be unable to accurately remember a former situation but instead extrapolate backwards from their present state invoking an implicit theory of change [28, 29]. A treatment received could trigger an implicit theory of positive change. Working back from their present health status, such an implicit positive change would lead patients to re-rate their baseline health status (then-test) as worse than their initial baseline rating independent of the actual effectiveness of the treatment.

That patients re-rate (by means of a then-test) their former health state as worse than previously after an improvement in health has been shown several times in the medical field [513]. However, there is very little evidence in the literature for predictions in patients who deteriorate in health compared to patients who improve in health. Response shift and prospect theory both predict that shifts in internal standards will be bi-directional: retrospective ratings of a former health state will be worse in patients who improve (health state A’ worse than previously (A)) and better in patients who deteriorate (health state B’ rated better than previously (B)). However, implicit theory of change would predict that both patients who objectively improve or deteriorate will rate their former health state as worse than previously rated if patients perceive there should have been an improvement.

The aim of this study was to explore the predictions from response shift and prospect theory by relating subjective change to objective change. This was done by comparing retrospective scores in patients with severe and prolonged rheumatoid arthritis who either deteriorated or improved after treatment with TNF-blocking agents.


Study site and subjects

Our hypothesis was tested in a population of chronically ill rheumatoid arthritis patients receiving anti-TNFα treatment. This patient population was chosen, because of the chronic character of the disease, which ensures the possibility of adaptation by patients to their imperfect health state, and thereby allows the reference level to shift. Anti-TNFα is a promising treatment to establish a significant decrease in disease activity [3032]. In addition, this population was chosen because of the possibility of categorizing patients by disease activity based on a validated objective measure of disease activity [33].

Data from a prospective registry were used. Since February 2003, all rheumatoid arthritis patients, from 11 Dutch hospitals, who started on an anti-TNFα agent (either adalimumab, etanercept or infliximab) for the first time, were included in this registry [34]. The primary aim of this registry is to evaluate and monitor anti-TNFα treatment in rheumatoid arthritis patients. Since January 2004, then-tests have been included in this registry.

Study design and outcome measures

Patients’ baseline characteristics were registered at the start of the first anti-TNFα treatment. These characteristics included age, gender, weight, disease duration, rheumatic factor status, the presence of one or more erosions in hand or feet and number of previous DMARDs (disease modifying anti-rheumatic drugs) used. In addition to these characteristics, the three main measures for this study, disease activity, the patient’s self-perceived general health and self-perceived pain, were registered at baseline and at a 3-month follow-up assessment. The 3-month follow-up assessment was chosen because at that point a clinical decision about the medication policy is made on the basis of the underlying disease state alteration (with the DAS28).

Disease activity was assessed using a modified Disease Activity Score 28 (DAS28), which is a statistically derived index combining the following variables: the 28 joint count for swelling (SW28), the 28 joint count for tenderness (TEN28) and the Erythrocyte Sedimentation Rate (ESR), with the ESR having the largest weight in the algorithm [35]. In order to minimize subjective interpretation of disease activity in our study, the VAS general health was left out of the original DAS28 algorithm. As a valid alternative to ESR, C-reactive Protein (CRP) scores were used in case of missing ESR values [36]. The research nurses were trained in giving a standardized amount of pressure (defined as causing whitening of the examiners nail beds) on the joints to measure tenderness and in using a standardized grip for assessing swelling [36]. For the purpose of describing the population, we also administered the disability index from the Health Assessment Questionnaire (HAQ-DI). The HAQ-DI is a self-assessed questionnaire asking about the ability of patients to perform several daily activities over the past week [37, 38]. The HAQ-DI is well validated and provides information on disease activity as well as joint damage [39, 40]. The questionnaire provides a score between zero and three, where a higher score indicates more functional impairment.

The patients’ self-perceived general health and pain were assessed on a 100-mm Visual Analogue Scale (VAS GH and VAS pain), which are feasible, valid and reliable methods to measure these constructs [13]. Patients were asked to rate their health or pain by placing a vertical line on a horizontal line, indicating the perceived amount of health or pain. This line ranged from zero to one hundred, where a score of zero indicates “worst imaginable general health” or “extreme pain”, and a score of one hundred indicates “best imaginable general health” or “no pain”, respectively. Patients were asked to rate their current general health at baseline and at 3 month follow-up. At 3 months, the patients were also asked to retrospectively re-rate the baseline general health and pain by means of a then-test [22, 41]. Patients were specifically instructed not to try and remember what they scored at baseline (recall), but to re-assess their health and pain at baseline from their current perspective. We chose to use the then-test for assessing response shift, because it is easy to use and it is most frequently used by others in longitudinal studies [42]. The actual DAS28 and the response category were calculated in the dataset afterwards and were, therefore, not known to the patient at the moment that he or she completed the scales and the then-test.

Statistical analysis

Descriptives of baseline characteristics were determined. Baseline values were subtracted from then-test score. A negative difference indicated that retrospective scores are worse than baseline scores (Fig. 1: A’-A), and a positive difference indicated that retrospective scores are better than baseline score (Fig. 1: B’-B). Changes over time in VAS scores were calculated as either prospective change or then-minus-post change. The prospective change was calculated by subtracting baseline scores from 3-month follow-up scores (Fig. 1: difference between A and B). A positive prospective change was interpreted as an improvement in health and a negative prospective change as deterioration in health. Post-minus-then change was calculated by subtracting then-test scores from the 3-month follow-up scores (Fig. 1: A-B’ or B-A’). One can consider the post-minus-then change as a prospective change corrected for a shift in internal standards (then-test score at 3 months referring to baseline). A positive post-minus-then change suggests a perceived improvement and a negative post-minus-then change suggests a perceived deterioration.

To test the hypothesis that patients who deteriorated retrospectively rate a former health state as better and patients who improved retrospectively rate a former have state as worse, patients were divided into three groups (non-responders, moderate responders and good responders to treatment) depending on their objectively determined European League Against Rheumatism (EULAR) response status at the 3-month follow-up moment. The EULAR criteria were used because these have shown good construct, criterion and discriminant validity [43]. The EULAR criteria are based on the modified DAS28 and combine prospective change and absolute level of attained disease activity (Table 1). By this definition, the non-response group is a mixture of patients who do not improve significantly and patients who deteriorate. Additionally, patients who deteriorated significantly, defined as at least one population standard deviation (0.6 DAS28) deterioration (negative prospective change), were analysed as a subgroup.

Table 1
EULAR response criteria (good, moderate, non-response)

Baseline characteristics were tested for equivalency between the groups by means of Chi-square tests, independent samples t-tests and Mann–Whitney U tests. The mean difference between then-test scores and baseline scores and the mean changes in VAS general health and VAS pain were tested for statistical significance using one-sample t-tests and were compared between groups using independent samples t-tests. All analyses were performed using the statistical software package SPSS 14.0.2 (SPSS Inc., Chicago, IL.)


Between January 2004 and December 2006, 212 patients had completed 3-month follow-up data. Fifteen patients were excluded from the analysis due to missing DAS28 and/or VAS data. Baseline characteristics of excluded patients did not differ from included patients. Thus, 197 patients remained eligible for the analyses: 51 (25.9%) patients were classified as good responders to the anti-TNFα therapy, 83 (42.1%) as moderate responders, and 63 patients (32.0%) were classified as non-responders to therapy. In this last group, 11 patients deteriorated significantly and were analysed as a subgroup. On the DAS28, responders had a mean improvement of 2.2 points (ranging from 1.2 to 4.5), moderate responders had a mean improvement of 1.5 points (ranging from 0.6 to 4.2), and non-responders had no improvement (mean −0.05, ranging from −3.0 to 1.1). Table 2 shows the baseline characteristics of clinically responding and non-responding patients. These data show that clinically classified responders did not differ from non-responders at baseline, except on the clinical outcome measures HAQ and DAS28. This was to be expected, because the EULAR response criteria combine change and an absolute level of attained DAS28 and, thus, baseline DAS28. The HAQ is strongly correlated with the DAS28.

Table 2
Baseline characteristics of patients grouped by response classification

Prospective change

Table 3 and the Figs. 2 and and33 show the changes in VAS scores on both general health (Fig. 2, solid line) and pain (Fig. 3, solid line). On average, clinical objective responders (good and moderate) to therapy scored an average of 31.0 points improvement on the VAS general health, which was a statistically significant improvement and a significantly bigger improvement than the average 9.3 points improvement that was reported by non-responders. On the VAS pain, good and moderate responders scored a significant mean improvement of 31.8 points and 30.9 points, respectively, whereas non-responders reported an insignificant improvement of 6.0 points. Patients who deteriorated according to their modified DAS28 scores deteriorated also according to their VAS GH (mean 12.3 points) and pain scores (mean 17.7 points).

Table 3
The prospective change (3 months minus baseline) and the response shift (then-test minus baseline value) for the VAS general health (GH) and pain for responders and non-responders
Fig. 2
Changes in VAS general health scores and then-test values of good responders (a), moderate responders (b), non-responders (c) and deteriorators (d). Note that a score of zero indicates a worst imaginable general health state, and a score of one hundred ...
Fig. 3
Changes in VAS Pain scores and then-test values of good responders (a), moderate responders (b), non-responders (c) and deteriorators (d). Note that a score of zero indicates extreme pain, and a score of one hundred indicates no pain

Retrospective scores

Objectively classified responders and non-responders showed significant worse retrospective scores compared to baseline values on both scales (Figs. 2 and and3,3, dotted lines). The identified difference between the retrospective scores and the baseline scores were of equal size for responders and non-responders (Table 3). The 11 deteriorated patients showed a significant difference between retrospective score and baseline values on general health of −14.5 (95% CI −28.3; −0.8) and on pain of −19.1 (95% CI −36.9; −1.3).

Then-minus-post scores

The calculated mean then-minus-post scores are also shown in Figs. 2 and and3.3. When applying the then-minus-post values, clinical good and moderate responders to therapy averaged a 40.7-point and 34.2-point improvement on VAS general health, respectively. This was statistically significant (P < 0.0001) and significantly more than the average 16.1-point improvement of the non-responders (P < 0.0001). Outcomes of the VAS Pain showed similar results. Good and moderate responders averaged a 42.6-point and 35.2-point improvement, respectively, compared to a significantly lower (P < 0.0001) improvement of 16.9-points for non-responders. The deteriorated patients showed stable disease activity when applying the then-minus-post score for the VAS GH (mean 2.3 points) and for the VAS pain (mean 1.4 points).

Discussion and conclusion

This study showed that both patients who improved (responders) or stayed the same/deteriorated (non-responders) rated their baseline health state worse than actually rated at baseline, when asked 3 months later with a then-test. Furthermore, paradoxical results occurred when the then-test was applied (then-minus-post) for the purpose of correcting for a shift in internal standards. For clinically stable or deteriorating patients, an improvement in health would be inferred when using then-minus-post ratings when compared to using only prospective ratings (Figs. 2 and and3).3). The fact that patients who stayed the same or deteriorated also reported worse health retrospectively conflicted with the predictions from response shift theory and prospect theory.

The negative response shifts in patients who improved were in line with the predictions derived from prospect theory and complied with the large amount of literature reporting on negative response shifts [512]. However, the negative response shifts in patients who stayed the same or deteriorated, conflicted with the prediction from prospect theory. Only two previous studies assessed patients who improved, who did not improve or who deteriorated in health. Results from one study were in line with prospect theory [44], while findings from the second study were not [5]. Ahmed and colleagues [5] also used objective criteria to determine the direction of disease change. In agreement with our study, they found mean response shifts in the same direction in 196 participants. These response shifts were independent of the direction of the disease change that had occurred. Janssen et al. [44] investigated response shift in 46 patients where the disease change was defined with a subjective change question. In contrast to Ahmed’s and our findings, their response shifts were dependent on the direction of the disease changes. The explanation for these conflicting findings may be the crucial difference in the way that disease change was defined: objective (Ahmed’s and our study) versus subjective (Janssen’s study).

The aforementioned implicit theory of change seems to explain our results more accurately than prospect theory and also the conflicting results in the literature. In our study and the study by Ahmed et al. [5], non-responders showed no objective disease change, but the treatment received could have triggered an implicit positive change. Working back from their health status, this implicit positive change would lead patients to re-rate their baseline health status (then-test) as worse than their initial baseline rating. In the study by Jansen et al. [44], patients self-reported a subjective deterioration in disease. Working back from their health status, they could have applied this implicit negative change to re-rate the baseline health status with the then-test as better than their initial baseline rating. In summary, an implicit positive change of patients in our and Ahmed’s study and the implicit negative change of patients in Janssen’s study can explain the conflicting retrospective scores more accurately than response shift or prospect theory. It has to be mentioned that an implicit positive change as described above could also have resulted from what is known as a placebo response, but either way change results in patients re-rating their baseline health status (then-test) as worse than their initial baseline rating.

The prediction of response shift theory, with respect to the fact that the internal standards by which patients rate their own health change over time, has clinical implications. This is especially true when subjective measures are used to monitor therapy effectiveness as was recently suggested in the field of rheumatology [4]. It can be suggested that physicians can try to correct for a shift in internal standards by asking patients to rerate their health state before they started therapy and rate their current health. By comparing those two ratings, therapy effectiveness can be evaluated. However, our results suggest that rather than changing internal standards, implicit theory of change is applied by patients to construct the value of a former health state. Therefore, if retrospective ratings are used, this will lead to the paradoxical result that patients who deteriorate or stay the same increase significantly on self-reported health and pain. For a clinician, this would make it more difficult to justify treatment change where from a more objective point of view or from the perspective of prospective self ratings, it would have been advisable.

Lenert et al. [45] did empirical research in health care to test predictions based on prospect theory. Patients from primary care practices with various medical illnesses and depression rated different hypothetical health states. They showed data that were consistent with the predictions from prospect theory: utility functions for health were “S” shaped and differed across levels of health [45]. The differences between their study design (cross-sectional) and our study design (longitudinal) may explain the differences in results. Their subjects rated hypothetical health states that were worse or better than their own health states, and they did this during one session. In our study, the patients had an actual change in their own health to either a worse or a better health state, and then rerated their baseline health with a then-test.

Another explanation for the fact that we did not confirm prospect theory may lie in the fact that the then-test may not be the appropriate instrument to evaluate a former health state and subsequently to measure response shift. Although the then-test is most frequently used as an instrument to measure response shifts [42], it has the limitation that it can be subject to recall bias [23], and it is dependent on the cognitive function of the subjects. Furthermore, it has to be mentioned that the results of this study apply to visual analogue scales, and the results cannot be generalized to other quality of life measures like the Short Form 36 (SF36) or EuroQol 5d (EQ5D) without further research. It might be possible that prospect theory fits the predicted outcomes measured by the EQ-5D or SF-36. On the other hand, response shifts as a result of changes in internal standards may not occur in measures like the SF36 or EQ5D, but response shift as a result of changes in values or re-conceptualization may occur [19]. Currently, there is no evidence that prospect theory predicts response shift as a result of changing values or conceptualizations, and it was not the focus of our study to deliver such evidence.

Some shortcomings concerning this study have to be mentioned. First, by using the EULAR criteria to define disease change (Table 1), the non-response group was a mixture of patients who did not improve significantly and patients who deteriorated. A subgroup of 11 patients who deteriorated on the objective measure DAS28 was analysed. These 11 deteriorated patients showed, consistently with the other groups of patients, a significant negative response shift on general health and pain, rating their baseline health state worse than initially rated. Such a subgroup analysis might be underpowered. Furthermore, the DAS28 is used as an objective measure, whilst it can be subject to interpretation especially in the components ‘joint pain’ and ‘joint swelling’. To minimize subjective interpretation in these components, all research nurses were trained twice per year for performing joint counts. Moreover, a modified DAS28 was calculated using the most objective components, the ESR and the joint counts, of which the ESR has the larger weight in the combined score. Another limitation is that the patients’ memory or perceptions of change were not assessed, and therefore the implicit perception of change could not be assessed.

In conclusion, the similar direction and magnitude of retrospective ratings in objectively defined improved and non-improved patients suggests that patients do not necessarily change their standards in line with their disease change. If a then-test is used to correct for shifts in internal standards, it might lead to the paradoxical result that patients who do not improve or even deteriorate increase significantly on self-reported health and pain making it more difficult for the clinician to justify treatment change. An alternative explanation for differences in retrospective and prospective ratings of health is the implicit theory of change which is more successful in explaining our results than prospect theory.


We are indebted to all research nurses and rheumatologist of the 11 departments of rheumatology for their participation and contribution in the data collection of the DREAM study. UMC ST Radboud, Nijmegen: J. Alberts, P. Barrera Rico, K. Bevers, A. Branten, M. Creemers, J. Deenen, A. van Ede, T. van Gaalen, E. de Groot, H. van Heereveld, F. van den Hoogen, R. Laan, P.van Riel, L. Schalkwijk, M. Veerkamp, C. Versteegden, C. Vogel, M. Vonk; Sint Maartens Kliniek, Nijmegen: H. Cats, A. Eijsbouts, M. Franssen, I. Geerdink, S. Herfkens, F van den Hoogen, M. Jeurissen, P. Koelmans, P. van ‘t Pad Bosch, D. de Rooij, A. Stenger, H. van Wijk; Rijnstate Ziekenhuis Arnhem: T. Berends, C. Bijkerk, C. De Gendt, J.Harbers, M. Janssen, A de Jong, H. Knaapen, H. Visser; Medisch Spectrum Twente, Enschede: A. ter Avest, K. Drossaers, M. Hoekstra, M. Kruijssen, I. Kuper, M. van de Laar, A. Mooij, H. Vonkeman; Twenteborg Ziekenhuis Almelo: H. Bernelot Moens, E. Bos, K. Drossaers, C. Haagsma, K. van de Hoeven, J. Oostveen; Streekziekenhuis Midden Twente, Hengelo: H. Bernelot Moens, M. Kleine Schaar; Medisch Centrum Leeuwarden, Leeuwarden: G. A. W. Bruyn, E. N. Griep, M. Horbeek, P. M. Houtman, T. L. Jansen, K. Rasing, A. Spoorenberg, A. Krol; Twee Steden Ziekenhuis, Tilburg: R. van Berkel, H. Brus, E. van Gorp, W. Hissink Muller, A. van Roy, M. Wijnands; Streekziekenhuis Zevenaar, Zevenaar: C. De Gendt, H. Visser, P. Brakenhoff. Furthermore, we would like to acknowledge Thea van Gaalen, Lia Schalkwijk and Carien Versteegden for the data processing and Hans Groenewoud for the assistance in managing the database. There was no financial funding for this study, and consequently complete independence in designing the study, interpreting the data, writing, and publishing the report.

Open Access This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.

List of abbreviations

C-reactive protein
Disease activity score for 28 joint counts
Disease modifying anti-rheumatic drug
Erythrocyte sedimentation rate
European league against rheumatism
General health
Health assessment questionnaire
28 joint count for swelling
28 joint count for tenderness
Tumour necrosis factor alpha
Visual analogue scale


1. Badia X, Monserrat S, Roset M, Herdman M. Feasibility, validity and test-retest reliability of scaling methods for health states: The visual analogue scale and the time trade-off. Quality of Life Research. 1999;8:303–310. doi: 10.1023/A:1008952423122. [PubMed] [Cross Ref]
2. Dworkin RH, Turk DC, Revicki DA, Harding G, Coyne KS, Peirce-Sandner S, Bhagwat D, Everton D, Burke LB, Cowan P, Farrar JT, Hertz S, Max MB, Rappaport BA, Melzack R. Development and initial validation of an expanded and revised version of the Short-form McGill Pain Questionnaire (SF-MPQ-2) Pain. 2009;144:35–42. doi: 10.1016/j.pain.2009.02.007. [PubMed] [Cross Ref]
3. Green C, Brazier J, Deverill M. Valuing health-related quality of life. A review of health state valuation techniques. Pharmacoeconomics. 2000;17:151–165. doi: 10.2165/00019053-200017020-00004. [PubMed] [Cross Ref]
4. Pincus T, Yazici Y, Bergman M, Maclean R, Harrington T. A proposed continuous quality improvement approach to assessment and management of patients with rheumatoid arthritis without formal joint counts, based on quantitative routine assessment of patient index data (RAPID) scores on a multidimensional health assessment questionnaire (MDHAQ) Best Practice & Research. Clinical Rheumatology. 2007;21:789–804. doi: 10.1016/j.berh.2007.02.009. [PubMed] [Cross Ref]
5. Ahmed S, Mayo NE, Wood-Dauphinee S, Hanley JA, Cohen SR. Response shift influenced estimates of change in health-related quality of life poststroke. Journal of Clinical Epidemiology. 2004;57:561–570. doi: 10.1016/j.jclinepi.2003.11.003. [PubMed] [Cross Ref]
6. Nieuwkerk PT, Tollenaar MS, Oort FJ, Sprangers MA. Are retrospective measures of change in quality of life more valid than prospective measures? Medical Care. 2007;45:199–205. doi: 10.1097/01.mlr.0000246613.49214.46. [PubMed] [Cross Ref]
7. Postulart D, Adang EM. Response shift and adaptation in chronically ill patients. Medical Decision Making. 2000;20:186–193. doi: 10.1177/0272989X0002000204. [PubMed] [Cross Ref]
8. Razmjou H, Yee A, Ford M, Finkelstein JA. Response shift in outcome assessment in patients undergoing total knee arthroplasty. Journal of Bone and Joint Surgery. American Volume. 2006;88:2590–2595. doi: 10.2106/JBJS.F.00283. [PubMed] [Cross Ref]
9. Rees J, Clarke MG, Waldron D, O’Boyle C, Ewings P, MacDonagh RP. The measurement of response shift in patients with advanced prostate cancer and their partners. Health Quality Life Outcomes. 2005;3:21. doi: 10.1186/1477-7525-3-21. [PMC free article] [PubMed] [Cross Ref]
10. Rees J, Waldron D, O’Boyle C, Ewings P, MacDonagh R. Prospective vs retrospective assessment of lower urinary tract symptoms in patients with advanced prostate cancer: The effect of ‘response shift’ BJU International. 2003;92:703–706. doi: 10.1046/j.1464-410X.2003.04462.x. [PubMed] [Cross Ref]
11. Ring L, Hofer S, Heuston F, Harris D, O’Boyle CA. Response shift masks the treatment impact on patient reported outcomes (PROs): The example of individual quality of life in edentulous patients. Health Quality Life Outcomes. 2005;3:55. doi: 10.1186/1477-7525-3-55. [PMC free article] [PubMed] [Cross Ref]
12. Smith DM, Sherriff RL, Damschroder L, Loewenstein G, Ubel PA. Misremembering colostomies? Former patients give lower utility ratings than do current patients. Health Psychology. 2006;25:688–695. doi: 10.1037/0278-6133.25.6.688. [PubMed] [Cross Ref]
13. ten Klooster PM, Drossaers-Bakker KW, Taal E, Laar MAFJ. Patient-perceived satisfactory improvement (PPSI): Interpreting meaningful change in pain from the patient’s perspective. Pain. 2006;121:151–157. doi: 10.1016/j.pain.2005.12.021. [PubMed] [Cross Ref]
14. Visser MR, Oort FJ, Sprangers MA. Methods to detect response shift in quality of life data: a convergent validity study. Quality of Life Research. 2005;14:629–639. doi: 10.1007/s11136-004-2577-x. [PubMed] [Cross Ref]
15. Sackett DL, Torrance GW. The utility of different health states as perceived by the general public. Journal of Chronic Diseases. 1978;31:697–704. doi: 10.1016/0021-9681(78)90072-3. [PubMed] [Cross Ref]
16. Boyd NF, Sutherland HJ, Heasman KZ, Tritchler DL, Cummings BJ. Whose utilities for decision analysis? Medical Decision Making. 1990;10:58–67. doi: 10.1177/0272989X9001000109. [PubMed] [Cross Ref]
17. Dolan P. The effect of experience of illness on health state valuations. Journal of Clinical Epidemiology. 1996;49:551–564. doi: 10.1016/0895-4356(95)00532-3. [PubMed] [Cross Ref]
18. Wit GA, Busschbach JJV, Charro FT. Sensitivity and perspective in the valuation of health status: Whose values count? Health Economics. 2000;9:109–126. doi: 10.1002/(SICI)1099-1050(200003)9:2<109::AID-HEC503>3.0.CO;2-L. [PubMed] [Cross Ref]
19. Sprangers MA, Schwartz CE. Integrating response shift into health-related quality of life research: A theoretical model. Social Science and Medicine. 1999;48:1507–1515. doi: 10.1016/S0277-9536(99)00045-3. [PubMed] [Cross Ref]
20. Allison PJ, Locker D, Feine JS. Quality of life: A dynamic construct. Social Science and Medicine. 1997;45:221–230. doi: 10.1016/S0277-9536(96)00339-5. [PubMed] [Cross Ref]
21. Adang EM. A comment on “response shift and adaptation in chronically ill patients” by Postulart and Adang. Medical Decision Making. 2001;21:423–424. [PubMed]
22. Schwartz CE, Sprangers MA. Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Social Science and Medicine. 1999;48:1531–1548. doi: 10.1016/S0277-9536(99)00047-7. [PubMed] [Cross Ref]
23. Schwartz CE, Sprangers MAG, Carey A, Reed G. Exploring response shift in longitudinal data. Psychology and Health. 2004;19:51–69. doi: 10.1080/0887044031000118456. [Cross Ref]
24. Tversky A, Kahneman D. The framing of decisions and the psychology of choice. Science. 1981;211:453–458. doi: 10.1126/science.7455683. [PubMed] [Cross Ref]
25. Treadwell JR, Lenert LA. Health values and prospect theory. Medical Decision Making. 1999;19:344–352. doi: 10.1177/0272989X9901900313. [PubMed] [Cross Ref]
26. Kahneman D, Tversky A. Prospect theory—analysis of decision under risk. Econometrica. 1979;47:263–291. doi: 10.2307/1914185. [Cross Ref]
27. Stalmeier PFM, Bezembinder TGG. The discrepancy between risky and riskless utilities: A matter of framing? Medical Decision Making. 1999;19:435–447. doi: 10.1177/0272989X9901900412. [PubMed] [Cross Ref]
28. Norman G. Hi! How are you? Response shift, implicit theories and differing epistemologies. Quality of Life Research. 2003;12:239–249. doi: 10.1023/A:1023211129926. [PubMed] [Cross Ref]
29. Ross M. Relation of implicit theories to the construction of personal histories. Psychological Review. 1989;96:341–357. doi: 10.1037/0033-295X.96.2.341. [Cross Ref]
30. Blumenauer, B., Judd, M., Cranney, A., Burls, A., Coyle, D., Hochberg, M., Tugwell, P., & Wells, G. (2003). Etanercept for the treatment of rheumatoid arthritis. Cochrane Database System Review, CD004525. [PubMed]
31. Navarro-Sarabia F, riza-Ariza R, Hernandez-Cruz B, Villanueva I. Adalimumab for treating rheumatoid arthritis. Journal of Rheumatology. 2006;33:1075–1081. [PubMed]
32. Blumenauer, B., Judd, M., Wells, G., Burls, A., Cranney, A., Hochberg, M., & Tugwell, P. (2002) Infliximab for the treatment of rheumatoid arthritis. Cochrane Database System Review, CD003785. [PubMed]
33. Gestel AM, Anderson JJ, Riel PL, Boers M, Haagsma CJ, Rich B, Wells G, Lange ML, Felson DT. ACR and EULAR improvement criteria have comparable validity in rheumatoid arthritis trials. American College of Rheumatology European League of Associations for Rheumatology. The Journal of Rheumatology. 1999;26:705–711. [PubMed]
34. Kievit W, Fransen J, Oerlemans AJ, Kuper HH, Laar MA, Rooij DJ, Gendt CM, Ronday KH, Jansen TL, Oijen PC, Brus HL, Adang EM, Riel PL. The efficacy of anti-TNF in rheumatoid arthritis, a comparison between randomised controlled trials and clinical practice. Annals of the Rheumatic Diseases. 2007;66:1473–1478. doi: 10.1136/ard.2007.072447. [PMC free article] [PubMed] [Cross Ref]
35. Prevoo MLL, VantHof MA, Kuper HH, Vanleeuwen MA, VandePutte LBA, Vanriel PLCM. Modified Disease-Activity Scores That Include 28-Joint Counts - Development and Validation in A Prospective Longitudinal-Study of Patients with Rheumatoid-Arthritis. Arthritis and Rheumatism. 1995;38:44–48. doi: 10.1002/art.1780380107. [PubMed] [Cross Ref]
36. Van Riel PLCM, Fransen J, Scott DL. EULAR handbook of clinical assessments in rheumatoid arthritis. Alphen aan de Rijn, The Netherlands: Van Zuiden Communications B.V.; 2004.
37. Fries JF, Spitz PW, Young DY. The dimensions of health outcomes—the health assessment questionnaire, disability and pain scales. Journal of Rheumatology. 1982;9:789–793. [PubMed]
38. Zandbelt MM, Welsing PMJ, Gestel AM, Riel PLCM. Health assessment questionnaire modifications: Is standardisation needed? Annals of the Rheumatic Diseases. 2001;60:841–845. doi: 10.1136/ard.60.5.511. [PMC free article] [PubMed] [Cross Ref]
39. Marra CA, Woolcott JC, Kopec JA, Shojania K, Offer R, Brazier JE, Esdaile JM, Anis AH. A comparison of generic, indirect utility measures (the HUI2, HUI3, SF-6D, and the EQ-5D) and disease-specific instruments (the RAQoL and the HAQ) in rheumatoid arthritis. Social Science and Medicine. 2005;60:1571–1582. doi: 10.1016/j.socscimed.2004.08.034. [PubMed] [Cross Ref]
40. Russell AS, Conner-Spady B, Mintz A, Mallon C, Maksymowych WP. The responsiveness of generic health status measures as assessed in patients with rheumatoid arthritis receiving infliximab. Journal of Rheumatology. 2003;30:941–947. [PubMed]
41. Howard GS, Schmeck RR, Bray JH. Internal invalidity in studies employing self-report instruments: A suggested remedy. Journal of Educational Measurement. 1979;16:129–135. doi: 10.1111/j.1745-3984.1979.tb00094.x. [Cross Ref]
42. Schwartz CE, Bode R, Repucci N, Becker J, Sprangers MA, Fayers PM. The clinical significance of adaptation to changing health: A meta-analysis of response shift. Quality of Life Research. 2006;15:1533–1550. doi: 10.1007/s11136-006-0025-9. [PubMed] [Cross Ref]
43. Gestel AM, Prevoo ML, ‘t Hof MA, Rijswijk MH, Putte LB, Riel PL. Development and validation of the European league against rheumatism response criteria for rheumatoid arthritis. Comparison with the preliminary American college of rheumatology and the World Health Organization/International league against Rheumatism criteria. Arthritis and Rheumatism. 1996;39:34–40. doi: 10.1002/art.1780390105. [PubMed] [Cross Ref]
44. Jansen SJ, Stiggelbout AM, Nooij MA, Noordijk EM, Kievit J. Response shift in quality of life measurement in early-stage breast cancer patients undergoing radiotherapy. Quality of Life Research. 2000;9:603–615. doi: 10.1023/A:1008928617014. [PubMed] [Cross Ref]
45. Lenert LA, Treadwell JR, Schwartz CE. Associations between health status and utilities implications for policy. Medical Care. 1999;37:479–489. doi: 10.1097/00005650-199905000-00007. [PubMed] [Cross Ref]

Articles from Springer Open Choice are provided here courtesy of Springer