There are several possible reasons that preference scores were lower in our patients with cognitive impairment. Patients with cognitive impairment may actually value their health state less because it includes a component of cognitive impairment. Alternatively, cognitive impairment may alter how patients respond to VAS, SG, and TTO and testing per se, biasing their responses downward independent of their "true" preferences. The two explanations are not mutually exclusive, and both could be operating in an additive or synergistic fashion. If our current measurement tools cannot accurately measure preferences in patients with cognitive impairment, then measuring the preferences of impaired individuals will require the development and validation of new instruments, and in the interim these individuals should be identified and excluded from preference analyses.

Cognitive impairment may well diminish preferences for current health – preferences vary with a variety of subject characteristics such as demographics [21,22], comorbid conditions [21,22], measurement instrument [23-25], mode of administration – computer versus personal interview [26], the population being tested – individuals with the condition of interest often provide higher values than others [27-29], and scale anchor points [30-32]. Neumann et al. used the Health Utilities Index Mark II to assess health values for Alzheimer's dementia from caregivers [10]. Health values were inversely related to patient health, ranging from 0.73 for questionable dementia to 0.14 for terminal dementia. Ekman and colleagues used the TTO and a postal survey to measure preferences for mild cognitive impairment and mild, moderate, and severe dementia health states in a cross section of the Swedish population [12]. Preferences varied inversely with cognitive functioning, ranging from 0.82 for mild cognitive impairment to 0.25 for severe dementia.

Jonsson and co-workers used the EuroQol 5D to measure preferences for current health in patients with Alzheimer's disease and proxy valuations from their primary caregivers [11]. Patient preferences varied little across MMSE-based severity levels, averaging 0.83. Proxy valuations were lower than patients' and varied inversely with the degree of dementia (range 0.69 for MMSE > 25 to 0.33 for MMSE < 10). In our regression models, cognitive impairment was associated with a 0.12 – 0.23 decrease in preference values, a substantial effect size. The consistent effect of cognitive impairment on preferences measured with three different techniques – SG, TTO, VAS – that differ widely in their cognitive demands provides cross-validating evidence in favour of a real detrimental effect of cognitive impairment on preferences for current health. We have no ready explanation why WTP preferences were not affected by cognitive impairment.

Cognitive impairment might interfere with comprehension and processing of information required to complete preference measurement tasks, leading to biased preference values. Woloshin and colleagues have shown that numeracy affects preferences measured with the SG, TTO, and VAS [33]. Bravata and colleagues showed that, even after excluding individuals with cognitive impairment based on the MMSE, the remaining subjects with relatively low MMSE scores were more likely to provide uniform preference values equal to 1.0 when asked to evaluate multiple hypothetical health states [13]. We found several differences between the patterns of responses of patients with cognitive impairment and those of unimpaired patients. The distributions of responses for our unimpaired subjects followed skewed-normal or skewed distributions with modal values at or near perfect health. In contrast, the preference distributions of our cognitively impaired subjects had non-standard morphologies and greater variance. This difference suggests that some cognitively impaired subjects may not have understood the test and given extreme or random responses (SG, TTO) or responses tending towards the middle of the visual scale (VAS). This pattern would result in lower mean preference scores compared to unimpaired patients, and may account for some of the differences between the two groups.

If there is a bias in preference reporting/measurement associated with cognitive impairment, one solution would be to exclude individuals with cognitive impairment from testing. Such a policy could be problematic for any assessments of societal preferences (which are recommended for use in cost-effectiveness analyses [23]), since it would exclude a substantial portion of the population – for example, an estimated 4.5 million people in the United States are afflicted with Alzheimer's disease [34]. The identification of cognitively impaired individuals would also be difficult. Adding a cognitive screening instrument to protocols collecting preference data would consume study resources and add to respondent burden. Our study used the MMSE, an 11-item instrument requiring 5–10 minutes and a face-to-face encounter. While widely used, the MMSE is not without its critics, and some authorities have suggested using a higher threshold to define cognitive impairment [35,36]. Other "bedside" alternatives to the MMSE are at least as complex and time consuming [37]. The 11-item Telephone Interview for Cognitive Status can be used for remote cognitive testing, but still requires 5–10 minutes to administer [38].

Twelve percent of our patient population had some difficulty understanding or completing the preference testing, although all provided responses for the VAS, SG, TTO, and WTP. Interestingly, we did not find that testing difficulties was associated with cognitive impairment as measured with the MMSE. Some investigators have excluded the responses of individuals who did not appear to understand the preference testing process [13,39,40], and others have developed techniques to detect and minimize inconsistencies during multiple preference measurements in the same subject [41]. Unfortunately, our study design did not provide us with sufficient data to allow a confident investigation of the effects of testing difficulties on preferences. Future investigations will include a more rigorous assessment of testing difficulties and enable investigation of the relationship between cognitive impairment and difficulty understanding and completing preference testing.

Most researchers have found that patient preferences vary depending on the measurement instrument, and our study is no exception – our patients had SG and TTO preferences significantly greater than VAS preferences (WTP values have a unique metric that precludes direct comparison with the other preference values).

These ubiquitous discrepancies have lead to a lively debate about their etiology and significance. Some believe that the SG is the "gold standard" in measuring patient preferences because it conforms to the axioms of von Neumann-Morgenstern utility theory; however, it is subject to bias and framing effects, and can be distorted by risk aversion [42-44]. The TTO has roots in decision theory and was developed as a more "user friendly" alternative to the SG, but TTO values can be confounded by time preferences [45-48]. While it is convenient to administer, the VAS has been criticized for lacking the theoretical underpinnings of the SG or TTO and may have limited applicability [49]. The VAS does not incorporate risk of death (SG) or certain reduced survival (TTO). Since most subjects are risk averse and somewhat reluctant to trade years of life, the VAS generally yields lower scores that the SG or TTO [50]. Finally, WTP responses are affected by economic resources, and WTP preferences are not expressed on a zero to one ratio scale, making it difficult to incorporate WTP values into decision analytic models [51,52]. Variations in risk aversion, time preferences, and economic resources are all likely contributing to the differences in preference values provided by the four instruments. We do not know whether one or more of these factors are asymmetrically distributed across our cognitively impaired and unimpaired patients, and it is unclear whether or how much these factors may be contributing to preference differences between cognitively impaired and unimpaired patients.