Inclusion of anchoring vignettes in health surveys is part of an integrated strategy of instrument design and analysis to make self reported measures more comparable between individuals, communities, and populations.12
Anchoring vignettes may be applied to many different problems in which ordered categorical self report data are collected. This approach enables examination of systematic differences in categorical cut points between populations, within populations across different socio-demographic groups, or within individuals or groups over time. The anchoring vignette method also allows comparisons between different questions relating to a common domain, enabling the interpretation of responses to these related questions on a single underlying scale, and thus providing a bridge between data collected using different instruments.
The use of vignettes has a long history in research for the social sciences, including applications in anthropology, sociology, and psychology since the 1950s18-20
and numerous applications of the factorial-survey technique.21
Recent examples of the use of vignettes in health and medicine include applications in nursing research, medical education, and research on clinical practice.22-25
Our anchoring vignette approach differs from those in previous studies in certain fundamental ways. Firstly, rather than generating random variants of the same vignette,21
our approach uses vignettes as scale anchors and therefore requires that a given vignette describes the same level to all respondents. Secondly, our strategy is based on explicit links between vignette ratings and self ratings through the use of identical questions and response categories.
Two important requirements for the use of anchoring vignettes are response consistency—which implies that an individual uses response categories for a particular question similarly when evaluating hypothetical scenarios as when providing self assessments—and vignette equivalence—which implies that the underlying domain level represented in each vignette is understood in approximately the same way by all respondents, irrespective of their age, sex, education, country of residence, or other characteristics. We note that even when vignette equivalence holds, the categorical ratings for a given vignette may vary systematically due to differences in expectations; our strategy is designed to identify these differences. Empirical investigations about the two requirements of the approach are essential elements of the research needed on anchoring vignettes. We present available evidence supporting both requirements; further research is underway to develop techniques for critically evaluating and comparing different vignettes.
Our examples show that variation in vignette ratings for mobility can reveal differences in expectations for health—for instance, between different age groups. Formal statistical models have been introduced to allow anchoring vignette data to be used in adjusting self rated measures of health,15,16
but fundamental insights can be gained into differences in the use of particular questions and their associated response categories by analysing distributions of vignette ratings, even before any models are applied. Anchoring vignettes have been developed for the World Health Survey for a range of different health domains, as well as for other areas that share similar methodological challenges, such as health system responsiveness and social capital. Although more work is needed to refine individual vignettes and identify those that work best, this study shows that the anchoring vignette strategy is feasible in a variety of settings and offers promise for more widespread application of the approach.
A number of limitations should be noted. Firstly, the sample size in this pilot study is small and cannot be assumed to represent general populations. Although we aim to show the types of empirical findings that are available through the use of anchoring vignettes, the data collected in the probability samples of the World Health Survey will allow further investigation on some of the questions that we raise. Cross validating the anchoring vignette approach will be useful—for example, using measured performance tests on selected health domains. Current understanding of the causes of differences in cut points is limited. Research on psychology and decision making has highlighted a range of biases and heuristics that shape responses to survey questions26
; similar quantitative understanding of how different health expectations influence self perceptions of health and key correlates of these differences would aid interpretation of self reported measures of health.
Interest has been rising recently in the challenges of interpreting self assessments of health, relating to issues of perception versus observation and experiences versus expectations.8,10
Anchoring vignettes can provide a useful tool for standardising perceptions of health and adjusting self reported measures to account for variation in norms and expectations for health. As self assessments continue to play a central role in the measurement of health outcomes in clinical trials and summary measures of population health, a strategy of including vignettes in national surveys and clinical research can improve the utility of these measures by confronting important problems of interpersonal comparability.
What is already known on this topic
Variation in perceptions of health and self assessments of health status may be related in part to different expectations for health
Standard methods for measuring health status do not distinguish changes in health from changes in expectations. Interpretation of self reported measures of health may be improved by using new methods that account for varying expectations
What this study adds
Application of a data collection strategy based on anchoring vignettes enables the investigation of different individual expectations for health and the adjustment of self reported measures of health to account for these differences
Empirical evidence from a multi-country survey study using the anchoring vignette strategy points to differences in health expectations across age groups and countries
By mapping responses to various questions on the same health domain to a common comparable scale, anchoring vignettes can provide a bridge between data collected using different instruments for measuring health status