Preprints and Reprints of Papers on Anchoring Vignettes

(To print PDF files, get the free Acrobat Reader.)

• Improving Anchoring Vignettes: Designing Surveys to Correct Interpersonal Incomparability by Daniel Hopkins and Gary King. We report the results of several randomized survey experiments designed to evaluate two intended improvements to anchoring vignettes, an increasingly common technique used to achieve interpersonal comparability in survey research.  This technique asks for respondent self-assessments followed by assessments of hypothetical people described in vignettes. Variation in respondents' assessments of the vignettes across people reveals interpersonal incomparability and allows researchers to make responses more comparable by rescaling them. Our experiments show first that switching the question order so that self-assessments follow the vignettes primes respondents to define the response scale in a similar way.  In this case, priming is not a bias to avoid but a means of better communicating the question's meaning.  We then demonstrate that combining vignettes and self-assessments in a single direct comparison induces inconsistent and less informative responses.  Since similar strategies are widely employed for related purposes, our results indicate that anchoring vignettes could reduce measurement error in many applications where they are not currently used.  Data for our experiments come from a national telephone survey and a separate on-line survey.
• Comparing Incomparable Survey Responses: New Tools for Anchoring Vignettes by Gary King and Jonathan Wand. When respondents use the ordinal response categories of standard survey questions in different ways, the validity of analyses based on the resulting data can be biased. Anchoring vignettes is a survey design technique, introduced by King, Murray, Salomon, and Tandon (2004), intended to correct for some of these problems. We develop new methods both for evaluating and choosing anchoring vignettes, and for analyzing the resulting data. With surveys on a diverse range of topics in a range of countries, we illustrate how our proposed methods can improve the ability of anchoring vignettes to extract information from survey data, as well as saving in survey administration costs. Political Analysis, 15, 1 (Winter, 2007): Pp. 46-66.
• Comparability of self rated health: cross sectional multi-country survey using anchoring vignettes Joshua A Salomon, Ajay Tandon, Christopher J L Murray, and the World Health Survey Pilot Study Collaborating Group. Objective To examine differences in expectations for health using anchoring vignettes, which describe fixed levels of health on dimensions such as mobility. Design Cross sectional survey of adults living in the community. Setting China, Myanmar, Sri Lanka, Pakistan, Turkey, and United Arab Emirates. Participants 3012 men and women aged 18 years and older (self ratings); subsample of 406 (vignette ratings). Main outcome measures Self rated mobility levels and ratings of hypothetical vignettes using the same questions and response categories. Results Consistent rankings of vignettes are evidence that vignettes are understood in similar ways in different settings, and internal consistency of orderings on two mobility questions indicates good comprehension. Variation in vignette ratings across age groups suggests that expectations for mobility decline with age. Comparison of responses to two different mobility questions supports the assumption that individual ratings of hypothetical vignettes relate to expectations for health in similar ways as self assessments. Conclusions Anchoring vignettes could provide a powerful tool for understanding and adjusting for the influence of different health expectations on self ratings of health. Incorporating anchoring vignettes in surveys can improve the comparability of self reported measusures. published in the BMJ 2004, 328, 258 (31 January.)
• Enhancing the Validity and Cross-cultural Comparability of Survey Research by Gary King, Christopher J.L. Murray, Joshua A. Salomon, and Ajay Tandon. We address two long-standing problems in survey research: measuring complicated concepts, such as political freedom or efficacy, that researchers define best with reference to examples; and what to do when respondents interpret identical survey questions in different ways. Scholars have long addressed these problems with approaches to reduce incomparability, such as writing more concrete questions -- with uneven success. Our alternative is to measure directly whatever incomparability exists and to correct for it. We measure incomparability via respondents' assessments, on the same scale as the self-assessments to be corrected, of hypothetical individuals described in short vignettes. Since actual levels of the vignettes are invariant over respondents, variability in vignette answers reveals incomparability. Our corrections require either simple recodes or a statistical model designed to save survey administration costs. With analysis, simulations, and cross-national surveys, we show how these problems can drastically mislead survey researchers and how our approach can fix them. Published in the American Political Science Review. [The November issue of the APSR included this article, but was based on the wrong set of proofs. The full corrected article was republished in the February issue. See the PDF above.]