Nonresponse bias in mail survey data: Salience vs. endogenous survey complexity

Abstract

The purpose of this chapter is to demonstrate the opportunities for, and utility of, explicit modeling of survey response/nomesponse. A good understanding of the relationship between survey response propensities and observable behavioral relationships within just the subsample of respondents can help inform researchers and policy makers about the likely nature of nomesponse biases. At best, however, a survey can only describe the characteristics of that group of individuals who were eligible to respond because they were contacted as part of an “intended sample” which has ideally been drawn at random from the population it is intended to represent. Any systematic nonresponse to a survey means that the resulting estimating sample will NOT be a random sample from the targeted population. Biases in the sample, if substantial, can mean significant biases in the implications of any analysis based on these nonrandom responses. Any researcher using mail survey data should be strongly encouraged to plan for, and then to undertake, explicit modeling of response/nonresponse to his or her survey instrument in a manner analogous to that presented here. This is especially important if one expects considerable heterogeneity in the socio-demographic characteristics of potential respondents, or if geographical proximity to the place(s) or object(s) featured in the subject matter of the survey varies substantially across potential respondents. It is also important if there are different versions of the survey, or if portions of the working sample consist of nonrandom convenience samples appended to a base sample that is reasonably representative. The key insight is that without formal nonresponse modeling and correction, the default presumption must be that substantial nonresponse biases could easily be present in any statistical work conducted using only a sample of mail survey respondents. Furthermore, if survey complexity is endogenous, it is not even possible to sign these potential biases ex ante. These biases can distort not only estimates of the level or elasticity of demand in the population, but also estimates of the degree of substitutability among goods and overall welfare calculations.

Publication
Chapter 8 in Valuing Recreation and the Environment: Revealed Preference Methods in Theory and Practice Joseph A. Herriges and Catherine L. Kling (eds.), Edward Elgar, pages 137-169 (1999)