January 2001 | Marianne Bertrand and Sendhil Mullainathan
This paper examines the reliability of subjective survey data and its implications for economic research. Marianne Bertrand and Sendhil Mullainathan argue that people often do not mean what they say when answering subjective questions, which can lead to significant measurement errors in economic analysis. They review experimental and empirical evidence showing that cognitive biases, social desirability, and other factors can distort responses to subjective questions. These biases include issues such as question wording, response scales, and the order of questions, which can all affect the accuracy of survey data.
The authors also discuss how these measurement errors can impact econometric analyses. If subjective variables are used as independent variables, they may still be useful as control variables, but their coefficients should be interpreted with caution. However, when subjective variables are used as dependent variables, the measurement errors are likely to be correlated with other characteristics and behaviors, making such analyses unreliable.
The paper presents empirical evidence from the High School and Beyond survey, showing that subjective questions can predict future income and job turnover. However, changes in responses over time do not necessarily reflect changes in behavior. The authors conclude that while subjective variables can be useful in explaining differences in behavior across individuals, they should not be used as dependent variables due to the potential for measurement error. The findings highlight the importance of carefully interpreting subjective survey data in economic research.This paper examines the reliability of subjective survey data and its implications for economic research. Marianne Bertrand and Sendhil Mullainathan argue that people often do not mean what they say when answering subjective questions, which can lead to significant measurement errors in economic analysis. They review experimental and empirical evidence showing that cognitive biases, social desirability, and other factors can distort responses to subjective questions. These biases include issues such as question wording, response scales, and the order of questions, which can all affect the accuracy of survey data.
The authors also discuss how these measurement errors can impact econometric analyses. If subjective variables are used as independent variables, they may still be useful as control variables, but their coefficients should be interpreted with caution. However, when subjective variables are used as dependent variables, the measurement errors are likely to be correlated with other characteristics and behaviors, making such analyses unreliable.
The paper presents empirical evidence from the High School and Beyond survey, showing that subjective questions can predict future income and job turnover. However, changes in responses over time do not necessarily reflect changes in behavior. The authors conclude that while subjective variables can be useful in explaining differences in behavior across individuals, they should not be used as dependent variables due to the potential for measurement error. The findings highlight the importance of carefully interpreting subjective survey data in economic research.