Addressing measurement error in social science data

Abstrakt

Measurement error is omnipresent in social science data. Assessing the sources and impact of the error is important for designing policies to increase measurement reliability, and for developing high-quality ratings. In this work, we discuss several statistical aspects addressing the measurement error in social science data. We introduce a flexible method for assessing heterogeneity in measurement error and reliability with variance component models. We also discuss the relationship between the reliability and the false positive rate and address the issue of zero estimates. Methods are demonstrated with real-data examples from teacher hiring and grant proposal peer-review.