A Note on Increases in Inattentive Online Survey-Takers Since 2020
Keywords:Lucid, crowdsourcing, attention checks, data quality, online surveys, inattentiveness
Lucid, a popular source of online convenience survey samples, has seen a significant increase in inattentive respondents since 2020. Inattentive participants – respondents who incorrectly answer directed query attention check questions – may be introducing substantial measurement error and attenuation bias. Using data from 152,967 survey respondents across multiple studies conducted between January 2020 and June 2021, we find that inattentive respondents report less reliable demographic data, less stable responses, and are systematically different from attentive respondents. We find some evidence of attenuation bias and mixed evidence that data quality has decreased slightly since 2020 even after filtering for inattentive respondents. We conclude that researchers using Lucid should report if they screened on attentiveness and consider replicating any null results. Such an unexpected increase in inattentiveness in a widely-used platform suggests that future researchers relying on online convenience survey samples should continuously assess data quality.
How to Cite
Copyright (c) 2022 John Ternovski, Lilla Orr, Joshua Kalla, Peter Aronow
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.