A recent study published in Discover Psychology has shed light on the peculiar effects of psychological interventions on survey honesty. Researchers in China set out to examine the factors related to misreporting in online health surveys and to assess the efficacy of using images of “watching eyes” to deter such behavior. Contrary to expectations, the study found that these images not only failed to decrease misreporting but actually increased the likelihood of false responses.
The internet has revolutionized the way researchers collect data, with platforms like Amazon Mechanical Turk facilitating easy access to millions of potential respondents. However, the integrity of survey responses remains a concern, especially when compensation is involved, tempting some to misreport for quicker completion.
Misreporting can severely undermine the validity of research findings, leading to misguided public health policies and resource misallocation. In China, the use of Sojump for participant recruitment has grown, yet little was known about the extent or factors contributing to survey misreporting within this context.
“When we started conducting another study in China using online surveys, I became curious about two issues. (1) How concerned should we be about deliberate misreporting by survey respondents? And (2) How effective would images of watching eyes be at discouraging deliberate misreporting? To the best of my knowledge, there was very limited research on survey misreporting in China, and no published studies that tested watching eyes to specifically discourage survey misreporting,” explained study author Stephen W. Pan, an assistant professor at The University of Texas at San Antonio.
The study included 1,655 Chinese adults who were recruited via Sojump. These participants were randomly divided into four groups. Specifically, participants were either asked to report their blood type directly or told they could skip the rest of the survey by claiming to have type AB blood, introducing a temptation to misreport.
Half of the groups were exposed to a “watching eyes” graphic before answering, intended to subconsciously encourage honesty. This setup aimed to both identify demographic predictors of misreporting and test the surveillance cue’s efficacy in a controlled online environment.
“We detected very limited evidence of deliberate survey misreporting when respondents were presented with an incentive to do so,” Pan told PsyPost. “Granted, the incentive to misreport was quite weak (an opportunity to end the survey early), but this finding is reassuring to us and perhaps other researchers who collect data through online survey organizations in China.”
A significant distinction emerged between participants exposed to the watching eyes imagery and those who were not, especially among those given an incentive to misreport their blood type. Contrary to the researchers’ hypothesis, the group exposed to the watching eyes and presented with the opportunity to misreport for a quicker survey completion were more likely to provide false information.
“We found that the effect of watching eyes images completely backfired,” Pan explained. “It seems that the watching eyes were so off-putting, that respondents exposed the watching eyes were actually MORE likely to misreport answers, simply so they could end the survey early. Hence, it appears that images of watching eyes can decrease positive engagement and cooperation, depending on the incentive structures.”
Across the board, no significant sociodemographic predictors of misreporting were identified, indicating that the propensity to misreport under the influence of watching eyes did not vary meaningfully with factors like age, gender, or education level.
The study is not without its limitations. The incentive for misreporting, tied to survey completion rather than a more tangible reward, may not have been strong enough to capture the full extent of dishonest behavior. Additionally, the sample lacked diversity in educational attainment, potentially overlooking the effects of socioeconomic status on survey honesty.
“We conducted a follow-up misreporting that included stronger incentives,” Pan noted. “Hoping to have that out in the near future.”
The study, “Who misreports on internet health surveys, and do images of watching eyes discourage misreporting? An online experiment from China,” was authored by Tianyu Guo, Chuqing Cao, Jing Wang, Xinwen Hu, Ying Chen, and Stephen W. Pan.