Surveys are an indispensable tool for any researcher. They’re quick, cost-effective, scalable, and at RIWI, we love them! Having pioneered a web-intercept technology (RDIT) designed to collect honest feedback by providing a secure, anonymous channel, we know a thing or two about the conditions required to promote honesty in research respondents. But, in many scenarios complete anonymity isn’t always possible, or even the goal.
So, how can researchers looking to engage a panel for instance, trust the truth is being told? This article explores the underlying reasons that cause respondents to misrepresent their own truths, and what researchers can do to get closer to what they really think and feel.
Why people aren’t always honest
The literature identifies several reasons why survey participants might hold back the truth:
1. Fear of judgment or repercussions.
Even online and from behind a keyboard, humans innately fear the stigma or repercussions of providing a response that may be perceived negatively. Social desirability bias can manifest itself in respondents editing personal information or beliefs to avoid judgment or embarrassment, especially when research topics are controversial or sensitive.
2. Sensitive or stigmatized topics.
Questions on subjects like health, finances, sexual behaviour or violence can trigger anxiety or shame. A randomized controlled trial found that individuals were significantly more likely to disclose stigmatized behaviours when their responses were anonymous rather than merely confidential – with disclosure rates nearly three times higher under anonymous conditions. This pattern aligns with RIWI’s own findings in sensitive research contexts, including studies in Afghanistan, where full anonymity enabled respondents to share views that are rarely expressed in identifiable surveys.
3. Unclear or leading questions.
The way that a question or test is worded can subtly suggest that there is a “right” answer or nature of response. The results of this can include respondents feeling nudged to agree, exaggerate, or choose the option that feels most acceptable. If unclear, respondents in an online scenario will often fill in the gaps themselves with their own assumptions rather than waiting for an email explanation from a moderator. In these scenarios, it is common for respondents to default to whatever sounds safest or most socially desirable.
4. Lack of trust in the survey sponsor or platform.
If participants doubt that their privacy will be protected or feel uncomfortable, they may withhold genuine views. A literature review on anonymity and confidentiality concluded that preserving these principles is essential for building trust and ensuring the integrity of survey responses.
Provoking Honesty: Tried And True Strategies
1. Prioritize anonymity and confidentiality
Anonymity and confidentiality are very different instruments, and respondents know that but may need help in their understanding. Calling out the difference carefully is important, and language used shouldn’t elude risk. The survey’s approach should be specified clearly and early, providing respondents with assurance that their identities won’t be linked to their responses – via anonymity or confidentiality.
2. Set expectations and reassure respondents
Communicating a survey’s purpose is a powerful and often underutilised tool. Doing so reassures participants that their feedback will actually be used and used responsibly. We recommend emphasizing that candid responses help improve products, services or policies – whatever it is that you’re studying.
3. Design neutral, non‑judgmental questions
Social desirability bias often occurs when questions sound accusatory or value‑laden. As a basic rule, avoid assuming the answer in the question. Wording like, “how helpful was the new feature,” clearly implies that a high helpfulness rating is desired. Whenever possible, look to avoid loaded or emotional words like, “right,” “wrong,” or “should,” and definitely ask for both positive and negative feedback to provide equal opportunity.
For sensitive demographic questions, provide ranges or categories (e.g., income brackets) rather than asking for precise figures.
4. Use validated scales and cross‑check data
Response-bias detection and social desirability scales can flag respondents who may be over-reporting socially acceptable behaviours. Used well, these additional metrics can be a useful lens for data quality and interpretation, allowing researchers to segment results (high vs low desirability scorers), adjust how much weight is placed on sensitive self-reports, and decide where to add complementary methods. For higher-stakes decisions, pairing self-report with genuinely implicit or behavioral signals can help close the gap between what respondents claim and what is more likely to drive real-world choice.
Online surveys will continue to shape business and policy decisions around the world. While biases and misreporting pose real challenges, researchers should feel encouraged by the strategies and tactics available to protect the privacy of respondents and design methods and questions that encourage candor.