A/B Testing the Online Survey Experience

Together with our partners, we are currently undertaking a large-scale research-on-research project into the participant’s experience with online surveys in order to A/B test how different approaches improve the user experience. We have already gathered a lot of knowledge about people’s attitudes to taking surveys. The aim of this new research-on-research is to build on that knowledge base and test in a controlled environment, how the survey-taker user experience can be improved.

Can a video by the client improve the user experience?

Our previous research has told us that many people join panels and take surveys in order to influence things and have their opinion count. Unfortunately, few surveys today give participants that feeling, so we want to test whether telling people who the survey is for and giving them examples of how the results will be used has an impact on the user experience, especially if we do it in an engaging format. Our research also tells us that people blame the end-client when they get a bad survey experience. We want to go a step further and test whether clients can create a positive impression on their brand by creating a positive survey experience, much in the same way as they can by creating a positive CX or community experience.

“Taking an online survey used to be one of the most fun things you could do on the internet”

This quote by Germán Loewe, Netquest’s CEO, has stuck in my mind. It seems hard to believe in today’s world of FaceBook, Instagram and Youtube that this indeed used to be the case. Whilst, it will never be true again, we don’t think that this is an excuse for accepting that surveys need to be boring, so we want to test some different ways of making surveys more interesting / less hard work for participants. What if we share data and insights on the topic at hand back with the survey-takers? What if we asked people about some things they are interested in alongside the things the client is interested in? Can we make a longer survey a better experience than a shorter one? As Richard Goosey of Kantar Health, one of the team members on this project, points out:

“If participants are prepared to give us their time, the least we should do is ensure our questions are relevant to them!”

Finally, can we make a survey a better experience by changing the language used: making it more conversational, more concise, more motivational?

AVOIDING UNINTENTIONAL IMPACTS

An important reason for doing this research using an A/B testing methodology in a controlled environment is to test whether or not making these changes will impact the survey results and more importantly impact the researcher’s recommendations to the client? If this is not the case, then we will succeed in breaking down one of researchers key objections to change.

Thanking our Partners

We would like to the opportunity to thank our partners on this research-on-research, without whose generous support, we couldn’t undertake this important work! In particular, we would like thank the workgroup members, who have worked hard on getting this project off the ground:

INPUT INTO THE PARTICIPANT ENGAGEMENT HANDBOOK

The results from this research-on-research will be used as input into the Participant Engagement Handbook we will be issuing in October. The handbook will give practical guidance on how to create a great user experience and avoid a poor one. Not only do people participating in research deserve it, so do the client-side decision-makers who rely on the data to inform their decisions.