I’ve heard it said in many corners that the Market Research industry is in a state of transition and transformation. That’s true, though I’d also argue that’s really always the case if we are doing things right. That aside, I see many firms trying new things and exploring new types of research. At the same time, however, I find it curious that there seems to be a growing lack of concern for and dearth of knowledge about research fundamentals, specifically who is participating in research and where they come from.
“There seems to be a growing lack of concern for research fundamentals”
Thus, I think it’s worth taking a step back and thinking about three important questions to ask your data provider(s). Before we dive in, it’s worth pointing out that the world of sample providers is more complex than it used to be. You have river sample or intercept providers who fish for participants online in a number of ways, aggregators or distributors who sell sample and hook into various sample sources without owning any of them, panel companies who own and manage their data sources, and some that don’t neatly fall into any of these categories. No matter where you get your sample, you should be asking…
1. Where do people come from?
Are people coming from the river, meaning that they may be “fished” (no, not catfished) from an online source, whereby you don’t know much about them and once they are surveyed and put back into the stream they are unlikely to be caught again? Or are they sourced from a panel whereby, to continue the metaphor, they can be fished out of a discrete pond where they are tagged and can be found again if need be? Participants from these discrete ponds add a level of comfort for researchers as they are vetted and typically thought to be of higher quality. But you shouldn’t check your curiosity at the door at this point. It is worth asking your provider how they recruit people. How do people make it into the pond? Panels should be meticulously managed and maintained by implementing consistent recruitment methodology from a variety of sources. Variety in recruitment is key, as you don’t want your participants to come from a single, specific source, one that lacks diversity.
2. Who are these people?
The more you know about who you are reaching, the better off you are. Thinking of how you source research participants, the depth to which they are profiled allows for more accurate, efficient, and quicker targeting while reducing the need to screen for everything. A well-maintained profiling database can also power impactful data appends. So, ask about profiling methodology, what areas relevant to your research they are profiled on, and how this data can be used to conduct smarter research.
3. What else?
Let’s be honest, I could have given you 5 more questions, but I’m taking the easy way out here and placing the burden on you now. Well, it may not be a burden to researchers as I’m just asking you to be inquisitive – as should be your nature. Just as you ask questions about your data, ask questions about where it comes from. You’re not going to capture everything you need to know by asking the two questions above, but it’s a start. You may want to also ask about quality control measures, variations in recruitment methodology across time, and panel/non-panel source blending. But I think the most important thing is for you, or perhaps someone you know, to think critically and ask questions.
Roddy Knowles,
Director, Product and Research Methodology
Research Now
