I first drew a version of this cartoon years ago sitting behind the focus group one-way mirror while working at General Mills. We held a lot of focus groups, many of them in a Minneapolis suburb called Eden Prairie. We used to jokingly refer to those focus group respondents as “The Oracles of Eden Prairie.”
We seemingly brought just about every marketing idea to the Oracles of Eden Prairie for inspiration and direction. But what frequently happened is that we would listen for statements that confirmed what we already believed and then disregard the rest. I learned that focus groups can be a great tool for confirmation bias.
The Wall Street Journal recently wrote about how focus groups are gradually being displaced by technology-driven tools like social listening. As Seventh Generation’s Maureen Wolpert put it: “It’s very expensive to find that progressive-leaning, millennial new mom who has the time to attend a focus group. This is a way to listen in on the conversation moms are having and to really see their problems.”
Focus groups can also give false reads because respondents aren’t reliable when describing their own behaviors and preferences, particularly in an artificial environment like a focus group.
Listening to how people naturally talk in social media led to ideas like sriracha-flavored Hidden Valley ranch dressing. Clorox heard people share how they mixed spicy flavors like sriracha into traditional ranch.
But confirmation bias can show up in any form of market research. Any market research tool is limited by how we use it. We have to take any Oracles of market research with a grain of salt.
Here are a couple related cartoons I’ve drawn over the years:
“All-knowing Focus Group”, October 2010
“Focus Group”, November 2013
“Confirmation Bias”, November 2014
Jacqueline Drew says
I am so jealous of your work- amazing insight and so beautifully conveyed in your visuals. Kudos as always!
Judy Bernstein says
One very exciting way to avoid common biases and still leverage the benefits that come from conferring with your (often expensively defined) specific target is to use creative problem solving methodologies in ‘focus group’ sessions rather than more conventional qualitative approaches. Aiming for group discovery via group invention vs. group reporting can yield lemony fresh insights that are uniquely feasible and viable for your brand, service or offering.
David Paull says
Entertaining, as always, Tom! As we’ve discussed before, what you describe can be a real problem and our work at Dialsmith specifically mitigates the impact of things like confirmation bias, group-think, and flawed recall/memory bias. By adding a real-time data capture tool in focus groups, we learn what each individual respondent thinks in-the-moment, then moderators use that immediate data to help drive richer and more substantive group discussions. It’s been saving clients from confirmation bias for years.