I first drew a version of this cartoon years ago sitting behind the focus group one-way mirror while working at General Mills. We held a lot of focus groups, many of them in a Minneapolis suburb called Eden Prairie. We used to jokingly refer to those focus group respondents as “The Oracles of Eden Prairie.”
We seemingly brought just about every marketing idea to the Oracles of Eden Prairie for inspiration and direction. But what frequently happened is that we would listen for statements that confirmed what we already believed and then disregard the rest. I learned that focus groups can be a great tool for confirmation bias.
The Wall Street Journal recently wrote about how focus groups are gradually being displaced by technology-driven tools like social listening. As Seventh Generation’s Maureen Wolpert put it: “It’s very expensive to find that progressive-leaning, millennial new mom who has the time to attend a focus group. This is a way to listen in on the conversation moms are having and to really see their problems.”
Focus groups can also give false reads because respondents aren’t reliable when describing their own behaviors and preferences, particularly in an artificial environment like a focus group.
Listening to how people naturally talk in social media led to ideas like sriracha-flavored Hidden Valley ranch dressing. Clorox heard people share how they mixed spicy flavors like sriracha into traditional ranch.
But confirmation bias can show up in any form of market research. Any market research tool is limited by how we use it. We have to take any Oracles of market research with a grain of salt.
Here are a couple related cartoons I’ve drawn over the years:
“All-knowing Focus Group”, October 2010
“Focus Group”, November 2013
“Confirmation Bias”, November 2014
Jacqueline Drew says
June 18, 2017 at 6:31 pmI am so jealous of your work- amazing insight and so beautifully conveyed in your visuals. Kudos as always!
Judy Bernstein says
June 19, 2017 at 8:00 amOne very exciting way to avoid common biases and still leverage the benefits that come from conferring with your (often expensively defined) specific target is to use creative problem solving methodologies in ‘focus group’ sessions rather than more conventional qualitative approaches. Aiming for group discovery via group invention vs. group reporting can yield lemony fresh insights that are uniquely feasible and viable for your brand, service or offering.
David Paull says
June 19, 2017 at 1:20 pmEntertaining, as always, Tom! As we’ve discussed before, what you describe can be a real problem and our work at Dialsmith specifically mitigates the impact of things like confirmation bias, group-think, and flawed recall/memory bias. By adding a real-time data capture tool in focus groups, we learn what each individual respondent thinks in-the-moment, then moderators use that immediate data to help drive richer and more substantive group discussions. It’s been saving clients from confirmation bias for years.
Sarah Montague says
June 29, 2017 at 10:31 amI just love your work. In addition to periodically licensing your marketoons for internal presentations, your site is often my go-to for humor breaks. I am still trying to find the budget to hire you as outside speaker at one of my client meetings. Maybe in 2018….. Cheers, Sarah
Mark Scott says
August 24, 2017 at 2:38 pmYour observations on the dangers of focus group have some truth. It is very popular to bash groups right now. However, social media listening does not represent a major improvement. Social media posts on products are even less representative. Despite all their flaws consumer research is as much an art as a science. At some point, to really understand what drives consumers you need to sit down and talk to them as people. a well run small focus group can be a very effective way to have that conversation.