“The correlation between stated intent and actual behavior is usually low and negative,” HBS Professor Gerald Zaltman wrote in his classic textbook, How Customers Think.
Years ago I took Zaltman’s class as a student and it made an impact on me. He talked about the conundrum that 80% of new products and services fail within six months of launch, usually after having been vetted by focus groups. There’s a gap, he taught us, between what people say and what they do.
The reason for the gap is not that focus group participants are lying maliciously. They may be saying what they think the moderator (or the rest of the group) wants to hear. They may be trying to project a certain image of themselves. Zaltman reported that 95% of what drives actual customer behavior is unconscious. He instructed us in a method called the Zaltman Metaphor Elicitation Technique (ZMET) as a way to surface some of those unconscious motivations.
Ultimately, I learned, whatever a focus group participant says about themselves is just their own opinions. It’s important to treat those opinions as opinions. And then supplement that research with what people actually do.
Here are a couple related cartoons I’ve drawn over the years:
“The Market Research Check List“ January 2012
“All-Knowing Focus Group“, October 2010
John says
I often find myself asking marketing professionals if they’ve ever particpated in a focus group – not for their own product, but just as a supposed member of the public. I’m amazed how few have.
If they had, they would know what a sham it all is. From the recruiters telling you what you should say your background is (not that I’ve noted anyone ever being asked) through to the conversations with other participants on the way out that reveal their true motivations, there is so much that is broken.
Colleagues argue that a good facilitator can break through all this and while I take the point, I take it with a huge dose of salt.
Boston Marketer says
Focus groups. Whose bright idea were they, anyway? I have participated in a couple, and my observation was that the questions were ill conceived and would not elicit the information supposedly needed. Rather than create an artificial public forum where participants define their own self-presentation, I’ve always found it more credible to meet with individuals one on one to obtain product and/or problem feedback. The individual being interviewed knows they have your full attention and you can steer the conversation down paths of (mutual) interest. People’s behavior in large groups is not the same as in one-on-one or very small groups. Perhaps that’s one reason why the USA finds itself in such a pickle today.
steve simon says
so good Tom, just love this one
Allen Roberts says
Absolutely true Tom.
I have commissioned many research projects, and always watched a few through the glass, and often been gobsmacked by the so called results that have emerged.
While some researchers are better than others, the average is a pretty low bar. However, occasionally there is one that as John noted, can cut through the sham, and those few are gold.
The basic problem is asking for views on things they have not seen, or are being asked to look at in an artificial environment in some sort of initial design stage.
Humans are really bad at seeing the future.