My big brand training taught me the “test, test, test, test, launch big” school of market research. I’m a big believer in the power of market research when done well. But I’ve also experienced the purgatory of over-testing. Over-testing limits our ability to move quickly and delays getting our ideas into the real world.
I find inspiration in the philosophy of Matt Mullenweg, founder of WordPress:
“Usage is like oxygen for ideas. You can never fully anticipate how an audience is going to react to something you’ve created until it’s out there. That means every moment you’re working on something without it being in the public it‘s actually dying, deprived of the oxygen of the real world … By shipping early and often you have the unique competitive advantage of hearing from real people what they think of your work, which in best case helps you anticipate market direction, and in worst case gives you a few people rooting for you that you can email when your team pivots to a new idea. Nothing can recreate the crucible of real usage.”
Internet companies commonly remain in Beta for extended periods of time, which allows them to adapt and evolve in the market. Yet, more frequently I see non-Internet companies doing the same thing. The aptly named BetaBrand launches a new clothing product every week. A Startup Store recently launched in New York as an ever-changing retail space that will completely change products and physical design every 4-6 weeks.
Rachel Shechtman, founder of A Startup Store, said: “We’re in beta because I don’t think it’s fair that digital companies are the only ones who get to be in beta.”
It’s not just an opportunity for startup companies either. I was struck by this recent quote from Marc Pritchard, global marketing officer at P&G: “I want P&G to adopt a ‘do-learn’ approach to brand-building. This means doing and learning simultaneously, rather than learning, learning, learning some more and then doing”
(Marketoonist Monday: I’m giving away a signed print of this week’s cartoon. Just share an insightful comment to this week’s post. I’ll pick one comment by 5:00 PST on Monday. Thanks!)
Zohare Haider says
January 15, 2012 at 11:38 pmNice write-up. Simple and to the point. The dilemma of GTM sometimes gets mixed up with how companies define 100% readiness in terms of product and after-sales. Fortunately, with the way social media has grown in ’10 and ’11 along with the rate at which companies are adopting guidelines to manage online communities, shows that the virtual environment of customer engagement is becoming more ‘real-time research’ based on then & there consumer feedback. Great stuff.
@jjbaybee on twitter
Ameena Falchetto says
January 16, 2012 at 1:02 amFirstly the cartoon made me laugh so hard.
I always hear people saying “Oh but we need to do our research” – yes, do your research but don’t let it cripple your ideas and prevent your ability to actually get started.
Research is and will always be a double edged sword – you need to do some research to make sure what you are planning is actually viable but you need to stop when you have found what you were looking for. Excessive researching makes you stop, over think, and doubt.
Go out there, do, make, create and if you need to tweak it along the way then great …
Sarah Jones says
January 16, 2012 at 5:19 amMost FMCG companies feel they have to get it right from the start. Nothing wrong with failure though as long as you learn from it and ideally do not waste too many resources. Over-researching can also be wasteful though!
Arun Prabhu says
January 16, 2012 at 5:20 amI attended a presentation by Camilla Barnard, co-founder of a starteup called Rude Health (www.rudehealth.com) and one of the points she made was that if they had more money when they started out they would have probably researched the market more. I sat there in the audience thinking, “If I had less money, I could launch much faster”.
Clark J says
January 16, 2012 at 5:23 amOne year after Sony launched the Walkman, the consumer electronics company I worked for thought it might offer something similar. Of course, we had focus groups. These fine citizens universally panned the idea of a personal music player. “Too dangerous,” “Too strange.” “Who would want something like this?” Heck, we were radicals, and we put out our own “fast follow” version anyway. Soured me on excess advance research.
vld says
January 16, 2012 at 5:26 amIt’s amazing how many people feel there’s a standard way to research every project – the people who say “I need to do that because Project X did that”…
Bill Carlson says
January 16, 2012 at 5:55 amAll things in moderation…
“Ready, fire, aim” has its place but comes at a cost. Research has its costs.
Many want to simply dismiss research and are partly right in their reasons. Others want to feel fully covered by research before making a decision and they are partly right in their reasons.
No right answer here, all a matter of degree — just need to consider the risk/reward balance. But note the word “risk” in particular — we tend to hear about the positive “reward” stories related to skipping research but there are certainly others who aren’t so inclined to ‘fess up.
Every decision, research or not, is a gamble but the more we know, the smarter and more confident we are — so like all things in life, some is good and too much can be bad…
Vince Skolny says
January 16, 2012 at 6:39 amI say it like this: Doing marketing is the only valid market research.
Because a market is not a defined, homogeneous, or static entity that exists independently, but the result of individual consumers making choices in the real world, the best market research can only tell us what the market was, not what the market is, which is the temptation to over-testing: Trying to learn what cannot be learned.
“Living in beta” on the other hand is not aiming to understand what the market is, but to understand, even guide, how the market is developing. Well done research can point us in the direction of where a market might emerge, but only in-the-field marketing will create the market and give the valid, measurable customer feedback necessary to move forward in a profitable direction.
Bonus Point: Beta marketing, properly done, creates customer relationships and builds loyalty as we go, something traditional research, much less over-testing, can never accomplish.
Rags Srinivasan, twitter: @rags says
January 16, 2012 at 7:04 am“Usage is like oxygen for ideas” – sticking with that analogy let us not forget that oxygen while required for us to live is also a prescription drug. Too much of oxygen causes blindness. On the same vein if we don’t decide on our target segment or tell them what job we want them to hire our product for we will end up getting too much usage information that aren’t actionable. Worse, with all the feedback from those who are not in our target segment (especially those who won’t pay for the product) we will be blind to the needs of our target segment.
Usage testing is required as long as we start with Segmentation, Targeting and Positioning, which the Marketing Research helps with.
-Rags
Paul Smith says
January 16, 2012 at 7:45 am“Make a little, sell a little. Learn a lot. Fail cheap.” A pithy summary of this mindset from former P&G CEO, Durk Jager
Arnold Seefeld says
January 16, 2012 at 8:14 amGreat post Tom! I think that testing is as well some kind of research. If you launch faster without researching theoretically the potential, you do it to research/test it practically. It’s the lean startup idea of Eric Ries. The bis mistake would be to launch without defined and measurable success indicators. I’d have loved to read more about how P&G is planning to implement this validated learning methodology, as testing is real life can be hyper-expensive in the FMCG world (listing fees, minimal production runs etc.) What’s your thought on it?
Karlis Nollendorfs says
January 16, 2012 at 8:42 amAs a Marketing Researcher, I could not agree more with the sentiment of your cartoon. Most Marketers that I speak to would agree that it takes too long to research all of the elements that go into a “big” product launch.
A good Marketing Research plan will develop really strong intuition EARLY in the process. The best insights create a strong sense of intuition and connection with the intended target. In those cases, I have found that less research is needed to validate, and decisions based on intuition are easier to make.
Just as bad as the team being retired, the audience for our idea will have changed as well, and the market no longer exists for our idea!
Andrew Padon says
January 16, 2012 at 7:46 pmI love the sentiment expressed in this cartoon. As a market research professional, one of my biggest wishes has always been to be better able to scale the magnitude of the testing to the magnitude of the risk. When companies have an excess of caution, great, inexpensive opportunities to do and learn get missed because we haven’t gotten the numbers. On the other hand, shooting from the hip in a case where you are literally risking the business is blatantly unwise. Finding the balance point is the magic.
Unfortunately, in many cases, all marketing research gets swept up in the same bucket. One of the worst outcomes I’ve seen of desires to be quicker and more lean at one point was eliminating the forward-looking research while maintaining all the development and optimization testing.
In order to best live in a lean, proactive environment, building a strong fundamental understanding of your consumers, their values, and their behaviors is essential. It’s the foundation that allows you to be intuitive and to move quickly.
Jono says
January 17, 2012 at 12:22 amOh but don’t forget about the amount of time us FMCGers spend testing our testing techniques!
Marcos - Brazil says
January 17, 2012 at 6:07 amFMG companies have a huge problem in their hands: lack of innovation. The products are all similar, and ultimately the battle is won with trade marketing and money to the trade. Low production costs help, too.
Too much research is just a way to justify marketing jobs and salaries ! It is hard to face the reality that one’s work consists mostly of filling reports to HQ and doing presentations for your boss.
After many years working in such companies, I hope I will never return.
tomfishburne says
January 18, 2012 at 6:59 amGreat commentary from everyone this week. Many thanks. I received emails from a number of my old market research colleagues too. One of them reminded me that it’s a poor craftsman who blames his tools. I think that’s true. It’s not the tests themselves. It’s how they are often applied.
This week’s cartoon goes to Bill Carlson. He reminds us of the risk/reward balance in deciding the right level of market research. Too little research is a problem too. Or research that isn’t applied well, which I tried to parody in this cartoon 5 years ago: https://marketoonist.com/2007/06/a-surefire-hit.html
I once worked in an environment that rebelled against any form of market research. It was a disaster. The pendulum was too far the other direction.
What I like about an “always in Beta” mindset is that the learning comes hand in hand with doing.
-Tom
Roberto Estreitinho says
January 18, 2012 at 3:38 pm“Launch, then iterate” has got to be one of the best pieces of advice regarding this. The more you research the more you’re looking for the average because you’re following a certain pattern. Don’t get me wrong — research is quite important! But knowing when to stop and when to act is vital.