As marketing evolves, one of the key shifts is the ability to test everything, all the time. Compared to my earliest marketing jobs in the late 90s, the tools of experimentation have been democratized and the cycle time between experiments have disappeared. We can get instant learning on just about any attribute we care to test on just about any aspect of marketing.
This testing mindset has the potential of taking out some of the guesswork out of marketing and brands are increasingly adopting a mindset to continually test and learn.
Yet this explosion in what we can test can lead to its own kind of analysis paralysis. Google couldn’t decide between two blues in a design, so they famously tested 41 shades between each blue to see which one performed better.
This data-driven environment led Google’s visual design leader, Douglas Bowman, to quit, penning an eloquent goodbye letter on the way out the door. He wrote,
“Data eventually becomes a crutch for every decision, paralyzing the company and preventing it from making any daring design decisions … I’ve grown tired of debating such minuscule design decisions. There are more exciting design problems in this world to tackle.”
One of my favorite brands is irreverent clothing company, Betabrand. They embody an “always be testing” mindset (symbolized by the word “beta” in their name), yet they also make constant “daring design decisions”.
Betabrand once parodied the absurd lengths testing can be taken, by running tests on the effect of beard length and style on click-through-rates in ads (the beekeeper look won out).
I think we all have to find the right balance of “daring decisions” and testing. A testing environment includes what we decide not to test. It also includes how we decide to frame the data of what we do test. Being data-driven doesn’t have to being data blinded.
Here are a few Halloween cartoons I’ve drawn over the years:
“Designing By Committee” October 2017
“Banner Ads” November 2015
“Trick or Treat” October 2008
Hutch says
Thank you, Tom, for your great insights and equally great cartoons. You are my marketing hero! LOL
Ronny Kohavi says
Many people cite Doug Bowman’s example as analysis-paralysis, but the reality was very different. Testing color schemes was wildly beneficial at both Google and Microsoft/Bing.
See https://www.quora.com/unanswered/How-much-did-Google-increase-conversions-after-its-41-shades-of-blue-test