Marketing is increasingly data-driven. But the data we choose matters. It’s tempting to cherry-pick the data that makes us look or feel good. Or the easiest data to persuade an audience to buy into whatever we want them to believe.
This is particularly the case in digital marketing, where over-simplified numbers like Likes or YouTube views are frequently used as proxies for success. There are many “vanity metrics” commonly reported in marketing that are easily gamed and have nothing to do with whether a campaign actually made a real impact.
Last year, an ad agency in Minneapolis called Solve ran an experiment where they uploaded a 4-minute completely blank video to YouTube (no sound, movement, or anything but a blank screen). They then managed to get over 100,000 YouTube views by simply buying viewership for 1.4 cents a view. In addition to the view count, they were able to cherry-pick a few engagement metrics (1-in-5 saw it all the way through, higher than average click-throughts, etc.) to make it seem compelling. But all along it was just a blank video.
Solve’s Neil James concluded:
“You could swap in any video, any brand, any objective, and we’d be able to spin a dazzling PowerPoint fable with elegant charts, eye-popping numbers, and poetic proclamations. Oh, the engagement! Oh, the dwell! Oh, the buzz we achieved! But despite the narrative, despite the numbers, we accomplished nothing. Absolutely nothing.”
Metrics only matter if they’re actionable. I found an interesting post from Lean Startup author Eric Ries in 2009 on the distinction between “actionable metrics” and “vanity metrics” in startup life. Eric shares three takeaways that I think are relevant for any form of marketing:
“1. Measure what matters. It’s tempting to think that, because some metrics is good, more metrics is better. That’s why vendors routinely list the thousands of reports they are capable of generating as a feature. The truth is, the key to actionable metrics is having as few as possible. Detailed reports are useful when we’ve diagnosed a problem and are looking for clues as to what’s gone wrong. But where does that diagnosis come from in the first place? Actionable metrics help us realize we have a problem and point us in the right direction to start solving it.
“2. Metrics are people, too. Great metrics tools allow us to audit their accuracy by tracing reports back to the individual people who generated their data. This improves accuracy, but its more important effect is that it lets us use the same customers for in-depth qualitative research. Not sure what the numbers mean? Get the customers on the phone and ask them.
“3. Measure the Macro. Lastly, even when we’re split testing the impact of a minor change, like a wording or a new button, it’s important not to get distracted by intermediate metrics like the click-through rate of the button itself. We don’t care about click-through rates, we only care about the customer behaviors that lead to something useful, whether purchase, retention for advertising CPM, or some other measurable “success” particular to your business model.”
I would love to hear your thoughts on how to navigate actionable metrics versus vanity metrics.