I recently stumbled across Goodhart’s Law:
“When a measure becomes a target, it ceases to be a good measure.”
The British economist Charles Goodhart originally framed this in a 1975 article on monetary policy, but it’s an important watch-out for business today.
When we optimize for a number instead of our true goal, we risk whatever we were trying to improve in the first place. Chasing metrics can bring unintended consequences or steer businesses in the wrong direction altogether.
Goodhart’s Law dates back 50 years, but it’s particularly timely in a world of AI. An optimization algorithm is only as good as the metrics it is designed to optimize. The routes AI finds can be technically correct, yet damaging in the long run.
Customer service is a case in point. Agents and chatbots promise efficiency and quicker resolution, measured by classic service metrics like average handle time. Yet, those metrics can lead AI to give fabricated or hallucinated information or make it harder to escalate. That could negatively impact brand trust in the long run, which may not be measured at all in the moment.
As DataNumen CEO Chongwei Chen put it in a recent Entrepreneur article:
“A 20% decrease in customer response time and drop in escalations are meaningless if wrong answers are given or handoffs to human agents are systemically made difficult.”
The more we drive our businesses with metrics and dashboards, the more we need to question what those metrics and dashboards measure.
We can measure the response time and forget the value of the response.
Here are a few related cartoons I’ve drawn over the years:
