Rather than implying a limit to the usefulness of data, I find it speaks more to the folly of substituting data for critical thinking.
You can reduce a fever by treating the underlying infection, soaking in ice water, or taking acetaminophen. No good doctor would judge a patient's health solely because a single metric, namely body temperature, was able to be moved into acceptable ranges. That doesn't mean temperature isn't extremely valuable data, and essential to decision making, but that it cannot be a substitute for understanding and solving the real problem.
I once knew of a SaaS company that had perpetually growing MRR (Monthly Recurring Revenue), great right? Except, churn was also growing. An increase in MRR was achieved by upselling a perpetually shrinking group of core customers. The core KPI of this company was MRR, and, unsurprisingly, this company does not exist anymore. Again, here is a case where we can see all that other data (churn, upselling) is very useful, as is the KPI. But the key to success or failure here is whether or not you want to really expend the effort to understand the problem or just chase a KPI.
KPIs are seductive because they make managing team's performance seem much easier: just get this number higher and you're doing good, get it lower and you're doing bad. But that's like playing a game of chess where each piece is controlled by a different person, and that person is judged solely on how many times they can get the king in check.
> No good doctor would judge a patient's health solely because a single metric, namely body temperature
That's a good example because as Strathern's formulation notes, the problem lies in make the metric the target. It would be folly to think that reducing a patient's body temperature to the normal range is sufficient for curing illness. GE's Jack Welch famously focused solely on the stock performance as a measure of success. It worked, by that measure GE was wildly successful. By almost any other measure Welch destroyed GE https://www.bnnbloomberg.ca/jack-welch-inflicted-great-damag...
You can reduce a fever by treating the underlying infection, soaking in ice water, or taking acetaminophen. No good doctor would judge a patient's health solely because a single metric, namely body temperature, was able to be moved into acceptable ranges. That doesn't mean temperature isn't extremely valuable data, and essential to decision making, but that it cannot be a substitute for understanding and solving the real problem.
I once knew of a SaaS company that had perpetually growing MRR (Monthly Recurring Revenue), great right? Except, churn was also growing. An increase in MRR was achieved by upselling a perpetually shrinking group of core customers. The core KPI of this company was MRR, and, unsurprisingly, this company does not exist anymore. Again, here is a case where we can see all that other data (churn, upselling) is very useful, as is the KPI. But the key to success or failure here is whether or not you want to really expend the effort to understand the problem or just chase a KPI.
KPIs are seductive because they make managing team's performance seem much easier: just get this number higher and you're doing good, get it lower and you're doing bad. But that's like playing a game of chess where each piece is controlled by a different person, and that person is judged solely on how many times they can get the king in check.