(Of note: updates to this blog are somewhat sporadic these days. Most of my spare writing time goes to my weekly web3 newsletter Block & Mortar or the occasional O’Reilly Radar piece on the intersection of data, marketplaces, and risk.)
For a while, my tagline on Twitter was “no vanity metrics.” And it’d still be there, if Twitter gave us more room to write a bio. But I digress …
“No vanity metrics” was my terse reminder: despite all we hear about the power of data analysis, it’s only valuable when we apply it to what matters. You don’t want to boastfully present some statistic (“we tripled page views last month!”) unless you can then tie that back to your business model (“we monetize page views”).
That’s probably why a colleague recently introduced me to Lloyd Tabb’s “I’m Sorry, But Those Are Vanity Metrics.” I highly recommend you read it. Tabb covers a lot of useful detail – consider his distinction between “vanity metrics” and “clarity metrics” – and hits on two points that are very important to me. And by that, I mean, two points I’ve ranted about before:
- The power of BI
- The value in data folks knowing the business domain
Why is BI so powerful? So long as your data is clean and correct, everything that comes out of BI is a fact. (I cover this in more detail in Question Marks and Periods in the World of Data.) “How many widgets did we sell last year? Broken down by month? And then by store?”
For a company that’s never done BI – or, if they’ve not done it well – getting access to this kind of knowledge can move mountains. These facts form a solid foundation on which to make decisions. “OK, based on these figures and our budget, we need to close our Union Square location.”
Why is it so important for data scientists and ML engineers to know the business domain? Without that knowledge, these very intelligent, highly-skilled people become passive members of your company. They’ll only provide the analyses that you specifically request, instead of proactively cooking up new and useful ideas to share with you.
For the people who ask me: “Why are you touting the benefits of BI? Aren’t you an ML/AI guy?” Sort of? Some of my services involve ML/AI, sure. But overall I’m interested in applying technology to solving problems. That means helping businesses move the needle, whatever the underlying toolset.
And, besides, you can’t do ML/AI until you’ve done BI. I’ve been saying that for years.
(“We’re doing AI” can be another vanity play, but I’ll save that rant for later …)