What you see here is the last week’s worth of links and quips I have shared on LinkedIn, from Monday through Sunday.
For now I’ll post the notes as they appeared on LinkedIn, including hashtags and sentence fragments. Over time I might expand on these thoughts as they land here on my blog.
2024/03/04: Can your models handle one more?
I wonder: how many data analyses and AI models were thrown off by Leap Day this year? And how does this compare to gaining/losing an hour around the Daylight Saving Time shifts?
“What Companies Need to Know When Accounting for Leap Day” (WSJ)
2024/03/05: Understanding the random
(Photo by John Gibbons on Unsplash )
When I talk to execs and product owners about AI, the biggest shock – and the strongest moment of enlightenment – hits when they understand just how much randomness is involved. A model’s path to that prediction or generated document is not always so straightforward and reliable.
Once they see that, the rest of these key points click:
1/ You can influence a model’s output (say, through careful choices in training data) but you can’t control it.
2/ When it comes to genAI, a model may take several probabilistic steps on the way to writing a single sentence. (“A is likely followed by B, which in some cases is followed by C but other times by D …”) That’s why we genAI bots write can documents that make grammatical sense but are factually inaccurate.
3/ If you absolutely need a deterministic system, go with software. (To be clear: software isn’t 100% deterministic, either. There’s still plenty of room for failures and corner cases, especially if your app has become a complex system as you’ve wired together different APIs and distributed calls. But even with that, it’s still more reliable than AI.)
This knowledge goes a long way in helping them sort out where – or even, “whether” – to apply AI in their business.
2024/03/06: Executive coaching slot
(Photo by Tabea Schimpf on Unsplash)
I have an exec coaching slot opening up in the coming weeks.
Are you in a leadership role for product (CPO, Head of Product) or tech (CTO, Dir of Eng)? Would you like to improve your company’s AI efforts?
Now’s the time to get started!
This is one-on-one coaching with me, an experienced AI practitioner. I know what it takes for a company to go from zero to AI and beyond.
Whether you’re taking your first steps want to take your existing AI work to the next level, this can help you:
- Develop your knowledge of what/when/how/why/why-not in AI
- Get guidance and answers specific to your situation
- Gain access to a trusted sounding board for your company’s AI challenges
- Reduce costly risk and dysfunction in your AI efforts
2024/03/07: Remember to follow up
It’s important to proactively test your AI models (especially genAI chatbots). Red-teaming exercises are a key element of that testing.
I’ve mentioned this before. But I did not point out that, once you’ve red-teamed a system, you’re supposed to … y’know … address the problems that surfaced.
You folks here on LinkedIn, you sure know this. But it seems, not everyone knows this:
“Microsoft accused of selling AI tool that spews violent, sexual images to kids” (Ars Technica)
[Microsoft engineer Shane Jones] told CNBC that he repeatedly warned Microsoft of the alarming content he was seeing while volunteering in red-teaming efforts to test the tool’s vulnerabilities. Microsoft failed to take the tool down or implement safeguards in response, Jones said, or even post disclosures to change the product’s rating to mature in the Android store.
(For more thoughts on handling the pointy bits in AI, check out my piece “Risk Management for AI Chatbots” and/or sign up for my newsletter, Complex Machinery)
2024/03/08: It’s not ready for the advanced class
Some publishers are looking into AI-generated translations for podcasts.
Not only do I work in AI, but I’ve also studied several languages. So I can assure you:
AI translation is no match for the casual, off-the-cuff speech of your typical podcast.
A news broadcast? Maybe. News tends to use clear, scripted speech and simple terms. (Which, by the by, is why watching the news is a helpful language-learning step on the way to watching TV shows or movies.) But a podcast? No.
You don’t have to take my word on this. The execs testing this technology will tell you the same thing! And yet, they’re … moving ahead anyway?
“Why podcast companies are investing in AI-generated podcast translations despite questionable quality” (Digiday)
So why are publishers testing this, if the quality isn’t up to snuff? Because it’s a cost-effective way to expand podcast shows internationally and into non-English language markets, execs said.
“It is uneconomic to do it manually – because there’s so many episodes of so many podcasts, there’s so many languages – and AI is really the solution,” [iHeartMedia CEO Bob] Pittman said during iHeartMedia’s recent earnings call.
2024/03/09: The models just repeat what they’ve seen
Over the years I’ve developed some plain-language techniques for explaining ML/AI to executives.
One of my favorites uses recruiting as the example use case. I show how a model will repeat any patterns it picked up during training, which is why you have to carefully curate your training data.
No idea why this comes to mind now. No idea! But a Bloomberg team has surfaced some rather damning examples of bias in GPT-3.5. Seems it’s ill-equipped for recruiting work.
“OpenAI GPT Sorts Resume Names With Racial Bias, Test Shows” (Bloomberg)