You can't hide behind a beta label
2025-11-05

This is not the sort of thing you want people to say about your product:

Gemini for Home, in its current form, is plagued by Al hallucinations, constantly prone to mislabeling people, colors, activities, objects, and animals, and that makes it fundamentally untrustworthy. And the less you trust your security device, the less you rely on it, even when you absolutely should.

And yet, it's precisely what someone has said about Google's Nest cameras:

People rely on cameras to be unbiased observers, but the addition of Al that interprets what it sees introduces bias - and if it's inaccurate, it's no longer useful. Gemini for Home, in its current form, is plagued by Al hallucinations, constantly prone to mislabeling people, colors, activities, objects, and animals, and that makes it fundamentally untrustworthy. And the less you trust your security device, the less you rely on it, even when you absolutely should.

We weren't the only ones having experiences like this. (As I write this, my Nest camera just labeled my 6-foot-3 husband as a child and claimed that his armful of laundry was a baby.)

(Source: New York Times / The Wirecutter, "Google Added AI to Its Nest Cameras, and All I Got Were Hallucinations of Rodents in My House")

AI-based products are under a lot of pressure. The companies selling them paint rosy pictures that, according to the people buying them, don't always match reality. That creates a compounding effect in which consumers are wary of anything related to AI, and more critical of the problems that crop up.

What are the takeaway lessons?

1/ If you're selling anything AI-related, you'd do well to double down on your testing. AI's probabilistic nature requires more robust testing than a typical software/app project.

2/ Maybe, just maybe, shift your marketing to focus less on the AI and more on the new functionality. Buyers are rarely excited about the technology behind their purchases. They just want something that works.

3/ Given item 2, don't hide behind the "beta" label if the product is available to the general public. Again, people expect things to work. It's not enough for your product to function well in a very limited, closed lab environment. It needs to work out in the real world, where real (paying!) customers actually use it.

Creating data out of thin air

Thoughts on synthetic data

It goes down as well as up

The impact genAI companies are having on the stock market