The folks at Anthropic turned some WSJ journalists loose on an AI-driven vending machine. The results were everything you'd imagine, and then some.

Beyond the laughs, this article holds four key lessons for companies adopting AI:

1/ You have to red-team the hell out of your app. It's better to surface problems in private, when you still have a chance to fix them.

2/ The red-teamers need autonomy. An external team, like the WSJ journalists, can be brutal in their testing and painfully honest in their assessment. An internal team may feel the pressure to soften their approach.

3/ Create standalone, isolated test projects. The article notes that Anthropic created the vending machine as a test of giving an AI agent authority in a business setting. Anthropic can now apply lessons from the (failed!) vending machine to its other work.

4/ Be careful with AI. Companies are eager to insert AI into critical business processes and put it in front of customers. The Anthropic/WSJ vending machine test should serve a dose of cold water on your AI plans (and in many vendors' marketing materials), stressing the need to think about the potential downside risks as well as the upside rewards.

Do you need help working through AI's risk/reward tradeoffs? Reach out.