(Photo by Frida Lannerström on Unsplash)

Your company's genAI adoption, like any other business decision, represents a risk/reward tradeoff.

It's too easy to focus on the potential rewards (business process automation) while ignoring the risks (loss of revenue and reputation). The harsh reality is that they are a package deal. And you're responsible for the downside risks even if you don't acknowledge them.

I've assembled this primer for executives and investors to shed more light on what drives downside risk – that is, what creates exposures – in genAI adoption. This can serve as your guide to evaluate projects and surface potential problems in your AI transformation.

(As a bonus: most of what I say here applies just as well to ML/AI. The technologies are quite similar, if not siblings.)

Top risk drivers

Lack of Planning: Without a plan, there's no real direction and no accountability. Just unrealistic goals that spiral out into various problems. This is the largest driver of risk when it comes to AI adoption, frankly. It's the parent of several items on this list.

This driver is most often rooted in a lack of AI literacy – an understanding of what AI really can and cannot do, in the context of a business – and manifests as executives going from the gut, refusing to speak with AI experts, or getting their information from questionable sources like vendors' marketing materials. Without AI literacy, executives may imagine rewards and then push the company into risky territory in the hopes of achieving them.

Failure to plan often takes the shape of Forced AI Adoption. This looks like a plan but it's really just an order from on high with no additional guidance. Since employees are throwing AI at everything for the sake of job security, they're not really thinking about the longer-term impact of their actions. They inadvertently create new risk exposures and waste money but surface few if any meaningful use cases. It's the business equivalent of introducing an invasive species to an ecosystem.

How to handle it: If the leadership team develops AI literacy, and from there develops a solid AI strategy, they'll head off most problems.

Model Error: Even if an AI system is right most of the time, it will still do the wrong thing some percentage of the time. The resultant problems can range from small annoyances which eat away at customer patience or confidence in your company, to large problems that capture media attention and legal action.

How to handle it: Be mindful of how you construct processes around genAI. In particular, design for fault tolerance and avoid sensitive use cases.

Insufficient Oversight: One compounding factor of Model Error is that AI-based automation is often buried deep inside a business process, with insufficient (or completely lacking in) human oversight. This is further compounded by AI systems operating at machine speeds. The combined effect is that problems may spread far and wide without anyone noticing until there's widespread damage.

As an example, I coined the term "Rogue Operator Threat" (ROT) to describe how an agentic AI system may cause damage similar to that of a rogue trader in a bank.

How to handle it: Design processes with observability in mind. That includes monitors, logs,and the ability for people to both inspect an AI bot's output as well as override any decisions it makes.

Model Economics: Operating an AI system will incur costs, whether you're running homegrown ML/AI models, self-hosting LLMs, or calling out to an external provider's LLM. Self-hosting requires up-front hardware spend and capacity planning; token spend for that external LLM will eat into your budget. In some cases you may wind up with an outsized LLM bill before your business has taken off.

How to handle it: The "right" choice depends on a variety of factors, including your business needs and the model's capabilities. It's best to do your homework up-front to avoid surprises down the road.

Unintentional misuse: When companies are unaware of model costs or capabilities, they may find use cases for which genAI does a passable job … but it's not the best tool for the job. LLM token spend cuts into profits while fixed rules (implemented in software), ML/AI, or a manual approach would be faster, cheaper, and/or more robust.

How to handle it: Thoroughly evaluate any proposed genAI use cases and compare them to alternatives.

Intentional Misuse (internal): AI is not always the best fit for the job. When companies force it into place (perhaps due to external pressure) they expose themselves to a high rate of Model Error, which will compound as those errors propagate through downstream consumers of the model's outputs – customers, business users, and so on. This risk driver usually stems from doing AI for the sake of AI, which in turn stems from Lack of Planning – especially the Forced AI Adoption kind.

How to handle it: Choose use cases for which AI is suitable. And since the definition of "suitable" will rely on several factors, you'd do well to loop in key parties to review use case plans.

Intentional misuse (external): Failure to secure your AI systems can lead joyriders to collect trophies (bearing your company logo, no less) and professional hackers to benefit (spending your tokens, turning your models against you, committing crimes for which you'll be blamed).

How to handle it: As with any other technology effort, security cannot be an afterthought. You must identity and address security concerns from the start, then regularly test your systems (e.g., red-team them) to stay ahead of bad actors. On the other side, develop procedures for tracking model misuse and for reporting incidents.

Completely Avoiding AI: Some execs are so fed up with the genAI hype, or they've seen too many scary headlines about genAI failures, so they've decided to pass on it altogether. While it's wise to skip a technology that won't help you, it's foolish to skip it without first doing your homework. You might be leaving opportunities on the table. This isn't about doing genAI for the sake of genAI. Nor is it about fear of "being left behind." This is about sitting down and understanding where genAI may help your business.

How to handle it: Work with a professional – someone who will take the time to understand your business model, your challenges, and your goals, then align that with genAI's capabilities – to explore what opportunities genAI may hold for you. From there you can devise a plan of adoption if you find something of value, or skip it if you don't. At least then you'll know for sure that genAI wasn't a fit.

Also look out for …

The risk drivers mentioned above feed into existing risk categories.

Operational Risk: This is a very broad category that covers risks created by people, processes, systems, and external events. genAI hits on all of them: it's meant to either take the place of people or to assist them; it's plugged into business processes; it runs on and/or connects to systems; and it's subject to external events, including everything that can happen with an upstream provider.

How to handle it: There's no way to completely eliminate operational risk. Your best bet is to surface potential problems in advance, establish various controls or safeguards to reduce the chance of occurrence, and continually review your systems for new exposures. You'll also need to develop clear oversight, both manual and automatic, over your systems.

Platform and third-party risk: While most ML/AI systems are home-grown and therefore under your control, genAI is most often outsourced to third parties. That can lead to data privacy concerns (your data and/or prompts being used for training) and operational issues for your business (if the provider goes under). I wrote about these issues in 2023, in "Risk management for AI chatbots".

Since then, the major genAI players face two additional downside exposures: competition from self-hostable, open-weight models on one side (which may take a bite out of their revenue) and massive financial debt on the other (issuing bonds and taking out loans for datacenter projects that are of questionable value). Combined, these increase the chances that a provider might suddenly change course or shut down.

How to handle it: Mitigation strategies aren't so clear-cut in this case; it's mostly a matter of thinking it through and understanding the road ahead. Two years ago you'd have chosen between "use an external LLM provider" or "don't use genAI at all." Self-hosting your LLMs has become a third option, but it may still not be the best one for your situation. An external provider may offer faster time to market and fewer technical headaches for you, in exchange for putting your business at the mercy of that provider and your network connectivity to their systems.

This is a technical matter in nature but will require leadership to make the ultimate decision.

Regulatory Risk: genAI has been around for four years as of this writing, and ML/AI for about twenty. That's an eternity in tech terms but very early days as far as rules and regulations. I'd go as far as to say that most of genAI is in its Wild West period right now. You could find yourself in the thorny situation of having built a system that runs but also runs afoul of local law. Especially if you and your end-users sit in different jurisdictions.

How to handle it: You'd do well to run any product plans and vendor TOS policies past your legal counsel. And do so early. It's better to find out there's a problem before you've launched.

Reputation Risk: AI failures can bring you unwanted public attention, especially if those failures stem from unwanted regulatory attention.

One under-appreciated form of negative PR is when your AI system works too well – this can lead to job cuts both inside and outside your company, or a feeling of creepiness among end-users. Even if you survive in the court of law, the court of public opinion is a different story.

How to handle it: Think through how your system will impact people, both when it's successful and also when it fails.

Smooth sailing

Companies are riding high on the promise of genAI's transformative power. They often forget that such transformation can go both ways. If you're unimaginative in surfacing downside risk exposures, or lax in handling those that you find, you may find yourself having to improvise your way through a disaster.

Your best bet is to surface risk exposures early and often, then handle them accordingly. That will help you to get the most out of your AI transformation while avoiding the painful parts.

Resources

In "Risk management for AI chatbots" (originally shared here, later reposted to O'Reilly Radar) I explored the threat landscape of the then-new genAI tools. That landscape has grown since I published that piece in 2023 but it's still a solid introduction to the risks and mitigation strategies.

In 2026 I published the AI Risk Weather Report (part 1, part 2) – a broad, comprehensive look at the upside opportunities and downside risks of the AI space.

I've also written shorter, focused pieces on risks like the alleged "ten percent chance of AI-driven extinction" and what I call Rogue Operator Threat (ROT) – a genAI agent's similarity to a rogue trader in a bank.