Nervous about meeting with your company's execs, legal counsel, or product team? Just answer these four questions.
ML/AI is often the newest function in an organization. As such, it is also the least-understood. That puts data scientists in the position of maintaining two skill sets: learning and keeping up-to-date on technical ML/AI tools, so they can build models and perform analyses; and learning communications skills, in order to speak about their work with business stakeholders and people in other departments.
Even if your company has developed an organzation-wide understanding of what ML/AI really is and what how it works -- and let's face it, this is painfully rare -- the onus is still on you, the data scientist, to translate concepts from your world to and from stakeholder-speak.
(While I don't think this is the ideal arrangement, it is the current state of affairs. So that's what I'll cover here. Stakeholders also need to learn how to speak ML/AI with their data scientists. I'll cover that in a future post.)
It may seem daunting to present your work to executives, in-house legal counsel, and product leaders when your core strength involves building models and analyzing data. There's no reason to worry. So long as you're able to answer these four questions, your presentation should go well:
It's rare that you will show a chart, model performance, or other result to the stakeholder and they will immediately understand. More often, they will pause for a moment then ask: "OK, so what am I looking at here? What is it you're trying to tell me?"
Well ... what are they looking at? You two may have the same chart or number in front of you, but you're not seeing the same thing. So what the stakeholder is asking is for you to provide additional context so that you can have a shared understanding.
This work actually begins before you enter their office or the conference room. As a first step, you should already be able to explain your work using the terminology and practices of your company's business domain. Next, develop notes that connect your work to some specific project, request, or use case. Include that contextual information as section headers in a document, or chart titles and axis labels in a data visualization. You can then point to those headers and labels as you guide the stakeholder through your findings.
Be prepared to explain what features you used, what training data you pulled or excluded, and why. In the event you have explore the techniques you used, keep the technical detail to a minimum. They're not asking you these questions as part of a technical interview; they're likely double-checking that you've covered your bases, and that you haven't strayed into any data that will prove troublesome.
(In the lending domain, for example, a company can get into serious legal trouble for using certain protected features in a model for creditworthiness. In other domains, an executive may need to demonstrate that your models don't use data from a vendor subscription that has since lapsed.)
Stakeholders live in a world of decisions. They focus on costs, outcomes, and impacts, so you need to be able to express your work in those terms. Doubly so, when the company is trying to decide between different courses of action. Here, your stakeholders will try to weigh each the relative costs and benefits of each approach.
When I say "cost," I don't just mean money. I sometimes use the acronym TERM -- time, effort, reputation, money -- to describe the wider sphere of resources a company can "spend" on a project. Even the notion of "money" has different dimensions, such as "present-day" (money we spend, now) and "possible future" (risks).
Costs are held in comparison to value, which uses many of the same dimensions. While value is typically another term for "present-day revenue," it can also involve (possible) future revenue, share price, customer goodwill, or public perception of the company.
As a data scientist, be prepared to explain things as:
The more concrete you are in your explanations -- providing specific numbers, dates, and outcomes -- the better.
Consistently explaining your work in terms of costs and benefits has an added bonus: over time, stakeholders should trust you more. They won't question your decisions as often, because they'll know that you're already thinking about these matters in the terms they deem useful. They may eventually just hand you a budget and a goal, and leave you the leeway to sort it out as you see fit.
This is a common follow-up question to "what's this worth?" but it deserves its own space.
Here, a stakeholder is asking how the company could achieve a similar outcome but through a different path. It helps to have done your homework in advance, so you're prepared to answer this question.
You could suggest using a different training dataset, different features, or a different modeling approach. Maybe you can get a similar result by outright purchasing a dataset or a service from a vendor that has more experience with the question your company wants to answer. Perhaps you could postpone this project a few months, to give a promising new open-source tool time to reach its next milestone release.
To round out that discussion, be sure to work with them to explore one particular alternative: "What is the cost of doing nothing?" What are the benefits and drawbacks of simply skipping this project altogether?
In other words: "So what?"
A stakeholder may ask you this after you've delivered your presentation or shared your idea, but it should have been the first thing on your mind as you began that analysis or modeling project.
Stakeholders care about increasing revenue, trimming losses, uncovering new avenues, and mitigating risk. By framing your work in those terms, you'll answer their "so what?" before they've even asked it. That dramatically improves your chances of getting your point across.
For example, you could tell a stakeholder that you want to delay a project for two weeks because of "data ethics." You understand the far-reaching ramifications of that term, but it may fall flat during a presentation if it's an unfamiliar concept to your audience.
It would be far more effective to explain that you need the extra time "to develop a more robust training dataset, which will improve model performance. In turn, that will reduce the company's reputational risk because the models will produce fewer incorrect predictions in a high-profile environment." Much better. You're now speaking your stakeholder's language.
Similarly, you could say that you want the company to focus on building "explainable models." Your CEO may write this off as a purely technical matter, and ask you to proceed with developing blackbox tools. You'll get a much better response if you tell them that "we need to understand how our models work under the hood, so that you can explain to the board why our AI-based system triggered those purchasing decisions."
Just about every data scientist job posting lists communication skills as a requirement. Whether for a one-off request, or as part of a formal presentation of your work, you'd do well to learn how to speak your stakeholders' language.
If you can answer these questions about your work:
then you'll always be prepared to meet with your company's executives, legal team, and product managers.
Now, for the stakeholders reading this: you're not off the hook. In a future post I'll explore communications skills for company leaders. Those will help you when interacting with your data scientists.
New Radar article: "Rebranding Data"
I've published an article on O'Reilly Radar: how many times will we rename the data field?
When "Constants" ... Aren't
Want to improve your risk assessment? Identify, then question, the constants in your world.