How to Make AI Decisions You Can Actually Defend
Maryna Popovichenko is a Senior UX Design Consultant at SAP and research specialist in AI-powered user experiences. We are happy to welcome her as a Ampli's guest writer — Ampli's Team
At a UX event last year, someone told me about an enterprise company that built a conversational AI assistant for their customer platform. The instinct was solid. Customers needed faster, more intuitive support. But the team shipped it without ever studying how customers actually solved the problem today. Within months, they quietly pulled it.
The chatbot wasn’t the mistake. Skipping the homework was.
I keep hearing versions of this story. A team adds conversational AI to a product, the pitch sounds great in a meeting, and a few months later they’re quietly removing it because users hated it. Or worse, ignored it entirely.
The pattern is always the same. The decision happens backwards. A team starts with the technology (”we should add AI here”) and works backward toward a justification. Evidence gathering, if it happens at all, comes in late to validate what’s already been decided. The result is AI features that are technically impressive and practically useless.
I’ve spent 15 years shaping complex digital products, and this keeps repeating everywhere. The harder question isn’t “can we build this?” It’s “should we?” And for whom? Under what constraints? With what boundaries?
That company’s chatbot didn’t fail because the technology was bad. It failed because nobody stopped to ask those questions first.
That’s what this framework is about. Three moves that turn “we need AI” into a clear, defensible product decision. Let me show you how it works with a real scenario.
The Scenario
An enterprise expense platform. Employees frequently don’t understand why their expense reports get rejected. Support ticket volume is high. Frustration is real.
The proposal: add a conversational AI assistant to explain rejection reasons and guide resubmission.
Sounds reasonable. The system already has deterministic, rule-based rejection logic. There are compliance implications. And there’s organizational pressure to reduce those support tickets.
This is the kind of scenario where “let’s add AI” feels like an obvious win. Which is exactly why it’s worth slowing down.
Move 1: Strengthen the Evidence
Use AI to challenge your assumptions before you use AI in your product.
Most proposals carry embedded assumptions that nobody has examined. Before you involve AI in your product, use AI to pressure-test your own thinking.
Start without AI. Write down your clarification questions about the problem. Form a hypothesis about the root cause. Then, and only then, bring AI in to challenge your work. Ask it: what assumptions are embedded in this framing? What might we be missing? What alternative explanations exist? What data would validate or invalidate this?
This means doing the actual homework first. Find out how users complete the task today, through interviews, observations, or usage data. Talk to the people filing those expense reports. Watch them struggle with the rejection screen. Look at the support tickets to see what language they use when they’re confused. That’s your evidence base. AI can help you stress-test it, but it can’t replace it.
AI sharpens evidence. Humans make decisions. AI is excellent at surfacing blind spots. It’s terrible at deciding what matters. Keep that separation clean.
When I ran this exercise with product teams, something interesting happened. Their initial questions were broad. “Why are users confused?” After AI pushed back on their assumptions, their questions got sharper: “Are rejection reasons written in policy language rather than user language?” and “Do repeat submitters have different comprehension patterns than first-timers?” The refinement was significant. Not because AI gave them answers, but because it pressured them to think more precisely.
Move 2: Evaluate the Interface Choice
Conversational AI is an interface decision, not a feature decision.
This is the step most teams skip entirely. And like any interface choice, it has tradeoffs that depend on context.
The question isn’t “would a chatbot be cool here?” It’s: what is the user cognitively experiencing at this moment? Does conversation reduce or increase their cognitive load? What happens if the AI misinterprets a rule? And where must AI stop?
To stress-test this, assign different constraint lenses to the same scenario. In the expense platform example, the answer looks very different depending on who you’re designing for:
High compliance / legal implications — A conversational AI that paraphrases policy language could introduce dangerous ambiguity. A misinterpretation isn’t a UX annoyance. It’s a compliance risk.
Deterministic, rule-based rejection logic — Conversation may be entirely unnecessary. If the answer is binary and the rules are clear, a structured UI showing “your receipt is missing” is faster and more trustworthy than a chat exchange.
First-time employees unfamiliar with policies — Conversation might genuinely help. But only if the AI knows when to hand off to a human, and never presents interpretation as fact.
Power users submitting expenses weekly — A chatbot is friction. They don’t need explanation. They need a faster workflow.
High support ticket volume pressure — There’s organizational temptation to deploy AI as deflection. But if the AI doesn’t actually resolve the confusion, you’ve just moved the frustration from the support queue into the product.
Conversational AI is an interface choice. Evaluate it like one. The patterns tend to cluster around two insights. Conversation helps when the user’s situation is ambiguous and contextual. Structured UX is stronger when the information is deterministic and the user needs speed, not dialogue. And trust tracks with predictability. Users trust systems that behave consistently more than systems that sound friendly.
Move 3: Make the Decision Explicit
If you can’t write one sentence explaining your AI decision, you’re not ready to build.
The final move is to formalize everything into a Decision Canvas. It’s a structured artifact that forces you to commit to specifics before writing a single line of code.
The canvas has six sections:
Problem Definition — described without mentioning AI at all. If you can’t articulate the problem without referencing the solution, you don’t understand the problem yet.
Evidence — what you actually know, pressure-tested from Move 1.
Risk — what can go wrong, informed by the constraint analysis from Move 2.
Boundaries — where the AI must stop. Non-negotiable lines.
Validation — what you need to test before committing to build.
Decision Statement — a single sentence in this format:
“We will use conversational AI to ___ for ___ because ___. We will not use it for ___ due to ___. Before building, we must validate ___.”
That sentence is the whole point. It’s explicit. It’s reviewable. It’s the kind of statement that can survive a stakeholder challenge because it was built on evidence, not enthusiasm.
AI becomes intentional when decisions are explicit.
Why This Matters Now
The expense platform scenario is a good example. After working through all three moves, some teams conclude that conversational AI is the right choice, but only for a specific user segment, with hard boundaries on what the AI can and cannot say. Others conclude that the real problem was never about AI at all. It was about unclear rejection messages that could be fixed with better content design in a week.
Both are good outcomes. Both are better than shipping a chatbot and hoping for the best.
Every product team I know is facing some version of this pressure. Leadership wants AI in the product. The timeline is tight. The easiest path is to skip the homework and build. This framework doesn’t slow that down. It redirects the energy that would go into building the wrong thing into building the right thing. Or, just as valuably, into deciding not to build at all.
Pick one AI feature your team is currently building or considering. Write down the decision statement from Move 3. Fill in every blank. If you can’t, that’s not a failure. That’s the framework working. You’ve found exactly where the thinking needs to happen before the building starts.
Decisions built on evidence are decisions you can defend.
We're always looking for guest writers. If you have a topic you're passionate about, we'd love to hear from you. Send your pitch to connect@ampli.xyz and we’ll get back to you.