Interview Drills — Applied AI Scenarios (AI-PM thinking in your domain)

Open-ended interview questions for AI-adjacent roles. Each has a Framework (the structure a strong answer follows), a Model answer (a concise example), and a Rubric (what an interviewer listens for). These drills pair the AI-product thinking from Topics 8–9 — design for uncertainty, define "good enough," design for failure, use AI only where it adds real value — with your domain (the field you already know). Practice thinking aloud. The app can role-play these as mock interviews (see mock-interview.md).

The universal AI-scenario structure: Restate the goal and who it's for → name where AI genuinely adds value (its strengths: language, summarizing, generating, finding patterns) → define "good enough" as a measurable bar, not "perfect" → keep a human in the loop and design for failure (what happens when it's wrong/unsure) → name what you'd watch out for (hallucination, bias, privacy, over-reliance) → pick a success metric. Use it on almost any "how would AI help in [domain]?" question.

D1

difficulty: easy
concept: ai-product-management How would AI help in your domain — and what would you watch out for?
Framework: Name your domain and a goal → pick one workflow where AI's strengths (language, summarizing, generating, pattern-finding) genuinely help → say how it assists rather than replaces the human → name 1–2 risks you'd watch for (hallucination, bias, privacy) → state a success metric.
Model answer: "In customer support, the goal is faster, accurate resolutions. AI shines at drafting replies and summarizing long ticket threads, so I'd use it to suggest a draft answer the agent reviews before sending — not auto-send. I'd watch for hallucinated policy details, so the draft cites the source article, and I'd keep customer data out of any tool that isn't approved. Metric: time-to-first-response and the share of AI drafts agents send with light edits."
Rubric: Strong answers pick a specific workflow where AI adds real value, keep a human in control, name concrete risks (not just "AI is risky"), and give a metric. Weak answers say "AI could do everything," skip the human-in-the-loop, ignore risks, or give no measure of success.

D2

difficulty: medium
concept: ai-product-management Define "good enough" for an AI feature in your domain. How would you know it's ready to ship?
Framework: Restate that AI quality is a range, not pass/fail → name the task and who relies on it → set a measurable bar tied to the stakes (accuracy threshold, acceptable error rate, what an error costs) → say how you'd measure it (evaluation rubric/sample, Topic 5) → note the fallback for the cases below the bar.
Model answer: "For an AI that summarizes legal contracts, 'good enough' isn't 100% — it's: on a held-out sample of 200 contracts, the summary captures every material clause with zero invented terms, judged against a rubric by a reviewer. Because a missed clause is costly, the bar is high and a lawyer always reviews. I'd ship when it clears that bar on the sample and the cases it's unsure about are flagged for human review rather than passed through silently."
Rubric: Strong answers treat quality as a measurable range, tie the bar to the stakes of an error, describe how they'd evaluate it (sample + rubric), and define what happens below the bar. Weak answers say "when it's accurate" with no number, no evaluation method, and no notion that the right bar depends on cost of failure.

D3

difficulty: medium
concept: ai-product-management Design for failure: walk me through what happens when the AI is wrong or unsure in your domain.
Framework: Name the feature and the realistic failure modes (wrong answer, low confidence, no useful output) → for each, design a graceful path (human review, fallback to the old flow, "I'm not sure" instead of a confident guess) → make errors low-stakes and recoverable → make it easy for the user to flag/correct it → tie back to trust.
Model answer: "For an AI that triages incoming medical-intake forms, failure modes are: mis-categorizes urgency, or is unsure. Design: it suggests a priority with a confidence level; anything below the threshold or flagged 'urgent' always routes to a nurse — never auto-resolved. If it produces nothing useful, the form falls back to the normal manual queue, so nothing is lost. Staff can one-click correct the category, which feeds the feedback loop. The point is no single AI mistake reaches a patient unchecked."
Rubric: Strong answers enumerate concrete failure modes, design a graceful fallback for each (not just the happy path), keep errors recoverable, and add an easy correction/feedback path. Weak answers assume the AI is usually right, have no fallback, or let high-stakes errors through without a human checkpoint.

D4

difficulty: medium
concept: ai-product-management Where should we NOT use AI in your domain? Talk me through a case where AI is the wrong tool.
Framework: State the principle — use AI only where it adds real value, not as a gimmick → pick a task in your domain where AI is a poor fit (high-stakes + unverifiable, deterministic rules already work, or unacceptable error cost) → explain why (probabilistic output, no understanding guarantees, bias/privacy exposure) → say what you'd use instead → note where in the same domain AI does fit, to show judgment.
Model answer: "In lending, I would not use a black-box AI to make the final approve/deny decision: the errors are high-stakes, hard to verify, and bias in the data can produce unfair, possibly illegal outcomes. A transparent rules-based or well-audited model fits better there. Where AI does add value in lending is the low-stakes, language-heavy work around it — summarizing an applicant's submitted documents or drafting the explanation letter — always with a human signing off."
Rubric: Strong answers show AI isn't always the answer, pick a genuinely poor-fit task with a real reason (stakes/verifiability/bias), suggest the better alternative, and still identify where AI does help. Weak answers either force AI everywhere or reject it everywhere, with no reasoning about value, stakes, or fit.

D5

difficulty: hard
concept: ai-limitations A stakeholder wants to fully automate a core task in your domain with AI. How do you respond?
Framework: Acknowledge the upside (don't be reflexively negative) → clarify the goal and the cost of an error for this task → assess fit against AI's limits (hallucination, no reasoning guarantees, bias, inconsistency) → propose a staged middle path (AI assists → human approves → measure → expand automation only where evals prove "good enough") → name the metric and the guardrail that would let you safely increase autonomy.
Model answer: "I'd start with the upside — automating this could cut turnaround a lot. Then I'd ask what one wrong output costs us here. For something like auto-publishing AI-written product descriptions, a confident hallucination ships an error to customers, so I'd propose: AI drafts, a human approves at first, and we measure the edit rate and error rate. As the evals show it clears our 'good enough' bar on a category, we can let that category auto-publish with spot-checks. We earn more autonomy with evidence, not on day one."
Rubric: Strong answers balance upside and risk, reason from the cost of failure, propose a staged human-in-the-loop rollout gated by evaluation, and define what evidence would justify more automation. Weak answers either rubber-stamp full automation or flatly refuse, without staging, metrics, or a sense that autonomy should be earned by measured quality.

D6

difficulty: hard
concept: ai-limitations You're building an AI feature in your domain. What could go wrong responsibly — bias, privacy, over-reliance — and how would you mitigate it?
Framework: Restate the feature and who it affects → walk the three risk axes: bias (does it touch people/decisions? where could skewed data cause unfair outcomes?), privacy (what data goes in, is it sensitive, is the tool approved?), over-reliance (will users stop verifying?) → give a concrete mitigation for each → add transparency (disclose it's AI) and a feedback loop → close on the throughline that AI augments human judgment, it doesn't replace it.
Model answer: "For an AI that screens job applicants' résumés in HR: Bias — trained on past hires, it could penalize non-traditional backgrounds, so I'd audit outcomes across groups and use it only to surface candidates, never to auto-reject. Privacy — résumés are personal data, so they only go into an approved, contractually safe tool, never a public chatbot. Over-reliance — recruiters might trust the ranking blindly, so I'd show why a candidate was surfaced and require a human to make every call. I'd disclose AI is used, let recruiters flag bad suggestions, and frame the tool as assisting judgment, not replacing it."
Rubric: Strong answers address bias, privacy, and over-reliance specifically for the domain, give a concrete mitigation for each, add transparency and a feedback loop, and land on human-in-the-loop. Weak answers name risks generically ("there could be bias") without domain-specific mitigations, ignore privacy or over-reliance, or treat responsible AI as a box-tick rather than design choices.