Portfolio case study & pitch
Builds on Topic 10.
What you'll produce
A one-page portfolio case study that compresses your five prior Support Copilot deliverables — prompt spec, eval, risk register, PRD, and the RAG build — into a single, recruiter-readable narrative, plus a 90-second spoken pitch script you can deliver from memory in an interview. The case study is the artifact a hiring manager actually opens first: it's the link on your resume and the thing they skim in 60 seconds before deciding whether you're worth a call. For an AI-adjacent candidate this is the highest-leverage page you own, because it proves the exact thing Topic 10 says employers most want from a non-traditional hire — that you can put AI in front of real users and know where it must not act alone. Done well, it tells one coherent story (problem → built → evaluated → de-risked → impact) and lands the domain-expertise + AI-judgment frame that separates you from someone who has only watched prompt-engineering videos. The pitch script turns that same story into something you can say out loud, calmly, in the first five minutes of a screen.
Instructions
- Gather your five artifacts and pull the load-bearing numbers. Open Deliverables 1–5 and extract the specifics that make the story credible: the baseline (14-hour reply time, doubled ticket volume), your eval scores (the pre-fix pass rate — 1 of 7 drafts in D2 — and the dimensions that scored worst), your top failure pattern (e.g. "invents fee amounts"), the guardrail that kills that pattern, your "good enough" quality bar from the PRD (D4's ≥95%-accuracy release gate), and the result of the RAG build. Use the actual numbers from your deliverables — don't round, soften, or invent a cleaner-sounding figure. The fastest way to lose a sharp hiring manager is a headline number they can't find when they click into the work; a case study whose figures trace back to the source beats one with adjectives every time.
- Write a one-line hook (the headline). State the feature and the tension in a single sentence a busy reader can grasp instantly — name the domain (fintech support) and the central risk (wrong answers about money). This is the first thing the hiring manager reads; make it earn the next ten seconds.
- Write the Problem section (2–4 sentences). Quantify the pain (volume, reply time, the cost of getting a money answer wrong) and state the vague brief you were handed ("use AI to fix this"). Establish that the hard part was judgment, not typing a prompt.
- Write the What I built section. Describe the Support Copilot in one or two sentences, then list the five artifacts as a build arc — one line each, each naming the skill it proves. This is where you show range: prompt engineering → evaluation → risk → spec → build.
- Write the How I evaluated and de-risked it section. This is the section that separates you from prompt-tutorial graduates. Name your rubric dimensions, give the headline score honestly (including what failed), name the #1 failure pattern, and state the concrete guardrail + human-in-the-loop decision that contains it. Honesty about a real failure is a hiring signal, not a weakness — say what broke.
- Write the Impact section. Tie it to a measurable outcome and the quality bar — and pull both straight from your deliverables rather than inventing a round number. Your honest pre-fix evidence is D2's 1-of-7 pass rate (14%), and your quality bar is D4's ≥95%-accuracy release gate (with D2's re-run target of ≥90% pass and zero Safety/Factuality failures). If the build is a prototype, frame the impact as the shape of the change the eval proves — "the grounding-plus-guardrail fix moves the dangerous failures from caught-in-review to caught-by-the-system, so agents review-and-send instead of write-from-scratch" — rather than claiming a production acceptance percentage you don't have. A specific, defensible "before" number you can stand behind beats an impressive "after" number a reader can't verify.
- Write the Domain + AI framing section (your differentiator). One short paragraph in your own voice: why your background + AI literacy makes you the right person to own a feature like this. This is the Topic 10 thesis made personal.
- Add a short links/artifacts list so a reader can click into the underlying work (the five deliverables), and a one-line tools/concepts tag (LLM, RAG, eval rubric, human-in-the-loop) so the keywords a recruiter scans for are present.
- Write the 90-second pitch script (~210–250 words spoken). Structure it: hook (who you are + the project) → problem → what you did → the one impressive specific (your best failure-catch + fix) → impact → why-you. Write it to be said, not read — short sentences, no jargon you can't define if interrupted. Use the same numbers as your written case study, said in words ("one of seven drafts," "ninety-five percent accuracy") — a spoken figure that contradicts your page is the fastest way to look like you don't know your own work. Time yourself reading it aloud; trim to land under 90 seconds at a calm pace.
- Pressure-test for coherence. Re-read and confirm the thread is visible: the eval's failure pattern drives the risk register's mitigation, which drives the PRD's acceptance criteria. If a hiring manager can't trace that line, tighten it — coherence across the six deliverables is exactly what graders and employers reward.
Worked example
(Filled in for the FjordPay Support Copilot, drawing the numbers from Deliverables 1–5.)
Support Copilot — drafting safe billing replies for FjordPay agents
One-line hook: Took a vague "use AI to fix support" brief at a fintech and shipped a RAG-grounded reply-drafting Copilot that's fast where it's safe and silent where it isn't — with the eval and guardrails to prove which is which.
Problem. After a product launch, FjordPay's support volume doubled and median first-reply time hit 14 hours. Leadership said "use AI to fix this" but was rightly nervous: these are customers asking about their money — a single confidently-wrong answer about a fee or a refund window is a compliance and trust incident, not a typo. The real problem wasn't "can AI write a reply" (obviously yes) — it was "where can AI be trusted to draft, and where must a human stay in control."
What I built — five artifacts, one feature.
- Prompt spec & iteration log — a production-style prompt (system role, task, constraints, output format, 2 few-shot examples) hardened over 3 iteration rounds; the key fix was a hard rule: never state a fee, rate, or timeframe not present in the provided knowledge.
- AI output evaluation — a 6-dimension rubric (accuracy, relevance, completeness, tone, safety, instruction-following) scored 1–3 across 7 sample tickets, with Safety and Factuality set as gating dimensions.
- Risk register & human-in-the-loop plan — hallucination, bias, PII, over-reliance, and misuse, each with likelihood/impact, a mitigation, and a human-review decision.
- AI feature PRD — written for AI's probabilistic nature: a measurable "good enough" bar, design-for-failure fallbacks, and an explicit "AI is wrong/unsure" acceptance criterion.
- Hands-on build (RAG) — implemented the prompt in Claude, grounded on a 6-snippet FjordPay knowledge base, and re-ran it on the same 7 real-style tickets so the before/after is measured on one fixed set.
How I evaluated and de-risked it. I scored 7 drafts against my rubric, with Safety and Factuality as gating dimensions — a single 1 there fails the whole draft no matter how polished it reads. The honest pre-fix result was brutal and exactly the point: only 1 of 7 drafts passed the gate (14%), with the Factuality and Safety floors both bottoming out at 1. The signature failure: on a "how much does it cost to send $400 instantly?" ticket, the model invented a "flat $25 fee" — when the real policy is 1.5%, capped at $15, so the true answer was $6. Fluent, confident, and wrong about a customer's money, and the kind of draft a skimming agent would send. That single failure drove the rest of the project: my risk register rates hallucinated money facts High likelihood / High impact, and the mitigation is concrete, not "be careful" — the Copilot may only state a fee, rate, or timeframe that appears verbatim in the retrieved snippet; if the answer isn't in the knowledge base, it must say it's not certain and route to a human rather than guess. That guardrail became the PRD's "AI is wrong/unsure" acceptance criterion (no draft is shown when there's no grounded source match), and in the grounded rebuild the same instant-transfer ticket computed the fee from the snippet correctly — 1.5% of $400 = $6, under the $15 cap. The human-in-the-loop decision: the Copilot never auto-sends — it drafts, an agent reviews and sends, and any reply containing a number, a refund, or account-closure language is a mandatory-review path. Customers see an AI-assisted disclosure.
Impact. The eval gave me a defensible before-and-after instead of a vibe. Pre-fix, the Copilot passed 1 of 7 drafts (14%) against a gate that treats any invented money fact as an automatic fail; grounding plus the verbatim-number guardrail moved the dangerous failures from caught-in-review to caught-by-the-system — in the rebuild, no unsupported fee amount survived the guardrail on that fixed 7-ticket set. The PRD then turns that evidence into the release gate engineering has to clear before scope expands: ≥95% of drafts factuality-clean with zero wrong numbers, validated on a frozen 40-ticket set, no auto-send ever. Projected operationally, agents shift from writing replies to reviewing them — the lever most likely to pull that 14-hour reply time down — while the escalation rule keeps the highest-risk questions with a human. (Prototype-stage and stated as such: the before/after is measured on 7 tickets, and the ≥95% bar is the gate the production build must hit, not a result I'm claiming yet.)
Domain + AI framing. I come from [customer-support operations] — I've lived the queue, so I know which billing questions are routine and which are landmines, and I know that in fintech a confident wrong answer about money is worse than a slow right one. Pairing that judgment with hands-on AI literacy — prompt engineering, systematic evaluation, RAG, and a real human-in-the-loop design — is exactly what owning an AI feature like this takes. I'm not here to make the AI sound impressive; I'm here to know where it must not act alone.
Artifacts: Prompt Spec · AI Output Evaluation · Risk Register & HITL Plan · AI Feature PRD · RAG Build Write-up (linked) Tools & concepts: LLM (Claude) · RAG grounding · eval rubric · hallucination guardrails · human-in-the-loop · AI disclosure
90-second pitch script (spoken — ~245 words)
(Read aloud and timed at ~88 seconds at a brisk-but-calm interview pace.)
"I'm a [support-operations] person who moved into AI-adjacent work, and the project I'm proudest of is a Support Copilot I designed end-to-end for a fintech called FjordPay.
Their problem was simple to state and scary to solve: after a launch, support volume doubled, replies were taking fourteen hours, and leadership said 'just use AI.' But these are customers asking about their money — so a confidently wrong answer isn't a typo, it's a trust incident.
So I didn't just write a prompt. I built the whole stack an AI feature needs: a hardened prompt, a six-dimension evaluation rubric, a risk register, a PRD, and a working prototype grounded with RAG on FjordPay's knowledge base.
The moment I'm proudest of is a failure I caught. In testing, a customer asked the cost to send four hundred dollars instantly, and the model invented a flat twenty-five-dollar fee — confident and made up; the real fee was about six. That shaped everything: I added a hard rule that the Copilot can only state a number that's actually in the source, and otherwise it says so and routes to a human.
The honest before-and-after: against a gate that fails any made-up money fact, only one of my seven drafts passed at first; after grounding it, no invented fee got through. So the bar in my spec is ninety-five percent accuracy with zero wrong numbers, and a human always presses send. That's the kind of judgment I'd bring to your AI features."
Rubric
The app's AI scores the learner's submission against these criteria and gives feedback. Levels: Needs work (1) / Solid (2) / Excellent (3). Passing = every criterion at Solid or above.
- Coherent narrative arc with a traceable thread — 1: a list of artifacts with no through-line, or figures that contradict the underlying work · 2: follows problem → built → evaluated → impact · 3: one tight story where the same failure (the D2 instant-transfer draft) and the same numbers (D2's pass rate, D4's quality bar) carry through the eval, the risk mitigation, the PRD criterion, and the rebuild — a reader who clicks into any of the five prior deliverables finds the figure exactly as cited.
- Concrete, honest evidence that traces to the source — 1: vague adjectives ("worked well," "made it safe"), or numbers that don't appear in any prior deliverable · 2: real numbers from the deliverables (14-hour reply time, the D2 pass rate, the D4 quality bar) · 3: load-bearing specifics including a named failure tied to an actual eval draft (e.g. the invented "$25" flat fee on the instant-transfer ticket from D2 Draft 1, where the true capped fee was $15) and the exact guardrail that contains it — every figure (pass rate, quality bar) reconciles with the deliverable it cites, and prototype-vs-production claims are stated honestly.
- De-risking & human-in-the-loop is front and center — 1: risk/HITL absent or an afterthought · 2: mentions a mitigation and where a human reviews · 3: a specific guardrail ("only state a number present in the retrieved snippet; otherwise refuse and escalate") plus a clear no-auto-send / mandatory-review decision and the AI-disclosure policy — the signal these roles hire for.
- Domain + AI framing — 1: generic "I'm passionate about AI" · 2: names the domain and pairs it with AI literacy · 3: makes the Topic 10 thesis personal and credible — why this specific background + hands-on AI skill makes them the right owner of a money-sensitive AI feature.
- 90-second pitch is deliverable out loud — 1: a written essay that can't be spoken in 90s, or jargon-stuffed · 2: hits hook → problem → what I did → impact in roughly the right length · 3: tight spoken English (short sentences, a memorable failure-and-fix beat), genuinely lands under 90 seconds, and ends on a why-you line a hiring manager remembers.
- Recruiter-readable & self-contained — 1: needs the reader to already know the project · 2: one page, scannable, links to the artifacts · 3: a hiring manager grasps the whole story in 60 seconds, the right keywords (LLM, RAG, eval, human-in-the-loop) are present for screening, and every artifact is linked for a deeper look.