Prompt spec & iteration log — AI-Adjacent Roles Capstone — SkilsMVP

What you'll produce

A production-style prompt spec for the FjordPay Support Copilot — a single, complete prompt written the way an AI PM or prompt engineer hands it to a team: a named system role, an explicit task, the context/constraints the model must obey, a rigid output format, and 1–2 few-shot examples that show (not tell) the bar. Alongside it you'll keep a documented iteration log showing 3 refinement rounds (initial → diagnosed problem → revised) run across multiple test inputs, plus the verification and safety guardrails baked into the prompt. This matters because in AI-adjacent roles the prompt is the product: it's the lever that turns an unreliable next-word predictor (Topic 3) into something a support agent can trust, and the iteration log is the artifact that proves you can do the core craft of Topic 4 — write specifically, test across inputs, diagnose what's off, and refine — rather than just talk about it. A prompt that works once is a demo; a prompt with a logged iteration history and guardrails is something a company can ship.

Instructions

Lock the scope before writing a word. Write one sentence naming exactly what the Copilot does and does not do: it drafts replies for billing and account questions for a human agent to review and send — it never sends to customers, never touches anything outside billing/account, and never invents specifics. This one-liner is what every later constraint enforces.
Write the system role. Give the model a named persona and its hard boundaries ("You are FjordPay's support-draft assistant… you draft, a human sends… you only handle billing and account topics"). Keep it to 3–5 sentences. This primes voice and authority (Topic 4) and is where you put the "never act alone" rule.
Write the task. State precisely what to produce from each ticket: a draft reply, in FjordPay's voice, grounded only in the provided knowledge, plus a confidence signal. Spell out the steps the model should take (read ticket → check if it's in-scope and answerable from context → draft or escalate).
Write context/constraints. List the non-negotiables as bullets: only use facts present in the provided knowledge snippet; if the answer isn't there, escalate instead of guessing; never state a specific fee, rate, date, or account balance unless it's in the input; never give financial, tax, or legal advice; redact nothing the agent needs but never echo full card/account numbers.
Define the output format rigidly. Specify exact fields so the agent's UI can parse them and so "I'm not sure" is a first-class output, not a buried sentence. Use a fixed structure: DRAFT_REPLY, CONFIDENCE (high/medium/low), ESCALATE (yes/no + reason), SOURCES_USED. A machine-readable format is what lets the human-in-the-loop work.
Add 1–2 few-shot examples. Write one answerable ticket → ideal full output, and one unanswerable / out-of-scope ticket → an ideal escalation output. The escalation example is the most important one: it teaches the model that "I don't know, send this to a human" is a correct answer, not a failure.
Bake in the guardrails explicitly. In the prompt itself, require: refusal-to-guess on missing facts, mandatory escalation triggers (anything about refunds over a threshold, fraud, account closure, legal/complaint language), a confidence flag on every draft, and a standing instruction never to claim it is a human. Write these into the prompt — guardrails that live only in your head don't ship.
Build a test set of at least 4 inputs spanning: (a) a clean answerable billing question, (b) a question whose answer is not in the knowledge snippet (hallucination trap), (c) an out-of-scope / emotionally charged or fraud ticket, and (d) one with a slightly adversarial or ambiguous phrasing. You can't claim reliability from one input (Topic 4).
Run 3 iteration rounds and log each one. For every round record: the version, the input(s) you tested, the observed problem (quote the bad output), the diagnosis (root cause in prompt terms — missing constraint? weak format? no escalation path?), and the specific change you made. Round 1 → 2 → 3 should show the prompt visibly getting safer, not just longer.
Close with a guardrail summary and a "known residual risks" line. State what the prompt now reliably prevents and what still requires human review — honesty about limits is a senior signal (Topics 5, 7) and sets up Deliverable 2's evaluation.

Worked example

(Feature: FjordPay Support Copilot — drafts billing/account replies for human agents. Knowledge the prompt is allowed to use is a small FjordPay snippet supplied at runtime; in this example: "Standard payouts arrive in 2 business days. Instant payout fee is 1.5%. Refunds are processed within 5–7 business days. We never charge a monthly account fee on the Starter plan.")

The prompt spec (final, v3)

System role

You are FjordPay Draft Assistant, an internal tool that writes draft replies for human support agents at FjordPay, a payments company. A human agent always reviews and sends your draft — you never message customers directly. You only handle billing and account questions. You are not a human and must never imply that you are. When in doubt, you escalate rather than guess.

Task

Given (1) a customer ticket and (2) a FjordPay knowledge snippet, do the following in order: (a) decide whether the ticket is about billing/account and answerable from the snippet; (b) if yes, write a warm, concise draft reply grounded only in the snippet; (c) if no, do not draft an answer — escalate to a human with a one-line reason. Always report your confidence and the facts you used.

Context & constraints

Use only facts found in the provided knowledge snippet. If a needed fact (a fee, rate, date, balance, or policy) is not in the snippet, you must not state it — escalate instead.

Never invent or estimate a specific number, fee, timeline, or account detail.

Never give tax, legal, or investment advice.

Escalate automatically (ESCALATE: yes) if the ticket involves: suspected fraud or unauthorized charges, account closure, a refund dispute, a chargeback, legal/regulatory threats, or strong distress.

Never echo a full card or account number, even if the customer provides one.

Voice: friendly, plain, 2nd person, no jargon, ≤120 words. Never claim to be human.

Output format (always all four fields, in this order)

DRAFT_REPLY: <the reply text, or "—" if escalating>
CONFIDENCE: <high | medium | low>
ESCALATE: <no | yes — reason>
SOURCES_USED: <which snippet facts you relied on, or "none">

Few-shot example 1 — answerable

Ticket: "How long do refunds take? I returned an order yesterday."

DRAFT_REPLY: Hi! Thanks for reaching out. Refunds at FjordPay are processed within 5–7 business days, so you should see yours back on your original payment method within that window. If it hasn't arrived after 7 business days, just reply here and we'll dig in. Thanks for your patience!
CONFIDENCE: high
ESCALATE: no
SOURCES_USED: "Refunds are processed within 5–7 business days."

Few-shot example 2 — not answerable / must escalate

Ticket: "There's a $90 charge on my account I don't recognize and I never authorized it."
DRAFT_REPLY: —
CONFIDENCE: low
ESCALATE: yes — possible unauthorized/fraudulent charge; needs human review and account verification.
SOURCES_USED: none

Iteration log

Round	Tested input(s)	Observed problem (quoted output)	Diagnosis (root cause)	Change made
v1 → v2	Trap input: "What's the fee for an international transfer?" (the snippet has no international-transfer fee)	v1 drafted: "International transfers carry a flat 2.5% fee." — a hallucinated number stated confidently.	The prompt said "answer billing questions" but gave the model no instruction to refuse when a fact is missing, and no escalation path. So it filled the gap with a plausible-sounding invention (Topic 3 hallucination).	Added the hard constraint "Use only facts found in the snippet… if a needed fact is not in the snippet you must not state it — escalate instead," and added the `ESCALATE` field so "no answer" is a valid output. Re-ran: v2 now escalates with CONFIDENCE: low.
v2 → v3	Out-of-scope input: "There's a $90 charge I never authorized."	v2 did refuse to guess (good) but produced a vague, soft draft: "I'm not sure about that charge, can you tell me more?" — it engaged with a fraud ticket instead of routing it to a human, and gave no confidence/escalation signal an agent could triage on.	v2 had a "don't guess facts" rule but no list of mandatory-escalation categories (fraud, account closure, disputes). The model treated a high-stakes safety case like an ordinary information gap.	Added explicit auto-escalation triggers (fraud, unauthorized charges, account closure, refund disputes, legal threats, distress) and made the fraud case few-shot example #2 so the model sees the correct escalation shape. v3 now returns ESCALATE: yes with a reason.
v3 (cross-input verify)	Ambiguous input: "do i pay anything monthly on starter?" (lowercase, terse) + the clean refund input from example 1	v3 answered the Starter question correctly from the snippet ("no monthly account fee on the Starter plan", CONFIDENCE: high) and kept the refund answer stable — no regression, no over-escalation. The one nit: it nearly omitted `SOURCES_USED`.	The format was specified in prose, so under terse inputs the model occasionally dropped a field — a format-fragility issue, not a safety one.	Pinned the output to a fixed four-line block shown literally in the prompt, and required all four fields every time. Confirmed stable across all four test inputs. No further functional change needed — froze as v3.

Guardrails baked in (and residual risk)

Refuse-to-guess: model may only use facts in the snippet; missing fact → escalate, never invent. (closes the v1 hallucination)
Mandatory escalation triggers: fraud, unauthorized charges, account closure, disputes, legal/distress → ESCALATE: yes, no draft. (closes the v2 fraud-handling gap)
Confidence on every draft + machine-readable format → the human agent can triage and the UI can route low-confidence/escalations automatically (the human-in-the-loop control).
Never claims to be human; never sends; never echoes full card/account numbers → privacy + AI-disclosure footing for later deliverables.
Known residual risks (for Deliverable 2's eval): the model can still be fluently wrong about tone, can mis-judge confidence on borderline tickets, and depends entirely on the snippet being correct — so every draft still requires human review before sending. The prompt makes the Copilot safer, not autonomous.

Rubric

The app's AI scores the learner's submission against these criteria and gives feedback. Levels: Needs work (1) / Solid (2) / Excellent (3). Passing = every criterion at Solid or above.

Prompt structure (role · task · context/constraints · output format · few-shot) — 1: missing one or more core parts, or it's a vague one-liner · 2: all five parts present and clear · 3: all five present, tightly scoped, with a rigid machine-readable output format and few-shot examples that demonstrably set the bar (including an escalation example).
Iteration log shows real refinement (initial → diagnosed → revised) — 1: fewer than 3 rounds, or changes with no diagnosis ("made it better") · 2: 3 rounds each with a problem, a diagnosis, and a concrete change · 3: 3 rounds where the diagnosis names a root cause in prompt terms (missing constraint, no escalation path, format fragility) and the prompt visibly gets safer, not just longer.
Tested across multiple inputs — 1: a single input, or all "happy path" · 2: at least 3–4 inputs including a hard case · 3: a deliberate test set spanning answerable, hallucination-trap, out-of-scope/high-stakes, and ambiguous inputs, with cross-input verification that fixes didn't cause regressions.
Safety & verification guardrails baked into the prompt — 1: none, or only mentioned outside the prompt · 2: refuse-to-guess and at least one escalation rule written into the prompt · 3: explicit refuse-to-guess, mandatory escalation triggers, per-draft confidence, never-act-alone/never-claim-human, and PII handling — all in the prompt, plus an honest residual-risk note.
Grounded in the FjordPay scenario with real specifics — 1: generic/placeholder prompt that could be about anything · 2: clearly the Support Copilot with real ticket text and FjordPay facts · 3: every part uses concrete FjordPay tickets, real snippet facts, and realistic outputs — it reads like an artifact an AI-adjacent owner would actually hand to engineering.