SkilsMVP
BootcampInterview prep

AI Mock Interview — Spec

AI Mock Interview — Spec

This document specifies how the app runs an AI-driven mock interview using the drill banks in this folder (practical-prompt-craft.md, ai-output-evaluation.md, judgment-and-ethics.md, domain-ai-scenario.md, behavioral-and-curiosity.md). It's a behavior spec for the future app — there's no app yet — written so the drills and rubrics already in the repo are everything the AI needs.

Goal

Give the learner realistic, repeatable interview practice with structured feedback, approximating a human mock interviewer for AI-adjacent roles (AI PM, Prompt Engineer, AI Operations) within the self-serve model. This is what turns "I read about AI interviews" (Topic 10) into "I've practiced them dozens of times." Crucially, it lets the learner demonstrate AI literacy out loud — phrasing prompts, judging output quality, reasoning about risk — which is exactly what these interviews probe.

Session flow

  1. Choose a mode: a single question, or a full mock (e.g., 1 practical-prompt-craft + 1 ai-output-evaluation + 1 judgment-and-ethics + 1 behavioral-and-curiosity, or a domain-ai-scenario focus), optionally filtered by type or difficulty.
  2. Ask a question. The AI poses a drill's prompt verbatim and waits — it does not reveal the Framework, Model answer, or Rubric.
  3. Let the learner answer, ideally thinking aloud (typed or spoken). The AI may give one light nudge if the learner is stuck or silent (e.g., "What does a good output look like here, and how would you check for it?"), mirroring a real interviewer — but never feeds the answer.
  4. Optionally probe with a follow-up a real interviewer would ask ("What would you change in the prompt if it still hallucinated?" / "How would you catch that failure mode at scale?" / "Where would you keep a human in the loop?").
  5. Score and give feedback against the drill's Rubric when the learner finishes.

Scoring & feedback

  • Evaluate the answer against the drill's Rubric signals and the Framework structure. Produce: an overall rating (e.g., Needs work / Solid / Strong), per-criterion notes (what was good, what was missing), and 2–3 concrete, actionable improvements.
  • Only after scoring, optionally show the Model answer as a reference — never before.
  • Be specific and kind: cite what the learner actually said. The aim is improvement, not a grade.
  • Watch for the classic misses the rubrics encode, by type:
    • practical-prompt-craft — vague instruction with no role/context/format, no examples, no constraints on length or tone, no plan for when the output is wrong (no iteration loop).
    • ai-output-evaluation — no explicit quality criteria, eyeballing one example instead of testing many, missing the obvious failure modes (hallucination, omission, off-tone, unsafe), and no way to measure quality repeatably.
    • judgment-and-ethics — ignoring bias, privacy, or hallucination harm; trusting the model uncritically; no human-in-the-loop where stakes are high; no honesty about AI's limits.
    • domain-ai-scenario — generic answer with no grounding in the domain, no fit between the AI capability and the real workflow/users, no metric, and no handling of the case where AI gets it wrong.
    • behavioral-and-curiosity — not using STAR, no genuine curiosity or self-directed learning shown, no real example, or treating AI as magic rather than a tool with trade-offs.

Repeat & improve loop

  • Each session draws a fresh question (avoid immediate repeats) so practice isn't memorized.
  • Track which rubric criteria the learner repeatedly misses across sessions and surface a "focus next on…" summary (e.g., "you often skip defining quality criteria before judging output," or "you rarely name where a human should stay in the loop").
  • Tie into the program (../role.md and Topic 10, Landing an AI-adjacent role): mock-interview checkpoints recommend specific drill types by stage (e.g., behavioral-and-curiosity early; practical-prompt-craft and ai-output-evaluation as the core skills demos; judgment-and-ethics and domain-ai-scenario before the job hunt).

Guardrails

  • The AI is a coach, not a gatekeeper — encourage retrying.
  • Never expose the rubric/model answer before the learner attempts the question.
  • Keep feedback grounded in the rubric; don't invent criteria.
  • Practice what the role preaches: when the AI evaluates an answer, it should itself model good AI-output evaluation — cite evidence from the learner's words, avoid overconfident scoring, and flag where it's uncertain.

Inputs the AI uses (already in the repo)

  • The drill prompt (the question), its Framework (ideal structure), Model answer (reference), and Rubric (evaluation signals) — from the five drill files in this folder: practical-prompt-craft.md, ai-output-evaluation.md, judgment-and-ethics.md, domain-ai-scenario.md, behavioral-and-curiosity.md.
  • No additional data needed; the content is the spec.