Topic 6 — Data, training, and the human-in-the-loop

Goal: Understand how humans shape AI through data and feedback.

Lesson 6.1 — Where the AI's "knowledge" actually comes from

In her first week at Trellix Health, Bina Velasquez watched Scribe — the company's AI that drafts clinical visit notes — confidently invent a medication dose for a patient who had never been prescribed it. Pri, the engineer who built the pipeline, shrugged: "The model's only as good as what it learned from." Bina, eight years a biology teacher, recognized the shrug. It's the same one she'd give a student who'd memorized the wrong textbook.

That sentence is the foundation of everything in this topic. An AI model is built from data — the examples it learns patterns from — and the oldest rule in the data world holds completely: garbage in, garbage out. Train a model on data that is biased, sloppy, or wrong, and it produces output that is biased, sloppy, or wrong, delivered in the same fluent, certain voice as a correct answer.

Which means a surprising share of "AI work" is really data work: gathering it, cleaning it, and curating it so the model learns from good examples instead of bad ones.

An AI doesn't have opinions about your data. It just becomes it.

When Scribe hallucinated that dose, the fix wasn't a cleverer algorithm. It was Bina tracing which training and reference examples had taught the model that pattern, and flagging that clinical sources were thin in that area. Curating, checking, and improving data is non-coding, judgment-and-detail work — exactly the kind a careful, literate career-changer can be excellent at on day one.

Lesson 6.2 — Labeling: teaching by example

Here's the puzzle Bina hit next. If nobody hand-codes rules into the model, how does it ever learn that "the patient denies chest pain" is a symptom note and not a medication note? How did the spam filter on her own inbox learn what spam looks like?

The answer is labeled (or annotated) data: examples that humans have tagged with the right answer or category. Someone marked thousands of emails spam or not-spam. Someone tagged this phrase is a diagnosis, that one is a dosage. Someone rated this AI reply was helpful, that one was not. The model studies those human-made examples and learns to imitate the judgment baked into them.

Labeling sounds menial until you try to do it well. Bina spends an afternoon tagging clinical phrases in Scribe's draft notes and discovers the hard cases everywhere: is "patient is stable" a reassuring observation or a vague non-answer a doctor would want rewritten? The label you choose is the lesson the model learns. Tag it carelessly and you've taught it something wrong, at scale.

This is why labeling is non-coding judgment-and-detail work, and why it's a real, paid entry point into AI. Pri can wire up the pipeline, but she's the first to admit she can't tell whether a clinical note reads correctly — that takes Bina's eye, and Dr. Marchetti's when the stakes climb. The person with domain sense and patience for detail is the one who makes the labels trustworthy.

Lesson 6.3 — Human-in-the-loop, across the whole lifecycle

People picture "AI" as a machine humming along alone. Trellix's actual workflow looks nothing like that. At four points every day, a human steps into the machine's path — and the phrase for that pattern is human-in-the-loop.

The loop spans the lifecycle:

• Giving feedback. Humans rate and rank AI outputs — this draft is better than that one — which teaches the system what "good" means (the evaluation skill from Topic 5).
• Labeling and curating data. Humans create and clean the tagged examples the model learns from (Lesson 6.2).
• Reviewing and correcting before use. In high-stakes products, a person checks the AI's output before it goes anywhere. No Scribe note reaches a patient until a clinician has reviewed it, because a confident hallucination here could reach a real person.
• Handling edge cases. When the AI is unsure, or hits something rare and weird, it routes the case to a human instead of guessing.

Bina lives at the review-and-correct station. The AI gives speed and scale; she gives judgment, correction, and a sense of what's safe. Her teacher's nose for a confident-but-wrong answer turns out to be the exact instinct the job rewards. The higher the stakes, the more the loop tightens — which is why these roles keep multiplying as AI spreads into medicine, law, and finance.

Lesson 6.4 — Your feedback becomes the model's next lesson

When Bina ranks two Scribe drafts — A reads cleanly, B buries the chief complaint — it feels like she's just doing quality control. She's doing more than that. Her ranking is about to become training signal.

This is the through-line from Topic 5 made concrete. In RLHF (reinforcement learning from human feedback), human pairwise preference rankings — A is better than B — are collected at scale and used to train a separate "reward model" that scores outputs the way the humans did. The main model is then tuned to chase that score. Your judgment, multiplied across thousands of comparisons, literally aligns the model toward what people prefer.

That makes this a real, hireable task. Pairwise preference ranking is standard paid annotation work, often priced roughly $0.50 to $5 per sample depending on how much expertise and care each comparison demands — and clinical comparisons like Bina's sit at the expert end.

A few more terms name the work around it:

• SFT (supervised fine-tuning): teaching the model from curated input → ideal-output pairs — show it the question and the answer you want.
• RLHF: the preference-ranking process above, where humans supply the "better/worse" signal.
• RLAIF (reinforcement learning from AI feedback): the same idea, but a model gives the feedback by judging outputs against a written set of principles (a "constitution"), which scales cheaper than paying humans for every comparison.
• Red-teaming: deliberately probing the AI to make it produce harmful, unsafe, or off-policy output — so the team finds the failure before a patient does.

Every one of those names a task someone is hired to do. RLAIF doesn't erase the human, either — a person still writes the principles the AI model judges against.

Lesson 6.5 — Golden datasets, and the question everyone asks

Marc, the AI PM, asks Bina the question that decides whether Scribe ships its next version: is it actually getting better, or does it just feel better this week? You can't answer that with vibes. You answer it with a golden dataset.

A golden dataset is a fixed, carefully reviewed set of test cases — usually around 200 to 500 examples — that the team runs every new model version against, so quality is measured the same way every time. Trellix builds Scribe's from three sources: real production failures (with patient data stripped out), human-crafted edge cases that should be hard, and a little synthetic expansion to cover gaps. Because the score is only trustworthy if the test never quietly changes, the golden dataset is versioned as strictly as code — and under regulations like the EU AI Act, high-stakes systems must document exactly this kind of data governance and traceability.

That brings up the worry Bina's career-changer friends always raise: won't AI just eliminate these jobs? The honest answer is that roles evolve, but the need for skilled humans is growing, not shrinking. Some narrow tasks shift as tools improve. Meanwhile, every clinic Theo onboards needs people to deploy Scribe well, judge its drafts, build the golden dataset, and keep it safe. The work moves up — from doing tasks by hand to directing and overseeing AI that does them.

The job is being promoted, not disappearing.

The skills that carry you through the churn are human ones: judgment, domain expertise, communication, critical thinking, ethics — and adaptability, the meta-skill, because the field keeps moving. The reassuring move is the one Bina made: stop fearing AI, and become the skilled human who makes it useful, accurate, and safe.

Worked example — Tracing one bad Scribe note all the way back

A pilot clinic files a complaint: Scribe drafted a visit note stating a patient was "advised to continue lisinopril," but the patient takes a different blood-pressure drug. Dr. Marchetti caught it before it reached the chart. Watch the whole topic show up in fixing one mistake.

Review caught it (human-in-the-loop). The note never reached the patient because a clinician reviews every draft before use — the tightest loop, reserved for the costliest mistakes. That's the safety net working exactly as designed.

Bina labels and traces it back (data work). She tags the error type — medication substitution — and traces it. The model had seen many notes mentioning lisinopril and learned to reach for it as a default. Garbage in: the reference data over-represented one drug, so the model guessed it when unsure.

She turns it into signal (feedback + golden dataset). Bina writes a corrected input → output pair for an SFT batch, and ranks the bad draft below a clean one to feed RLHF. Then she adds the scrubbed case to the golden dataset as an edge case, bumping its version, so every future model is tested against this exact failure.

The team checks for siblings (red-teaming). Bina and Pri probe Scribe with similar prompts to see what else it defaults to wrongly — finding the next failure before a clinic does.

One note. Labeling, human review, feedback-as-training-signal, a versioned golden dataset, and red-teaming — every concept in this topic, doing real work. And the person at the center of it is the careful human who can tell a confident answer from a correct one, not the engineer.