Topic 9 — Tools and the AI ecosystem

Goal: Know the AI landscape, tools, and key terms you'll work with.

Lesson 9.1 — The landscape, by category not by name

Bina Velasquez taught high-school biology for eight years before she became an AI quality analyst at Trellix Health, the 40-person startup behind Scribe — an assistant that drafts clinical visit notes and patient-message replies for small clinics. In her first week, an engineer mentioned offhand that they'd "swapped the note-drafter onto a cheaper tier and put Opus behind the patient-message path." Bina nodded and wrote the sentence down to decode later. By Friday she could have said it herself.

Here's what she'd worked out. The flagship AI tools you keep hearing about are chat assistants, and under the hood each one runs on a large language model (LLM) — software trained to predict and generate text. The big three by name: Claude (by Anthropic), ChatGPT (by OpenAI), and Gemini (by Google). You'll use these daily and build products on top of them.

The trick that saves you is to learn the categories, because the names churn every few months:

• Chat assistants / LLMs — general-purpose text AI. Claude, ChatGPT, Gemini, and others.
• Media-generation tools — for images, audio, video, and code. You should know these exist; LLMs sit at the center of most AI-adjacent work.
• AI features built into everyday products — the writing helper in your email, the meeting summarizer, the coding copilot. Increasingly, "using AI" just means using software you already had.

One detail matters across all of them: each model comes in tiers. A faster, cheaper version for high volume, and a more capable, pricier version for hard tasks. As of mid-2026 that looks like Claude Haiku, Sonnet, and Opus; Gemini Flash versus Pro; ChatGPT's Instant, Thinking, and Pro modes. The exact names will have moved by the time you read this. The pattern — fast-and-cheap versus slow-and-smart — will not.

That tiering is exactly what Bina's engineer was describing. Cheap tier for the routine note draft; the most capable model where a wrong word could reach a patient.

Lesson 9.2 — How a product actually "uses" a model: the API

When Bina opens Claude in a browser and types a question, that's the chat assistant. But Scribe isn't a person typing into a chat box. Scribe is software that, thousands of times a day, sends a clinic's appointment data to a model and gets a drafted note back — no human in the loop for that step. How?

Through an API (Application Programming Interface) — the doorway that lets one piece of software talk to another. When a company says it "builds on Claude" or "builds on GPT," it means its code calls the model's API: it sends a prompt over the internet and gets a response back, programmatically.

Two things about APIs that show up in job descriptions and product specs, and that you should recognize on sight:

• Providers. The major AI labs — Anthropic, OpenAI, Google, and others — each offer their models through their own API. A product can switch providers, or use different ones for different jobs. Trellix uses one provider for Scribe's note-drafting and could move tomorrow if another got cheaper or better.
• Billing is per token. A token is a chunk of text, roughly three-quarters of a word (you met tokens in Topic 3). API usage is billed by the token, and the durable thing to remember is this: input and output are priced separately, and output costs several times more than input. The exact figures move — don't memorize today's, just check the provider's current rates — but to make it concrete: as of this writing, a top-tier model like Claude Opus runs roughly $5 per million input tokens and about five times that, around $25, per million output tokens; a cheaper tier like Haiku is a fraction of both. You will never compute these by hand. But when Marc, the AI product manager, says "the patient-message path is getting expensive because the model's replies are too long," you'll know he means output tokens, and you'll know why that's the costly half.

You don't implement the API. You read the sentence with the word "API" in it and know exactly what's happening behind it.

Lesson 9.3 — RAG: giving the model something to look at

Early on, Bina caught Scribe confidently citing a medication dosage that wasn't in the patient's chart. The model had invented something plausible. In a clinical product, that's the nightmare. So she asked Pri, the ML engineer, the obvious question: how do we make it answer from the real chart instead of from thin air?

Pri's answer was three letters Bina now hears constantly: RAG — Retrieval-Augmented Generation. Instead of asking the model to answer from memory, the system first retrieves the relevant documents — this patient's actual chart, the clinic's own protocols — and hands them to the model along with the question. The model then answers from that material.

Walk it slowly:

1. A request comes in: "draft a follow-up note for this visit."
2. The system searches a private knowledge base (the patient's records, the clinic's templates) for the relevant pieces.
3. It feeds those matches to the model together with the prompt.
4. The model answers using the real data in front of it.

Why this is everywhere: RAG sharply reduces hallucination (the model is reading facts, not guessing them), and it lets the AI use up-to-date or private information the base model was never trained on — your company's documents, last week's data. Trellix could never have trained a model on every clinic's private charts. RAG means it doesn't have to.

This is the single most important architecture term in the topic. Nearly every enterprise AI application today is a RAG system. When a job description says "we're building a RAG pipeline over our internal docs," picture exactly Bina's four steps — that's the whole job, on a poster.

Lesson 9.4 — Agents: when the AI stops chatting and starts doing

A chat assistant answers and waits for you. But Marc keeps using a word that means something more: agents.

An agent is an AI system that doesn't just produce text — it plans and takes actions using tools to finish a multi-step task. Give it a goal, and it can browse the web, run a calculation, call another piece of software, check its own result, and try again — looping until the job's done, with far less hand-holding per step.

Picture the difference at Trellix. Ask the chat assistant "what's this patient's last A1C reading?" and it answers if you paste the data in. An agent version could, on its own, look up the record, pull the last three readings, draft a follow-up message, and flag it for Dr. Marchetti's review — several actions chained toward one outcome.

It's the fastest-growing corner of the field, and Bina's role grew straight into it. Because an agent acts rather than just suggests, a wrong step doesn't sit harmlessly in a chat window — it can do something. That raises the stakes on the exact work she does: guardrails and evaluation. Someone has to define what the agent is allowed to touch and rigorously test whether it behaves. In a clinical product, that someone is not optional.

The more an AI system can do on its own, the more it needs someone whose whole job is checking that it does the right thing.

Lesson 9.5 — Fine-tuning vs. prompting and RAG (and how to get fluent)

Dr. Soledad Marchetti, the family physician piloting Scribe, told Bina the drafts didn't "sound like a clinician." Bina's instinct was to ask whether they should fine-tune the model. Pri walked her through the trade-off, and it's one worth carrying into any AI conversation.

There are three ways to make a model behave the way you want, cheapest to most expensive:

• Prompting — write better instructions and examples. No training, changes in minutes, costs almost nothing. Always try this first.
• RAG — feed the model the right documents so it answers from real data. Still no retraining; you just update the knowledge base.
• Fine-tuning — actually retrain a base model on your own examples so the behavior gets baked in. Powerful for teaching a consistent style or specialized task, but it's slow, costs real money, and has to be redone when your needs change.

The practical rule teams follow: try prompting and RAG first, and reserve fine-tuning for when those aren't enough. Prompting and RAG are cheaper and far faster to change; fine-tuning is the heavy tool you reach for once you've outgrown the light ones. For "make it sound more like a clinician," Bina and Pri got most of the way with a sharper prompt and a few good examples — no fine-tuning needed.

Knowing that ordering is what lets you sound credible in an interview when someone asks "would you fine-tune for this?" — the seasoned answer is usually "not yet; let's see how far prompting and RAG get us."

And the way you actually earn that fluency isn't reading. Get hands-on with one or two leading chat assistants until prompting and evaluating them is second nature — most have free tiers, so it costs you nothing but time. Then build something small and real: a custom assistant that drafts your weekly update, a tested prompt that reliably summarizes a document the way you need it. Theo, the enablement consultant who onboards clinics onto Scribe, says the new hires who land are never the ones who can list model names — they're the ones who can pull up a tool and show a thing they made it do.

Be tool-aware, not tool-obsessed. Names and versions change constantly; the durable value is your AI literacy, judgment, and ability to apply AI to a real problem — with adaptability as the meta-skill underneath it all.

Worked example — Bina reads a job description

Six months in, Bina sees a posting that would once have been a wall of jargon. Now she reads it like a translation she already speaks. The ad wants an AI Quality Analyst who can:

"Evaluate outputs from our LLM-based assistant across model tiers, design eval sets for our RAG pipeline over proprietary documents, write and refine prompts, define guardrails for early agentic workflows, and help the team decide when fine-tuning is warranted. API and token costs a plus."

She decodes it in one pass. LLM-based assistant — a chat-assistant product like Scribe, built on a provider's model. Model tiers — they run faster/cheaper and slower/smarter versions and want someone judging quality across both. RAG pipeline over proprietary documents — the system retrieves the company's private data and feeds it to the model; her evals would check it's answering from the real documents, not hallucinating. Prompts — the cheap, fast lever she'd reach for first. Agentic workflows + guardrails — AI that takes actions, needing exactly the testing she does. When fine-tuning is warranted — they know the trade-off and want someone who can tell them "not yet." API and token costs — she understands input versus output billing and why long responses get expensive.

One posting. Every term in this topic, in the wild. She doesn't need to build any of it. She can read the whole job, speak to it in the interview, and know she'd fit — which is exactly what this topic was built to give her.