Why an appendix like this exists
A surprising share of expensive AI mistakes are vocabulary failures.
Someone says "agent" meaning a scripted workflow. Someone hears "training" and budgets for a machine-learning project when they needed a search index. A vendor says "we log everything," a buyer hears "we can reconstruct any case," and both leave the room satisfied and wrong.
This appendix defines every term the course relies on, in one place, in the sense the course uses it. Where a word is genuinely contested in the market, that is stated rather than hidden.
A. How the machine works
| Term | Definition |
|---|---|
| Token | The unit a model reads and writes — roughly a short word or word fragment. Pricing, context limits and speed are all measured in tokens, which is why they appear on invoices |
| Prompt | Everything sent to the model for one request: instructions, retrieved material and the user's question |
| Context window | The maximum amount of text a model can consider at once. Exceeding it means something must be dropped or summarised, and what gets dropped is a design decision |
| Inference | One act of producing output from a model. The recurring cost of running AI, as opposed to the one-off cost of building it |
| Hallucination | Confident output that is not true. Not a malfunction — a consequence of a system optimising for plausible continuations rather than verified ones |
| Temperature / sampling | The setting governing how much variation the model allows when choosing each next token. Lower is more repeatable, never fully deterministic |
| Embedding | A numerical representation of a piece of text such that similar meanings sit close together. The mechanism behind semantic search |
| Vector search | Finding text by meaning rather than by keyword, by comparing embeddings |
| Retrieval (RAG) | Fetching relevant documents at question time and placing them in the prompt, so answers reflect current sources and can cite them |
| Fine-tuning | Adjusting an existing model on your own examples. Reliable for form and style; unreliable for installing facts |
| Open-weight model | A model whose parameters are published, so it can be run on your own infrastructure. Not the same as open source, which would also cover the training data and code |
| Frontier model | The largest, most capable models available at a given moment. A moving label, not a category |
| Small model | A model chosen for a narrow task, usually run close to your data. Often the right answer for high-volume, sensitive, well-defined steps |
| Jagged frontier | The observation that model capability has an irregular boundary: tasks of apparently equal difficulty can fall on opposite sides of it |
B. Systems and architecture
| Term | Definition |
|---|---|
| Model call | A single request to a model and its response. The atomic unit of judgement in a system |
| Workflow | Models and tools orchestrated through predefined code paths. You wrote the sequence; the model fills in the judgement steps |
| Agent | A system in which the model directs its own process, choosing which tools to call and in what order. Flexible, and unpredictable by construction |
| Autonomy level | How much a step may do without a human: suggest, draft, act-with-approval, act-and-report, act-autonomously |
| Tool call | A structured request the model emits — "look up order 4471" — that the surrounding system executes against a real API, returning the result to the model |
| MCP (Model Context Protocol) | A standard for describing tools to models so a capability built once can be offered to different systems without bespoke glue |
| API | A contract between two systems: ask in this exact form, get an answer in that exact form |
| Connector | A prebuilt integration to a specific system. Convenient, and worth checking against the four API questions in Module 4 |
| System of record | The authoritative, change-controlled home of data — ERP, EHR, LIMS, core banking. Read from it freely; write to it as a separate, slower project |
| System of engagement | Where people meet the work: portals, inboxes, chat. Usually the right home for AI |
| Latency | Time from request to response. Model calls answer in seconds; queries answer in milliseconds, which is why placement changes the user experience |
| Batch window | The fixed schedule on which an upstream system will hand over data. Frequently the real constraint on how "real-time" a workflow can be |
C. Observability and reliability
| Term | Definition |
|---|---|
| Log | A record that something happened. Tells you an event occurred, not the story of a case |
| Trace | The complete record of one run of a workflow, end to end. The unit you open when asked "what happened to this case?" |
| Span | One timed step inside a trace — a retrieval, a model call, an approval. Spans nest, so a trace is a tree rather than a list |
| Attribute | A labelled value attached to a span: model version, token count, document id, approver. What makes cost and behaviour attributable to a specific step |
| Trace id | The identifier tying every span of a run together, and tying the run to your other systems |
| Evaluation set (eval set) | A curated collection of cases with known-good answers, used to measure whether a change helped. The most valuable asset the course asks you to build, and the one to keep in your own systems |
| Benchmark | A public test of general model capability. Useful for shortlisting, weak evidence about your documents |
| Drift | Quality degrading over time without anyone changing anything — because the model, the data or the world moved |
| Regression | A change that makes previously correct behaviour incorrect. Detected by the eval set, or by a customer |
| Override rate | How often a human approver changes the system's output. A rate of zero over months means the gate is decorative, not that the system is perfect |
| Acceptance criterion | The measurable claim, agreed before launch, that decides whether a system goes live or is rolled back |
| End-to-end reliability | The success rate of a whole chain, which is the product of its steps rather than the average. Ten steps at 95% is about 60% |
D. Governance
| Term | Definition |
|---|---|
| Personal data / PII | Information relating to an identifiable person. The trigger for most of Module 6 |
| Special category data | Sensitive classes under the GDPR — health, biometrics, beliefs and others — requiring an Article 9 condition in addition to an Article 6 lawful basis |
| PHI | Protected health information under HIPAA: health data held by a covered entity or its business associate |
| De-identification | Removing identifiers so data falls outside a regime. Under HIPAA, either Safe Harbor (remove eighteen identifier types) or Expert Determination |
| Lawful basis | The Article 6 justification for processing personal data — contract, legitimate interests, consent and others. Chosen deliberately, because each carries different rights |
| Purpose limitation | Data collected for one purpose may not be freely reused for another. The principle that most often blocks training on operational data |
| Article 22 | The GDPR right not to be subject to a decision based solely on automated processing with legal or similarly significant effects. A nominal human step that cannot change the outcome does not escape it |
| DPIA | Data protection impact assessment. Required for high-risk processing; most useful as a design tool run early rather than paperwork filed late |
| BAA | Business associate agreement. The contract required before a vendor — including a model provider — may process PHI on your behalf |
| Data residency | Where data is physically processed and stored. Often the constraint that decides the deployment model rather than merely inconveniencing it |
| Sub-processor | A third party your vendor uses to deliver the service. Their list is part of your compliance perimeter and your concentration risk |
| Guardrail | An automated check constraining what goes into or out of a model — masking identifiers, blocking categories, validating format |
| Approval gate | An existing organisational decision right made explicit in a system: a named person who can see the inputs, change the output, and whose decision is recorded |
| Human oversight | Meaningful human control over an automated decision. Meaningful is the operative word; see the override test |
| Audit trail | The durable record of who decided what, on what evidence, and when. In practice, your traces plus your approvals |
| Risk tier | The EU AI Act's classification — prohibited, high-risk, limited/transparency, minimal — which determines what obligations attach and when |
| Conformity assessment | The procedure by which a high-risk AI system is shown to meet its requirements before being placed on the EU market |
| GPAI | General-purpose AI model. Carries its own obligations under the AI Act, distinct from the risk tier of any system built on it |
| Agent washing | Relabelling existing chatbots or robotic automation as agents. Gartner's term for the supply side of the lemon market |
| AI washing | Overstating AI capability to investors or customers. The SEC's term, and it has been enforced |
| Shadow AI | Unsanctioned AI use by staff. Best measured before it is legislated, and governed by data class rather than by tool name |
E. Commercial and strategic
| Term | Definition |
|---|---|
| Total cost of ownership | Licence plus consumption plus integration plus ownership, over the life of the system. The last two are usually the ones that surprise |
| Consumption pricing | Paying per unit of usage. Costs rise with adoption, so success raises the bill — ask what it costs at ten times today's volume |
| Lock-in | The cost of leaving: where your prompts, evaluation sets and traces live, and what you can take with you |
| Evidence ladder | The ranking of claim strength from demonstration to production evidence, and the size of commitment each justifies |
| Pre-registration | Fixing the metric, baseline, sample and decision rule before a pilot runs. The single step that separates a pilot from a demonstration |
| Baseline | The current value of the number you intend to move, measured before deployment. Without it, no claim about improvement survives scrutiny |
| Counterfactual | What would have happened anyway. Naming it is what makes an improvement attributable rather than coincidental |
| Data flywheel | A loop where using the product generates data that measurably improves the product, attracting more use. Rare; most claimed flywheels break at "measurably" |
| VRIN | The test for a durable advantage: valuable, rare, inimitable, non-substitutable. Access to a frontier model passes one of the four |
| Transport vs judgement | The distinction underneath organisational redesign: moving and reformatting information, whose price is collapsing, versus deciding and taking responsibility, whose value is rising |
Terms this course deliberately avoids
Three words appear constantly in the market and are not used as load-bearing terms here, because they carry more heat than meaning.
"Intelligent." It describes an impression, not a capability. Replace it with what the system does: reads unstructured text, classifies into fuzzy categories, drafts language.
"Autonomous," used unqualified. Autonomy is a level assigned per step, not a property of a product. Ask which level, at which step, with which gate.
"AI-powered." True of a four-step workflow with one model call and of a system that is entirely a model. It is the phrase the bill of materials exists to unpack.
Self-check
- A vendor says "our agent handles denials end to end." Which two terms in section B would you ask them to pin down, and what would each answer tell you?
- Explain the difference between a log and a trace to a colleague in one sentence each.
- Your team reports "the model call was slow." Which unit of observability would locate the actual cause, and why does nesting matter?
- Which two documents does a model provider need to sign before it may process European health data on your behalf?
- Give an example from your own organisation of something described as "AI-powered" that the bill of materials would show to be mostly not.