Designing Agentic Workflows

Why this module exists

"Agentic" has become a category label rather than a design decision. Systems are described as agents the way software was once described as cloud — as a statement of modernity rather than of architecture.

This module treats autonomy as what it is: a set of choices, made step by step, each with a cost. How much independence at which step, guarded by whom, failing into what, and evidenced how.

The output is a specification. Not a diagram of boxes, but a document precise enough that a vendor could build against it and an auditor could read it.

5.1 Autonomy is a dial, not a switch

The first mistake is applying one autonomy setting to a whole process. Real designs vary it step by step.

Two rules that come from watching this go wrong:

Most of the value is at levels 2 and 3. They capture the labour saving — the drafting, the assembly, the reading of four thousand messages — while keeping a human on the consequence. Levels 4 and 5 add modest additional efficiency and a great deal of risk surface.

Most incidents come from a step being set at 5 because the demo worked. A demonstration exercises the common case. Autonomy is priced by the uncommon one.

5.2 Approval gates are decision rights, made explicit

A gate is not a checkbox added for comfort. It is an organisational decision right that already exists — someone in your company is already accountable for this class of decision — written into the system so that it survives automation.

That framing does real work. It tells you who the approver must be (the person who holds the right today, not whoever is free), and it tells you when a gate is missing (when the system takes an action nobody currently has the authority to take unilaterally).

Four rules cover almost every case, and they compose:

The fourth is the one most often neglected and most often fatal. A gate that requires the approver to log into a new system will be batched, rushed at the end of the week, and eventually delegated to whoever has time. Convenience is not a nicety here; it is the control.

The override test

The hard question, and the one that connects this module to Module 6:

Can the approver actually change the outcome?

Three sub-questions establish it:

1. Do they see the inputs the system used — not just its output?
2. Do they have time at real volume? Four hundred approvals a day at thirty seconds each is three and a half hours. If nobody has that time, the gate is fiction.
3. Has anyone ever overridden it? Pull the number. A gate with a 0% override rate over months is either a perfect system or a rubber stamp, and it is not a perfect system.

A gate that fails the override test is worse than no gate, for two reasons. Operationally, it creates false assurance — everyone believes the output was reviewed. Legally, as Module 6 shows through the SCHUFA reasoning, a nominal human step does not remove a decision from the automated-decision rules. You carry the obligation and get none of the protection.

5.3 Design the failure path first

Systems without a designed failure path do not fail gracefully. They fail silently, which is worse, and silent failure is the usual reason a promising pilot is quietly abandoned six months later — not because it broke loudly, but because trust drained without anyone being able to point at an incident.

For every step, answer four questions before deployment:

1. What does wrong output look like here? Be specific. Not "an error" — "an appeal letter citing the wrong policy section," "a summary that omits the complaint."
2. Who notices, and how? A person, a validation rule, a downstream system, or nobody.
3. How quickly? Same minute, next day, next quarter, never.
4. What happens next? The recovery path, and who owns it.

Two patterns are worth building in by default.

Deterministic validation after judgement. The cheapest reliability improvement available: a rule that checks the model's output before it moves on. Does the drafted letter reference a policy section that exists? Does the extracted date fall in a plausible range? Is the total equal to the sum of the lines? These catch a large share of failures at almost no cost, and they convert an unbounded error into a caught one.

A designed escape hatch. Every workflow needs a path for "this case is not like the others." Without one, the system will handle unusual cases badly rather than declining them. Instruct the system to escalate on low confidence or on inputs outside its defined scope, and — critically — make sure the escalation queue has an owner. An escalation queue nobody reads is a silent failure with extra steps.

5.4 Traces and spans: the evidence layer

This section introduces vocabulary you will be held to in vendor conversations, so it is worth being precise rather than approximate.

A trace is the complete record of one run of a workflow, end to end — one denial processed, one quote produced, one ticket answered. It is the unit you look at when someone asks what happened to this case?

A span is one timed step inside that trace. Retrieving the policy document is a span. The model call that drafts the letter is a span. The approval is a span. Each span records when it started, how long it took, what went in, what came out, and whether it failed.

Spans nest. A trace is not a flat list but a tree: the drafting step contains the retrieval it triggered, which contains the two database queries the retrieval made. That structure is the whole point — it is what lets you answer which part was slow or wrong, rather than only that the run was.

Traces serve three masters at once, which is why they are the highest-return piece of infrastructure in this entire course:

The minimum trace schema:

Note the retention consequence: traces contain the data the system processed. They inherit its residency, retention and deletion obligations, and they are frequently forgotten in the data map. Include them in the Module 6 control map explicitly.

5.5 Reliability engineering for chains

Module 3 established the arithmetic: chained step reliabilities multiply. Five steps at 95% is 77%; twenty is 36%. Three design moves follow directly.

Shorten the model chain. Prefer one model step doing a well-specified job over five model steps reasoning their way along. Every additional model step is another multiplication by a number below one. Length is risk.

Interleave deterministic steps. A rule between two model steps is a 1.0 in the product and, if it validates, it truncates error propagation. The strongest workflows alternate: judge, check, act, check.

Set the acceptance threshold before launch. Decide in advance what performance makes this worth deploying — and what performance triggers rollback. Negotiating that number after the results are in is how systems get deployed at levels nobody would have approved in advance.

One further point, often missed: measure the end-to-end rate, not the per-step rate. Vendors quote per-step accuracy because it is flattering. Your customer experiences the product of the chain.

5.6 Where customer engagement fits

Module 3 showed that customer-facing work is where the shape test most often returns yes. Design adds a caution: customer-facing steps almost never belong above autonomy level 3.

The asymmetry is that a customer interaction is both irreversible and reputationally amplified. A wrong internal summary is corrected in a meeting. A wrong message to a customer is screenshotted.

The pattern that works: the model does the reading, the retrieval and the drafting — the parts that consume the agent's day — and the human does the sending. That typically removes 70–80% of handling time while leaving every outbound communication signed by a person. It is less impressive in a demo than full automation, and it is the version that is still running a year later.

Exercise — Write the specification

Time: 90 minutes. Produces the artefact for this module.

Take the workflow you labelled in Module 3 and mapped in Module 4, and specify it.

1. Step table. Every step, its layer, its autonomy level, and one sentence on why that level.
2. Gate specification. For each gate: the rule (threshold, counterparty, content, channel), the named approver, the channel they already work in, and the expected volume in minutes per day.
3. Override test. For each gate, answer the three sub-questions. If any fails, redesign the gate or lower the autonomy level.
4. Failure table. For each step: what wrong looks like, who notices, how fast, recovery path.
5. Escape hatch. The condition that routes a case out, and the named owner of that queue.
6. Trace schema. Which fields you will record, and their retention period.
7. Acceptance criterion. One measurable claim with a baseline, a date, a threshold and a rollback action.

The specification is complete when someone who was not in the room could build against it and disagree with it. Vagueness is what makes a document unarguable, and unarguable documents are how bad systems get approved.

Self-check

1. Why is autonomy assigned per step rather than per process?
2. A gate exists, an approver is named, and the override rate has been zero for six months. What do you conclude, and what do you do?
3. Your vendor reports 96% per-step accuracy across a seven-step agent. What is the end-to-end figure, and what would you ask them to change?
4. Name one step in your own workflow where a deterministic check after a model step would catch a realistic failure.
5. What is the retention obligation on your traces, and who currently owns it?

Further reading

• Anthropic, Building Effective Agents, 2024 — workflow patterns and when to stop adding autonomy.
• Module 6 of this course, on the legal status of human oversight that cannot change the outcome.