Human in the loop in practice: designing the approval step and reading rubber-stamping off the approval log

In July 2026 a wave of articles went through the Polish business press arguing that an employee supervising AI cannot simply click “approve” — Infor was among the outlets that ran the story on 10 July 2026, in Polish, citing Articles 13, 14 and 26 of the AI Act. On the substance that piece is right, but it leaves out two things: when those provisions begin to apply, and how to check whether approval in your own company has already turned into clicking. Human in the loop is not a property of a model or an option in a tool. It is the design of one step in a process — a screen, a queue and a log — and each of them can be designed badly in a way the diagram never shows.

This piece describes how to build that step and how to measure it. It does not settle how your system is classified under the AI Act, nor the lawful basis for the processing — that belongs to a lawyer, not the contractor. Where the person stands in the process we set out in step four of an AI rollout; here we start one question further on. Polish writing on human in the loop is not scarce — from definitions and vendor guides through to a peer-reviewed paper by Martyna Kaczmarczyk on how hollow such oversight can be; we return to it below. One thing is missing from all of it: a method for reading that hollowness off your own approval log. We describe what a project of this kind looks like from end to end under AI automation for business.

Three supervision modes — which one is human in the loop, and which is human on the loop

There are three variants, and we have already named all three in step four of the rollout in terms of the work rather than the jargon. “A person approves every result” is human in the loop: the decision comes before the effect, and until it exists nothing is executed. “A person approves exceptions” is its narrower form — still a decision before the effect, only for a subset of cases. “A person supervises statistically” is human on the loop: the system runs by itself and oversight rests on samples, indicators and a stop button. That is also where the argument sits for beginning with the first variant even when the target is the last.

The two English terms are sometimes used interchangeably, but the difference is operational rather than terminological. Statistical oversight needs three things at once: a designed way of drawing the sample, a tested stop, and somebody who looks at the alerts. Moving to it without them is not a loosening of oversight but its removal. A fourth possibility — human out of the loop, meaning an audit after the fact and nothing else — is not a variant here, because there is nothing in it to design. (The Polish calque “człowiek w pętli” does exist, but Polish Wikipedia describes it in the context of autonomous weapons.)

What the approver has to see

Four things, on a single screen: the source — the document, message or recording the result came from; the system’s proposal together with its confidence score; the difference against the data already in the ERP or CRM; and the reason the case landed in the queue instead of passing through on its own. If any one of those four has to be looked up in another window, in practice nobody will check it.

A useful reference point comes from the CJEU judgment in C-203/22 (Dun & Bradstreet Austria, 27 February 2025): a person subject to a decision under Article 22 of the GDPR must be given, under Article 15(1)(h), an explanation of “the procedure and principles actually applied”, not a disclosure of the algorithm. Nobody owes the approver that. But where the approver sees less than the person the decision concerns would hear, it is hard to speak of real influence over the outcome.

A better screen is not a safeguard in itself. In the study by Bansal and colleagues (CHI 2021), adding explanations to an AI recommendation increased its acceptance regardless of whether it was correct. A screen specification is a hypothesis to be measured, not a proof — hence the second half of this text.

Who owns the queue, and what its SLA is

The owner of the queue is a person with a first name and a surname, not a department. Then four things to settle: the coverage window, the backlog ceiling above which the queue counts as failing, what happens to a case once its deadline passes, and who covers for the owner during leave. Without that last point, the SLA holds for eleven months of the year.

Article 26(2) of the AI Act — a provision that will begin to apply in December 2027 at the earliest, as set out below — says that the deployer assigns human oversight to natural persons who have “the necessary competence, training and authority, as well as the necessary support”. The first three are quoted everywhere and the fourth is skipped, yet it is the one that means time in the rota, access to the source system, and the right to state that a case cannot be settled on this screen.

The deadline is set in policy rather than left to the tool; what each platform offers in that respect is collected in the tooling table below. This is not the same as the split of decision rights in a project, described in our implementation methodology: here it is one queue, and the question of who is answerable for keeping it from growing.

The capabilities real oversight needs — and four designs that will not survive an audit

Article 14(4) of the AI Act — also deferred, to the same dates — lists five things the overseeing person must be able to do. We group them into three; that grouping is ours, not the Regulation’s: understand (points (a) to (c): knowing the system’s limitations well enough to catch an anomaly, remaining aware of automation bias, and interpreting the output correctly), intervene (point (d): disregarding, overriding or reversing the output) and halt (point (e): interrupting the system with a “stop” button so that it comes to rest in a safe state).

Four designs of the approval step fall short of that, and every one of them looks entirely normal inside a company:

• An “approve all” button. Bulk acceptance usually means nobody saw the individual case. There is no good scope for it: a category that has passed the promotion rule set out later in this text no longer reaches the queue at all, and one that does reach it is there for a reason.
• A queue where the median time to decide is two seconds. Two seconds are enough for a click and not enough to read anything. That is a charge against a design which — to take an example — has somebody work through two hundred cases a day and calls it oversight, not against the people working inside it.
• Overrides with no trace. A person changed the amount or the category, and only the final result reached the log. Without a field recording what was changed, you can neither demonstrate that the oversight was real nor learn anything from the corrections.
• A stop that nobody has ever tried. The test is simple: once a quarter somebody presses stop on a live queue, during working hours, and writes down three numbers — how many seconds passed before it actually stopped, how many cases were left half-finished, and how many people found out without asking. The date of the last test is a field in the approval policy; without it the button is a declaration. Where agents hold rights to act inside systems, the same test applies to the scope of those rights — we write about that on our AI agent implementation page, and what separates an agent from an ordinary bot we settled in the piece on the difference between an AI agent and a chatbot.

Three numbers from the approval log

All three are counted separately for every category of case — an averaged queue is a queue in which nothing is visible.

1. The share of corrections and rejections. The proportion of cases in which the person changed anything in the proposal or rejected it.
2. The median time to decide. From opening the case to saving the decision. The median, not the mean — one case left over a weekend ruins the mean.
3. Disagreement in a sampled audit. The share of cases, in a random sample of approved ones, where a second person looking at the same material would have decided differently.

The first number is the most widely reported and the weakest at settling anything. A correction rate close to zero has four explanations: the model is right in this category, the queue receives only easy cases, the approver is clicking, or the approver cannot see enough to disagree. No single number separates them. The third asks a different question — whether the approved outcomes were right — and it falls silent exactly when the model is right, because a clicking approver and a reading one then produce the same empty sample; an auditor who sees no more than the approver did shares the same blind spot. It takes the second number beside it to say whether anybody was looking, which is why we treat a sampled audit as a compulsory part of designing the queue rather than an extra, and never read it on its own. The categories where a log like this pays for itself soonest are among our examples of AI automation — the point about replies prepared for approval ends with a promise that this text keeps.

The clicking test: median time to decide against reading time

Start by calculating the lower bound for your own screen, because it is the one number you do not have to obtain from anybody. An approval screen carrying roughly 180 words of content, at a reading rate of 175 to 300 words per minute, gives 36 to 62 seconds of reading alone, before anyone compares anything against the system.

The calculation is illustrative. The range of 175 to 300 words per minute comes from Marc Brysbaert’s 2019 meta-analysis covering 190 studies and 18,573 participants, and it is measured on English non-fiction — so it fits an English approval screen directly, while a Polish screen carries the same content in fewer words and the same arithmetic understates it there. Either way the result is a floor and not an expected time, because reading is not the same thing as verifying.

Then set that bound against what the log shows:

The table is an illustration and not measured data — it shows the shape of an approval log, not the result of a particular deployment; every category has a different screen and therefore a different reading floor, calculated in the same way as above. What matters in it is the first row: a median of six seconds against a floor of thirty-five seconds calculated for that screen is a strong signal that the category is not being approved but clicked through, and the 1.2% correction rate sitting beside it proves nothing, because it comes out of those same six seconds. It is the only row that looks ready for loosening on paper, and the only one whose numbers must not be trusted. The second row is its opposite: a median above the floor and 14% corrections make a queue where somebody really is looking — and that is precisely why the category is not promoted, because a person is still changing one case in seven.

The test itself — comparing the median against a calculated reading floor — is our own idea rather than a method from the literature; the literature supplies only the reading rates. It works in one direction: a median clearly below the floor is a strong signal that nobody is reading, while a median above the floor proves nothing yet. Agreement between a person and the system that is too good is a reason to check, never a diagnosis.

Automation bias: why a good system spoils oversight

Oversight degrades faster the better the system works: the less often something needs correcting, the less it pays to check. The Regulation names this outright — Article 14(4)(b) requires the overseeing person to remain aware of that tendency, which the Polish text of the Act renders as “błąd automatyzacji”. A review of 74 studies by Goddard, Roudsari and Wyatt (JAMIA 2012) points to training, an explicit assignment of accountability to the approver, and screen design — including whether the system offers information or goes straight to a recommendation — as mitigating factors. Lyell and Coiera (JAMIA 2017) add that the phenomenon appears in single tasks too, wherever verifying the result is costly, and immediately caution that the literature is fragmented and that few studies report statistical significance against a control group.

So be careful with a diagnosis reached in advance. Three pre-registered experiments by Alon-Barkat and Busuioc (JPART 2023) on a combined sample of 2,854 people found no automation bias: algorithmic advice was followed at roughly the same rate as equivalent advice from a human expert. Designs that do curb over-reliance effectively — such as forcing the reviewer to commit to a decision before the suggestion is shown, in the study by Buçinca, Malaya and Gajos (2021) — collected the worst user ratings in that same study. That is a laboratory rather than a real workplace, but expect the solution that works for you to be the one your team protests against most loudly. The diagnosis itself — that human oversight is in practice often merely formal and amounts to passively authorising the algorithm’s decision — was made by Martyna Kaczmarczyk in the Polish public-law journal Studia Prawa Publicznego (2/2026); all that is new here is how to measure it.

The rule for promoting and demoting a category

Promoting a category — from “we approve every case” to “we approve exceptions” — has to have a numeric threshold, or it happens by feel in a week when the queue has grown. We take the sample size from the rule of three (Hanley and Lippman-Hand, JAMA 1983): after a run of n cases with no error, the upper bound of the 95% confidence interval for the error rate is roughly 3/n.

That is the upper bound of the 95% confidence interval after a clean run, not a threshold of acceptable quality. The latter is a business decision, and for some categories it reads “we do not promote at all”.

Our starting values, revised after a pilot: a category is promoted once it has at least 200 consecutive cases behind it with no error that had to be unwound, a correction rate below 2% across that run, sampled-audit disagreement below 1 in 50, and a median time to decide above the reading floor — that last condition exists so a run without errors does not turn out to be a run without looking. Demotion has a single trigger and takes effect immediately: one approval whose consequence left the company and had to be recalled — a correction at a counterparty, a withdrawn shipment, a commitment reversed — returns the category to approving every case the same day. The statistics come afterwards.

How to draw the sample in statistical oversight

The sample is drawn from the whole population of cases approved in the period, not from the ones that struck somebody as odd — otherwise you are measuring the alertness of whoever picks them rather than the quality of the queue. Three conditions are enough to start: the draw is automatic and recorded in the log alongside the decision, the sample is pulled separately for each category, and the cases in it are reviewed by somebody other than the person who approved them.

The rest — sample size at different volumes, what to do with a run that produces no disagreement at all, and how not to pay more for the audit than the process itself is worth — is material for a separate piece.

Articles 14 and 26 of the AI Act, and “solely” in the GDPR — as log fields, not as a lecture

The date first, because it is missing from the article we began with. Articles 14 and 26 of the AI Act do not yet apply: they apply to the high-risk systems in Annex III from 2 December 2027, and to the systems in Annex I from 2 August 2028 — Article 113(c) of the consolidated text as at 27 July 2026, checked on 8 September 2026. That is not a reason to wait: retrofitting a screen, a queue and a log onto a running process costs more than designing them from the outset. It is a reason not to buy anything under pressure from a deadline that does not exist. What applies sooner we have collected under our AI Act technical compliance sprint.

What settles the matter today is Article 22 of the GDPR and its word solely — we set the doctrine out in the piece on GDPR and an LLM rollout, and here a single sentence from the guidelines is what counts. The Article 29 Working Party, in WP251rev.01 — subsequently endorsed by the European Data Protection Board — writes the thing at the heart of this text: a controller “cannot avoid the Article 22 provisions by fabricating human involvement”, and routinely applying automatically generated profiles “without any actual influence on the result” remains a decision based solely on automated processing. Oversight is to be “meaningful, rather than just a token gesture”, carried out by somebody with the authority and competence to change the decision. Whether your queue fits that description is for a lawyer to settle; what we answer for is that it can be demonstrated from the data.

It is demonstrated through fields in the log — but that is our design, not a list from the Regulation. The AI Act requires the recording of events (Article 12) and enumerates a minimum set of fields only for the biometric systems in Annex III, point 1(a). Our list: who decided, when, what they saw on the screen at the moment of the decision, what confidence the system reported, what they changed, how long they spent on the case, and whether a halt or an escalation was triggered. When buying, remember that Article 14 binds the provider and Article 26 the deployer, which normally means you. Log retention is settled together with the rest of the decisions about data, which we write about under data security and GDPR. This describes the technical and organisational side; it is not legal advice.

How this looks in the tool: n8n, Make and Power Automate (as at 8 September 2026)

The rows come from vendor documentation read on 8 September 2026 (n8n, Make, Microsoft Learn) and may change. The third column is the important one: you buy the approval mechanism ready-made, and you still add the log fields yourself. We described how to build the flow in our n8n guide — that is the construction, this is what has to go inside it.

For agents, n8n has a separate gate for approving tool calls: the person sees which tool the model wants to run and with what parameters. The gate itself operates in the node — approval runs the tool, refusal cancels it. The documentation notes, however, that the system prompt must additionally describe which tools require consent and what the model should do after a refusal (n8n documentation, as at 8 September 2026) — and that part is an instruction, not a control. Only what works independently of the model is a control.

When not to loosen oversight — and when a queue is the wrong design

The promotion rule has exceptions that no number of error-free cases overrides:

• Irreversible actions. A payment, a shipment, a publication, a deletion of data, a commitment towards a counterparty. There is no promotion here — at most a narrower scope within which the action may be carried out.
• Determinations with a significant effect on a particular person. Employment, credit, access to a service. A separate legal regime and separate accountability; some of these applications we do not build at all, which we write about under what we do not do.
• Categories with no log of their own. If you cannot count the three numbers for that category separately, you have no basis for promotion, only an impression.

A queue can also simply be the wrong design. At a few cases a day it usually costs more than the cases themselves — the screen, the rota and the audit have to be maintained regardless of volume; work out where exactly that boundary lies for your own process, because there is no public figure for it. A queue with no owner is worse than no queue, because it looks like a safeguard and is not one. And where a process has no settled course yet, the first step is not a queue but a determination of who approves what and what happens in the exceptions — one of the things we advise against building on our page on AI automation for business.

A one-page approval policy

Everything above fits on a single page, and it is the only document this step requires. For each category of case, write down eleven fields:

1. The category and its volume.
2. The supervision mode: every case, exceptions, or a sample.
3. The queue owner by name, and their deputy.
4. The coverage window, the deadline, and what happens once it passes.
5. What is visible on the decision screen.
6. Who is entitled to approve.
7. What is written to the log.
8. The halt procedure and the date it was last tested.
9. Promotion criteria: number of cases, correction threshold, disagreement threshold, time threshold.
10. The single event that demotes a category immediately.
11. The policy review date.

This is an operational template, not a model legal document — its value lies in every field having somebody to fill it in, and an empty row being visible at once.

What to do next

Take one queue you already have and fill in those eleven fields for it. Three are usually blank: the owner, the halt procedure and the date it was last tested. Then calculate the reading floor for the decision screen and compare it with the median from the past month. If the median is lower, you have your answer before the first sampled audit — and a concrete reason to start by changing the screen rather than the model.