The EU AI Act wants your logs to prove something. Can they?

Since 2 August 2026, the AI Act’s obligations for high-risk systems apply, with penalties that reach into percentages of global turnover. Most of the compliance conversation is about paperwork. That’s the wrong end of the problem: nearly everything the Act demands of a high-risk system is an architecture property — your system either has it or it doesn’t, and no document can retrofit it.

What the auditor will actually ask

Strip the recitals away and the high-risk obligations converge on four questions someone will eventually ask about a specific output of your system:

1 · Where did this result come from? (traceability)
2 · Can you reproduce it? (record-keeping)
3 · Who was responsible for accepting it? (human oversight)
4 · How do you know it’s right? (accuracy and robustness)

Notice what kind of questions these are. None of them is answered by a risk-management PDF. Each is answered — or not — by what your system actually recorded at the moment it produced the result, months before anyone thought to ask.

Record-keeping is not logging

Every system has logs. Almost no system has records — in the sense the Act cares about, where a specific past output can be explained and reproduced.

The difference is what you store. Logs capture what happened around a computation. Records capture what went into it. In the filing engine I built for Saudi tax compliance, every computation begins by freezing all of its inputs into a versioned checkpoint — persisted before the engine runs, so even a crash mid-computation leaves a replayable record of what was attempted. The computation itself is a pure function over that checkpoint: same inputs in, bit-identical result out, and a test suite that asserts exactly that.

That last point matters more than it looks. Rules change; regulators issue corrections. A corrected rule should change your answer. What you need is to know that it did — which is only possible if you can re-run yesterday’s inputs under today’s rules and diff the results.

Traceability: every figure answers for itself

“Where did this number come from?” has to be answerable per figure, not per system. The standard I build to: any number in any output can be interrogated back to three things — the rule that produced it (down to the clause it implements), the input it consumed (down to the source row), and the run that computed it (replayable, as above). If any of the three is missing, the number doesn’t leave the system.

This is cheap to build in from the start and brutally expensive to retrofit — which is precisely why it’s the thing to check before you ship, not after you’re asked.

Human oversight that can’t be skipped

The Act requires effective human oversight of high-risk systems. There are two ways to implement that, and they are not equivalent.

The weak way is policy: “a person reviews model outputs before they take effect.” Policies live in people’s heads, and they die under deadline pressure.

The strong way is structure. In the filing engine, models propose how a client’s accounts map onto tax-form lines — but the computation engine’s parameter is literally named approved_mappings. There is no code path on which a model’s suggestion reaches the arithmetic without a person having accepted it. The oversight isn’t a step in a process someone could skip; it’s the type signature of the system.

Accuracy you can demonstrate

“How do you know it’s right?” has a bad answer (“we tested it”) and a good one: the system checks itself, and refuses when it can’t. Two patterns carry most of the weight:

Self-auditing arithmetic. Controls that re-derive key results by an independent route and compare, every run — not once in CI, but as part of producing each output. A control that is right most of the time is not a control; it’s an opinion with good manners.

Refusal at the boundary. Where a rule exists but isn’t implemented, the system rejects the request before computing — because a pre-flight rejection is a support ticket, while an approximate answer is a finding waiting to surface. An auditor who sees your system decline out-of-scope work trusts every answer it did give more, not less.

The documentation trap

The compliance industry will happily sell you the paper: risk registers, conformity templates, model cards. You’ll need some of it. But there’s a directional choice hiding here: either the paper describes the system, or the system produces the paper. In the first case every audit is an archaeology project and every document drifts out of date the day it’s signed. In the second, the audit trail is a by-product of normal operation — the checkpoint, the trace, the control results, generated fresh for any output on demand.

Five questions to ask your own system

1 · Pick any number your system produced last quarter. Can you show the rule, the input and the run behind it — today?
2 · Can you re-run it and get the identical result? How do you know?
3 · Can a model output reach anything consequential without a recorded human acceptance?
4 · When your system is asked something outside its implemented scope, does it refuse — or approximate?
5 · If a rule changed tomorrow, could you list which past outputs are affected?

Five yeses and the Act is mostly a formality of writing down what your architecture already does. Any nos, and you know exactly where the engineering work is — better to find it now than in a finding.

← All insights