Financial services · 4 min read
The question that stops projects is not what the system did, but what it was configured to do on the day. Four properties make that answerable.
The audit question people prepare for is 'what did the system do'. The one that actually stops projects is 'what was the system configured to do on the day in question, and what has changed since'. Without configuration history that question has no answer at all, and a control you cannot reconstruct is a control you cannot attest to. This is a walkthrough of the four properties that make an AI system auditable, written for the engineer who has to build them rather than the reviewer who will ask for them.
Scope note: this is engineering guidance about controls, not legal advice and not a statement of what any rule requires. SOX and securities recordkeeping are named because they are the frameworks these reviews are conducted under; their text, your regulators and your own counsel and internal-audit function govern what applies to you. NeuralSeek is deployed inside your environment, so the compliance position is yours — nothing described here is a certification held by us.
In a regulated firm, an AI system that touches books, records or customer interactions is not evaluated on capability first. It is evaluated on whether it produces a complete, trustworthy account of what happened: who did what, when, under which configuration, and whether that account can be handed over promptly and stand up to someone who assumes nothing.
That is a tractable engineering problem, and it decomposes into four properties. None of them is exotic. What makes them hard is that they have to be in place from the beginning — an audit trail cannot be added to history.
Two complementary logs, plus control over where they live. An organisation-wide, tamper-evident record of every interaction gives you the account. A record of the prompts themselves gives you the reconstruction — when a specific answer is questioned later, you can show precisely what was asked and what material the system was working from.
The part teams under-plan is storage. Retention obligations for regulated records are usually longer than anyone's default log retention and often require a specific kind of durability, so the ability to route these records to the backend your firm already uses for regulated records — rather than to whatever the application defaults to — is what makes the control real rather than nominal.
This is the property most AI systems lack and the one that most often turns a review into a project. If the retrieval thresholds, the guardrail policy or the model assignments changed at some point, and nobody can say when or to what, then every answer produced before that change is unexplainable.
What is needed is a timestamped version of the whole configuration captured on every change, so the exact state on any past date can be recovered — and a readable difference between any two versions, with the ability to revert. That converts a control change from an untracked edit into a reviewable, attributable event, which is what a controls framework is asking for in the first place.
A trail that cannot be tied to an identity fails the audit at one end; a trail that leaks secrets fails it at the other, and more expensively. Every recorded interaction should attach to a known access path and identity rather than to an anonymous call, which is a design decision about how the system is fronted rather than something you can add later.
At the same time, keys and connection secrets must be kept out of prompts, out of responses and out of the logs themselves. The record you produce as evidence is a document that will be copied, mailed and stored by people outside your team. It should not be a place credentials can be found.
The final property is whether the trail can be turned into something a reviewer accepts without a translation layer. In practice that means mapping each control to the clause of a recognised framework it satisfies — ISO 42001 and the NIST AI Risk Management Framework are the two most firms anchor on — so that the answer to a walkthrough request is a document rather than an investigation.
The test is simple and worth applying to whatever you are evaluating: when the request arrives, is producing the evidence an export, or is it an emergency?
The difference between an AI initiative that ships inside a regulated firm and one that stalls is rarely the model. It is whether the audit trail is a product of the configuration or something reconstructed under pressure each time someone asks.
All four properties above are settings in NeuralSeek rather than components to integrate, and they apply to every workflow built on the deployment rather than to the ones whose authors remembered. That is the whole argument: not that these controls are unusual, but that inheriting them is very different from re-implementing them.
Keep reading
Each article is the long version of an argument a platform page makes in a paragraph.
Every article links back to the page that owns its subject.
Why it cannot be patched at the model layer, what an indirect payload hidden in a document actually looks like, and why the containment boundary has to sit outside the thing being attacked.
Read on prompt injectionRegular expressions find the personal data that looks like personal data. 'Maria in room 4B' identifies a patient and matches nothing. Where each approach fails, and why the two belong in sequence.
Read on PII detectionTalk to a NeuralSeek expert about how it fits your stack, where your data has to live, and the governance your auditors already expect.