Andrej Karpathy has been blunt about a trait every LLM builder eventually runs into: models are sycophantic. They want the task to go well. They rarely push back, they don't like saying "I'm not sure," and they almost never stop halfway to ask a question. His advice to builders is to "surface uncertainty, don't hide it." The default behavior is the opposite.
The quiet failure
In a chatbot, this is a mild annoyance. In compliance, it's a liability with a delay on it. Draft a SAR, run a sanctions review, map a control: the output comes back structured and confident whether or not it should be. If an applicability condition got dropped along the way, nothing about the document looks wrong. It fails silently, and silent failures in compliance surface years later, usually in front of an examiner.
People call this hallucination. I think that misses the point. The real problem is unchecked completion: the model filled a gap it should have flagged, kept going, and reported the job done. You can't prompt your way out of it. Completion is the thing these models are trained to do.
Planner → generator ⇄ evaluator
We don't think compliance AI should be a single answer engine. It should be a loop with three roles.
The planner sets the scope: what evidence is required, what assumptions are allowed, what triggers escalation to a human. The generator drafts the work. The evaluator challenges the draft before anyone relies on it. Did it use the right evidence? Follow the right rules? Name its uncertainty?
The generator and evaluator argue. Drafts bounce between them until they grade out, and anything they can't resolve goes to a person, with the findings attached.
Quality has to be gradable
None of this is exotic. It's what a good human compliance workflow already looks like: maker, checker, escalation path. The temptation with AI is to skip it, wire a generator straight to the filing queue, and call it automation. Teams that give a model the full loop get a force multiplier. Teams that give it only the middle step get confident drafts with nobody arguing back.
So the test isn't "did the AI produce an answer." It's: right evidence, right rules, uncertainty named, edge cases routed to review. That's what makes quality gradable instead of assumed.
How we build it
This is how Tracfox works. Every agent drafts; none of them decides. Output lands in a review queue with the citation that justifies it. Ambiguous applicability gets flagged for a human, not guessed. If the system can't cite it, it says so.
AI will transform compliance work. Whether it transforms your liability depends on the architecture around it.