Fraud stopped being a purely human exercise someday within the final yr. Not all of it, clearly; I imply, most fraud continues to be somebody typing on a cellphone someplace attempting to maneuver cash that is not theirs.
However sufficient of it has shifted that Experian’s 2026 Way forward for Fraud Forecast put a reputation on it. AI brokers transacting and buying on folks’s behalf at the moment are an actual driver of monetary fraud.
The report calls it “machine-to-machine mayhem,” which feels like a headline written to get clicks, and truthfully, it labored.
As a result of a official buying agent and a fraudulent bot can present up in a transaction log trying principally equivalent. No person’s discovered the inform but.
I need to sit with why that is an even bigger deal than it sounds. Most fraud detection, at each scale, from a neighborhood fintech to a Fortune 500 financial institution, has quietly rested on the identical assumption for a decade or extra.
Which is that there is a human on the opposite finish of the transaction, and people depart fingerprints.
Issues like how they kind, after they change gadgets, principally the way in which somebody behaves after they’re anxious or in a rush.
Each fraud mannequin, whether or not anybody constructing it says so out loud, is educated to note when a human stops appearing like themselves.
I ran right into a smaller, dumber model of this actual drawback whereas constructing a fraud detection system, one which leaned laborious on SHAP to elucidate itself.
It is price telling as a result of the factor that received me out of bother on the time is the precise factor I now assume is working out of street.
The query that wasn’t actually in regards to the mannequin
At my presentation, one of many panelists leaned again in his chair and requested a query that had nothing to do with precision or recall.
If a CEO makes a one-million-dollar transaction, and a college scholar makes the identical transaction, does your system flag them the identical means?
I keep in mind the room going quiet, and I keep in mind taking longer to reply than I wished to.
My mannequin, a Random Forest classifier I would educated on the PaySim dataset after benchmarking it towards XGBoost and LightGBM, did not know or care who was transacting.
It appeared on the transaction: quantity, timing, gadget, velocity.
Mainly the identical options for a CEO’s account as for a broke undergrad’s.
On paper, that is what equity in ML is meant to appear to be. In entrance of a panel whose complete job is discovering holes in danger fashions, it appeared like I would missed one thing apparent.
This is what really saved me, and it wasn’t a better mannequin.
I would wired SHAP into the pipeline sitting on prime of the classifier, principally as a result of a supervisor earlier within the yr had pushed again on treating the mannequin as a black field, and it turned out to be the rationale I survived that query.
I might pull up the precise transaction, present the ranked SHAP values, and say: here is what really drove this prediction, so as, and with numbers hooked up.
That query has stayed with me longer than virtually anything from that presentation. Not due to what it uncovered about my mannequin particularly, however due to the belief sitting beneath the entire subject: that the factor making a transaction has a life exterior the transaction, a job, a historical past, a purpose to be shifting that specific sum of money.
In easier phrases, context.
If you’d like the complete construct, the mannequin comparisons, the PaySim dataset quirks, and what my supervisor really mentioned when he noticed the SHAP output for the primary time, I wrote all of it up right here: I Educated Six Fashions for Fraud Detection, and the Finest One Is not in Manufacturing. This piece picks up from that one however does not require you to have learn it.
···
Why explainability stopped being optionally available
Black-box fraud fashions have all the time had a belief drawback, agentic AI or not.
The fashions that carry out greatest, deep ensembles, gradient-boosted bushes, are usually the toughest to interpret, and that is a genuinely annoying trade-off for those who’ve ever needed to clarify a flagged transaction to somebody who is not a knowledge scientist.
If a compliance analyst cannot inform whether or not a flag is an actual sign or simply noise the mannequin latched onto, they’re going to both rubber-stamp every little thing or ignore the mannequin totally.
I’ve seen each occur, and neither one is why you constructed the factor.
SHAP earned its spot in fraud stacks by answering a slender, unglamorous query, not “is that this mannequin good” however “why did it do that, for this one transaction, proper now.”
It is much less thrilling than mannequin structure, which is possibly why it does not come up in dialog as a lot because it in all probability ought to.
A danger rating with no reasoning hooked up is near ineffective the second a human has to behave on it. That was my actual drawback on the presentation.
As an alternative of “92% fraud chance,” you get one thing like: the quantity is excessive for this account, the account is beneath 48 hours outdated, or the service provider has prior flags.
Individuals belief that.
Or a minimum of they belief it greater than a quantity with nothing beneath it.
The bottom is shifting beneath the fingerprint
This is the half that bugs me.
Each explainability technique constructed for fraud detection, SHAP included, explains a choice utilizing options that describe a transaction: quantity, time, gadget, velocity.
Now stick with me right here.
These options exist as a result of they correlate with how people behave when one thing’s incorrect.
A scholar’s account abruptly shifting CEO-sized cash appears to be like unusual exactly as a result of we assume a secure human behind it, with a sample of life the anomaly is breaking from.
An agent does not have a sample of life in any means that maps onto these options.
It does not get drained at 2 am. It does not fat-finger a switch or panic-send cash to the incorrect account.
It will probably transact at a velocity and consistency no human might maintain for 5 minutes, and that breaks the entire premise fraud detection has quietly run on: that deviation from a human baseline is the sign.
This is not a sometime drawback. It is already exhibiting up within the numbers.
The quantity I discover extra attention-grabbing than the headline 84% is the 60% who anticipate AI-mediated banking to make conventional fraud defenses much less efficient.
That factors to one thing deeper than criminals merely getting higher instruments. The indicators these defenses had been constructed round could themselves be altering.
Regulators are already circling this, and never within the ordinary glacial means.
NIST’s AI Agent Requirements Initiative, launched in February 2026, treats brokers as identifiable non-human principals that want their very own credential lifecycles and audit trails, on the reasoning that current auth frameworks had been constructed for people and do not stretch to cowl autonomous software program.
Within the US, it is already additional alongside than a requirements initiative. Senator Mark Warner’s draft AI AGENT Act, floated in mid-2026, would create a federal registry of trusted brokers and require them to behave transparently in a consumer’s curiosity on main platforms.
I do not know if this actual invoice survives contact with Congress; most do not of their first draft.
However the intuition behind it’s the factor price noticing: lawmakers have landed on id and accountability because the load-bearing query, earlier than the trade has agreed on how you can even log an agent’s reasoning in a means a human might audit later.
What breaks, and what does not
I need to be clear: I do not assume SHAP is useless or dying, belief me, not even shut.
Function-level explanations nonetheless matter for the transaction-shaped slice of fraud, and people aren’t going anyplace as fraud actors, sadly.
However function attribution on transaction fields solely ever answered why did this look incorrect. Agentic fraud is dragging a distinct query onto the desk.
Why did this actor do that in any respect?
That second query desires a distinct type of hint totally.
Not transaction options however the agent’s precise choice path, what instruments it referred to as, what it was approved to do, and whether or not the motion it took falls inside or three steps exterior the scope somebody delegated to it.
A system constructed to elucidate a Random Forest’s verdict on a greenback quantity has actually nothing to say about whether or not an agent went off-script upstream of the transaction it will definitely made.
That is nearer to auditing a trajectory than scoring a knowledge level.
I’ve gone on the lookout for somebody who’s solved this correctly and principally discovered papers that gesture on the drawback with out really closing it, which both means I am dangerous at discovering issues or it is genuinely nonetheless open. Presumably each.
(Additionally, and it is a facet thought I can not totally again up: I ponder if a part of the rationale no one’s cracked this but is that agent trajectories are logged by the identical corporations constructing the brokers, and there is not quite a lot of incentive to make that logging legible to an out of doors auditor. That is extra of a suspicion than an argument, so take it for what it is price.)
My guess, and it is only a guess, is that this does not get fastened by making SHAP extra subtle.
It in all probability will get fastened by borrowing from a totally totally different nook of AI security, agent monitoring and trajectory analysis, after which stitching it onto the explainability self-discipline fraud detection already has.
Or possibly these two fields by no means actually merge and somebody builds a 3rd factor from scratch. I shuttle on which one is extra doubtless.
···
Remaining ideas
I constructed a system to catch a human doing one thing a human would not usually do, and SHAP was the factor that made it defensible in entrance of a room of individuals paid to search out its weak factors.
That drawback continues to be actual, and nonetheless not totally solved, so this is not me saying my fraud detection system or others prefer it are performed for.
However the goal has moved someplace the present toolkit wasn’t initially constructed to comply with, and I do not assume that is a small footnote.
It is the hole between explaining a rating and explaining an actor. Proper now most of us, me included, solely actually know how you can do the primary one.
When you’re constructing or evaluating fraud detection, the query I would really sit with is not whether or not your mannequin is correct.
It is whether or not your explanations nonetheless imply something as soon as the factor on the opposite finish of the transaction stops being an individual.
···
Earlier than you go!
I write in regards to the hole between what a mannequin does and what you’ll be able to really show it did, particularly as soon as the factor making selections stops being clearly human. You’ll be able to subscribe to my publication if you would like extra of that.
Join With Me
