Explainability in Financial Services AI: What Regulators Are Actually Asking For

Every financial services client we work with eventually asks the same question: how do we make our AI explainable? It is the right question, but it is incomplete, because explainability is not one requirement. It is several different requirements wearing the same word, and each regulatory body asking for it wants a different answer.

Why the same word means different things

A model risk management examiner, a fair lending regulator, and a consumer protection auditor are not asking the same question when they ask you to explain your model. The model risk examiner wants to understand the model's behavior across its full range of inputs and confirm someone at your organization can defend design choices under review. The fair lending regulator wants to know whether the model produces disparate outcomes across protected classes and whether you can demonstrate why, in terms a compliance officer can act on. The consumer protection auditor, or a plaintiff's attorney in a dispute, wants a specific, individual answer: why was this customer denied, at this decision, on this date. Building one artifact and calling it "explainability" satisfies none of these audiences well, because each one is asking a structurally different question.

What examiners and regulators are actually asking for, body by body

In our engagements, we have found it useful to map explainability requirements directly to the regulatory function asking for them, rather than treating explainability as a single deliverable.

• Model risk management (SR 11-7 and equivalent frameworks): documentation of model design, data lineage, validation methodology, performance monitoring, and evidence of ongoing independent review. This is explainability as governance process, not as a single output.
• Fair lending and ECOA/Regulation B: statistically defensible adverse action reasons, disparate impact analysis, and the ability to show that a denial reason is both accurate and one of the actual drivers of the model's output, not a plausible-sounding proxy.
• Consumer-facing disclosure requirements: a specific, individual-level explanation a customer can understand, delivered in plain language, tied to the actual decision made about them.
• Internal audit and the board: a level of explanation aimed at demonstrating oversight capability, showing that senior leadership understands what the model does and does not do well enough to be accountable for it.

Most organizations build for the first audience and assume it covers the rest. It does not. A model validation report that satisfies an SR 11-7 examiner will not produce an individual adverse action notice a customer can understand, and it will not, on its own, demonstrate the absence of disparate impact.

What a practical explainability program actually looks like

The organizations that get this right build explainability in layers, matched to audience, rather than chasing a single technical solution like SHAP values or LIME outputs and assuming that satisfies everyone. Feature attribution methods are necessary infrastructure, but they are inputs to an explanation, not the explanation itself. Someone still has to translate attribution scores into a defensible adverse action reason, and someone still has to aggregate individual explanations into a portfolio-level fairness analysis a board can review.

We build explainability programs around the decision, not the model. What decision is being made, who is entitled to an explanation of it, and what regulatory function will eventually review it. That framing produces the documentation, the individual disclosures, and the aggregate analysis in parallel, instead of retrofitting three deliverables out of one technical artifact after an examiner asks a question you were not prepared for.