A new regulatory reality

The EU AI Act, which entered into force in 2024, represents the world’s first comprehensive legal framework for artificial intelligence. For financial institutions operating in the EU, this is not an abstract policy document — it has direct, immediate implications for how AI models are built, documented, tested, and deployed.

At the heart of the regulation is a simple but demanding principle: AI systems that affect people’s lives — including credit decisions, insurance pricing, and fraud detection — must be explainable, auditable, and fair. Black box models that produce outputs without traceable reasoning are no longer acceptable in high-risk use cases.

What “explainable” actually means in practice

Explainability is not a binary property. It exists on a spectrum, and the EU AI Act’s requirements are nuanced. At minimum, a financial institution must be able to:

  • Describe the data the model was trained on and any known biases in that data
  • Explain how the model reaches a decision at the individual case level
  • Demonstrate that the model has been tested for discriminatory outcomes
  • Maintain documentation sufficient for a regulator to audit the model’s behaviour

Techniques such as SHAP and LIME have become standard tools for post-hoc explainability. But increasingly, regulators are also interested in intrinsically interpretable models for the highest-risk decisions.

The compliance burden is real — but manageable

Many institutions are finding that building explainable AI is not dramatically more difficult than building opaque AI — provided it is treated as a design requirement from the start rather than an afterthought. The challenge is primarily organisational: documentation, governance, and testing processes need to be established and maintained.

“Explainability is not the enemy of performance. In our experience, models designed to be explainable often generalise better precisely because interpretability constraints prevent overfitting to noise.”

What this means for financial institutions

Practically speaking, every financial institution deploying AI in credit, insurance, fraud, or customer-facing decision-making needs to conduct an AI Act compliance audit. This involves mapping AI systems against the regulation’s risk categories, assessing documentation gaps, and developing remediation roadmaps. At Cytrus, we build AI systems that are compliant from day one — with explainability as a core feature, not an afterthought.