Responsibility, Law & Governance
Explainability
Why this decision? With deep networks there is rarely a complete answer — but there are useful approximations: which inputs were decisive, how confident the model was, which rules applied. Where decisions affect people, explainability is a legal obligation and a question of fairness at once. Sometimes the simpler, explainable model is the more responsible choice.
An example
A loan application is rejected. The answer “the model looked at income and length of employment” is not a justification — it names features, not the decision rule. A justification would be: with this income and this length of employment, the application would have been approved at a net income 300 euros higher. That statement is verifiable, contestable and usable for the person affected. The first is not.
A closer look
The methods range from feature weighting to counterfactuals: what would have had to be different for the decision to flip? With large language models, genuine explanation is a research topic; in operation one makes do with source citations and logged intermediate steps.
Why it matters
In practice it pays to keep two things apart. Traceability of the process — what happened, in what order, on what data basis — is achievable and a matter of good logging. Explainability of the decision — why this result and not another — is in principle difficult with deep models, and all common methods deliver approximations, not truth. When a regulator demands justifications, the honest answer is therefore sometimes to use a simpler, inspectable method for that purpose.
Commonly misunderstood
A justification supplied by the model is itself an output. It describes what sounds plausible, not necessarily what led to the answer.
Where this term does its work in the magazine.
1 article uses the term or takes up its principle.