Responsibility, Law & Governance
AI Incident
When an AI system causes real harm: the wrong answer with consequences, the discriminatory decision, the agent with too much access. Mature organizations treat this like IT security incidents — report, contain, review, without a blame theater. Whoever collects and analyzes incidents learns from small harms before large ones occur.
An example
The customer assistant stated a wrong cancellation period for three weeks. It came to light through a complaint. The questions afterward are always the same: How many customers were affected? What exactly was said? Since when? What changed — model, prompt, document? Without a log none of these questions can be answered, and then all that remains is a blanket apology to everyone.
A closer look
An incident is more than an error: an output or action from which harm has arisen or could have arisen. Handling includes a reporting route, reconstruction from the log, an immediate measure and the question of what was structurally missing.
Why it matters
For AI systems, most organizations lack what is a matter of course for IT outages: a definition of what an incident even is, a reporting route, an owner and a follow-up review. This is not a large structure — one page is enough. It should settle three things: what must be reported (near misses too), to whom, and that every incident turns into an automated test. The last point is what turns annoyance into learning. The EU AI Act requires reporting for high-risk systems anyway.
Commonly misunderstood
Without a reporting route, incidents are cleaned up one by one and not counted. Then the very frequency is missing that would show it was not an isolated case.
Where this term does its work in the magazine.
1 article uses the term or takes up its principle.