Validation Is No Longer Enough
A scalpel sits on a shelf until someone needs it. An AI model keeps learning, drifts, adapts, and in doing so falls through every grid built for static medical devices. Why the EU approval system has to be fundamentally rethought for adaptive AI.
Article 1 of the European Medical Device Regulation states its goal plainly: patient safety. Over decades, Europe has built a comprehensive system around it: risk classes, conformity assessment, CE marking, post-market surveillance. The system works. It has prevented scandals, set standards, built trust.
And it has a fundamental problem: it was designed for a scalpel.
A scalpel has a defined purpose. It sits on a shelf until someone needs it. It doesn't change. It doesn't learn. It doesn't drift. At some point it is disposed of. The regulatory requirements that follow from this model (a one-time approval, a defined redesign procedure for changes, reactive post-market monitoring) fit it perfectly.
They don't fit AI.
The Adaptivity Problem
An approved AI model can change in three ways without any human actively intervening.
The first is the most obvious: functional extensions such as new indications, new data inputs, new outputs. In the US, there has been the Predetermined Change Control Plan (PCCP) for some years now, and the EU has a similar, less established concept. That is a step in the right direction.
The second is trickier: site adaptation and personalization. A model deployed in a clinic can adapt to that clinic's data: its patient populations, diagnostic habits, technical infrastructure. After six months of operation, it is no longer the same as on day one. The current regulatory model treats it as if it were.
The third is the most dangerous: performance degradation. A model that performed validly on historical data at the time of approval can get worse in live operation, for instance through sample drift (new data no longer matches the training data), guideline changes or device updates. Without continuous monitoring, you often notice only once harm has occurred.
What Post-Market Surveillance Does Today, and What It Doesn't
One could object: there are monitoring obligations after approval, after all. True. Manufacturers have to collect data, report regularly, operate a quality management system.
The problem: this data isn't really used. It isn't systematically analyzed. It doesn't feed into regulatory decisions. And it isn't used to continuously improve the device itself. In today's system, post-market surveillance is a compliance instrument, not a learning mechanism.
That is the core of the critique: not that the system is too strict, but that it intervenes at the wrong point.
From Validation to Continuous Assurance
The thesis I am working on with my research group at EKFZ Dresden: Validation alone is no longer enough. The idea of checking a device once, certifying it and then letting it run is conceptually wrong for adaptive systems, not badly executed but set up wrongly from the start.
What is needed instead is Continuous Assurance: a regime that learns from operation itself, detects drift signals, monitors subgroup performance and systematically feeds these findings back to manufacturers and regulators.
This isn't science fiction. Much of the data required is already being collected today; it just isn't used. Clinics record outcomes, error rates, usage behavior. What is missing is the regulatory infrastructure that turns this into a continuous GlossaryClosed Loopa system that learns from its results (action → measurement → insight → action).Go to entry ↗.
One concrete building block: the Broad GlossaryModel Carda model’s profile (capabilities, limits, risks).Go to entry ↗ concept our research team is currently working on. It extends the familiar Model Cards (which the FDA also recommends) to systems that can theoretically cover thousands of diagnoses, and communicates performance not as an average but granularly across the entire disease atlas. What a system does well for internal medicine, it may not be able to do at all for rare diseases. That has to be visible.
What MDR 2.0 Brings, and What Remains Open
The European Commission has put a reform proposal on the table. MDR 2.0 is the most serious revision effort in years: it tries to untangle the double regulation by the AI Act and the MDR, revises device classes and clinical evidence requirements, and contains the concept of a regulatory sandbox.
These are real improvements. But the proposal fails to address three core problems:
It doesn't close the gap between the MDR and the AI Act. The fragmentation remains. Manufacturers continue to navigate two regimes that aren't aligned with each other.
It ignores continuously learning systems. Open learning, continuous learning, adaptive training after deployment: none of it appears.
It has no concept for agentic AI. Systems that autonomously decide which tools to use, that intervene at multiple workflow points, that can no longer be described as single, purpose-bound devices: the regulatory model simply doesn't know them.
A Bright Spot: Hospitals
There is an interesting approach in the reform: hospitals are to get more room to develop their own systems, much as is already possible today for in vitro diagnostics (Article 5(5) MDR). Colleagues from the Netherlands and our team in Dresden are working on systematically expanding this path.
It is no panacea. But it creates room for innovation within a controlled framework: with a quality system, without Notified Body involvement, with the close coupling to clinical practice that regulated devices often lack.
What Remains
The regulatory system isn't broken. Its core principles are sound. But it was designed for a world of static devices. AI is the opposite of that.
The direction is clear: away from one-time validation, toward continuous monitoring. Away from reactive post-market surveillance, toward a closed-loop system that learns from live operation. And away from a regulatory model that simply doesn't know agentic AI.
How fast that happens doesn't depend on the EU alone. It also depends on whether manufacturers stay in Europe rather than choosing the route to America, and work together on better solutions. In the end, only a system that grows with the technology protects anyone, and that doesn't happen on its own.
Oscar is a postdoctoral researcher at the Else Kröner Fresenius Center for Digital Health (TU Dresden) and researches the regulation and implementation of AI as a medical device. He is a co-founder of Denkmal.