AI in Service of the Doctor — Under Human Control
Artificial intelligence is progressively entering medical practices. But behind the term "medical AI agent" lie very different uses, with risk levels and regulatory obligations that differ depending on what the system actually does.
The right question is not "Should AI be used?" but "For which task, with which data, under what human oversight and within what regulatory framework?"
What Is a Medical AI Agent?
A medical AI agent is a system using artificial intelligence to accomplish or assist certain tasks in a healthcare environment. It can understand natural language requests, structure data, transcribe a consultation, prepare a draft or automate certain workflows.
But the term "medical AI agent" is not a regulatory classification. What matters legally is the purpose, the data processed and the intended use.
The Main Uses
Patient assistant (ELYSIUM Patient) — answers administrative questions, explains how the practice works, directs to validated content. It must not be presented as capable of making a diagnosis.
Medical Scribe (ELYSIUM Scribe) — transcribes the consultation, structures key elements, prepares a draft report or letter. The generated document remains a draft subject to validation.
Operative documentation (ELYSIUM Bloc) — assists documentary traceability in the operating theatre. Transcribing is not certifying: the operative report remains an engaging medical document.
Administrative AI — organizes requests, prepares forms, manages certain workflows. Different risk profile from clinical AI.
AI Prepares — the Doctor Controls
AI-generated text can contain an omission, a misinterpretation, speaker confusion or a terminology error. A clinical document does not become reliable simply because it is well written.
The responsible workflow keeps the professional in the loop:
Transcription → structuring → draft → doctor's review → corrections → validation
The danger of automation bias — automatically validating what the machine proposes — must be taken into account from design.
GDPR, HDS and AI Act
Using AI does not make personal data obligations disappear. In a medical environment, one must examine: purpose, legal basis, minimization, recipients, retention periods, processors and transfers.
HDS: using an HDS-certified host may be essential for certain health data, but does not mean overall compliance.
AI Act: since August 2, 2026, new provisions are applicable, including certain transparency obligations. It is no longer enough to ask whether the AI works, but what its regulatory classification is and which obligations apply to this specific use.
Medical device: not all AI used by a doctor is automatically a medical device. Classification depends on the intended purpose and the software's actual functions.
Cybersecurity and Traceability
The security of an AI agent does not depend solely on encryption. The entire environment must be protected: identity, authentication, data, model, API, database, logs and integrations.
Traceability allows reconstruction of what happened: user, date, action, system version, generated output, modifications and human validation. It is essential for safety, continuous improvement and governance.
FAQ
Can AI replace a doctor?
An AI agent can assist certain tasks, but medical practice involves clinical judgment, professional responsibility and a patient relationship that cannot be assimilated to automated content generation.
Must a report generated by AI be verified?
Yes. Automatic generation can contain errors. Any clinical document must be subject to the necessary controls and validations before use.
Is HDS sufficient to make medical AI compliant?
No. Host certification does not cover all obligations relating to processing, security, GDPR or AI use.
Does the AI Act apply to doctors?
The AI Act imposes different obligations depending on the system and the actor's role. The system and its specific use must be analyzed rather than treating all medical AI uses identically.
Can consumer AI be used with patient data?
Using health data in an AI service requires prior analysis of processing, contractual terms, hosting and applicable guarantees.
The Elysium MedTech Approach
At Elysium MedTech, we build an architecture around four principles: Utility (automate what brings real value), Human Control (the professional remains at the center of decisions), Traceability (understanding system actions) and Compliance by Design (GDPR, HDS, AI Act integrated from design).
The goal is not to replace the doctor. It is to give them back time, while preserving their judgment, their responsibility and the relationship with their patient.