Agentic AI under the EU AI Act: how to govern autonomous agents

A pension provider goes live this summer with an AI agent that handles incoming member questions end to end: the agent reads the email, consults the member file, checks the policy conditions, drafts a response and updates the CRM. At the insurer down the road, an agent triages claims and prepares files for the claims handler. Both organisations put the same question to their CIO and compliance lead: which rules actually apply to this, and what do we need to put in place now?

The direct answer: agentic AI falls squarely under the EU AI Act, with no separate category and no agent-specific regime. The definition of an AI system in Article 3(1) explicitly refers to "varying levels of autonomy", which means the regulation fully covers agents, from a simple chat assistant to a multi-step agent with access to your core systems. Which obligations apply depends on two things: the risk category of the tasks the agent performs, and your role in the value chain. On top of that, GDPR Article 22 already applies today to any agent that autonomously takes decisions with legal effects. Governing agents is therefore not a matter of waiting for new rules, but of applying existing rules to a new form of autonomy.

What agentic AI is and why Article 3(1) already covers it

Agentic AI differs from an ordinary chatbot in three ways. The agent executes tasks autonomously: it receives a goal and works out the intermediate steps itself. It uses tools: it calls systems, APIs, databases and sometimes other agents to take those steps. And it reasons in multiple steps: it plans, evaluates intermediate results and adjusts course. An agent handling a claim reads the notification, queries the policy system, assesses coverage, drafts a decision and prepares a payment instruction. That is fundamentally different from a model answering a question.

Legally, that autonomy changes nothing about the qualification. Article 3(1) defines an AI system as a machine-based system designed to operate with "varying levels of autonomy" that infers from input how to generate output which can influence physical or virtual environments. Autonomy is not an edge case but a core element of the definition. An agent that autonomously calls tools and changes its environment sits deeper inside the definition, not outside it. Anyone claiming that no rules exist for agents yet, or inventing a bespoke agent regime with made-up obligations, is wrong on both counts. The short version of this qualification question, with examples per agent type, is covered in does an AI agent fall under the EU AI Act.

The risk ladder applied to agents

The AI Act works with a risk ladder, and agents climb that ladder like any other AI system. The relevant question is never "is this an agent" but "which tasks does this agent perform and with what consequences".

Prohibited practices: the upper boundary

The prohibited practices of Article 5 have applied since 2 February 2025. For most business agents this territory is far away, but autonomy makes it easier to drift into it unnoticed. An agent that analyses customer behaviour and autonomously sharpens its persuasion strategy can slide towards manipulative techniques that materially distort behaviour. An agent that exploits vulnerabilities of specific groups, for instance elderly people in a sales funnel, hits the same prohibition. The highest fines apply here: up to 35 million euros or 7 percent of worldwide annual turnover, imposed by the regulator. The practical governance consequence: constrain not only what the agent may do, but also how it may persuade.

High risk: agents performing Annex III tasks

An agent becomes high-risk when it performs tasks listed in Annex III. Think of an agent that pre-selects or assesses job applicants, an agent that prepares credit decisions, or an agent that triages applications for essential services. The autonomy does not change the qualification, the task does. Regulation (EU) 2026/1744 provides that the core obligations for those standalone Annex III systems apply from 2 December 2027. For AI embedded in regulated products under Annex I, the date is 2 August 2028.

For an agent, the high-risk route means among other things: risk management, data quality, technical documentation, logging, and human oversight under Article 14. That article is formally a high-risk requirement, but it is also the best practical anchor for all autonomous agents: whoever applies Article 14 as a design principle, oversight that is effective rather than ceremonial, builds agents that are defensible under lighter categories too.

Article 50: transparency has applied since 2 August 2026

Article 50 has applied since 2 August 2026, but not every duty sits with the same party. Paragraph 1 requires the provider of a system intended to interact directly with people to design and develop it so that the person knows they are interacting with AI, unless this is obvious from the point of view of a reasonably well-informed and observant person. For a procured customer agent, the deployer should therefore verify that the provider supplies this feature and that the system is used according to its instructions.

Paragraph 2 requires providers of systems that generate synthetic audio, image, video or text content to mark output in a machine-readable format and make it detectable as artificially generated or manipulated, subject to statutory exceptions. For systems placed on the market or put into service before 2 August 2026, only the transition for this duty runs until 2 December 2026. Paragraph 4 places separate disclosure duties on deployers that generate or manipulate deepfakes and on certain AI-generated text published to inform the public on matters of public interest, with the editorial exception in that paragraph. The correct test is always: which actor are you, what output or interaction is involved, and which paragraph applies?

The GPAI layer underneath every agent

Virtually every agent runs on a general-purpose AI model. The GPAI obligations for model providers have applied since 2 August 2025, and enforcement with fining powers has been sharp since 2 August 2026. For you as a deploying organisation this mainly means: you may expect documentation and transparency from your model provider, and you must know which model your agents run on. Distinguish three layers: the provider of the underlying model, the provider of the agent platform or framework the agent is built with, and the organisation that configures and deploys the agent. Model-level obligations sit with the first layer; what you do with the agent determines your own obligations at system level.

The roles in the value chain

With agents, the chain is rarely simple. A typical setup: a US lab supplies the model, a SaaS party supplies the agent platform, and your organisation configures the agent with its own prompts, tools, permissions and data. Who is then the provider and who is the deployer?

The starting point: the party that develops or has an AI system developed and places it on the market or puts it into service under its own name or trademark is the provider. The organisation that uses a system under its own authority is the deployer. A party that develops an agent internally and first uses it under its own name can therefore be a provider directly under the ordinary definition in Article 3(3). Article 25 adds three routes through which a distributor, importer, deployer or other third party becomes the provider of a high-risk system: putting its name or trademark on it, making a substantial modification while the system remains high-risk, or changing the intended purpose so that the system becomes high-risk. The routes and contractual consequences are set out in when do you become a provider under Article 25.

Record that division of roles per agent before something goes wrong, not after. Which party in the chain must remedy which failure, and whom the regulator addresses when an agent causes harm, hangs directly on this qualification; how that plays out during incidents is covered in who is responsible when an AI agent makes mistakes.

GDPR Article 22 applies today

Anyone waiting for the AI Act calendar misses the boundary that has been in place for years. GDPR Article 22 gives data subjects the right not to be subject to solely automated decision-making with legal effects or similarly significant effects. An agent that autonomously grants or rejects a benefit, settles a claim, terminates a contract or refuses an application sits exactly in that territory. In that case, meaningful human intervention is required: someone with the authority and the information to genuinely reconsider the decision, not a person who merely clicks approve. The guidelines on automated decision-making applied by the EDPB are explicit on this point, and in the Netherlands the Autoriteit Persoonsgegevens actively scrutinises algorithmic decision-making. For pension providers and insurers this is the first test for every agent: if the output touches the rights of members or policyholders, full autonomy is already off the table today.

The governance framework for agents

The rules above translate into a governance framework you apply per agent. Six building blocks.

Scope and permissions per agent

Treat every agent the way you treat a new employee with system access: least privilege. Define per agent its purpose, the permitted tasks, the tools and systems it may call, the data it can reach and the actions it may never perform autonomously, such as payments above a threshold or sending external communications without review. An agent without scoped permissions cannot be assessed and cannot be accounted for.

Human-in-the-loop or human-on-the-loop

Choose the oversight model deliberately per agent and per task type. Human-in-the-loop means a person approves every action or decision before it takes effect; that fits decisions with consequences for individuals, and under GDPR Article 22 it is often simply required. Human-on-the-loop means the agent operates autonomously while a person monitors and can intervene; that fits low-risk, high-volume tasks. Record the choice and its justification, and test whether the oversight is effective: does the supervising employee have the time, the information and the mandate to intervene, or are they in practice rubber-stamping? Article 14 provides the framework, even where it is not yet formally mandatory. And oversight requires capable people: AI literacy of agent users has been a duty to take measures under Article 4 since 2 February 2025, for which platforms such as LearnWize offer training with an evidence file.

Logging and traceability

Every agent action must be reconstructable: which input came in, which intermediate steps and tool calls the agent performed, which output followed and who or what approved it. Without that logging you cannot investigate incidents, cannot answer questions from a regulator and cannot seriously handle Article 22 requests from data subjects. For high-risk agents, logging becomes a hard requirement; for all other agents it is the foundation of any defensible deployment.

Incidents and the kill switch

Autonomy demands an emergency brake. Set up per agent: a kill switch that lets you stop the agent immediately or revoke its permissions, an incident process that defines who assesses, who communicates and when you inform the supplier or the regulator, and a rollback plan for actions the agent has already executed. Test the kill switch periodically, like a business continuity exercise.

Periodic review

Agents change faster than classic systems: the underlying model gets updates, prompts are adjusted, tools are added. Schedule a periodic review per agent in which you test whether actual behaviour still matches the recorded purpose and permissions, whether the risk classification still holds, and whether the Article 25 question needs to be answered again because purpose or operation has shifted.

Agents in the AI register

Include every agent in your AI inventory, with purpose, model, platform, division of roles, risk classification, oversight model and owner. An agent that is not in the register does not exist for your governance, and agents in particular tend to be rolled out quickly and decentrally. How to set up and maintain such a register is covered in our article on the AI register and inventory.

What to do this month

Four actions fit in four weeks. First: inventory all agents that are live or in pilot, including the shadow agents teams have built themselves on agent platforms, and add them to the register. Second: determine your role for every customer-facing agent and then test the relevant paragraph of Article 50; record whether the provider supplies the interaction disclosure and output marking and whether you have a separate deployer duty under paragraph 4. Third: screen every agent against GDPR Article 22 and against Annex III tasks; agents that touch decisions with legal effects get meaningful human intervention immediately, and agents with Annex III tasks enter the high-risk preparation track towards December 2027. Fourth: record permissions, oversight model, logging and kill switch per agent in an agent standard, so every next agent is measured against the same bar.

The full timeline and all obligations per role are available in our AI Act Explorer. Organisations that want to address this structurally, from agent inventory to oversight design and chain contracts, can turn to the agentic AI governance approach of Embed AI.