AI for fraud signals in social assistance: DPIA, FRIA and what the AI Act prohibits

Imagine you are the data protection officer (DPO) of a Dutch municipality. The Work and Income department no longer wants to pick social assistance files for review at random, but with a model. The vendor offers "smart risk selection": a score per file showing how likely it is that someone receives too much assistance. The files with the highest score go to an enforcement officer first. Your manager asks: do we need a DPIA, does this fall under the AI Act, and is it allowed at all?

There is no short answer with a single article. Three layers apply: the GDPR, the AI Act prohibitions that already apply, and the rules for high-risk AI that apply from 2 December 2027. This article sets them out.

Lesson 1: the DPIA is required, and beforehand

The GDPR requires a data protection impact assessment (DPIA) where processing is likely to result in a high risk, and it must be done before the processing starts (Article 35(1) GDPR). For fraud selection in social assistance there is little to debate. The AP has adopted a list of processing operations for which the starting point is that a DPIA is required (Stcrt. 2019, 64418, in Dutch). Category 3 is "fraud prevention": large-scale processing and/or systematic monitoring for fraud prevention, with "fraud prevention by social services" as an express example. Category 15 is profiling, including the assessment of someone's economic situation, reliability or behaviour. If you exchange sensitive data in a partnership, such as data on debts, care or poverty, category 8 counts as well.

The GDPR itself also names a systematic and extensive evaluation of personal aspects based on automated processing, including profiling, on which decisions producing legal effects are based (Article 35(3)(a)). If a high risk remains after the DPIA that you cannot mitigate, you consult the AP first (Article 36(1)).

Lesson 2: the data the model uses decides almost everything

A municipality cannot rely on legitimate interests for its tasks (Article 6(1), last subparagraph GDPR). The legal basis is the public task (Article 6(1)(e) and (3)). The municipal executive may investigate the accuracy and completeness of the data provided and, where necessary, other data needed for granting or continuing assistance, and may revise or revoke the assistance on that basis (Article 53a(6) Participation Act, in Dutch). Together with data minimisation (Article 5(1)(c) GDPR) that is the limit for the model: data not needed for the assistance do not belong in the score.

The SyRI case showed how strictly the court looks at this. In ECLI:NL:RBDHA:2020:865 (5 February 2020, English translation ECLI:NL:RBDHA:2020:1878) the District Court of The Hague held that the legislation on the System Risk Indication (SyRI), a statutory instrument to detect fraud in areas including benefits, allowances and taxes, breached Article 8(2) ECHR. According to the court its use was insufficiently transparent and verifiable. The court drew on the GDPR principles of transparency, purpose limitation and data minimisation and declared Article 65 of the SUWI Act and Chapter 5a of the SUWI Decree non-binding. The lesson for a municipal model: record which indicators the model uses, why each indicator says something about the right to assistance, and who can check that.

Lesson 3: the line of the ban on social scoring

The ban on social scoring applies since 2 February 2025, including to public authorities (Article 5(1)(c) AI Act). It prohibits evaluating or classifying people based on their social behaviour or personal characteristics where the score leads to detrimental treatment in a context unrelated to the one in which the data were collected, or to treatment that is unjustified or disproportionate to that behaviour or its gravity.

The Commission's guidelines on prohibited AI (C(2025) 5052 final) work this out with examples close to social assistance. Paragraph (166) describes a social welfare agency that estimates the probability of fraud by benefit recipients using characteristics with no apparent connection to fraud, such as having a spouse of a certain nationality or ethnic origin, having an internet connection, behaviour on social platforms or performance at work. That is prohibited. According to the same paragraph, data that is relevant for the allocation of the benefits and lawfully collected could be used to determine the risk of fraud, because public authorities pursue a legitimate aim in verifying that benefits are correctly allocated.

Paragraph (167) describes a municipality that scores the trustworthiness of residents based on behaviour in many contexts, such as library books returned late or rubbish put out on the wrong day, and uses the score to withdraw benefits. Paragraph (168) describes a tax authority that checks childcare benefit for fraud using criteria such as low income and dual nationality. Both are prohibited. At the same time, paragraph (175) says that fraud prevention and detection are not prohibited as such, if lawful and in line with the AI Act and other law. Even with relevant data, the detrimental treatment must still be justified and proportionate (paragraph (176)). The question for your DPIA is therefore, per indicator: does this data point have a demonstrable link with the right to assistance, and is the inference proportionate?

Lesson 4: the ban on predicting criminal offences usually does not reach social assistance

Article 5(1)(d) AI Act prohibits risk assessments that predict whether someone will commit a criminal offence based solely on profiling or personality traits. The Commission reads that ban as limited to criminal offences. A public authority that uses AI in an administrative investigation to assess the risk of irregularities in expenditure processes falls outside it according to paragraph (217) of the guidelines, even where information about possible criminal offences comes up. Failing to comply with the duty to provide information in social assistance as a rule leads to an administrative fine imposed by the municipal executive (Article 18a Participation Act, in Dutch).

Two caveats. Paragraph (218) says that the national classification of an offence is not conclusive: "criminal offence" has its own EU meaning, so a fine that can reach the full amount wrongly received is not automatically out of the picture. That the ban usually does not reach social assistance is therefore our reading. And if the municipality also uses the score to select files for a criminal complaint, test the model against point (d) after all.

Lesson 5: in our reading, a high-risk system

Annex III point 5(a) covers AI systems used by or on behalf of public authorities to evaluate the eligibility of people for essential public assistance benefits and services, "as well as to grant, reduce, revoke, or reclaim such benefits and services" (Annex III AI Act). A model that decides which files are reviewed, with revocation and recovery as possible outcomes, comes close to that.

The Commission points the same way. According to paragraph (290) of its draft guidelines, eligibility can also be assessed again after benefits have been granted, which may lead to repayment. In the same draft guidelines on the classification of high-risk AI of 19 May 2026 it gives, under Annex III point 5(a), the example of a system that analyses benefit applications for irregularities or fraud indicators and flags them for further investigation, after which a case handler assesses whether the claim is fraudulent. According to the Commission that system is high-risk, because flagging determines whether an application is subject to further scrutiny, delay, suspension or potential refusal. According to paragraph (291), all decisions on granting, reducing, revoking and reclaiming may fall under this point, and via Recital 58 denial as well. The draft guidelines are non-binding and not yet final; the Commission should have adopted them by 2 February 2026 (Article 6(5)). The example concerns applications, but the same reasoning fits the selection of running files for review.

The exception in Article 6(3) for a "preparatory task" does not help here. The last subparagraph of that paragraph provides that an Annex III system is always high-risk where it profiles natural persons, and a risk score per recipient is profiling.

What follows from 2 December 2027 for the municipality as deployer (Article 26)? Use in line with the instructions, human oversight by people with the necessary competence, training and authority (paragraph 2), input data that is relevant and sufficiently representative to the extent you control it (paragraph 4), monitoring its operation and reporting serious incidents (paragraph 5), keeping logs for at least six months (paragraph 6), registration in the EU database (paragraph 8) and informing recipients that the system is used on them (paragraph 11). As a body governed by public law you carry out a fundamental rights impact assessment, the FRIA, before first use (Article 27). You may refer to the parts your DPIA already covers (paragraph 4), and you notify the market surveillance authority of the results (paragraph 3). Anyone affected by a decision with legal effects based on such a system gets, in addition to the GDPR, a right to an explanation of the role of the system (Article 86).

If the model is already in use before 2 December 2027, those duties only apply after a significant change in its design. For systems intended to be used by public authorities, the deadline is in any case 2 August 2030 (Article 111(2)). The GDPR duties and the Article 5 prohibitions already apply now, also to a system that is already running.

Lesson 6: the enforcement officer decides, not the score

A score that only determines who gets a review is usually not yet a decision on the assistance. That decision only follows after the investigation. But Article 22 GDPR also covers decisions that similarly significantly affect someone. If the selection directly leads to a suspension of payment or an intrusive investigation, or if the score in effect decides, Article 22 comes into play. In the SCHUFA judgment (C-634/21, 7 December 2023) the Court of Justice held that a score can be an automated decision even if someone else takes the final decision, where the score plays a determining role in it. That case concerned a score from another company; within one municipality it is an analogy, but the Commission refers to the same judgment in its guidelines.

The Dutch exception in Article 40 of the GDPR Implementation Act for automated decisions in a statutory task does not apply to decisions based on profiling. So set up the process so that the officer investigates and records the facts, that a decision to revoke or recover rests on those facts and not on the score, and that the officer knows what the score does and does not say. That also fits the human oversight of Article 26(2) AI Act.

Lesson 7: what you ask the vendor

The vendor is usually the provider under the AI Act and the processor under the GDPR. Ask now for the list of indicators and the origin of each data point, the test results on errors (how often does the model wrongly flag someone, and does that differ between groups), and a processing agreement that excludes use of the data for other purposes (Article 28 GDPR). Also make sure the officers and the staff who configure the model know how it works: Article 4 AI Act requires measures that support the AI literacy of your staff.

For banks and insurers detecting fraud, part of the same rules applies, with one important difference: Annex III point 5(b) excludes AI used to detect financial fraud from the high-risk category of creditworthiness.

What you do now

1. Carry out the DPIA before the pilot, with AP categories 3 and 15 as a starting point.
2. Record for each indicator its link with the right to assistance; remove what lacks that link.
3. Test the model against Article 5(1)(c) with the examples in paragraphs (166) to (168) of the guidelines.
4. Record that the score only selects, and that decisions rest on facts from the investigation.
5. Prepare the FRIA and the registration, with 2 December 2027 for a new system; an existing system after a significant change and in any case by 2 August 2030.
6. Ask the vendor for indicators, test results and a processing agreement.
7. Put the system in the municipality's algorithm register, so residents can see that it exists.

Want to check which duties apply to your own use case under the GDPR and the AI Act? Use the free case check.

What remains open

• The high-risk guidelines. The Article 6(5) guidelines are still a draft (19 May 2026). The example on flagging applications for a fraud investigation may still change in the final text, and it concerns applications, not expressly the review of running files.
• Amendments to the Participation Act. wetten.overheid.nl shows amendments to the Participation Act that do not yet have an entry-into-force date. Check the current text of Articles 18a and 53a before quoting them in your DPIA.
• The status of the guidelines. The Commission's guidelines are not binding; the authoritative interpretation of the AI Act lies with the Court of Justice.