Abuse deepfakes: ban on AI apps in the EU is coming

AI applications to the abusive Creating sexualized deepfakes should in the EU banned in the future be. Representatives of the member states and the European Parliament agreed on a corresponding adjustment of the AI ​​law, as the Cypriot EU Council Presidency announced. At the same time should other AI rules simplified to exploit the economic potential of artificial intelligence in Europe.

Deepfakes: Creating deceptively real content is very easy

Before the changes can come into force, the agreement must be confirmed by the plenary session of the EU Parliament and the Council of Member States’ Governments. As a rule, this is considered a formality. If the reform goes through, that would Ban from December 2nd In 2026, the EU AI Office, which was created two years ago, will be responsible for enforcement.

In so-called deepfakes, a woman’s face or voice is inserted into another video – such as a porn sequence artificially imitatedmaking it seem like she’s doing or saying things that never actually happened. Through the development of AI, the Creating deceptively genuine content is much easier become. The new ban should also explicitly target the creation of content that depicts child sexual abuse.

Grok scandals and debate about digital violence

The issue came to the fore at the EU level at the end of last year KI-Chatbot Grok in focus: Until the US company behind the software, led by tech billionaire Elon Musk, restricted this function, people repeatedly ordered the AI ​​to undress women in images they selected. On New Year’s Eve, the chatbot itself apologized for posting a picture of two teenage girls “in sexualized outfits” to have created.

The debate about sexualized digital violence took on new urgency at the end of March Collien Fernandes went public with allegations against her ex-husband Christian Ulmen. She accuses him of creating fake profiles in her name and using them to distribute pornographic content. The presumption of innocence applies to elms.

The EU presented a directive on the subject as early as 2024

The planned tightening of the law is not the first EU rules Combating digital violence. Existing regulations already stipulate that the creation and distribution of manipulated depictions of sexual acts without the consent of those affected should be punishable in all member states. The corresponding EU directive has been in force since May 2024, but in many cases not yet translated into national law become.

The new ban in the AI ​​regulation at EU level that is now emerging would change the perspective of the punishment for the crime switch to banning the tool, i.e. the AI ​​application itself. Participants in the negotiations emphasize that the ban is not intended to excessively restrict the creation or manipulation of images.

Non-consensual bikini pictures could remain permitted

The new law should precisely define what is sexualized content and what is not. Non-consensual bikini picturesas created by Grok and distributed on X, could therefore remain allowed.

The EU Commission originally proposed changes to the AI ​​law in order to relieve the burden on the economy and in particular the AI ​​industry. However, companies have recently repeatedly asked for more time to make the necessary adjustments stricter requirements to implement. According to the agreement, providers of chatbots and other services should be given this time.

Deepfakes are “not a gimmick” but violence

The companies behind ChatGPT, Claude and Co. now only have to comply with parts of the law that the European AI Office was originally supposed to enforce from August onwards from December 2026. By then at the latest, providers should clearly mark AI content as such – so they would have to mark generated images and videos with watermarks. The AI ​​office is only scheduled to enforce other rules from December 2027.

Politicians welcomed the agreement. “Women also need to be protected from violence in the digital space and we now have one for exactly that important step set. Deepfakes are not a gimmick, they are violence“, said the SPÖ EU MP and women’s policy spokeswoman Elisabeth Grossmann.

“Sexualized violence on the internet has real effects on people in the real world. That’s why we have to fight it there as well,” reacted Sophia Kircherinternal market spokesperson for the ÖVP in the European Parliament. The Green MPs Lena Schilling expressed relief “that the Green proposal for a ban on these apps prevailed in the negotiations.”