The Hugging Face Artificial Intelligence (AI) model repository is being used as one of the ways for some users to generate ‘deepfakes’ non-consensual acts that undress women and minors, with hardly any measures to prevent it.
A report by the European non-profit organization AI Forensics found that seven of the nine main models dedicated to image editing hosted in the Hugging Face repository allowed access to this type of edition to undress women.
The organization details that the ability to edit an image with these models was as simple as entering several words that simply detailed the goal of undressing a person.
That is, unlike closed models such as Google’s Gemini and OpenAI’s ChatGPT, There is no measure that prevents these types of requests. by users on Hugging Face.
“No type of safeguard is being implemented at the platform level. Only developers can, if they wish, integrate one, and most of them prefer not to do so,” says AI Forensics principal researcher Paul Bouchaud, in statements to the specialized media Wired.
In fact, AI Forensics share some data obtained thanks to the creation of a ‘trap’ tool he posted on Hugging Face’s Spaces to track what type of images and ‘prompt’ requests would be received.
The tool created in Spaces (a section dedicated to creating web demos) was specially designed not to process image requests, and received more than a thousand prompts and images during a week.
He 73 percent of the requests analyzed were of a sexual naturewhile 83 percent sought to undress a person, and 95 percent of all those cases targeted women. Additionally, 7 percent of sexual requests targeted minors.
AI Forensics has made it clear that at no point is he accusing Hugging Face to be the source of the AI models it hosts, although Bouchaud insists that the platform could easily filter what goes in and out.
In addition, offers some moderation recommendations to the repository to prevent the generation of non-consensual intimate images and child sexual abuse material, such as the integration of centralized filters at the platform level both in the input of ‘prompts’ and uploaded photos and in the output of images and videos, instead of waiting for the user or developer to take responsibility for it.
Above all alludes to the fact that 0 percent of the audited Spaces moderated or filtered those ‘prompts’ or input images, while only 3 percent of the generated image outputs had any type of control (a feature available in the Stable Diffusion model).