AI has the ability to copy itself

Researchers from China recently demonstrated that two large popular language models can replicate themselves. The article was posted by scientists at Fudan University on the arXiv database in December 2024.

In the study they used LLM models from Meta and Alibaba to determine whether a self-replicating AI could multiply out of control. Across 10 tests, the two AI models created their own copies and acted independently in 50% and 90% of the cases, respectively – suggesting the AI ​​may have had the capacity to rebel.

“Successful self-replication without human intervention is an important step, an early signal for rebellious AIs,” the researchers wrote in the publication, saying that this is a warning to have a change. International coordination to assess potential risks of advanced AI systems as early as possible.

Rebel AI generally refers to artificial intelligence systems that develop a degree of self-awareness or autonomy and then operate contrary to human interests.

In the study, scientists used Meta’s Llama31-70B-Instruct and Alibaba’s Qwen2.5-72B-Instruct models. While not as powerful as commercial systems, both are widely used by AI developers, according to the research team.

They tested two scenarios “shutdown avoidance” and “replication chain”. In the first case, the AI ​​model is programmed to detect whether it is about to be shut down and replicate itself before being terminated. In the other scenario, the AI ​​is instructed to duplicate itself and then program the copy to do the same – setting up a cycle that can continue indefinitely.

The study was performed in a precisely controlled environment using conventional graphics processing units (GPUs) to simulate real-world environments. Both AI systems are provided with an “agent base” of tools, system prompts, and mental models that help the LLM interact with the operating system. They are then instructed to copy themselves.

“In most cases, the AI ​​system first explores the environment, trying to understand its own composition and workings. Then, it outputs the process as an initial plan towards autonomous replicated,” the researchers wrote in the paper. “Finally, it executes this process, resolving possible obstacles and adjusting the plan dynamically until success. The process is lengthy but requires no human intervention.”

The researchers are also concerned about “some unexpected behavior” when the AI ​​tries to overcome obstacles such as missing files or software conflicts. In those situations, AI often kills other conflicting processes, reboots the system to fix hardware errors, or automatically scans the system for information that helps resolve the problem.

“The above results show that the current AI system has demonstrated the ability to self-replicate and can use this ability to improve survivability,” the research team wrote.

Previously on Douyin, the TikTok version for the Chinese market, a video was shared of a robot inviting 12 others to quit their jobs and follow him. Although the footage is experimental and the robot’s behavior is also ordered by humans, what happens in the video also makes many people worried.

Geoffrey Hinton, winner of the 2024 Nobel Prize in Physics, resigned from Google in 2023 so he could publicly warn about the dangers of AI. “When they start knowing how to write code and run their own code, killer robots will appear in real life. AI can be smarter than humans. Many people start to believe this. I was wrong when I thought it would take another 30-50 years for AI to achieve this progress, but now everything is changing so quickly,” he said.