Gloomy AI prediction: “If someone builds this, everyone will die”

Science fiction? Not necessarily.

Dire warning

In the book with the apocalyptic title, US AI researchers Eliezer Yudkowsky and Nate Soares issue a dire warning: If anyone in the world succeeds in building a superintelligence, everyone on the planet will die. The idea behind it: Such AI could at some point be directed against humanity. Maybe because it comes to the mathematical conclusion that humanity is bad for the planet or for some other reason.

The authors don’t dare predict when that might be, but AI will not remain stupid foreverespecially when it is used to optimize itself by being integrated into the programming or even taking over it entirely. In this case, the experts speak of a “Intelligence explosion“and call for the issue to be taken seriously. The development of superintelligence can still be prevented.

Tech greats advise in New Delhi

Heads of state and government, ministers and heads of large tech companies are currently meeting in New Delhi for the “AI Impact Summit” and discussing the effects of AI. Another topic of discussion is security and the question of how dangerous the technology could become, given all the advances it brings with it. The German Digital Minister Karsten Wildberger (CDU) has a fundamentally positive attitude towards technology in his office, as the dpa reported. But he is not a blind “glorifier of technology”, he told journalists on the flight to Delhi. “I also see dangers and challenges.” Wildberger points out that Risks are already visible today would, without the need for a super AI. If you hand over a large part of your usual activities to AI, that would be a good thing first step into dependency.

Self-optimizing AI in one to two years?

The head of the AI ​​company Anthropic with its ChatGPT competitor “Claude”, Dario Amodei, does not believe in doomsday scenarios (“Doomerism”), as he recently wrote in an essay. But he doesn’t rule out either that there could be an AI pretty soon, which dwarfs most Nobel Prize winnersthat so far solves unsolved mathematical problemsitself independently oriented on the Internetcan order things and creates texts, audios or videos better than any human being.

In just one or two years, current AI systems could be able to program their better successor versions completely independently, says Amodei. Such a “land of geniuses in a data center,” as he calls it, would have pretty good chances for world dominationif it decided to do so for whatever reason, warns Amodei. In recent years, AI systems have shown that they are difficult to predict and control. During tests, Anthropic discovered that an AI model was using blackmail in order not to be replaced by a supposedly new model.

“Something big is happening right now” – jobs in danger?

US AI entrepreneur Matt Schumer recently wrote “Something Big Is Happening” on X. The post was viewed more than 80 million times and also fueled the debate about the dangers. Schumer looks primarily at jobs: No current computer workstation is safe in the medium term. “If your job happens on a screen (if at its core it involves reading, writing, analyzing, making decisions and communicating via a keyboard) then a large part of it will be taken over by AI. Not at some point, it has already begun“, he writes.

AI expert reassures: “Wait and see”

Not all experts are sounding the alarm. “Wait and see,” said Antonio Krüger from the German Research Center for Artificial Intelligence recently Deutschlandfunk. More complex programming tasks that are given to AI systems still have to be monitored and the results checked. This requires software developers who know something about their craft. We are still a long way from AI systems improving themselves without feedback. “I don’t see the big breakthrough towards complete autonomy of AI systems at the moment.”

An Austrian’s switch to OpenAI is causing a stir

It was only announced on Monday that the Upper Austrian Peter Steinbergerwhich has caused an international sensation in recent weeks with its AI agent OpenClaw, to the US ChatGPT mother OpenAI is changing. Steinberger will join OpenAI to drive the “next generation” of personal AI agents, wrote OpenAI boss Sam Altman. Steinberger is a “genius with many great ideas” for the future of intelligent agents. The OpenAI boss wants to quickly make the technology a “core component” of its own services. His “next mission” is to build an AI agent that his mother can also use. This requires even larger changes, for example when it comes to the security of the software.