Amazon Security Chief Warns: “AI Agents and Models Are Great: Don’t Trust Them”

The phrase was said by Steve Schmidt, senior vice president and Chief Security Officer of Amazon, during one of the first sessions of the AWS Summit Washington, D.C. and echoed in the political capital of the United States: during the last months, MythosAnthropic’s model specialized in cybersecurity, was at the center of a dispute between the company, the United States government and its technological partners over its ability to find flaws in sensitive systems. In April of this year, Anthropic set off alarms in systems around the world for its offensive capabilities.

AWS, Amazon’s cloud division, appears in this discussion as Anthropic’s key infrastructure and as an access channel to these models through Amazon Bedrock, the Amazon service that allows the use of artificial intelligence models from different companies from the cloud.

The example was Mythos, the Anthropic model that Schmidt mentioned to dimension the change in scale. During the conference, the executive said that systems of this type already allow many more failures to be found than traditional processes. And this has operational consequences for companies and governments: if before fixing vulnerable systems could take 30, 60 or 90 days, Today the windows go from 6 a.m. to 12 p.m.

The session was also accompanied by Brandon Williams, Undersecretary of Nuclear Security of the United States and administrator of the National Nuclear Security Administration (NNSA), the agency that is in charge of areas such as nuclear weapons, nuclear propulsion, counterterrorism and nonproliferation. Williams described that work as “infinitely complex” and with “very serious consequences” in the event of a failure. The common point was the tension that runs through the State, which involves protecting secrets, maintaining safe environments and keeping pace with an industry that invests in AI on a much larger scale than the public sector.

For Schmidt, cybersecurity teams can no longer limit themselves to blocking processes. Their role, he said, is to find a way to safely advance an organization’s mission. In AI, that means leveraging automated agents to accelerate defenses, generate detections, search for patches, and simulate attacks, with tight controls over each step. “Agents are cool, AI models are cool – don’t trust them“, he warned. Amazon uses, as in almost the entire cybersecurity industry, agents that work with each other: an automated “red team” attempts to exploit systems and a “blue team” identifies gaps, builds defenses and escalates the result to a person.

Another of the myth that fell during the conference was that of the absolute autonomy of AI: “We need a human in the loop (loop), in the process,” said Schmidt, because the current models, according to his estimate, have an accuracy of between 83% and 84%. And he assured that, in terms of security, “You need to reach 100%.”

The other focus was internal risk. Schmidt argued that compromised human access can be as relevant as software vulnerabilities. State actors, he said, “look for weaknesses in identity and permissions to access the data.” In this sense, he highlighted the crucial importance of containing the access of those who are part of a network, whether public employees or private companies. “If an employee assigned to Washington DC logs in from Prague, for example, the system can require immediate re-authentication with a physical hardware token,” he explained.

To a large extent, this principle, called “of least privilege,” occupied a central place. Schmidt revealed that since April 2024, Amazon identified more than 2,400 attempts by North Korean citizens of being hired by the company, in a recurring problem in companies throughout the West (which even registered cases in Mexico). “And those are only the ones we detected,” he warned.

For the executive, no organization can rely on a perfect rejection strategy. Defense begins when it is assumed that someone can eventually get in: segmentation, minimum permissions, data traceability and the ability to contain the damage matter here.

He also mentioned attempts by Russian and Chinese actors against Amazon’s Microsoft Entra environment in early 2024 (something that already has precedents in the 2020 US election campaign), stopped, he assured, by a non-negotiable internal control: valid identity and physical hardware token.

At the main conference (keynote), Dave Levy, vice president of Worldwide Public Sector at AWS, placed that technical discussion on a broader scale: the history of America’s public infrastructure. The executive opened with a reference to 250th anniversary of the countryan almost unavoidable event in Washington, which this week has an installation with a state fair to celebrate the next 4th of July.

“Think about what has been built in that time: not just the institutions, but the ideas; the notion that ordinary people, with the right tools, can do extraordinary things,” he said. Then he returned to 1876, to the Philadelphia Exposition, when the United States was emerging from the Civil War and seeking to show itself to the world: “In one building, Alexander Graham Bell transmitted the human voice over a cable, in another, the Remington typewriter was presented. At the center of it all was the Corliss steam engine, which powered every machine in the room from a single source.”

For Levy, the cloud today occupies a place similar to that of that common infrastructure: a shared foundation on which governments, laboratories, defense agencies and regulated organizations can develop new capabilities. “We built AWS the same way, based on culture,” he said. His point was that digital transformation of the public sector requires bridging the gap between what a mission calls for and what legacy systems allow to be done.

However, one of the current difficulties has to do with the implementation of AI, in the midst of the discussion about layoffs. In his talk, Levy said that 90% of attempts Using artificial intelligence in organizations does not go beyond the proof of concept (that is, testing) and does not “go into production.” For this reason, the company launched FDE (Forward Deployed Engineering) to assist companies that want to apply AI.

Later consulted by Clarín, in a round table with journalists from different Latin American media, he explained: “I would not say that implementing AI is more difficult than previously believed. I would say that executing it operationally, at scale and throughout an organization, is probably more challenging than many clients thought at the beginning.”

“It’s pretty simple to deploy a chatbot or a desktop tool that generates responses. But if you want to rethink, for example, how to do inspection and security in a hospital, that’s a much bigger project: there are security controls, documentation and many parts of the organization that have to work together. Technology can support it and generate real returns, but organizations need to execute those projects with experts at their side before they can fully see the benefits.”

The closing of keynote had an unusual appearance: John Ratcliffe, director of the CIAwho opened his speech by clarifying that he does not speak in public frequently.

Ratcliffe took the speech into the same territory he had traversed the rest of the day: emerging technologies, national security and speed. According to him, when he took over the leadership of the CIA he put China and emerging technologies as central priorities. AI, quantum computing and biotechnologyhe stated, are already modifying economies, conflicts and forms of asymmetric warfare.

Ratcliffe’s reading touched a sensitive point for public administration: he said that the agency has to incorporate technology with less bureaucracy. He said that previously the acquisition of a new tool could take up to 24 months, plus another nine months of security evaluation. The CIA, he said, is now looking to complete most of its acquisitions within six months, with nearly 400 purchases made in the last six months.

The final stretch returned to the AI. Ratcliffe said the broad adoption of these tools is transforming the way the CIA works and argued that the capabilities of frontier models can be thought of as “digital nuclear weapons”. The phrase fits with the concern that surrounds models such as Mythos, de Anthropicmentioned during the conference for its ability to find vulnerabilities on an unprecedented scale. That discussion also touches AWS: Amazon and Anthropic expanded their strategic alliancewith Trainium capability and using AWS infrastructure to train and serve advanced models.

Ratcliffe closed with a nuance that dialogues with Schmidt’s warning about agents and models. The CIA, he said, has to move fast, take “smart risks,” experiment and correct course as it goes. He also maintained that good intelligence will continue to require good human judgment.

In a conference dominated by cloud, supercomputing and AI announcements, that closing left the background that the State’s technological race is measured by the ability to integrate new tools without handing over control of decisions.