Openai’s operator agent has human capabilities to navigate and execute internet tasks

Openai  has presented the preliminary version of Operator, an agent who combines the capabilities of the GPT-4O model with advanced reasoning through learning and can interact with graphic user interfaces (GUI) to navigate the Internet in the Internet Autonomous, as if it were a human.

This tool works with a new model called Computer-Useing Agent (CUA) and is based “on years of fundamental research on the intersection between multimodal understanding and reasoning,” the developer explained in a statement.

By combining the advanced perception of the User Graphical Interface with problem solving, this Chase It is able to divide the tasks to execute them into several steps, Adapt dynamically when unexpected changes arise and self -correction if you run any action incorrectly.

More specifically, the model, than Combine GPT-4O with advanced reasoningprocess data from the screen pixels to understand what happens in it and use a mouse and virtual keyboard to complete the actions autonomously, with actions such as clicking, traveling or writing.

In this way, you can act with human abilities in a wide range of digital environments, interact with buttons, menus and text fields (GUI), perform tasks such as complementing forms and navigating websites without the need for applications programming interfaces (API ) specialized.

However, the technology company has indicated that although this model manages most steps automatically, seeks user confirmation for sensitive actions. For example, enter login data, answer captcha forms or execute a bank transaction.

On the other hand, Openai has clarified that it stores the chats, the navigation history and the screenshots that Operator performs to offer their service until the users are deleted, which can do so through the tool configuration page.

Also, by eliminating a chat, all screenshots taken during that chat are erased. Once all this information is deleted, they will be deleted from the company’s systems within 90 days.

Due to all the commented capabilities, OpenAI has indicated that his new work “marks the next step in the development of AI, because it allows the models to use the same tools that humans depend daily and open the door to a wide range of new applications. ”

Cua is still at an early stage of development and has some limitations. It is also endorsed by Webarena and Webvoyager, achieving a success rate of 58.1 percent in the first and 87 percent in the second.

On the other hand, Openai has indicated that Osworld, a reference point that evaluates the capacity of the models to control complete operating systems, such as Ubuntu, Windows and Macos, has given a success rate of 38.1 percent. However, the company has observed an escalation in time of trial. which means that model performance improves when you must execute more steps or actions.

The firm has also stressed that Cua has developed with security as the highest priority to address the challenges posed by an agent to the real world. This way, is trained to reject harmful tasks and illegal activities the irregular.

Nor can we access websites that Openai itself has blocked preventively, such as adult content websites or gambling, and is able to identify fraudulent activities. In this way, monitor the activity and pause the execution of an action if it detects suspicious content in the panel.

At the moment, this agent is being implemented through a preview of research through Operator.chatgpt.com and already have access to it the subscribers of the PRO level in the United States. Soon, plan to take it to users of Plus, Team and Enterprise, as well as integrate their capacities in Chatgpt.