AI ‘agents’ are capable of using a computer just like you do

Prompt-driven artificial intelligence chatbots have exploded in popularity and user engagements since OpenAI launched its ChatGPT platform to the public in November 2022. Competitors, including tech giants like Google, Meta and others, quickly followed with similar AI-driven platforms that can answer queries, author papers and even create images and video footage.
Now, those tools are on the verge of taking an evolutionary step, advancing to capabilities that include operating programs and computer systems in much the same way the software’s human progenitors function.
Artificial intelligence software startup Anthropic released its own chatbot, Claude, in March 2023, and has since risen as one of the top developers in its category. On Tuesday, the company announced an upgraded release of Claude that includes “computer use” capability. Anthropic says the new functionality, which is currently only available in a beta version to developers, can use computers “the way people do” including looking at a screen, moving a cursor, clicking buttons and typing text.
For the moment, Anthropic says Claude’s computer user capabilities are “experimental” and still cumbersome and error-prone, but the developer has shared the technology with companies like Asan, Canva, Cognition and DoorDash, among others, to test out its effectiveness.
“Computer use capabilities have the potential to change how tasks that require navigation across multiple applications are performed,” Mike Krieger, chief product officer at Anthropic, told VentureBeat. “This could lead to more innovative product experiences and streamlined back-office processes.
“We anticipate it being particularly useful for tasks like conducting online research, performing repetitive processes like testing new software, and automating complex multi-step tasks.”
Microsoft’s Copilot Vision feature and OpenAI’s desktop app for ChatGPT have shown what their AI tools can do based on seeing your computer’s screen, and Google has similar capabilities in its Gemini app on Android phones, according to a report from The Verge. But those developers haven’t gone to the next step of widely releasing tools ready to click around and perform tasks for users like Anthropic has done.
In a demonstration video of Claude’s new computer use functionality, conducted in a “controlled environment”, Anthropic researcher Sam Ringer showed the system filling out a fictional vendor request form, completing tasks including searching for necessary information on spreadsheets, completing the form and returning it to the information requestor.
“This example is representative of a lot of drudge work that people have to do,” Ringer said. “We’re excited for people to try it and we should expect things to get a lot better over the coming months.”
The U.S. market for artificial intelligence software development is on an absolute tear, projected to generate $214 billion in revenues by the end of 2024, up $50 billion from last year, and is on pace to hit $1.3 trillion in annual revenues by 2030, according to a report released last week by Forbes.
The same report estimates the domestic AI market will add a 21% net increase to the U.S. gross domestic product by 2030. But while a Forbes survey found 64% of businesses believe that artificial intelligence will help increase their overall productivity, more than 75% of consumers are worried about the impact that AI has on the ability to trust information found on the internet.