AI Is Unleashing a New Human-Machine Interaction Model: LLMs + RPA

Omar Shaya
3 min readMar 4, 2023

LLMs are unleashing a new human-machine interaction model and will redefine the RPA category.

Adept, Inflection AI, and Maya Labs are a few of the companies at the intersection of LLMs and RPA

A new human-machine interaction model at the intersection of large language models (LLMs) and robotic process automation (RPA)

Robotic Process Automation (RPA) provides “software robots” that use computers on our behalf. The category existed for a long time with notable companies such as UiPath and Zapier. However, with the development of Large Language Models, a new type of RPA-companies is emerging. Existing RPA solutions rely on a lot of manual work that has to be done upfront to create rules and processes to get the job done. The new solutions pioneered by a few startups rely on natural language (NL) text description as the input to automate what the end-user wants to do.

Keyboards, mouse cursors, and touch screens are the bottleneck for human-machine interaction. People are limited by those methods to interact with computers. With the new startups and LLMs, people will be able to extract more value from computers, faster, and with less effort and friction than they have today using traditional input/interaction methods. Traditionally, many existing GUI-based software offer scripting languages to allow users to execute complex tasks–e.g. Excel Macros, or Maya (3D animation software) Embedded Language (MEL). Those scripting languages go beyond what even pro-users can do with these apps, and heavy training is required to master those scripts. A better and easier way is required.

LLM-enabled RPA companies will solve those problems by enabling users to describe, in plain English, what they want the machine to do. This new user interface will reduce human training time to learn new software, will make complex software more accessible to a wider range of people, and as a result for businesses, costs will go down and productivity will go up.

Companies in The Space

Adept: the company is building text-to-action models to automate actions that otherwise users have to do themselves in apps or on the web. Adept has a focused approach on automating time-consuming tasks within existing software. For example, one can add a new lead to Salesforce CRM by simply describing that in plain English. It is rumored that they have deals with Salesforce and Adobe to create NL interfaces for their apps. Adept has a strong founding team of former AI scientists from Open AI and Google Brain.

Examples from Adept

Maya Labs: while similar to Adept, this Y-Combinator company focuses on scenarios that require integration. Using natural language, the user can define a flow for the machine to extract data from a sheet, manipulate that data, and insert it into a template. Maya Labs tackles scenarios similar to existing RPA players such as UiPath. This positions it as a more familiar tool to companies that already rely on RPA.

Inflection AI: there is little known about this mysterious company other than its prominent co-founders (e.g. Mustafa Suleyman, Reid Hoffman) and huge seed round ($265 million). They claim to want to redefine human-computer interaction through natural language. And some sources claim they are building a web assistant or the next iteration of Siri.

See also: Inflection seeks up to $675mn as funding for AI groups heats up | Financial Times (ft.com)

--

--