Fully autonomous AI agents “should not be deployed” warn researchers

PEX Network’s weekly news bulletin rounds up the latest research, reports and publications in operational excellence (OPEX), digital transformation, artificial intelligence (AI), business process management (BPM), process mining and process intelligence and more.

This week includes:

• Researchers find “no clear benefit” of fully autonomous AI agents
• Workers risk overreliance on generative AI
• Robotic process automation market set for major growth
• Synergized process mining and BPM enhances operational performance and decision-making
• IDC examines key functionalities of global IDP software

Researchers find “no clear benefit” of fully autonomous AI agents

Researchers warned that fully autonomous AI agents should not be developed. “We find no clear benefit of fully autonomous AI agents, but many foreseeable harms from ceding full human control,” they stated. As detailed in a new paper, the researchers reviewed prior scientific literature and current product marketing to delineate different AI agent levels, detailing the ethical values at play in each and documenting trade-offs in potential benefits and risks.

“Our analysis reveals that risks to people increase with the autonomy of a system,” they wrote. “The more control a user cedes to an AI agent, the more risks to people arise. Particularly concerning are safety risks, which affect human life and impact further values.” The researchers highlighted three critical directions for the future: adoption of agent levels, human control mechanisms and safety verification.

“While increased autonomy can offer genuine benefits in specific contexts, human judgment and contextual understanding remain essential, particularly for high-stakes decisions,” the researchers concluded. “The ability to access the environments an AI agent is operating in is essential, providing humans with the ability to say ‘no’ when a system’s autonomy drives it well away from human values and goals.”

Watch Tyrone Smith Jr, program faculty member/adjunct associate professor at University of Southern California (USC) and founding member of Society of People Analytics (SPA), explore building trust in AI

Workers risk overreliance on generative AI

Some workers risk becoming overly reliant on generative AI, with their problem-solving skills declining as a result. That’s according to a new study by Microsoft and Carnegie Mellon University. The pair surveyed 319 knowledge workers to investigate when and how they perceive the enaction of critical thinking when using generative AI and when and why generative AI affects their effort to do so.

Participants shared 936 first-hand examples of using generative AI in work tasks. “Quantitatively, when considering both task- and user-specific factors, a user’s task-specific self-confidence and confidence in generative AI are predictive of whether critical thinking is enacted and the effort of doing so in generative AI-assisted tasks,” the study noted.

Higher confidence in generative AI is associated with less critical thinking, while higher self-confidence is associated with more critical thinking. This essentially means that workers who are confident tackling a task are more likely to apply critical thinking around generative AI outputs, while those less comfortable with a task are more likely to assume generative AI outputs are adequate.

“Generative AI shifts the nature of critical thinking toward information verification, response integration and task stewardship,” the researchers stated, pointing to a need to re-think the design of enterprise AI tools.

Download Generative AI & the transformation of operational excellence: Industry report

Robotic process automation market set for major growth

The robotic process automation (RPA) market is predicted to reach a value of $172.77 billion by 2032, according to a report from Orion Market Research. The study provides a quantitative analysis of RPA trends and dynamics to determine the most promising opportunities over the next seven years. It also offers valuable insights into different regions and the key players operating within each of them, taking into consideration economic, social, environmental, technological and political factors.

“The process automation market encompasses a wide range of applications, including manufacturing, supply chain management and customer service, among others,” the report read. “As businesses continue to seek innovative solutions to streamline operations and maintain competitiveness, the demand for process automation solutions is expected to rise steadily.”

Watch Ramnath Natarajan, director at Johnson Controls, discuss running an automation center of excellence

Synergized process mining and BPM enhances operational performance, decision-making

Researchers assessed how process mining supports the improvement of BPM outcomes in terms of operational performance, flexibility and decision-making. “Enterprises, in the current volatile and fast-paced business environment, have shifted more towards data-driven approaches towards business process enhancement,” they wrote. The integration of process mining with BPM promotes visibility of processes and performance for the purpose of optimization, resulting in enhancements in effectiveness, stability and flexibility, they added.

“Although issues like the quality of data and resistance to change are always present, there are distinct advantages of this integration in the long run.” Subsequent studies should consider the use of other algorithms and techniques of the machine learning family in process mining to increase the accuracy of predictions, according to the researchers.

“In the same regard, more research could be done on how industries other than the ones explored in this paper can adopt this kind of integration.” Consequently, process mining and BPM are set for further development on the crossroads of these two approaches to open up the next levels of efficiency and innovate the business future.

Register for All Access: Process Intelligence and Process Mining 2025

IDC examines key functionalities of global IDP software

Market intelligence firm International Data Corporation (IDC) released a new report examining the key functionalities of worldwide intelligent document processing (IDP) software. The IDC ProductScape is designed to help technology purchasers identify which vendors align with their changing requirements, categorizing functionalities as fully supported, partially supported, partner provided, road map or not supported.

The report features products from a range of IDP providers including ABBYY, Appian, AWS, IBM, Rossum, SER Group, Tungsten Automation, UiPath and Veryfi.

“IDP software is designed to improve the accuracy and speed of processing documents,” wrote Andrew Gens, research analyst a IDC. “However, no single vendor is uniquely positioned to address all the needs of potential technology buyers. This ProductScape should help your organization shortlist the best IDP vendors for your needs, including providing a resource for outreach and/or creating your own specific questionnaires.”

Watch Lisa Williams, global operations talent strategy and employee experience director at Dow, reflect on embracing new technology

READ MORE PEX RESEARCH & REPORTS NEWS

• New age of AI brings unprecedented autonomy to businesses

• Agentic AI drives investment to tackle complex business workflows

• Intelligent automation market set for large growth as process orchestration drives success

PEX Report 2024 global state of process excellence

PEX Network's annual report reveals the current state and future trends of process excellence. Based on a study profiling hundreds of global process excellence leaders, the report provides insights on how to leverage innovation, generative AI, and process intelligence to achieve operational excellence, customer-centricity, and organizational growth. Download your copy of the PEX Report 2024/2025 today and benchmark your process excellence journey against your peers.

Download Now