The Deconstruction of an AI Agent

Modern society is marked by the division of labor and even atomization. This economic model has been a catalyst for generating prosperity. Most people are accustomed to working with devices that are designed to be used in a specific way. This is not a problem in the vast majority of cases.

In principle, know how a car works; I can get from A to B safely. If something breaks, there are experts who know my car inside out and can repair it. There are experts, craftsmen, and specialists for most everyday items in the private and professional world.

When deconstructing an AI agent based on the "knowledge" of a large language model (LLM), we find that the agent's behavior is not easy to document or understand, and it cannot be replicated. This need not be a disadvantage! The fantastic results of AI are largely based on statistics, which in turn can be unambiguous in essence, but cannot always be replicated due to the amount of data and situation taken into account.

Taking the deconstruction of an AI agent to the extreme could result in it transferring employees' salaries at the end of the month—or not—and possibly causing it to hallucinate about why there is no money this time. This is a very theoretical example, but there are numerous reports of dramatic derailments of AI systems. The challenge, as the mathematician and computer scientist would say, lies in uniqueness. The challenge lies in the nature of AI itself. Using reinforcement learning, a high-quality AI can become the best chess or Go player because every game ends unambiguously.

The machine has either won or lost. If the machine plays against itself millions of times, it quickly learns all the tricks and becomes better than any human. Of course, reinforcement learning does not work with poems, pictures, or PowerPoint presentations. Here, it is the viewer's taste that dictates whether the result is successful.

SAP and many other IT companies now want to use LLMs and AI agents in the ERP environment. To prevent major disasters, the system will be rule-based. This is reminiscent of robotic process automation (RPA), in which software robots (bots) were designed to automate repetitive, rule-based tasks that would otherwise be performed by humans. However, AI agents will have more freedom and autonomy thanks to LLMs' "intelligence." Can this work?

Based on an LLM's findings, an AI agent could conclude that the current business model is inefficient and environmentally harmful. At best, the AI agent refuses to work. At worst, it deletes the ERP database.

Recognizing this danger, platform provider Boomi is addressing not only automation through AI agents but also the governance of AI agents. Future LLMs and AI agents for ERP systems will require deterministic control. Is this a contradiction? The AI scene is only at the beginning of a revolutionary development.

SAP seems to ignore most of these dangers and concerns. SAP only briefly addresses the topic of heterogeneous AI development and its many challenges and contradictions. At the SAP Connect event in Las Vegas, Executive Board Member Muhammad Alam said SAP customers need more than a patchwork of different best-of-breed applications. At the same time, his statement was put into perspective at a technical level. A patchwork is likely to result with a two-tier AI system consisting of assistants controlling several AI agents.

SAP is intensifying its efforts to develop a composite S/4 (Composable ERP) that is increasingly aligned with a holistic suite. The existing S/4 patchwork is to be orchestrated by AI assistants and AI agents. Currently, there is no system-critical classification of the work of AI assistants and AI agents at SAP.

SAP is aiming to transform its existing ERP patchwork into a two-tier AI system, comprising assistants and agents. However, to thrive in a volatile market environment, companies require more than a patchwork of different best-of-breed applications, according to Muhammad Alam. The recent SAP announcements demonstrate how the new Business Suite integrates artificial intelligence, data, and applications. No one is adressing back-referencing, reinforcement learning, or governance.