Nothing but the Truth About Process Mining

Like annual reports, statements of claim are well reviewed in terms of content, as they have to stand up to scrutiny by lawyers, auditors and judges. They therefore provide valuable insights into the current market situation. A current example of the effect of court statements is the statement by an Apple manager on the search function, which triggered a fall in Alphabet's share price². 

Leading PM providers

Particularly explosive: Celonis, according to Gartner the leading provider in the field of process mining (PM), feels compelled to take legal action against SAP. This points to considerable difficulties in the interaction between the two companies. Specifically, Celonis accuses SAP of deliberately blocking access to SAP data with third-party tools - including solutions from Celonis. Many of the allegations mentioned in the statement of claim are of a legal nature and are of secondary importance for this article. Of more interest are the statements that provide insight into the practical status of Process Mining use: Are there any new findings? Technological or substantive advances?

This article follows on from an article³ from 2022 in the E3 magazine. At the time, it was about six misunderstandings between customer requirements and the possibilities and limitations of process mining. The conclusion at the time was that process mining tools are assumed to have certain capabilities for which there are more suitable approaches in other tools or directly in the SAP ERP system. Why does process mining often reach its limits in SAP? 

The truth about event logs

A central promise of PM is the analysis of real business processes on the basis of so-called event logs - digital traces that every transaction leaves behind in the system. The statement of claim (point 24) emphasizes precisely this: every business process creates a digital footprint that should be analyzable. However, the reality in SAP systems is different. Such event logs usually do not exist there in the depth and structure that would be necessary for effective PM. There are tables that map certain processes - such as predecessor-successor relationships (e.g. table VBFA) or change logs (e.g. CDHDR). However, this information only shows part of the overall picture.

One example: Numerous follow-up processes are automatically triggered as part of customer order processing - such as the transfer of requirements to the planning department. These integrative processes, often carried out by so-called "accountants", are often only incompletely recorded in the standard logs. It becomes particularly complex when price calculations or account determinations come into play - areas that are hardly taken into account in many process mining tools. In practice, this means that companies have to resort to complex additional analyses. Depending on the issue at hand, additional data sources have to be evaluated and individually combined. This is possible - but only with in-depth SAP expertise and considerable technical effort.

Conclusion: Anyone who believes that process mining is simply possible "at the touch of a button" in SAP systems is mistaken. The reality is complex - and the truth about event logs is much less straightforward than marketing promises suggest.

Process mining or ERP?

In practice, PM is often advertised with the argument that it brings concrete benefits in day-to-day business - for example, when checking vendor invoices or avoiding double payments (point 27 of the statement of claim). However, this example shows just how redundant some PM analyses actually are: This is because such checking mechanisms have long existed in every common ERP system - including SAP. If PM manufacturers nevertheless advertise such functions, they quickly find themselves in need of an explanation. This is because they run the risk of merely replicating existing ERP features - often at higher costs and with potential gaps in up-to-dateness and data quality. 

Especially in comparison to a modern ERP system such as SAP S/4 Hana, which is equipped with powerful analysis tools, it is becoming increasingly difficult for external PM solutions to offer real added value. SAP S/4 Hana offers around 100 standardized analysis apps, for example for tracking customer orders, which enable precisely such insights - directly in the ERP system, without external tools.

Conclusion: When PM systems promise functions that ERP has long since mastered, this not only creates confusion, but also a questionable duplicate infrastructure. With the technical progress of modern ERP platforms, the room for maneuver for external PM tools is therefore becoming increasingly smaller. 

Where is the truth about the target process? Point 30 of the statement of claim claims that the original target process can be derived from the real actual data of an event log. But this is not tenable. An event log - especially an incomplete one - only shows what actually happened, not what was planned or intended.

In order to understand the intended flow (the "target flow") of a process, the customizing settings in the ERP system must be analysed. These define the specific process specifications that were defined by the customer during system implementation or ongoing customization. Without this configuration, the view of the target process remains incomplete.

Another problem is that the process mining world often lacks well-founded cause-and-effect analyses. Control parameters from the master data - such as how a material is planned or how an order is evaluated - are rarely taken into account. Yet it is precisely these parameters that have a significant influence on process behavior.

This neglect is not a new phenomenon: even classic process modeling had to struggle with it. The PM approach traditionally focuses on observable processes, not on the underlying system-side control mechanisms.

Conclusion: If you really want to understand the target state of a process, there is no getting around the analysis of customizing and master data. However, this requires specialized methods - such as reverse business engineering (RBE)⁴ - that were developed precisely for this purpose. Process mining alone is not enough here.

12 to 36 hours until the truth

A central weakness in the practice of PM appears to be data extraction. Point 124 shows that Celonis used an outdated SAP technology for the data connection: the Remote Function Call (RFC) with Abap. The problem is that SAP no longer supports this method. Point 141 shows how inefficient the old solution was: extracting data from 100 tables took around twelve hours. 

New approaches, for example via OData, even take up to 36 hours. This shows that a real real-time data connection between the PM system and ERP system never existed - because the analyzed data was always at least half a day old. A (hopefully exaggerated) cost estimate shows that the alternative download of 100 GB of data via SAP Datasphere could cost up to 30,000 dollars. So it seems that in the cloud world, data provision is no longer free. 

In addition, the question arises as to why PM analysis must be carried out across the board.
100 GB of data to be transferred? With a more targeted analysis approach, significantly less data could be sufficient - often just 1 GB per query would be enough.

One old and two new truths

The process mining approach is now outdated. You could draw parallels with the history of online store providers such as Intershop: In the 2000s, they were seen as the beacons of hope of the digital market. Today, if they still exist at all, they are usually only niche providers with limited commercial success. It was already clear at the end of the 2000s that the integrated ERP suite would prevail in the long term. PM tool providers now seem to be experiencing a similar fate.

In addition, PM tools have long since given up their original focus - analyzing event logs. Their growing "hunger for data" is placing an increasing burden on systems. At the same time, S/4 Hana and its in-memory database have greatly improved the analysis capabilities within the ERP system. Analyses in terms of process monitoring can now be mapped directly in ERP - PM tools are becoming increasingly superfluous. Their useful area of application is therefore significantly smaller.

What's more: In the cloud world, unlimited data queries and high resource utilization are no longer a matter of course. If AI models are also to be trained for PM in the future - which is likely - competition for computing power and data access will continue to increase. It remains to be seen what legal solutions or compromises will be found here. To be continued ...

Misunderstanding vs. reality

Misunderstanding 

• Quick problem diagnosis
• Comprehensive overview
• Compliance/risk analysis
• Analysis across multiple systems
• End-to-end view
• Automation

Reality

• High effort with PM tools
• Other tools easier
• Not part of PM
• Limits available
• Only possible to a limited extent
• Mostly via ERP or add-ons

(1) Link to the statement of claim.

(2) Link to the Frankfurter Allgemeine article

(3) Reverse engineering for software libraries such as SAP R/3; (4) Article: Hufgard, Wenzel (1999): Reverse Business Engineering - Deriving models from productive R/3 systems. In: Electronic Business Engineering. 4th International Conference on Information Systems. S. 425-441.