Mega Trend - Big Data

The barrel is overflowing. SAP is barely able to handle all the construction sites anymore. Innovation is all well and good, but SAP's reckless pace is disruptive. Now "real-time" is taking its revenge.

SAP has neglected its duty of care to existing customers in favor of "Run Simple". The result is numerous projects that are getting out of hand and can only be rescued with the intensive help of external consultants and partners.

Without naming names here, there are SAP partners who have leased the majority of their consultants and programmers to SAP.

Some time ago, I reported here about a colleague in the retail sector whose Hana system was down for almost ten days - he was not yet productive with it.

Even SAP couldn't help, it took bringing in an outside Hana expert to get the system flying again - guessed right, it was the SAP partner that also powers the world-renowned brewer and was on the Sapphire stage in Madrid in 2011.

This and other external assistance for SAP construction sites have taken on such dramatic proportions that they have left their mark on the balance sheet for the first quarter of this year.

CFO Luka Mucic has promised that these are one-time expenses, and everyone in Walldorf was obviously shocked at the scale of the external support.

Not without gloating, our SAP regulars' table noted: Now SAP is also feeling how bitter it is to pay maintenance fees for enterprise support.

I do not agree with SAP CFO Mucic that the enormous financial outlay for external assistance with SAP projects can be localized to the first quarter of this year. We are only at the beginning of a mega-data development.

If 150 S/4 systems at new and existing customers already overwhelm SAP's consulting capacities, so that massive support from partners has to be relied on, what can be expected in the coming years?

In a previous column, I predicted that the 2025 deadline would be missed. Due to the available resources, the deadline cannot be met, neither by SAP itself, nor by its partners, nor by its existing customers.

Our regulars' table can be a good gauge: some are not averse to the concept of "Hana & S/4", but this and next year will be characterized by smaller PoCs, then an evaluation phase begins, budget funds must be made available, and a release change for infrastructure, for Linux, for Hana, for Abap, and for the ERP must be planned.

Likewise, Mega Data needs to be planned: We had an internal conference with the production managers, CCC leaders, CDOs and CIOs. One of the topics was IoT and how to manage the data volumes. We have CNC milling machines that deliver up to two terabytes of data per day.

Should the data be transported further or will it remain on site?

Can you provide enough memory and server power locally to perform evaluations and interactions in real time?

Modern CNC machines are extremely fast, what if the data line is too thin and Hana is too slow?

We found good answers to most of the questions, but one mega-data question remained open at the end: Whether local or cloud computing, where to put the data volumes - and what must, should, can be archived?

Hana is sufficiently fast for IoT, but CNC data cannot be stored here, nor archived in IQ/ASE, because it is far too expensive license-wise. SAP itself knows this challenge and has made the community happy with Vora, an in-memory query engine.

Vora is built on the Apache Spark Execution Framework and enables analytics on Hadoop data. Unfortunately, we are already reaching the limits of Hadoop, so we will need more alternatives in the foreseeable future. Hadoop in combination with Hana is excellent, but IoT is a mega-data project.

I think it boils down to a concept similar to the data collection at the Cern International Research Center. Nowhere is so much sensor data generated as in the underground detectors of Cern.

There are two challenges here: quantity and speed.

The storage capacity of the Cern data centers is not even close to being sufficient for the flood of data that is generated in a very short time. Thus, a theoretically simple trick is used: sorting out and forgetting before storing.

In practice, this process is extremely complex and can only be managed with the highest technical effort - but the principle is convincing and could also help us with thousands of CNC machines. So before Hana/Hadoop gets its hands on the data, you should clean it out. I wonder if anyone at SAP has thought of that.