Information flows in real time

Hello, I have a package for you, are you home? I am at work right now, I was expecting you tomorrow. Your website still says that my order is being processed.

What sounds like a trivial matter, when thought of on a larger scale, costs unnecessary time, money and reputation. The courier will have to drive the package back and forth unnecessarily, while at the same time the customer will not have the best experience. The data was not updated in time and therefore did not reflect reality - that the package was already on its way.

Event streaming offers (almost) unlimited application possibilities: For companies, real-time data is the key to optimizing the processes and services they offer their customers. The answer is event streaming - collecting, analyzing and managing information streams the moment they are created.

The streaming architecture can do everything from simple event notifications, such as sending an alert when a sensor's temperature rises to a dangerous level, to real-time machine learning models that detect fraudulent activity.

Application design and analysis

Organizations are already starting to think in terms of events and incorporate event streaming into their application design. This allows them to take their data analytics to the next level, deliver an excellent customer experience, create intuitive interfaces, develop machine learning scenarios, and future-proof their business.

Monitoring shipments in real time through mobile push notifications: Datavard recently implemented a prototype solution for a large company in the building materials industry. This planned to stream shipment events from SAP to native Azure cloud services, with the goal of enabling employees to monitor the status of a shipment through mobile push notifications.

In simple words, they would receive a message on their cell phone the second the status of the broadcast changes. To implement this kind of event streaming, you need a tool for real-time application integration.

Glue and Data Lakes

For this event streaming scenario, Datavard uses the Glue tool, a solution for native SAP data integration with data lakes deployed on big data platforms. Glue is already used by many customers to replicate meaningful SAP business objects in real-time to modern cloud data stores such as Azure Data Lake Gen2. This enables SAP data to be analyzed and insights to be gained with modern and powerful analytics services from Microsoft Azure.

Deploying an event-driven architecture with (only) one lightweight plug-in! In the above scenario, event streaming technology was required for several reasons. First, SAP data had to be adapted and combined with IoT sensor data.

Second, retrieving SAP data via the SAP gateway placed an additional burden on the already heavily loaded production system. Third, latency requirements (the time delay between a change in shipment status and the receipt of a notification) were set globally at a demanding four to six seconds.

In this particular use case, Datavard leveraged tools such as Azure Event Hubs, Azure Cosmos DB, and Microsoft Azure capabilities to orchestrate a cloud native architecture designed to ingest and process a high throughput of small data sets such as SAP transaction events.

Microsoft Azure integration with SAP used Glue, a solution for delivering SAP data to a platform of choice. With recent enhancements, Glue has the unique ability to integrate SAP directly with Azure Event Hubs.

When streaming mode is enabled, an optimized process is triggered and replications are executed in real time as soon as the business event and corresponding data are generated in the system. So much for the theory. When the innovative solution was put to the test, no one was disappointed - the system successfully generated push notifications the moment an event occurred.

Even though this new innovative integration scenario was only made possible by a lightweight SAP plug-in, many other use cases are still possible. Event streaming offers companies numerous opportunities, and existing technologies are ready to take advantage of them.