Adopting Event Streaming for Monitoring Shipments in Real Time

Hello, I have a package for you, are you home? I am at work right now, I was expecting you tomorrow. Your website still says that my order is being processed.

It might sound like a small problem, a nuisance soon forgotten, but on a larger scale, these situations cost time, money and reputation. The delivery service will need to drive back and forth with the package while the customer is not getting the best experience. The data did not refresh in time and as a result they did not reflect reality—that the package was already on its way.

For organizations, having real-time data is key to streamlining their work and services they deliver to their customers. The answer to this is event streaming—collecting, analyzing and managing streams of information the moment they are created.

Streaming architecture can power anything from simple event notifications—e.g. sending an alert when the temperature rises to a dangerous level—to real-time machine learning models that detect fraudulent activities. Modern stream processing systems are also a source of truth for business.

Application design and analysis

Organizations are already starting to think in terms of events and to adopt event streaming into their application design. It allows them to take data analytics to the next level, deliver excellent customer experience, build intuitive interfaces, develop machine learning scenarios, and future-proof their business.

Datavard recently prototyped a solution for a major construction company who planned to stream shipment events from SAP to the native Azure cloud services. The goal was to enable employees to monitor the status of a shipment through mobile push notifications.

Put simply, they would get a message on their phones the second the status of the shipment changed. To implement this type of event streaming, they needed a tool for real-time application integration.

Glue and Data Lakes

For this event streaming scenario, the company used Datavard Glue—a strong solution for a tight, native SAP data integration with data lakes running on big data platforms. It is already used by many customers to replicate meaningful SAP business objects in real time to modern cloud data storages like Azure Data Lake Gen2. This enables them to analyze their SAP data and get insights using the modern and powerful analytics services of Microsoft Azure.

In the described scenario, event streaming technology was needed for several reasons. Firstly, SAP data had to be adjusted and combined with IoT sensor data. Secondly, fetching SAP data through SAP Gateway put a strain on the productive system which was already under high strain. Thirdly, the latency requirements (the time delay between a change of shipment status and getting a notification) were set to a challenging 4 to 6 seconds globally.

In this specific use case, the company leveraged tools like Azure Event Hubs, Azure Cosmos DB, and Microsoft Azure functions to orchestrate a cloud-native architecture designed for the ingestion and processing of high throughput of small batches of data such as SAP transactions events.

To integrate Microsoft Azure with SAP, the company used Datavard Glue—a solution for deploying SAP data on a platform of choice. With the recent enhancements, Datavard Glue has the unique capability to directly integrate SAP with Azure Event Hubs.

When the streaming mode is activated, an optimized process is triggered, and replications are executed in real time as soon as the business event and corresponding data are generated in the system. So much for the theory. When the company put it to the test, they were not disappointed—the system successfully generated push notifications the moment an event occurred.

This new innovative integration scenario was facilitated by just a lightweight SAP plugin, and there are plenty of other use cases possible. Event streaming creates ample opportunities for organizations, and existing technologies are ready to tap into them.