All processes in harmony

When it comes to orchestration services, it's worth taking a look at the open source world

The digital transformation is bringing companies more and more digital processes. These are more efficient than their analog predecessors and, in the best case, can be automated; their sequence is always traceable and the quality of the results can be checked at any time.

Many systems come with their own mechanisms for the automated control of processes, for example the SAP Business Warehouse with programmable process chains. These are generally limited to their own system context, and at the latest when processes extend across several systems, reliability and traceability are sacrificed and parachutes begin to be knitted.

Most systems provide advanced users with programming interfaces via which certain functions can be controlled by external programs. This circumstance prepares the field for orchestration services. Such a service is the framework for users to define complex process flows, have them executed based on configurable conditions, react to errors and deviations, and finally keep an overview of all processes and states.

What sounds so simple is a complex challenge. The very term orchestration contains the metaphor of a symphony orchestra, in which a multitude of instrument groups and individual musicians are guided to work together to create a harmony. A central orchestration service is as crucial to the quality of digital processes as the conductor is to the performance of a symphony.

So there is no way around the orchestration service. We inevitably come to the famous question: Make or Buy? Or more precisely, since a complete in-house development is not advisable in view of the given complexity: Which software fits my requirements best?

For every company, the answer may be different due to the specific framework conditions. Our experience with our customers is that the adaptability and extensibility of modern open source orchestration services are a decisive competitive advantage over proprietary products. The base of freely available extensions and connection modules to common business applications is enormous in the open source ecosystem and is constantly growing. If a specific connection is missing, it can be added without jeopardizing compatibility with the core system.

Apache Airflow is a lighthouse project among the open source orchestration services. Airflow is based on the Python programming language and goes so far as to define all process flows in source code. The entire operation can be based on fully versioned configuration files according to DevOps principles.

The possibilities for automating business processes with this framework are practically unlimited. Many companies use precisely these strengths to control cross-system processes between SAP and third-party systems. For example, data from SAP BW can be retrieved for automated machine learning model training in a non-SAP context and forecast data calculated on the basis of the models can be written back.

Integrating and operating such an open source orchestration service in the system landscape requires technical know-how and provides independence from the product lifecycle of proprietary software products. Last but not least, this path is an investment in the capabilities of your own team. Durable, open technologies as the basis for business process orchestration are a significant factor for the sustainability of deployed components.