AI in the SAP Environment Redesigned with a Side-by-Side Architecture

SAP clearly positions its AI innovations within its own cloud and platform strategy. Features such as Joule are available to users who are running current S/4 releases and using the SAP Cloud at the same time. For many companies, this is not yet a realistic option. ECC systems still in use, high transformation costs, regulatory requirements for data storage, and concerns regarding data sovereignty in hyperscaler clouds mean that many SAP customers remain effectively excluded from these innovations. In regulated industries, operating business-critical data outside of controlled environments is often not permitted at all.

Separating AI Logic from the SAP Core System

This is exactly where the side-by-side model comes in. This approach deliberately separates the AI logic from the core SAP system, making it possible to run AI in a separate but closely integrated architecture. This allows companies to leverage AI innovations without fundamentally transforming their existing SAP landscape. Instead, SAP remains the leading system for business data, while AI operates as a standalone layer alongside it.

With Red Hat AI, Red Hat offers a solution that supports precisely this architectural model. The integrated platform for deploying AI in hybrid cloud environments runs outside of SAP systems yet still ensures seamless integration. Technologically, Red Hat AI is based, among other things, on components such as Red Hat OpenShift AI as the operating platform and an inference server that handles the execution of the AI models.

From an architectural standpoint, SAP remains the backend where the relevant business data is stored, whether in ECC, S/4, or a Business Warehouse. Through standardized interfaces such as OData or REST, this data can be transferred to an LLM and an inference server for processing and generating results. The results can then be fed back into the SAP interface via APIs—typically in JSON format—such as the SAP Fiori front end. This creates a seamless user experience for users, even though the actual intelligence is operated outside the SAP system. One advantage of such an architecture is its openness. Companies are not locked into a specific model but can use different LLMs depending on the use case. These include, for example, SAP RPT-1—an AI model provided by SAP for business data—as well as IBM Granite or models such as Llama or Mistral. This flexibility allows companies to tailor AI strategies to their specific needs and design them independently of individual vendors.

Furthermore, companies using side-by-side architectures can ensure data sovereignty, as they can run their AI solutions entirely within their own private cloud or data center. This allows them to retain control over their data, comply with regulatory requirements, and avoid dependencies on hyperscalers. This factor is becoming increasingly important from a strategic perspective, particularly in Europe.

From Fraud Detection to the Supply Chain

The business value of the side-by-side approach to AI deployment—which leverages both SAP and external data—is significant. Applications range from automating fraud detection in finance to optimizing inventory levels in the supply chain and increasing customer loyalty. The side-by-side model offers a pragmatic and immediately actionable solution to current challenges. Companies can continue to use existing systems while integrating AI capabilities and driving innovation independently of release cycles or cloud strategies. AI thus evolves from a platform-specific function into a flexible tool that can be deployed across the entire value chain.

To the partner entry: