What Modern Data Architectures Require Today

At its core, SAP data integration has always meant the same thing: extracting data from operational SAP systems and making it available for reporting, analytics, or data warehouses. Nothing has changed in that regard—but what is now additionally required has certainly changed. Cloud and lakehouse architectures, automated processes, and new data-driven applications significantly expand the scope of requirements.

Data must not only be reliably available, but also as up-to-date as possible, traceable, and usable across different platforms—such as in cloud and analytics environments like Microsoft Fabric, Databricks, or Snowflake. This is also changing the role of data integration. The simple connection between source and target systems is increasingly becoming the foundation for enterprise-wide use of SAP data, including as a basis for AI agents and applications.

This, in turn, involves a complex interplay of various requirements: Companies need SAP-compliant access paths, must provide data that is as up-to-date as possible depending on the use case, and must preserve the data’s business context. At the same time, they must avoid creating new dependencies. Last but not least, the question arises as to how data and the insights derived from it can be fed back into operational processes. With these new usage scenarios, the focus shifts to how companies access SAP data.

New Rules for Data Access

In the future, modern integration solutions must combine performance with SAP-compliant access methods.

CDS Views play a crucial role in this process and form the foundation for the structured and targeted provision of SAP data. This raises the question for companies of which access method is best suited to their specific use case. Factors such as data volume, timeliness requirements, and existing system landscapes all play a role in this decision.

The latest version, Xtract Universal.iQ from Theobald Software, supports various approaches: In addition to ODP OData—the established SAP standard interface for accessing CDS views—CDSFlow offers another approach for targeted, selective extraction and stable delta processes. Even the best data access is of limited help if decisions are based on outdated information. Traditional batch processing reaches its limits as soon as operational processes require up-to-date information. Instead of regular data transfers, data must increasingly be available with low latency for analyses, applications, and automated processes.

The availability of up-to-date data does not depend solely on the platform used. Rather, the decisive factor is the frequency with which changes are transferred from the SAP systems. Methods such as Table CDC or CDSFlow can deliver changes continuously or at short intervals.

Apache Kafka can serve as a central data hub through which data is continuously processed and distributed to downstream applications or data lakes. The shorter the time between a change in the SAP system and its delivery to downstream systems, the faster this information can be incorporated into analyses or operational processes. This makes timeliness a key aspect of modern SAP data integration, regardless of the technology used for further processing.

As data sets grow, so does the need for organization. After all, just because data is technically available doesn't mean it's actually usable.

Companies need to know what data is available, which of those data points they actually need, what they mean, and how they relate to one another. Metadata provides this context: It makes data sources and relationships transparent and helps identify the SAP data relevant to a specific use case more quickly.

Data Needs Context

A centralized metadata and terminology management system can classify SAP data objects by business domain, clarify responsibilities, and translate technical field names into terms that business users can understand. Lineage information supplements this classification with information about the data’s origin—an important prerequisite for control, compliance, and data quality.

Xtract Universal.iQ brings this contextual knowledge together, thereby laying the foundation for reusable data products. Instead of reprocessing raw SAP data for each use case, the system creates documented and cataloged data products that business units can access directly.

This additional context not only makes SAP data easier for business users to understand, but also increasingly usable for AI agents. In addition to usability, the focus is shifting to control over where and how data is processed—keyword: data sovereignty. This is becoming increasingly important, especially in light of modern cloud and lakehouse architectures. Data sovereignty is about more than just the question of where data is physically stored. The decisive factor is the extent to which an architecture is tied to a specific provider—for example, through vendor-specific cloud integrations, proprietary interfaces, or closed storage formats.

The tighter this integration, the more difficult and expensive it will be to switch platforms later on. On the other hand, consistently separating data sources, the integration layer, and target systems reduces this dependency and preserves the ability to flexibly develop your own data infrastructure.

This is where open data formats like Apache Iceberg come into play. They make data available in a way that allows different analytics and data platforms to access it. As a result, the choice of data format becomes a key factor in the long-term flexibility of a data architecture.

Seamless Data Integration

Xtract Universal.iQ can automatically and incrementally deliver SAP data in this open format for data lakehouse architectures. However, data sovereignty does not end with the choice of data format and architecture. Equally important is who actually operates the data integration infrastructure.

Flexible, containerized deployment models give companies the freedom to choose where they run their data integration even at this level. For example, Xtract Universal.iQ can be deployed on-premises or in a private cloud. Modern SAP data integration does not end when data leaves the SAP system. Insights gained from analyses must also be able to feed back into operational processes.

From Data Flow to Process

For this to work, integration must be bidirectional. Data integration is thus increasingly becoming process integration as well. Depending on the application, it must also be possible to trigger processes or write data back in a targeted manner. Xtract Universal.iQ supports this bidirectional approach.

AI scenarios further expand on this approach: Through an optional MCP server, SAP data and services can be made available to AI agents and LLM-based applications in a controlled manner.

Access to data is just the beginning

The demands placed on SAP data integration are becoming increasingly diverse, and there is no end in sight to this trend. Data has long since ceased to be merely the basis for reports or analyses; rather, it now forms the foundation for data-driven decisions, processes, and new business models.

This makes the question of how to ensure that this data remains available and usable for companies in the long term all the more important. Accordingly, the future of SAP data integration lies in the interplay of various requirements and technologies.

SAP-compliant access paths, real-time data, metadata, open formats, and bidirectional process integration are the key building blocks today. It is not yet possible to fully foresee what additional requirements will arise in the future.

This makes it all the more important to have an integration architecture that can accommodate new target platforms, access methods, and usage scenarios without having to fundamentally rebuild existing data flows.