In-memory is not enough

Even SAP has now realized that the original, seemingly ingeniously simple idea of keeping all data permanently in memory is partly not feasible and partly not enough.

It is not feasible, for example, if a database contains a large amount of historical data, the loading of which into memory would lead to an explosion in the cost of the memory required.

It is also not enough to keep data permanently in memory, because every database must guarantee the persistence of the data stored in it.

It is therefore necessary to use persistent storage media and it makes sense to think about how to use this storage as efficiently as possible. For many versions of the Oracle database, there are already two technologies that serve this efficient use:

Partitioning and compression. And there are two technologies, because the word "efficient" can have two different meanings.

What belongs together: Partitioning

"Efficient use of memory" - this can initially mean that data should be stored in such a way that as few I/O operations as possible have to take place when accessing information that is not yet in memory.

"Optimization of data storage" then means that data is not stored in random order (as is done by default), but rather taking into account the question of which data groups will be needed together again later.

Partitioning a table means breaking it down into subgroups. A partition then contains, for example, all data that was added in a certain month or is assigned to a certain branch.

For SAP-on-Oracle customers, partitioning is activated by default in SAP BW, so they benefit from it immediately. However, Oracle Partitioning is released and supported for all SAP NetWeaver applications. It can therefore also be used in non-BW systems. The SAP Partitioning Engine, for example, is then available for implementation.

Compression

However, "efficient use of storage space" can also mean that data should be stored in such a way that it takes up as little storage space as possible and that the gigantic growth of databases can be slowed down.

Across several versions of the database software, this means that the efficiency of data storage must be permanently increased so that the same data stock requires less and less storage space from version to version.

And an additional requirement means that all this should happen without the customer having to pay the price of a deterioration in performance.

Oracle Database 11g already relied on the concept of not writing values that appear more than once. This applies to tables as well as indexes. The compression rate that can be achieved in this way depends on the characteristics of the data and the application.

Data from SAP BW (BI) can usually be compressed more than data from SAP ERP (ECC), and SAP CRM allows even greater savings. On average, a database fully compressed with Oracle Database 11g in the SAP environment requires 55 percent less storage space than the corresponding uncompressed database.

It all depends on the temperature

A frequently asked question is: Why is compressed data storage not actually made the standard?

Part of the answer can be found by looking at the SAP BW tables that are used to load new data. We would like to compress such tables, but this would significantly delay the loading process.

This is where Oracle Database 12c comes in, with the introduction of a new parameter. In version 11g, the user can answer the question as to whether this object should be compressed for each table and each index.

Possible answers are "Yes" or "No". Version 12c also asks when new or changed data should be compressed. Answers such as: "Yes, but not for another week" are now possible.

Only with this new parameter can the tables required for loading data be included in the compression: Uncompressed data is loaded first (no extension of the runtime), which is only compressed later, i.e. at a suitable time (deferred compression).

This solution has been designed in Oracle Database 12c in such a general way that it can be used to implement complete Information Lifecycle Management (ILM). It is based on two new features:

• The so-called heat map automatically monitors how intensively data is used. It divides data into "hot data", which is frequently used for reading and writing, "warm data", which is only used for reading, and "cold data", which is very rarely used or no longer used at all.
• Automatic Data Optimization (ADO) makes it possible to define more precisely what "hot", "warm" and "cold" should mean and to specify what should happen when the data transitions from a hot to a warm or from a warm to a cold state.

Data can be relocated to other storage systems when its "temperature" changes, a process also known as "storage tiering". For example, cold data can be relocated to slower and therefore more cost-effective disks.

The database administrator uses Automatic Data Optimization to define rules that describe the different "temperature states", for example that data that has not been changed for 180 days is to be regarded as "cold".

"Compression tiering" then also decides how strong the compression applied should be; the time factor can also play a role here. If data has not been touched for more than 360 days, for example, the strongest compression would be applied.

"Within" is better than "nearby"

These comprehensive compression concepts and technologies for Oracle Database 12c offer particular advantages compared to SAP's own solutions. The continuously growing database also becomes a problem with Hana.

In order not to hinder productive operations, SAP relies on "near-line storage", which means nothing other than extracting data from the productive database and storing it separately, albeit "close" to the database. With Oracle Database, data can be kept in the production database for much longer, as it can be compressed considerably more.

Oracle Exadata

If you want to exploit the Oracle database even further, Oracle's Engineered Systems are a good choice. The Exadata database engine, which is optimized for database operation, is capable of even more optimization and efficiency enhancement methods:

With Hybrid Columnar Compression, Exadata systems offer additional, stronger compression algorithms so that compression tiering is possible in numerous levels.

In addition, an Exadata extends the database with "Smart Storage". This shifts some of the data-intensive calculations from the database server to the storage server. For example, tables and indices that are not relevant for queries can be filtered out at storage level to significantly reduce I/O.

Oracle Database therefore offers a wide range of options for optimizing and increasing the efficiency of database storage. Those who use these options for their SAP systems can make better use of resources without sacrificing database performance.