Software and performance testing of SAP systems

The FIS-ASP test team had no real doubts when they asked themselves this question. They were well aware of the special requirements placed on the software.

To say it in advance: The test did not turn out to be a disappointment, on the contrary. The task was to perform software and performance tests of SAP systems using open source resources. The OpenQa project selected for this purpose was originally developed by Suse.

Many companies use OpenQa for automated testing of software and integrate it into the continuous integration processes of source code management. Entire Linux distributions are tested in a fully automated manner in this way.

Since OpenQa can perform and verify GUI input using image recognition as well as analyze played sound samples, it quickly became clear that the tasks would not be unsolvable.

The tests themselves take place in prepared images that represent a complete virtual machine. The virtual machines are created with QEMU - in its KVM extension, this is not a foreign product for the FIS-ASP test team, as it is also approved for the virtualization of SAP systems and is also used in the OpenStack Cloud.

An originally highly estimated effort for such a project is therefore reduced enormously if the technologies used in the process are already known.

During the initial tests, it became apparent that OpenQa can be used to test not only software, but also performance in an automated manner. To this end, FIS-ASP developed various scenarios: First, a test was performed that creates a predefined number of users in the SAP system.

All interactions with SAP took place through the SAP GUI for Java, which runs on a small Linux VM. After that, a series of other "tests" started in parallel, each representing a user.

All that had to be done was to take into account the company-specific use cases in order to reliably measure the SAP system to be tested under load situations and to collect performance data in a targeted manner.

The different tests are defined in easy-to-edit JSON files and can thus also be adapted script-controlled. After all tests have been run, all previously created users in SAP can be removed again with another.

Tests of browser-based interfaces such as Fiori apps are just as manageable as tests through the SAP GUI. The testers at FIS-ASP used both Chrome and Firefox as browsers supported by SAP.

In terms of scalability, the requirement was to be able to run performance tests in the range of more than 10,000 users with a reasonable amount of hardware.

Scalability in hardware was already given by OpenQa, multinode could be configured there in the interface. However, since each test represents a virtual machine on the OpenQa server, FIS-ASP investigated some options in Linux memory management here.

The best results were achieved by using KSM (Kernel Samepage Merging). Here, memory pages that do not differ are only stored once in memory and presented to all processes that need them.

The variety of different tasks that can be solved with OpenQa significantly reduces the amount of different tools that would otherwise have to be used.

The solution can be used to create useful additions to SAP and surrounding systems, while keeping the workload low due to the high level of automation.

Furthermore, when working with OpenQa, there are always new approaches to use, be it performing standardized rework after system copies or even simple activities such as password changes.

Through the integrated control of all performed steps in the respective tests, the success of the activities can be controlled and also documented.