Ten rules for SAP test management

Not all, but many obstacles in test management can be removed in advance with the following rules.

Rule 1

Write a test concept: With scarce resources and tight deadlines, why should the test management team go to the trouble of creating a test concept and having it accepted?

The trick is to keep the concept as short and concise as possible and to use graphics to represent complex issues such as a test environment with many interfaces.

The essential components of a test concept include, for example, the definition of the test objects and objectives, the acceptance criteria and the acceptance procedure, as well as the presentation of the test infrastructure (SAP systems, transport routes, transport deadlines). Have the concept approved by development and the business side.

Rule 2

Invest enough time in the quality of the test cases. The business departments often lack the time to create the test cases. Consequently, they are "recycled" from old projects, are too general, too granular, or do not meet the functions to be tested and accepted.

The result: the expert testers test too much or too little, but not the necessary features. Progress is too slow, the error rate is too high, and the quality improvements are too small. The business departments (and not IT) should definitely take the necessary time to create the test cases.

Rule 3

Prioritize your test cases. Often, business department A wants to test all new features down to the last detail and subject the already implemented software to a complete integration test including batch testing and end-2-end testing involving third-party and other systems.

Department B, on the other hand, chooses a pragmatic approach with only a few test cases, depending on the deadline and resource situation. It is important to find the right measure for the test scope.

Best practice is to prioritize the test cases. A good approach is risk-based testing. Here, the test cases are weighted in particular with regard to the extent of damage in the event of malfunctions.

Rule 4

Schedule multiple test cycles. The first test cycle usually gets off to a bumpy start. There are blockages, the testers do not know the application yet and many errors result.

The high number of corrections can lead to side effects that require retesting of test cases that have already been successfully completed. Before you know it, the test cycle is over.

Therefore, take a break after the first run and give development a chance to fix all critical errors. Then, with a consolidated software and data status, start another cycle in which you test at least all high- and medium-rated test cases from the first cycle.

Rule 5

Take care of the test environment in good time. A typical situation: The test cases are ready, the department employees want to start, but the test cannot be started because the SAP system is not available.

There are many reasons for this. The system was not reserved in time and is occupied by other projects. Or the development was not able to roll out the software to the test systems or not completely.

Often, important and necessary interfaces or suitable user IDs and authorizations are also missing. Therefore, plan your test environment early and monitor the complete and correct deployment.

You set the first pillar by defining test entry criteria in the test concept. If essential conditions are not met, do not start at all.

Define in the concept or in a test environment plan exactly which SAP systems with which dataset are required when, which interfaces must be connected via which third-party systems, which user IDs with which authorizations are to be provided, and when software corrections are to be transported.

Rule 6

Use an integrated test and defect management tool. The most commonly used tool for test and defect management is Excel.

It works quite well up to a number of 100 to 150 test cases with a low defect volume. As soon as the volume increases, it becomes increasingly problematic to cope with own Excel solutions.

In particular, there is often no overview of which defects affect which test cases. It is also difficult to generate the current overall status or the test progress, if necessary with forecasts.

Often the Excel tables (and especially the macros) can only be maintained by a few or in the worst case by an expert.The use of an integrated test and defect management tool is the better choice.

The SAP Solution Manager components Test Workbench and Help Desk are ideal for SAP users. They do not incur any license fees, the implementation effort is manageable at three to five person days, and users are familiar with the SAP interfaces.

Rule 7

Conduct a test kick-off. The effort you invest in a well-prepared kick-off will be recouped in daily test operations.

The agenda should include topics such as goals, procedures, deadlines, quality gates, test constraints, test and defect reporting, as well as regular meetings and conference calls.

Even if there will be many participants, invite all stakeholders such as subject matter testers, the test management team, development, batch control if applicable, and project management to the kick-off.

This facilitates subsequent communication and helps to avoid many unnecessary queries. It has proven useful to make the kick-off documents and status reports, as well as concepts, available in an easily accessible medium such as MS SharePoint.

Rule 8

Do not start testing too early. The usual scenario: The test and acceptance dates are fixed. The expert testers are scheduled. The decision-makers want the project to be a success.

And the development team is still struggling with severe and/or many deficits in the software. The development manager wants to postpone the test; the project management wants to keep the deadline at any cost.

Often, those in charge give in to the desire and the test is started with knowledge of the deficits and the associated risks. What happens?

The first acceptance tests may still be positive. But then many errors are found, test blockages occur. The testers are frustrated. And above all, this has a negative impact on the project committees, the departments involved, and possibly even the management board.

Therefore the recommendation: Even if you are under a lot of pressure to meet the planned start date for the specialist tests, postpone the tests if the software is not yet ready for testing. You will save yourself and the testers a lot of trouble and, above all, avoid a high image damage for your project.

What can be done preventively? Defining and checking quality gates has proven to be effective. Define in the test concept which preconditions must be fulfilled for the start of the tests.

This includes, for example, proof of documented developer tests, the creation of user documentation, the provision of the test infrastructure, and the listing of unresolved errors with a correction date. If the quality gate is not met or not met to a sufficient degree, there is no go for the test.

Rule 9

Interrupt testing in case of test blockages. The absence of important features, a high number of bugs or limitations in the test infrastructure such as non-performing systems can lead to test blockades.

The testers cannot start due to the current overall situation. In parallel, there is high pressure to complete routine work at the desk. Of course, it is no fun to send testers home in tightly timed windows.

But the negative consequences, such as dissatisfied testers and panic in the specialist departments or possibly with the management board due to the poor software quality, are to be valued more highly in case of doubt. You can use the interruption to consolidate the software.

Rule 10

Communicate with the testers. A key success factor for your project is regular communication with the testers. It is important to be transparent about the current situation, bug status, constraints, etc. Hold regular status and bug meetings with the testers and development.

For example, daily stand-up meetings have proven effective. Try to speak directly with everyone on a daily basis to learn more about specific test and supposed error situations. It has also proven useful to bring in developers to test or retest bugs in complex situations.

Conclusion

Every test management project in the SAP environment is different and the perfect project process will probably never exist. But many things become easier with the above-mentioned basic rules.

Ultimately, the aim is to bring the quality of the software to a level necessary for productive operation in a process accepted by all those involved. The fact that not all errors are eliminated before going live is the rule rather than the exception.