Test Strategy for modular Kyma

[arturskorupa]

Description

According to the concept of modularized Kyma it is not needed to run integration tests on top because all needed E2E test are supposed to be covered by respective test scenarios run on a module level. On top of kyma modules only a basic smoke tests are planned (as a part of conformance tests).

A test strategy is needed to describe requirements in terms of tests for particular module to assure good overall quality of the product.

Acceptance Criteria

• Test Strategy document is created and it covers aspects of approach for testing in terms of an individual module
• Fast integration tests are removed
• Quality measures are defined for every module
• Module teams prepared their test scenarios to include versions of dependant components from Fast and Stable channels

Links

• Concept document
• Transition plan
• Lifecycle manager
• Module manager

[kasiakepka]

Test Strategy document is created and it covers aspects of approach for testing in terms of an individual module

What would be the purpose of having such document? How the strategy will be implemented in real life, not just on paper?
Creating long documentation that is hard to implement brings little value, long documents tend to outdate quickly.

[jakobmoellerdev]

I agree, this test strategy is also in Phase 2a when we likely already had to implement pipelines and procedures for the frontrunner module. I think we should be more pragmatic here and forego this strategy in favor of actually supported and fitting tests for the given development phase and let the teams own that. If there is a need for central testing and strategies again, for me this is a sign that modularization did not succeed.

[tobiscr]

Checklist for the teams for the general testing strategy of Kyma modules:

https://github.tools.sap/pandas/testing-strategy/tree/master/modularisation/governance/operational_test_management

[ameteiko]

He is the Testing Strategy concept document.