Skip to content

Commit

Permalink
doc: added importance and custom markers section
Browse files Browse the repository at this point in the history
  • Loading branch information
Dan Lavu committed Sep 15, 2023
1 parent 8c63ec1 commit 1fe53f9
Showing 1 changed file with 75 additions and 18 deletions.
93 changes: 75 additions & 18 deletions docs/marks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -113,42 +113,99 @@ You can then run tests that are relevant only to the selected ticket:
cd src/tests/system
pytest --mh-config=mhc.yaml --mh-lazy-ssh -v --ticket=gh#3433
@pytest.mark.tier
=================
@pytest.mark.importance
=======================

The `tier mark <https://github.com/next-actions/pytest-tier>`__ can
associate a test with a specific tier.
The `importance mark <https://github.com/next-actions/pytest-importance>`__ can
associate a test with a level of importance. This is used by quality engineers to
prioritize and group the test determined by the level of impact to the customer.

The ``@pytest.mark.tier`` takes single number as an argument.
The ``@pytest.mark.importance`` takes a string as an argument. The values used
are "critical", "high", "medium" and "low".

.. code-block:: python
:caption: Examples
@pytest.mark.tier(0)
def test_tier0()
@pytest.mark.importance("critical")
def test_importance_critical()
pass
@pytest.mark.tier(1)
def test_tier1()
@pytest.mark.importance("high")
def test_importance_high()
pass
You can then run tests that are relevant only to the selected ticket:
You can then run the tests by importance:

.. code-block:: text
cd src/tests/system
pytest --mh-config=mhc.yaml --mh-lazy-ssh -v --tier=1
pytest --mh-config=mhc.yaml --mh-lazy-ssh -v --importance="critical"
Tier definition
===============
Importance definition
=====================

.. list-table:: Tier definition
:align: center
.. list-table::
:widths: 10 90
:header-rows: 1
:stub-columns: 1

* - Tier
* - Importance
- Description
* - @TODO
-
* - critical
- Core subset of tests that covers all operational features. This is used for gating where it maybe ran several
times a day. To manage resources, the execution all these tests should be kept under an hour.
* - high
- The most comprehensive set of tests, that covers all operational features.
* - medium
- Tests that do not impact the operational function, like the CLI commands included in sss-tools package.
* - low
- Tests that may have a long execution time or are heavy on resources so will be executed during our comprehensive
test cycle.

@pytest.mark.custom
===================

The use of `custom markers <https://docs.pytest.org/en/latest/how-to/mark.html>`__
help group tests into categories; identity, authentication, authorization are
some examples.

.. code-block:: python
:caption: Examples
@pytest.mark.authentication
def test_authenticate_user()
pass
You can run tests by custom markers:

.. code-block:: text
cd src/tests/system
pytest --mh-config-mhc=mhc.yaml --mh-lazy-ssh -v -m authentication
Custom Marker definitions
=========================

.. list-table::
:widths: 20 80
:header-rows: 1
:stub-columns: 1

* - Marker
- Description
* - authentication
- Tests checking user and password policies and anything to do with the login prompt.
* - authorization
- Tests checking user access after login like sudo and credential delegation.
* - cache
- Test checking the local cache like timeout, negative cache and refresh.
* - contains_workaround
- Test requires workaround for an existing bug. (gh=...,bz=...)
* - identity
- Tests checking user identity lookups, group memberships, domain priority and id mapping.
* - schema
- Tests checking ldap schemas, rfc2307, rfc2307bis, AD and directory attributes.
* - slow
- Tests that are slow. (deselect with '-m "not slow"')
* - tools
- Tests for all SSSD CLI commands; sssctl, sss_cache, sss_override, etc.

0 comments on commit 1fe53f9

Please sign in to comment.