SSG test suite fails when rules marked with 'platform: machine' are run in container environment

[mpurg]

Description of problem:

The ssg test suite does not take into account the platform: machine definitions in the rule, when deciding which tests to run.
The evaluation of applicability (machine vs container) is performed only after the test environment is already prepared,
which can result in the test failing if the environment is not setup correctly in a non-supported environment.

An example rule which tests fine in a VM but fails to test in a container is all_apparmor_profiles_in_enforce_complain_mode, when the machine profile is added to its rule defintion.

$ tests/test_rule_in_container.sh --no-make-applicable-in-containers --dontclean --logdir logs_bash --remediate-using bash --name ssg_test_suite --datastream build/ssg-ubuntu2204-ds.xml all_apparmor_profiles_in_enforce_complain_mode

ERROR - Rule 'all_apparmor_profiles_in_enforce_complain_mode' test setup script 'correct_all_apparmor_profiles_in_enforce.pass.sh' failed with exit code 1
ERROR - Environment failed to prepare, skipping test

By adding the following to the test scenario, the issue is resolved:

if [ -f /.dockerenv ] || [ -f /run/.containerenv ]; then
    exit 0
fi

$ tests/test_rule_in_container.sh --no-make-applicable-in-containers --dontclean --logdir logs_bash --remediate-using bash --name ssg_test_suite --datastream build/ssg-ubuntu2204-ds.xml all_apparmor_profiles_in_enforce_complain_mode

WARNING - Script correct_all_apparmor_profiles_in_enforce.pass.sh using profile (all) notapplicable
WARNING - Rule xccdf_org.ssgproject.content_rule_all_apparmor_profiles_in_enforce_complain_mode evaluation resulted in notapplicable

SCAP Security Guide Version:

master branch (a11711d)

Operating System Version:

Steps to Reproduce:

1. Add platform: machine to rule all_apparmor_profiles_in_enforce_complain_mode
2. ./build_product ubuntu2204
3. podman build --build-arg "CLIENT_PUBLIC_KEY=$(cat ~/.ssh/id_rsa.pub)" --build-arg "ADDITIONAL_PACKAGES=cmake ninja-build python3-yaml python3-jinja2 python3-deepdiff python3-requests jq python3-pip libxml2-utils xsltproc ansible-lint wget libdbus-1-dev libdbus-glib-1-dev libcurl4-openssl-dev libgcrypt20-dev libselinux1-dev libxslt1-dev libgconf2-dev libacl1-dev libblkid-dev libcap-dev libxml2-dev libldap2-dev libpcre3-dev python3 swig libxml-parser-perl libxml-xpath-perl libperl-dev libbz2-dev librpm-dev g++ libyaml-dev libxmlsec1-dev libxmlsec1-openssl" -t ssg_test_suite -f Dockefiles/test_suite-ubuntu2204
4. tests/test_rule_in_container.sh --no-make-applicable-in-containers --dontclean --logdir logs_bash --remediate-using bash --name ssg_test_suite --datastream build/ssg-ubuntu2204-ds.xml all_apparmor_profiles_in_enforce_complain_mode

Actual Results:

See error above

Expected Results:

The expected result is that the tests pass when inside a container because the rule should not be evaluated.

Additional Information/Debugging Steps:

Possible solutions that come to mind:

• the tests should be removed from the set of applicable tests in a container environment
• scenarios are appended with the conditional to return 0 if running in a container environment