add tier-0 test case #486

pengshanyu · 2024-07-09T14:36:15Z

We have several test cases for FFI, but we are missing some basic/simple sanity tests for qm, if it is installed properly and/or if environment started successfully.
This is a draft version. Since the naming of the test files and test methods are simple, I hope to use this version as a starting point for improving the sanity test.
We would like to name this test plan tier-0, but need to rename the current tier-0 first, so I temporarily named it tier-2

resolve #342

Yarboa

Please add the tests to .packit.yaml
to be visible on github tests

plans/e2e/tier-2.fmf

tests/qm-sanity-test/test.sh

pengshanyu · 2024-07-11T08:08:53Z

Please add the tests to .packit.yaml to be visible on github tests

Should I do it now in this PR or should I wait until this PR is merged before doing it?

Yarboa · 2024-07-11T08:50:00Z

Please add the tests to .packit.yaml to be visible on github tests

Should I do it now in this PR or should I wait until this PR is merged before doing it?

MHO, if this new packit test introduced, why not let it run, this way we confirm that test merged are working

Now, i see you calked the tests tier-2
I think that current tier-0 tests should be renamed to a different name, like
multi-bluchi-agents, that is the real purpose of it, maybe we will remove it from tests later on.

tests/qm-sanity-test/test.sh

pengshanyu · 2024-07-16T06:02:43Z

MHO, if this new packit test introduced, why not let it run, this way we confirm that test merged are working

Now, i see you calked the tests tier-2 I think that current tier-0 tests should be renamed to a different name, like multi-bluchi-agents, that is the real purpose of it, maybe we will remove it from tests later on.

Hi @Yarboa, I renamed current tier-0 tests to multi-bluechi-agents, but there are also tier-0 tests in qm-connectivity and qm-unit-files, how should they be renamed or do you want them to be tier-0 along with the sanity tests?

Yarboa · 2024-07-16T09:55:45Z

MHO, if this new packit test introduced, why not let it run, this way we confirm that test merged are working
Now, i see you calked the tests tier-2 I think that current tier-0 tests should be renamed to a different name, like multi-bluchi-agents, that is the real purpose of it, maybe we will remove it from tests later on.

Hi @Yarboa, I renamed current tier-0 tests to multi-bluechi-agents, but there are also tier-0 tests in qm-connectivity and qm-unit-files, how should they be renamed or do you want them to be tier-0 along with the sanity tests?

qm-connectivity exist in new tier-0 but it could run in the test, since it checks for multi connectivity

related to qm-unit-files, i think it is already running in ffi

Please update the test of tier-0 similar to ffi tests
Please use this one
https://github.com/containers/qm/blob/main/.packit.yaml#L75

Please as you mentioned rename tier-0 tests as mentioned
and please enable old tier-0 for manual trigger only as per this
https://packit.dev/posts/manual-triggering#manual-only-triggering-of-jobs

pengshanyu · 2024-07-16T10:35:09Z

@Yarboa thank you for the reply.
qm-connectivity run with run-test-e2e
the new tier-0 test run with set-ffi-env-e2e
they are different test plan, I'm not sure how to run them both as tier-0 at one tmt run

.packit.yaml

tests/qm-sanity-test/main.fmf

tests/qm-sanity-test/test.sh

Yarboa

Very nice work @pengshanyu
Please see the comments above,
I would not choose to number the tests, but it is up to you

tests/qm-sanity-test/README.md

.packit.yaml

plans/e2e/multi-bluechi-agents.fmf

plans/e2e/tier-0.fmf

Yarboa · 2024-07-19T10:27:15Z

/packit test --identifier qm-tier-0

Yarboa · 2024-07-21T05:05:21Z

LGTM,
@pengshanyu well done
Please squash all comments, to have clean commits history before merge

pengshanyu · 2024-07-22T11:06:46Z

@Yarboa, thank you so much for the review. I added the Polarion-id.

Signed-off-by: pengshanyu <[email protected]> delete blank Lines Signed-off-by: pengshanyu <[email protected]> trim trailing whitespace Signed-off-by: pengshanyu <[email protected]> update function name and implementation Signed-off-by: pengshanyu <[email protected]> added:check /var on ostree Signed-off-by: pengshanyu <[email protected]> rename old tier-0 to multi-bluechi-agents;rename sanity test to tier-0 Signed-off-by: pengshanyu <[email protected]> trailing-whitespace Signed-off-by: pengshanyu <[email protected]> remove old tier 0 Signed-off-by: pengshanyu <[email protected]> split sanity test to 6 tests Signed-off-by: pengshanyu <[email protected]> trailing-whitespace Signed-off-by: pengshanyu <[email protected]> not number the tests;add main.fmf;manual_trigger multi-bluechi-agents Signed-off-by: pengshanyu <[email protected]> correct typo Signed-off-by: pengshanyu <[email protected]> add polarion id Signed-off-by: pengshanyu <[email protected]>

Yarboa · 2024-07-23T07:26:51Z

/packit test --identifier qm-tier-0

pengshanyu · 2024-07-23T08:49:02Z

Hi @Yarboa , after this PR merged, we should also exclude this sanity test cases in the p-l-p test plan, right?

Yarboa · 2024-07-23T11:41:50Z

/packit test --identifier e2e-multi-bluechi-agents

Yarboa · 2024-07-23T12:26:30Z

/packit test --identifier e2e-multi-bluechi-agents

seems that tests is failing
need to check maybe we have an issue with default values
https://artifacts.dev.testing-farm.io/ea25bbf3-3563-4e9f-a7f4-5058fb393c4e/work-multi-bluechi-agentsg24c48ar/log.txt

plans/main.fmf

Signed-off-by: pengshanyu <[email protected]>

pengshanyu · 2024-07-24T05:55:58Z

/packit test --identifier e2e-multi-bluechi-agents

Signed-off-by: pengshanyu <[email protected]>

pengshanyu · 2024-07-24T08:25:26Z

/packit test --identifier e2e-multi-bluechi-agents

Yarboa

LGTM

pengshanyu requested review from rhatdan, lsm5, dougsland, Yarboa, odra, sandrobonazzola, dennisbrendel and priyverm as code owners July 9, 2024 14:36

Yarboa reviewed Jul 10, 2024

View reviewed changes

Yarboa reviewed Jul 11, 2024

View reviewed changes

tests/qm-sanity-test/test.sh Outdated Show resolved Hide resolved

tests/qm-sanity-test/test.sh Outdated Show resolved Hide resolved

tests/qm-sanity-test/test.sh Outdated Show resolved Hide resolved

pengshanyu force-pushed the add-tier0-test branch from d12e330 to 24ddc88 Compare July 16, 2024 01:58

Yarboa reviewed Jul 16, 2024

View reviewed changes

.packit.yaml Outdated Show resolved Hide resolved

tests/qm-sanity-test/main.fmf Outdated Show resolved Hide resolved

tests/qm-sanity-test/test.sh Outdated Show resolved Hide resolved

Yarboa reviewed Jul 17, 2024

View reviewed changes

tests/qm-sanity-test/README.md Outdated Show resolved Hide resolved

.packit.yaml Show resolved Hide resolved

plans/e2e/multi-bluechi-agents.fmf Outdated Show resolved Hide resolved

plans/e2e/tier-0.fmf Show resolved Hide resolved

pengshanyu force-pushed the add-tier0-test branch from a1db407 to 6ea8f46 Compare July 18, 2024 00:53

pengshanyu force-pushed the add-tier0-test branch from dab24a3 to cc45a79 Compare July 22, 2024 02:27

pengshanyu force-pushed the add-tier0-test branch from fe8e436 to 4dfad9c Compare July 23, 2024 02:08

Yarboa reviewed Jul 23, 2024

View reviewed changes

plans/main.fmf Outdated Show resolved Hide resolved

remove PACKIT_COPR_PROJECT

3792942

Signed-off-by: pengshanyu <[email protected]>

remove skip_build

f1ef968

Signed-off-by: pengshanyu <[email protected]>

Yarboa approved these changes Jul 24, 2024

View reviewed changes

Yarboa merged commit 0322896 into containers:main Jul 24, 2024
8 checks passed

pengshanyu deleted the add-tier0-test branch July 25, 2024 05:06

dougsland mentioned this pull request Sep 15, 2024

tag and release QM v0.6.7 upstream #572

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add tier-0 test case #486

add tier-0 test case #486

pengshanyu commented Jul 9, 2024

Yarboa left a comment

pengshanyu commented Jul 11, 2024 •

edited

Loading

Yarboa commented Jul 11, 2024

pengshanyu commented Jul 16, 2024

Yarboa commented Jul 16, 2024 •

edited

Loading

pengshanyu commented Jul 16, 2024 •

edited

Loading

Yarboa left a comment

Yarboa commented Jul 19, 2024

Yarboa commented Jul 21, 2024

pengshanyu commented Jul 22, 2024

Yarboa commented Jul 23, 2024 •

edited

Loading

pengshanyu commented Jul 23, 2024 •

edited

Loading

Yarboa commented Jul 23, 2024

Yarboa commented Jul 23, 2024

pengshanyu commented Jul 24, 2024

pengshanyu commented Jul 24, 2024 •

edited by Yarboa

Loading

Yarboa left a comment

add tier-0 test case #486

add tier-0 test case #486

Conversation

pengshanyu commented Jul 9, 2024

Yarboa left a comment

Choose a reason for hiding this comment

pengshanyu commented Jul 11, 2024 • edited Loading

Yarboa commented Jul 11, 2024

pengshanyu commented Jul 16, 2024

Yarboa commented Jul 16, 2024 • edited Loading

pengshanyu commented Jul 16, 2024 • edited Loading

Yarboa left a comment

Choose a reason for hiding this comment

Yarboa commented Jul 19, 2024

Yarboa commented Jul 21, 2024

pengshanyu commented Jul 22, 2024

Yarboa commented Jul 23, 2024 • edited Loading

pengshanyu commented Jul 23, 2024 • edited Loading

Yarboa commented Jul 23, 2024

Yarboa commented Jul 23, 2024

pengshanyu commented Jul 24, 2024

pengshanyu commented Jul 24, 2024 • edited by Yarboa Loading

Yarboa left a comment

Choose a reason for hiding this comment

pengshanyu commented Jul 11, 2024 •

edited

Loading

Yarboa commented Jul 16, 2024 •

edited

Loading

pengshanyu commented Jul 16, 2024 •

edited

Loading

Yarboa commented Jul 23, 2024 •

edited

Loading

pengshanyu commented Jul 23, 2024 •

edited

Loading

pengshanyu commented Jul 24, 2024 •

edited by Yarboa

Loading