[TEST] Test BSS within the HPE CSM pipeline to identify regressions

[alexlovelltroy]

This is a placeholder for @mtupitsyn to add any errors he finds with running SMD at HPE.

[mtupitsyn]

FYI - vulnerability of level "high" is found in ghcr.io/openchami/bss:sha256-49cfc147beee3480bd99301e34016a43fbde06ed4b6d8041cd607315d1ac0331 by Snyk scan:

Testing artifactory.algol60.net/csm-docker/stable/ghcr.io/openchami/bss@sha256:d96e569bb4d54e05f9ff70d4b6db903eccb60b4ea5b1690db11205d50b8b617a...

✗ High severity vulnerability found in google.golang.org/grpc
  Description: Denial of Service (DoS)
  Info: https://security.snyk.io/vuln/SNYK-GOLANG-GOOGLEGOLANGORGGRPC-5953328
  Introduced through: google.golang.org/[email protected]
  From: google.golang.org/[email protected]
  Fixed in: 1.56.3, 1.57.1, 1.58.3

[alexlovelltroy]

Looks like that's related to hms-hmetcd. I've created a PR for that package at Cray-HPE/hms-hmetcd#13 which unvendors and updates the library. All tests pass locally with the changes.

[mtupitsyn]

Similar to SMD reconciliation test (OpenCHAMI/smd#42), I've deployed CSM based version from scratch and modified deployment to run OpenCHAMI version of BSS container. Container started without errors. However, unlike SMD test, there were some test failures. I'm attaching archived logs of pods and test outputs: vshasta-bss-logs.tar.gz

Test definitions:

• smoke: https://github.com/Cray-HPE/hms-bss/blob/master/test/ct/smoke/smoke.json
• non-disruptive API tests: https://github.com/Cray-HPE/hms-bss/tree/master/test/ct/api/1-non-disruptive

[alexlovelltroy]

We're going to need more than a bunch of logs and some json. We're not familiar with your testing framework. What tests failed and what were the messages associated with each failure?

[mtupitsyn]

It's in a file named logs-bss/openchami-bss-test-stdout.txt in attached archive. It's too big to put it entirely in issue comment. Short info is:

=========================== short test summary info ============================
FAILED api/1-non-disruptive/test_service.negative.tavern.yaml::bssAPIServiceAPIsNegative - Verify POST, PUT, PATCH, DELETE against service endpoints[service/status]
FAILED api/1-non-disruptive/test_service.negative.tavern.yaml::bssAPIServiceAPIsNegative - Verify POST, PUT, PATCH, DELETE against service endpoints[service/etcd]
FAILED api/1-non-disruptive/test_service.negative.tavern.yaml::bssAPIServiceAPIsNegative - Verify POST, PUT, PATCH, DELETE against service endpoints[service/hsm]
FAILED api/1-non-disruptive/test_service.negative.tavern.yaml::bssAPIServiceAPIsNegative - Verify POST, PUT, PATCH, DELETE against service endpoints[service/status/all]
FAILED api/1-non-disruptive/test_service.tavern.yaml::bssAPIServiceAPIs

First 4 API tests fail on HTTP code 405 returned, where HTTP code 200 is expected. The last test got 404 at http://cray-bss/boot/v1/service/etcd while test expects 200.

[alexlovelltroy]

Let's get your BSS SMEs involved here.

In the case of the 405 errors, I think the tests are invalid. Why would we want to support POST,PUT, PATCH on status urls? Given what I understand of those endpoints, they exist solely for reading health information.

The 404 on service/etcd endpoint may be a regression. Since OpenCHAMI doesn't use etcd, I think this is a bug you can assign to an HPE SME to review and address. I'm guessing that some of our work to switch backends may have altered the way that endpoint works and/or removed it.

Good to see that so many of your integration tests pass correctly. That should indicate that our work on BSS hasn't disrupted BSS functinality within CSM and that you can switch upstream without issue.

[mtupitsyn]

In the case of the 405 errors, I think the tests are invalid. Why would we want to support POST,PUT, PATCH on status urls? Given what I understand of those endpoints, they exist solely for reading health information.

I looked into test descriptions - my bad, I misinterpreted test results. These are so called "negative tests", which expect 405 HTTP code on invalid request. So outcome should be treated as "returned HTTP code 200 where 405 is expected", not the other way around.

I'll check with Curt to see who is our BSS expert these days.