test(scorecard): add container logs to scorecard results #758

tthvo · 2024-03-09T00:19:36Z

Welcome to Cryostat! 👋

Before contributing, make sure you have:

Read the contributing guidelines
Linked a relevant issue which this PR resolves
Linked any other relevant issues, PR's, or documentation, if any
Resolved all conflicts, if any
Rebased your branch PR on top of the latest upstream main branch
Attached at least one of the following labels to the PR: [chore, ci, docs, feat, fix, test]
Signed all commits: git commit -S -m "YOUR_COMMIT_MESSAGE"

Fixes: #750

Description of the change:

Added goroutines to asynchronously fetch container logs and appended them to test results.

The idea works very nicely: https://gist.github.com/tthvo/fc6b06c948e410c3048610498b0ae09c. Maybe additional formatting is needed?

Motivation for the change:

See #750

How to manually test:

export IMAGE_NAMESPACE=quay.io/thvo
make scorecard-build bundle bundle-build && \
  podman push $IMAGE_NAMESPACE/cryostat-operator-bundle:2.5.0-dev && \
  make test-scorecard

andrewazores · 2024-03-11T14:31:03Z

Looks pretty good to me. Maybe there's some more room to refactor and reuse some duplicated code, or perhaps defining some constants for container names rather than having individual functions for each?

tthvo · 2024-03-11T17:30:06Z

I refactored to remove duplicate codes now. Also, reserved the full container name (with CR name prefix).

ebaron

This is fantastic work, thanks @tthvo! It would be cool in a follow up PR if we could get this working for other tests too. Like for the reports test, it would good to see the logs from the reports container too.

ebaron · 2024-03-19T20:51:29Z

@Mergifyio backport cryostat3

mergify · 2024-03-19T20:51:43Z

backport cryostat3

✅ Backports have been created

#770 test(scorecard): add container logs to scorecard results (backport #758) has been created for branch cryostat3 but encountered conflicts

* test(scorecard): add container logs to scorecard results * build(bundle): regenerate bundle with new scorecard tags * chore(scorecard): refactor to remove duplicate codes (cherry picked from commit d01e0d2) # Conflicts: # bundle/manifests/cryostat-operator.clusterserviceversion.yaml # bundle/tests/scorecard/config.yaml

…) (#770) * test(scorecard): add container logs to scorecard results (#758) * test(scorecard): add container logs to scorecard results * build(bundle): regenerate bundle with new scorecard tags * chore(scorecard): refactor to remove duplicate codes (cherry picked from commit d01e0d2) # Conflicts: # bundle/manifests/cryostat-operator.clusterserviceversion.yaml # bundle/tests/scorecard/config.yaml * Fix conflicts --------- Co-authored-by: Thuan Vo <[email protected]> Co-authored-by: Elliott Baron <[email protected]>

* feat(discovery): options to configure discovery port names and numbers (backport #715) (#725) * feat(discovery): options to configure discovery port names and numbers (#715) Signed-off-by: Thuan Vo <[email protected]> (cherry picked from commit a552021) * resolve conflict --------- Co-authored-by: Thuan Vo <[email protected]> Co-authored-by: Andrew Azores <[email protected]> * Deploy cryostat 3.0 * Remove extraneous file * test adjustments * feat(discovery): options to configure discovery port names and numbers (#715) Signed-off-by: Thuan Vo <[email protected]> * Fix typo in environment variable breaking reconciler test, fix missing SecurityContext * Fix conflict with cluster cryostat removal * ci(gh): add comment when /build_test is finished (#745) * add scorecard test/suite selection (#746) * test(scorecard): scorecard tests for recording management (#698) * test(scorecard): scorecard tests for recording management Signed-off-by: Thuan Vo <[email protected]> * fixup(scorecard): fix cr cleanup func * test(scorecard): registry recording test to suite * chore(scorecard): reorganize client def * chore(scorecard): clean up common setup func * chore(bundle): regenerate bundle with scorecard tag * chore(bundle): correct image tag in bundle * fix(bundle): add missing scorecard test config patch * feat(scorecard): scaffold cryostat API client * chore(scorecard): clean up API client * test(scorecard): implement recording scorecard test * fixup(scorecard): correctly add scorecard test via hack templates * fix(client): ignore unverified tls certs and base64 oauth token * chore(bundle): split cryostat tests to separate stage * fix(scorecard): extend default transport instead of overwriting * chore(scorecard): refactor client to support multi-part * fixup(client): fix request verb * fix(client): fix recording create form format * fix(scorecard): create stored credentials for target JVM * fix(scorecard): fix 502 status error * chore(scorecard): simplify client def * chore(scorecard): fetch recordings to ensure action is correctly performed * test(scorecard): test generating report for a recording * chore(scorecard): clean up * test(scorecard): list archives in tests * ci(scorecard): reconfigure ingress for kind * ci(k8s): correct cluster name * test(scorecard): use role instead of clusterrole for oauth rules * test(scorecard): parse health response for additional checks * chore(scorecard): add missing newline in logs * chore(scorecard): check status code before parsing body in health check * test(scorecard): add custom target discovery to recording scorecard test * add EOF wait and resp headers * add resp headers * chore(client): configure all clients to send safe requests * fix(clients): add missing content-type header * fix(scorecard): add missing test name in help message * chore(client): create new http requests when retrying * chore(bundle): update scorecard image tags --------- Signed-off-by: Thuan Vo <[email protected]> Co-authored-by: Ming Yu Wang <[email protected]> Co-authored-by: Ming Wang <[email protected]> * test(scorecard): scorecard test for Cryostat CR configuration changes (#739) * CR config scorecard * reformat * reviews * add kubectl license * test(scorecard): scorecard test for report generator (#753) * deploy reports sidecar * report scorecard test * update * rebase fix * query health * fix(build-ci): fix scorecard image tag returned as null (#760) Signed-off-by: Thuan Vo <[email protected]> Co-authored-by: Elliott Baron <[email protected]> * test(scorecard): add container logs to scorecard results (#758) * test(scorecard): add container logs to scorecard results * build(bundle): regenerate bundle with new scorecard tags * chore(scorecard): refactor to remove duplicate codes * add permission to publish comment when ci fails (#769) Co-authored-by: Elliott Baron <[email protected]> * Update NewCoreContainer and associated tests * build(go): update Golang to 1.21 (#777) * test(scorecard): logWorkloadEvent for cryostat-recording errors (#759) * logWorkLoadEvent for cryostat-recording errors * reviews * tr.LogChannel --------- Co-authored-by: Elliott Baron <[email protected]> * test(scorecard): fix rebasing skipped commit (#780) * Merge pull request #8 from ebaron/scorecard-methods test(scorecard): use methods for more easily passing data * update bundle image * Review fixes * generate storage key, create expected Secret * fixup! generate storage key, create expected Secret * database secret handling corrections * combine database connection password and encryption key into one secret * correct storage secret key/access key * update datasource port number to not conflict with storage * precreate eventtemplates bucket * remove storage volume parameter overrides * use HTTP for Cryostat probe even when TLS is enabled - TLS will be done via auth proxy later * correct environment variable names for proxy awareness * Fix remaining merge conflict * Fix makefile * config cleanup and test fixup --------- Signed-off-by: Thuan Vo <[email protected]> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: Thuan Vo <[email protected]> Co-authored-by: Andrew Azores <[email protected]> Co-authored-by: Ming Yu Wang <[email protected]> Co-authored-by: Ming Wang <[email protected]> Co-authored-by: Elliott Baron <[email protected]>

tthvo added test safe-to-test labels Mar 9, 2024

tthvo force-pushed the container-log branch 3 times, most recently from 83e457a to 8a2c819 Compare March 11, 2024 08:32

tthvo marked this pull request as ready for review March 11, 2024 08:42

tthvo requested review from ebaron and a team March 11, 2024 08:42

tthvo added 3 commits March 17, 2024 23:43

test(scorecard): add container logs to scorecard results

6e9020a

build(bundle): regenerate bundle with new scorecard tags

d4b7184

chore(scorecard): refactor to remove duplicate codes

64fb057

tthvo force-pushed the container-log branch from fb28a3f to 64fb057 Compare March 18, 2024 06:56

ebaron approved these changes Mar 19, 2024

View reviewed changes

ebaron merged commit d01e0d2 into cryostatio:main Mar 19, 2024
5 checks passed

tthvo deleted the container-log branch March 19, 2024 22:48

tthvo mentioned this pull request Mar 19, 2024

test(scorecard): add container logs for report sidecard test #772

Merged

7 tasks

mergify bot mentioned this pull request Apr 16, 2024

test(scorecard): add container logs for report sidecard test (backport #772) #796

Merged

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test(scorecard): add container logs to scorecard results #758

test(scorecard): add container logs to scorecard results #758

tthvo commented Mar 9, 2024 •

edited

Loading

andrewazores commented Mar 11, 2024

tthvo commented Mar 11, 2024

ebaron left a comment

ebaron commented Mar 19, 2024

mergify bot commented Mar 19, 2024 •

edited

Loading

test(scorecard): add container logs to scorecard results #758

test(scorecard): add container logs to scorecard results #758

Conversation

tthvo commented Mar 9, 2024 • edited Loading

Welcome to Cryostat! 👋

Before contributing, make sure you have:

Description of the change:

Motivation for the change:

How to manually test:

andrewazores commented Mar 11, 2024

tthvo commented Mar 11, 2024

ebaron left a comment

Choose a reason for hiding this comment

ebaron commented Mar 19, 2024

mergify bot commented Mar 19, 2024 • edited Loading

✅ Backports have been created

tthvo commented Mar 9, 2024 •

edited

Loading

mergify bot commented Mar 19, 2024 •

edited

Loading