Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using dws-test-driver for DWS state progression integration tests #28

Merged
merged 5 commits into from
Feb 2, 2023

Conversation

nathandotleeathpe
Copy link
Contributor

Signed-off-by: Nathan Lee [email protected]

@github-actions
Copy link

github-actions bot commented Dec 14, 2022

Code Coverage

Package Line Rate Health
burst_buffer 89%
Summary 89% (283 / 318)

@nathandotleeathpe nathandotleeathpe force-pushed the main branch 2 times, most recently from 33c26a0 to 593a298 Compare December 14, 2022 21:52
.github/workflows/integration-test.yml Outdated Show resolved Hide resolved
testsuite/integration/src/tests/slurmctld.py Outdated Show resolved Hide resolved
testsuite/integration/src/tests/slurmctld.py Outdated Show resolved Hide resolved
src/burst_buffer/burst_buffer.lua Show resolved Hide resolved
@nathandotleeathpe nathandotleeathpe force-pushed the main branch 4 times, most recently from 74ea65b to 793d98e Compare December 15, 2022 15:58
@roehrich-hpe
Copy link
Contributor

"nathandotleeathpe force-pushed the main branch from 793d98e to c5a29c8 Compare 4 hours ago"

When you do the force-push, I can't use github gui to look at what you've changed since the last time I was here.

@roehrich-hpe
Copy link
Contributor

Update .gitignore for these pycache dirs:

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	testsuite/integration/src/tests/__pycache__/
	testsuite/integration/src/tests/dws_bb_plugin/__pycache__/
	testsuite/integration/src/tests/environment/__pycache__/

@roehrich-hpe
Copy link
Contributor

I'm finished reviewing and testing this, up to commit c5a29c8.

@roehrich-hpe
Copy link
Contributor

Update .gitignore for these pycache dirs:

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	testsuite/integration/src/tests/__pycache__/
	testsuite/integration/src/tests/dws_bb_plugin/__pycache__/
	testsuite/integration/src/tests/environment/__pycache__/

And "reports/" as well:

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	testsuite/integration/reports/
	testsuite/integration/src/tests/__pycache__/
	testsuite/integration/src/tests/dws_bb_plugin/__pycache__/
	testsuite/integration/src/tests/environment/__pycache__/

Signed-off-by: Nathan Lee <[email protected]>
@roehrich-hpe
Copy link
Contributor

The docker-compose from the 'setup-kind' target...

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
[+] Running 2/3
 ⠿ Network slurm_default  Created                                          0.0s
 ⠿ Container slurmctld    Started                                          0.5s
 ⠿ Container c1           Waiting                                        195.0s

The containers ....

$ docker container ls -a | grep slurm
a9d3d975faed   slurm-bb:test          "/usr/local/bin/dock…"   About a minute ago   Exited (1) About a minute ago                               c1
58213079cb9f   slurm-bb:test          "/usr/local/bin/dock…"   About a minute ago   Exited (1) About a minute ago                               slurmctld

$ docker logs slurmctld
---> Starting the MUNGE Authentication service (munged) ...
munged: Error: Failed to check keyfile "/etc/munge/munge.key": Permission denied

$ docker logs c1       
---> Starting the MUNGE Authentication service (munged) ...
munged: Error: Failed to check keyfile "/etc/munge/munge.key": Permission denied

@roehrich-hpe
Copy link
Contributor

The docker-compose from the 'setup-kind' target...

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
[+] Running 2/3
 ⠿ Network slurm_default  Created                                          0.0s
 ⠿ Container slurmctld    Started                                          0.5s
 ⠿ Container c1           Waiting                                        195.0s

The containers ....

$ docker container ls -a | grep slurm
a9d3d975faed   slurm-bb:test          "/usr/local/bin/dock…"   About a minute ago   Exited (1) About a minute ago                               c1
58213079cb9f   slurm-bb:test          "/usr/local/bin/dock…"   About a minute ago   Exited (1) About a minute ago                               slurmctld

$ docker logs slurmctld
---> Starting the MUNGE Authentication service (munged) ...
munged: Error: Failed to check keyfile "/etc/munge/munge.key": Permission denied

$ docker logs c1       
---> Starting the MUNGE Authentication service (munged) ...
munged: Error: Failed to check keyfile "/etc/munge/munge.key": Permission denied

Add a docker volume rm slurm_etc_munge slurm_var_log_slurm to the 'clean' target in testsuite/integration/Makefile. That fixed my probs.

Copy link
Contributor

@roehrich-hpe roehrich-hpe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add the 'docker volume rm' to the 'clean' target. I have no other issues.

@roehrich-hpe
Copy link
Contributor

Add the 'docker volume rm' to the 'clean' target. I have no other issues.

Actually, rather than adding the 'docker volume rm', change the 'docker compose down' and add a '--volumes' arg to it.

@nathandotleeathpe nathandotleeathpe merged commit 8562dde into DataWorkflowServices:main Feb 2, 2023
nathandotleeathpe added a commit to nathandotleeathpe/dws-slurm-bb-plugin that referenced this pull request Feb 3, 2023
…taWorkflowServices#28)

* Using dws-test-driver for DWS state progression integration tests

Signed-off-by: Nathan Lee <[email protected]>

* Fixed integration test errors

Signed-off-by: Nathan Lee <[email protected]>

* code review changes

Signed-off-by: Nathan Lee <[email protected]>

* Updated dws-test-driver to main branch HEAD

Signed-off-by: Nathan Lee <[email protected]>

* Code review

Signed-off-by: Nathan Lee <[email protected]>

---------

Signed-off-by: Nathan Lee <[email protected]>
nathandotleeathpe added a commit that referenced this pull request Feb 10, 2023
…lity to plugin (#30)

* Using dws-test-driver for DWS state progression integration tests

Signed-off-by: Nathan Lee <[email protected]>

* Fixed integration test errors

Signed-off-by: Nathan Lee <[email protected]>

* code review changes

Signed-off-by: Nathan Lee <[email protected]>

* Added integration tests for error scenarios

Signed-off-by: Nathan Lee <[email protected]>

* Updated dws-test-driver to main branch HEAD

Signed-off-by: Nathan Lee <[email protected]>

* Code review

Signed-off-by: Nathan Lee <[email protected]>

* Remove option to run in real slurm or k8s

Signed-off-by: Nathan Lee <[email protected]>

* Refactored unit test before adding error scenarios

Signed-off-by: Nathan Lee <[email protected]>

* Added driver error observability

Signed-off-by: Nathan Lee <[email protected]>

* Using dws-test-driver for DWS state progression integration tests (#28)

* Using dws-test-driver for DWS state progression integration tests

Signed-off-by: Nathan Lee <[email protected]>

* Fixed integration test errors

Signed-off-by: Nathan Lee <[email protected]>

* code review changes

Signed-off-by: Nathan Lee <[email protected]>

* Updated dws-test-driver to main branch HEAD

Signed-off-by: Nathan Lee <[email protected]>

* Code review

Signed-off-by: Nathan Lee <[email protected]>

---------

Signed-off-by: Nathan Lee <[email protected]>

* Pre-code review fixes

Signed-off-by: Nathan Lee <[email protected]>

* PR change

Signed-off-by: Nathan Lee <[email protected]>

* Update submodules

Signed-off-by: Nathan Lee <[email protected]>

* Updating dws submodules

Signed-off-by: Nathan Lee <[email protected]>

* Removed unused test function

Signed-off-by: Nathan Lee <[email protected]>

* Added sleep before cancelling job

Signed-off-by: Nathan Lee <[email protected]>

* Added documentation about test error transformation

Signed-off-by: Nathan Lee <[email protected]>

---------

Signed-off-by: Nathan Lee <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants