GSoC 2023: Slurm fixes and development

[adi611]

With reference to discussion #664 and weekly meeting discussion on 15/06/23, I needed to find a suitable docker container with newer version of slurm and update the GitHub Actions accordingly. One of the more relevant containers that I could find on DockerHub was this which runs on slurm version 21.08.5 which released on December of 2021. Here are the pytest coverage for the said container:

= 103 failed, 745 passed, 88 skipped, 5 xfailed, 383 warnings, 28 rerun in 907.68s (0:15:07) =

Please let me know if I should open an issue to change the GitHub Actions workflow.

[djarecka]

let's try to understand why tests are failing

[djarecka]

looks like the issue is due to Slurm accounting storage is disabled. I have to read more

[adi611]

@djarecka - I tried running the GitHub Action pytest code for the current Slurm container (mgxd/slurm:19.05.1) on my laptop and here is the result:

----------- coverage: platform linux, python 3.7.3-final-0 -----------
Coverage XML written to file /pydra/cov.xml

====== 846 passed, 88 skipped, 5 xfailed, 4 warnings in 744.44s (0:12:24) ======

pydra version: 0.1.dev1890+g426564e.d20230627

which is different from that on the Actions:

---------- coverage: platform linux, python 3.7.16-final-0 -----------
Coverage XML written to file /pydra/cov.xml

====== 910 passed, 88 skipped, 12 xfailed, 1 warning in 776.98s (0:12:56) ======

pydra version: 0.1.dev1+ge9e32f1

meanwhile the result of the proposed container with newer version of slurm on my laptop is:

= 101 failed, 745 passed, 88 skipped, 5 xfailed, 379 warnings, 28 rerun in 919.35s (0:15:19) =

[adi611]

@djarecka @ghisvail - Here are a few examples of docker images for which I am unable to run the testslurm.yml workflow. All the examples have errors at the Display previous jobs with sacct stage of the workflow.

• hongminc1/slurm_container_python_11:latest (GA workflow file, GA workflow run)
• nathanhess/slurm:full (GA workflow file, GA workflow run)
• cytomineuliege/slurm:v1.1.0 (GA workflow file, GA workflow run)

[djarecka]

could you please provide specific command you run? did you have any setup steps?

[adi611]

I followed the exact steps and commands as mentioned in the current testslurm.yml file. The links GA workflow file go into more depth on the steps I followed (which are the same for all, only change is the DOCKER_IMAGE value) and the links to GA workflow run shows the errors which I encountered. Please let me know if I understood the question correctly.

[adi611]

There appears to be an issue with the updated Slurm GA workflow, as it continues to run endlessly. I will check and resolve the problem.