Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use "run" with singularity/apptainer instead of "exec", when possible #2065

Merged
merged 1 commit into from
Nov 12, 2024

Conversation

sameeul
Copy link
Contributor

@sameeul sameeul commented Nov 10, 2024

Context for the PR:
Currently if the user uses a docker image in a workflow and depending on the choice of the container engine (docker vs singularity), the execution can be different, if the docker image has an entrypoint.

I understand that CWL community encourages docker images w/o entrypoints, however there are a good number of images produced by various scientific communities that comes with entrypoints. When this images are used in HPC environment, in most cases, due to security policies (not having access to root), singularity is chosen as the container engine. In these cases, docker images with entrypoint fails to execute.

I also know that creating CommandLineTools with a baseCommand can bypass the entrypoint issue when running with singularity but that also makes those CommandLineTools container engine dependent, in turn making the CWL workflows themselves container engine dependent.

With those observations, if there is no other strong reason for choosing singularity exec in the initial implementation, I would propose to change it to singularity run.

Here is also a link of relevant discussion: https://cwl.discourse.group/t/singularity-exec-vs-docker-run/956/4

@sameeul sameeul changed the title Swhich "singularity exec" to "singularity run", similar to docker. Switch "singularity exec" to "singularity run", similar to docker. Nov 10, 2024
Copy link

codecov bot commented Nov 10, 2024

Codecov Report

Attention: Patch coverage is 70.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 83.98%. Comparing base (048eb55) to head (d2f9f24).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
cwltool/singularity.py 70.00% 1 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2065      +/-   ##
==========================================
- Coverage   84.05%   83.98%   -0.07%     
==========================================
  Files          46       46              
  Lines        8302     8312      +10     
  Branches     1957     1959       +2     
==========================================
+ Hits         6978     6981       +3     
- Misses        848      854       +6     
- Partials      476      477       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mr-c
Copy link
Member

mr-c commented Nov 11, 2024

cwltool/singularity.py Outdated Show resolved Hide resolved
cwltool/singularity.py Outdated Show resolved Hide resolved
@sameeul
Copy link
Contributor Author

sameeul commented Nov 12, 2024

Thanks for the feedback. I ran the CI on my fork and looks like the singularity tests are passing now.

@mr-c mr-c changed the title Switch "singularity exec" to "singularity run", similar to docker. Use "run" with singularity/apptainer instead of "exec", when possible Nov 12, 2024
@mr-c mr-c merged commit 1557c8d into common-workflow-language:main Nov 12, 2024
44 of 46 checks passed
@mr-c
Copy link
Member

mr-c commented Nov 12, 2024

A new cwltool release has been made with your fix; thank you @sameeul !

https://github.com/common-workflow-language/cwltool/releases/tag/3.1.20241112140730 / https://github.com/common-workflow-language/cwltool/releases/tag/3.1.20241112140730

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants