Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reuse SbatchStep and QsubSteps functions when creating DragonBatchSteps #595

Open
al-rigazzi opened this issue May 21, 2024 · 0 comments
Open
Labels
area: Dragon area: job Issues related to job management type: feature Issues that include feature request or feature idea

Comments

@al-rigazzi
Copy link
Collaborator

Description

DragonBatchStep should convert the step to a SbatchStep or to a QsubStep and directly use the corresponding write_script methods, instead of re-implementing them.

Justification

The DragonBatchStep has to create either a Sbatch or a Qsub script to be run through the corresponding scheduler. Currently, the functions to create such scripts are implemented in DragonBatchStep, but they are only slight variations of the SbatchStep and QsubStep counterparts. It would be better to convert the step internally to the appropriate batch step, and by doing so, being able to re-use its methods.

Implementation Strategy

The DragonBatchStep needs to create the request file, which is not created by Slurm or PBS batch steps, usually. This needs to happen before the step is converted. Moreover, the internal step (which would usually be a SrunStep on Slurm and one of the equivalent types on PBS), will need to be a LocalStep, used to start the dragon_client.py entrypoint as we currently do. But this LocalStep will have to be proxied - no indirect.py entrypoint should be invoked, as the job will still be managed through Dragon.

@al-rigazzi al-rigazzi added type: feature Issues that include feature request or feature idea area: job Issues related to job management area: Dragon labels May 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: Dragon area: job Issues related to job management type: feature Issues that include feature request or feature idea
Projects
None yet
Development

No branches or pull requests

1 participant