[New Feature] Slurm training script

[rayleizhu]

How can I start training on a slurm cluster? Do I need to take care of ddp initialization routines by myself?

[FengLi-ust]

Yes, you can refer to DINO slurm scripts as an example. We will upload this script in the future.

[rayleizhu]

I’ve implemented slurm launcher using submitit. It supports:

• one-line launch command on slurm cluster, just like existing trainer provided by current detrex.
• automatic experiment name generation according to hyper-parameter overrides.
• automatically requeue and resume from latest checkpoint when a running job is preempted or reaches maximum execution time.

Are you interested in merging my implementation to detrex? If yes, I will create a pull request by the end of this week.

[FengLi-ust]

Of course, that would be great!

[rentainhe]

Thanks for your contribution, we will close this issue~ reopen this at anytime if you need more help~