No Progress bar showing in Slurm output file #19227
-
The title explains exactly the problem I have. However, when I run a sbatch script of the model I do not see the progress bar updates in the slurm output file or somewhere else in the logs directory. I tried both the tqdm and rich progress bar but with both I do not see any output. Is this the intended behaviour for running in a slurm system? I made sure I am using the latest Ligthning and tqdm version. I also search everywhere on the lightning docs and github for a similar question but i could not find any. Any help or solutions are appreciated. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
Okay, I found my problem. It was a mistake from my part, I am sorry for the issue. |
Beta Was this translation helpful? Give feedback.
Okay, I found my problem. It was a mistake from my part, I am sorry for the issue.
I was using the pytorch ligthning + hydra template to construct my code and I used hydra to change the progress bar which was default set to rich to tqdm, however, I did this wrong and thus I was still using the rich progress bar on the server. This one does not provide intermediate step updates.