esds/posts/2021/casper_pbs_dask/

[utterances-bot]

Using Dask on the New Casper PBS Scheduler — NCAR-ESDS 0.1 documentation

https://ncar.github.io/esds/posts/2021/casper_pbs_dask/

[Yefee]

Hi Max,

Thanks for the nice post! I tried it on casper and made everything work except visualizing the dashboard.
I added my user name (che43) and port (8889) in the link
dask.config.set({'distributed.dashboard.link':'https://jupyterhub.hpc.ucar.edu/stable/user/che43/proxy/8889/status'})

However after the initialization of client, the link doesn't work. Any thoughts on this to get it work?

Thanks!
CF

[kmsampson]

You can try this on casper:

dask.config.config.get('distributed').get('dashboard').update({'link':'{JUPYTERHUB_SERVICE_PREFIX}/proxy/{port}/status'})

[alifallahm]

Hi Max,

Many thanks for your nice update as I was looking for such deployment for a long time. As local directory, can I put this way?
/glade/scratch/${USER}/dask/casper-dav/local-dir

Following your setup, I wonder why when I run the client, still I can see 0 workers, cores, and memory!

[alifallahm]

It worked perfectly!