Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC: adapts documentation for first-time users #28

Merged
merged 8 commits into from
Jul 23, 2020

Conversation

stsievert
Copy link
Contributor

What does this PR implement?
This is my first time running Dask-CHTC. I've adapted the documentation for items I'd like to see in the documentation. So far, these are:

  • Adds note about installing Jupyter before running
  • Only show dask-chtc jupyter run lab instead of that and the unuseful jupyter lab.
  • Renames "Example Notebook" to "Dask Cluster Creation."

Reference issues/PRs
Other issues that came up in this first usage are #25, #26 and #27.

@stsievert
Copy link
Contributor Author

I did have one question on the "Using the start, status, and stop subcommands" subsection: why can't I run dask-chtc jupyter run lab through tmux? That's how I normally persist Jupyter sessions on remote servers.

Copy link
Contributor

@JoshKarpel JoshKarpel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nitpicks, and one thought about the already-running-job message, but otherwise LGTM

docs/source/jupyter.rst Outdated Show resolved Hide resolved
docs/source/jupyter.rst Outdated Show resolved Hide resolved
docs/source/jupyter.rst Outdated Show resolved Hide resolved
dask_chtc/cli.py Show resolved Hide resolved
@JoshKarpel
Copy link
Contributor

JoshKarpel commented Jul 20, 2020

I did have one question on the "Using the start, status, and stop subcommands" subsection: why can't I run dask-chtc jupyter run lab through tmux? That's how I normally persist Jupyter sessions on remote servers.

Yep, that should work just fine. I want there to be a workflow that doesn't involve tmux for those who don't like it, but for those who do, tmux is a great way to make a detached session.

Do note that it won't be quite as protected as the fully-detached dask-chtc jupyter start job would be. CHTC admins do reserve the right to kill userland programs; if you accidentally start behaving badly in some part of your tmux, they might kill the whole thing as a safety measure.

@stsievert
Copy link
Contributor Author

CHTC admins do reserve the right to kill userland programs;

How can I be a good user so CHTC admins don't kill my jobs? I'm interested in running jupyter lab instead of dask-chtc jupyter run lab in a tmux session for maybe a couple days. What other downsides come from running jupyter lab instead of dask-chtc jupyter run lab?

@JoshKarpel
Copy link
Contributor

How can I be a good user so CHTC admins don't kill my jobs?

The main consideration is: try not to use large amounts of CPU, memory, or disk I/O on the submit node for longer than a few minutes at a time. The HTCondor daemons need those resources to serve other people's jobs. For a Dask workflow, this means making sure that you do all the heavy-duty computation through Dask, pulling only the minimal information back to the scheduler.

What other downsides come from running jupyter lab instead of dask-chtc jupyter run lab?

Right now, very few (for you; there are some incidental benefits for us). But we're hoping to be able to enforce resource limits by doing this, which will make CHTC admins much happier with this workflow. As far as I know, there shouldn't be a difference in the behavior of Jupyter once you've got it running - let me know if there is, so that we can fix it :)

@JoshKarpel JoshKarpel added documentation Improvements or additions to documentation enhancement New feature or request jupyter labels Jul 21, 2020
@JoshKarpel JoshKarpel merged commit debd3b9 into CHTC:master Jul 23, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation enhancement New feature or request jupyter
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants