port forwarding failing on windows

[rsignell-usgs]

When I launch jupyter-forward to connect to our USGS HPC machines

jupyter-forward [email protected] --launch-command "srun -p compute -N 1 -t 00:30:00" --conda-env jupyter --port 8889

from an Anaconda prompt on Windows, there are no errors, but nothing loads in the new http://localhost:8889 browser tab: the connection is refused, so I guess the port forwarding is not working correctly.

I can do port forwarding the old fashioned way from Windows, using git bash and :

 ssh -N -L 8889:`hostname`:8888 [email protected]"

and I verified that the jupyter-forward invocation above works fine from Linux.

Is there some other package I need to load into my custom conda environment on Windows to get things working?
Or something else I could do to help debug the problem?

[mnlevy1981]

When you specify --port 8889, it's going to forward to your local 8889 port. Does anything load in http://localhost:8889?

[rsignell-usgs]

@mnlevy1981 , oops, the 8888 above was a typo (I just fixed it).
Indeed, nothing loads on http://localhost:8889 (when I specify port 8889). The browser tab pops up, but connection is refused.

[rsignell-usgs]

Has anyone successfully used jupyter-forward on Windows?

[andersy005]

Has anyone successfully used jupyter-forward on Windows?

Not that I know of. I am going to set up a windows VM to see if I can reproduce the issue, and figure out what's going on

[mnlevy1981]

@mnlevy1981 , oops, the 8888 above was a typo (I just fixed it).
Indeed, nothing loads on http://localhost:8889 (when I specify port 8889). The browser tab pops up, but connection is refused.

bummer! (I figured it was a typo, but was hoping for an easy fix :)

@andersy005 -- I have access to a windows machine at home and am happy to help test this as well. You might be better off on your own, though; I installed conda on it at some point to test things for our tutorials, but that's about the extent of my familiarity with scientific computing in Win10...

[andersy005]

Something strange is going on... I was able to create a VM with windows on it. Launching jupyter on a remote cluster with nodes with public IP addresses works:

jupyter-forward [email protected] --no-port-forwarding

Launching jupyter on the same cluster but enabling the port forwarding via localhost doesn't work:

jupyter-forward [email protected]

In the first tab, you can see that everything is working fine when I access jupyter via the public IP address.

When I check whether the (8888) port is in the LISTENING state on localhost, I get the following

I will keep looking and will report back if I find something...

[rsignell-usgs]

@andersy005 , are there any Windows experts at NCAR?
If not, perhaps @TomAugspurger could help us find someone!
It would be so great to use jupyter-forward from git-bash in Windows!

[rsignell-usgs]

@ocefpaf, do you think one of the windows guys at conda-forge might know the answer here?

[rsignell-usgs]

@cgohlke, do you how how to address the Windows issue here?

[cgohlke]

Sorry, I don't have the setup to try this right now.

[rsignell-usgs]

It's worth mentioning again that the "standard" remote Jupyter tunneling works fine on Windows:

• Windows: open an Anaconda command prompt terminal

• Windows: type ssh denali.cr.usgs.gov

• Denali (HPC): switch to a compute node (srun command)

• Denali: type jupyter lab --no-browser --ip=$HOST--port=$JPORT

• Windows: open another Anaconda command prompt terminal

• Windows: type ssh -N -L 8889:$HOST:$JPORT [email protected]

• Windows: open localhost:8889

So rather than a firewall issue, there must be some issue with the jupyter-forward script, right?

[andersy005]

I now have access to a windows computer... I'll be looking into this in the coming days

[rsignell-usgs]

@andersy005, I'm guessing you are a tad busy, but just letting you know we are still very interested in getting this problem solved.

[andersy005]

@rsignell-usgs, thank you for your patience... I was on vacation for the last week. i still have this on my TODO list though, and i'm hoping to be able to report what i learn in tomorrow or thursday.

[andersy005]

i can confirm that i can reproduce this error on my local windows PC. i'm looking into what's going on exactly

[andersy005]

@rsignell-usgs, it turns out this is a known bug/issue in Fabric (the library we are using under the hood). For some reason, the tunneling doesn't work on windows

• Cannot port forward fabric/fabric#2102

it appears someone put together a fix. This hasn't been merged:

• forward_local() does not work on windows. fabric/fabric#2221

[rsignell-usgs]

OMG @andersy005 and @ocefpaf -- YOU GUYS ROCK !!!!

Windows users can now get a successful jupyter-forward conda environment with:

conda create -n jupyter-forward -c conda-forge jupyter-forward 

and here's proof:

[rsignell-usgs]

Tip: As you can see above, I was able to activate my jupyter-forward environment in powershell, after doing:

powershell -ExecutionPolicy ByPass -NoExit -Command "& 'C:\Users\rsignell\miniconda3\shell\condabin\conda-hook.ps1' ; conda activate 'C:\Users\rsignell\miniconda3' "

But the Anaconda command prompt works fine also -- and perhaps therefore preferable unless there is a reason to use powershell.

[andersy005]

thank you for your patience, @rsignell-usgs! @ocefpaf, thank you for the patch! So glad to see this working...