Can't connect when RemoteCommand is very slow

[SimeonEhrig]

Since today, I use the to run a custom .bashrc for a specific ssh connection to setup a environment for my remote vsc running on an Ubuntu 20.04:

Host vsc-v100-alpaka
ForwardAgent no
Hostname ***
ServerAliveInterval 15
ServerAliveCountMax 4
User ***
ForwardX11 yes
Compression yes
IdentityFile ~/.ssh/***
IdentitiesOnly yes
RemoteCommand /bin/bash --rcfile ~/workspace/caravan/alpaka/.vscode/env_init.sh -i
Proxycommand sh -c "ssh loginNode1 -W %h:%p || exec ssh loginNode2 -W %h:%p"

Since today, it is not working anymore, because vsc detects the system as Windows and tries to execute the powershell. I already tried the option remote.SSH.remotePlatform without success:

"remote.SSH.remotePlatform": {
        "vsc-v100-alpaka": "linux",
        "vsc-v100-vikunja": "linux",
        "v100": "linux"
    },

If I disable useLocalServer, the remote system is correct detected as Linux but than enableRemoteCommand is not working anymore.

• VSCode Version: 1.64.1
• Extension Version: v0.73.2022020915
• Local OS Version: Linux Mint 20.3 (Ubuntu 20.04 base)
• Remote OS Version: Ubuntu 20.04.3
• Remote Extension/Connection Type: SSH
• Logs:

[14:07:39.175] Log Level: 2
[14:07:39.176] [email protected]
[14:07:39.176] linux x64
[14:07:39.197] SSH Resolver called for "ssh-remote+vsc-v100-vikunja", attempt 1
[14:07:39.198] "remote.SSH.useLocalServer": true
[14:07:39.199] "remote.SSH.path": undefined
[14:07:39.199] "remote.SSH.configFile": undefined
[14:07:39.199] "remote.SSH.useFlock": true
[14:07:39.199] "remote.SSH.lockfilesInTmp": false
[14:07:39.200] "remote.SSH.localServerDownload": auto
[14:07:39.200] "remote.SSH.remoteServerListenOnSocket": false
[14:07:39.200] "remote.SSH.showLoginTerminal": false
[14:07:39.201] "remote.SSH.defaultExtensions": []
[14:07:39.201] "remote.SSH.loglevel": 2
[14:07:39.201] "remote.SSH.enableDynamicForwarding": true
[14:07:39.202] "remote.SSH.enableRemoteCommand": true
[14:07:39.202] "remote.SSH.serverPickPortsFromRange": {}
[14:07:39.202] "remote.SSH.serverInstallPath": {}
[14:07:39.210] SSH Resolver called for host: vsc-v100-vikunja
[14:07:39.210] Setting up SSH remote "vsc-v100-vikunja"
[14:07:39.215] Acquiring local install lock: /tmp/vscode-remote-ssh-27b54bc2-install.lock
[14:07:39.218] Looking for existing server data file at /home/user/.config/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-27b54bc2-d6ee99e4c045a6716e5c653d7da8e9ae6f5a8b03-0.73.2022020915/data.json
[14:07:39.219] Using commit id "d6ee99e4c045a6716e5c653d7da8e9ae6f5a8b03" and quality "stable" for server
[14:07:39.223] Install and start server if needed
[14:07:39.232] PATH: /home/user/.local/bin:/home/user/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/local/cuda/bin/:/home/user/.emacs.d/elpa/rtags-20200221.36/rtags-2.38/bin/:/home/user/.local/bin
[14:07:39.232] Checking ssh with "ssh -V"
[14:07:39.241] > OpenSSH_8.2p1 Ubuntu-4ubuntu0.4, OpenSSL 1.1.1f  31 Mar 2020

[14:07:39.248] askpass server listening on /run/user/1000/vscode-ssh-askpass-d896945542334e4562f2148bc1c5376501d5951d.sock
[14:07:39.249] Spawning local server with {"serverId":1,"ipcHandlePath":"/run/user/1000/vscode-ssh-askpass-e535fe489c460b21da64b83ce436810c43c9f569.sock","sshCommand":"ssh","sshArgs":["-v","-T","-D","39031","-o","ConnectTimeout=30","vsc-v100-vikunja"],"serverDataFolderName":".vscode-server","dataFilePath":"/home/user/.config/Code/User/globalStorage/ms-vscode-remote.remote-ssh/vscode-ssh-host-27b54bc2-d6ee99e4c045a6716e5c653d7da8e9ae6f5a8b03-0.73.2022020915/data.json"}
[14:07:39.249] Local server env: {"DISPLAY":":0","ELECTRON_RUN_AS_NODE":"1","SSH_ASKPASS":"/home/user/.vscode/extensions/ms-vscode-remote.remote-ssh-0.73.2022020915/out/local-server/askpass.sh","VSCODE_SSH_ASKPASS_NODE":"/usr/share/code/code","VSCODE_SSH_ASKPASS_EXTRA_ARGS":"--ms-enable-electron-run-as-node","VSCODE_SSH_ASKPASS_MAIN":"/home/user/.vscode/extensions/ms-vscode-remote.remote-ssh-0.73.2022020915/out/askpass-main.js","VSCODE_SSH_ASKPASS_HANDLE":"/run/user/1000/vscode-ssh-askpass-d896945542334e4562f2148bc1c5376501d5951d.sock"}
[14:07:39.254] Spawned 26334
[14:07:39.379] > local-server-1> Spawned ssh, pid=26342
[14:07:39.381] stderr> OpenSSH_8.2p1 Ubuntu-4ubuntu0.4, OpenSSL 1.1.1f  31 Mar 2020
[14:07:39.409] stderr> mux_client_request_session: session request failed: Session open refused by peer
[14:07:39.553] stderr> debug1: Server host key: ecdsa-sha2-nistp256 SHA256:jJ5MNeSKjfrOlQTGEhjMFSpd29zh+fGdNGs9JcfU27U
[14:07:39.733] stderr> Authenticated to fwk394 (via proxy).
[14:07:39.733] stderr> bind [::1]:39031: Address already in use
[14:07:39.733] stderr> bind [127.0.0.1]:39031: Address already in use
[14:07:39.733] stderr> channel_setup_fwd_listener_tcpip: cannot listen to port: 39031
[14:07:39.733] stderr> Could not request local forwarding.
[14:07:39.734] stderr> ControlSocket /tmp/remoteuser@fwk394:22 already exists, disabling multiplexing
[14:07:40.326] stderr> bash: cannot set terminal process group (-1): Inappropriate ioctl for device
[14:07:40.326] stderr> bash: no job control in this shell
[14:07:50.471] stderr> (vikunja-dev) remoteuser@fwk394:~$ 
[14:07:50.497] > ready: e3b128ee2c71
[14:07:50.500] stderr> echo 'ready: e3b128ee2c71'
[14:07:50.500] stderr> (vikunja-dev) remoteuser@fwk394:~$ 
[14:07:50.535] Platform: windows
[14:07:50.536] Remote command length: 8140/8192 characters
[14:07:50.568] > Linux 5.4.0-89-generic #100-Ubuntu SMP Fri Sep 24 14:50:10 UTC 2021
[14:07:50.569] stderr> (vikunja-dev) remoteuser@fwk394:~$ 
[14:07:50.638] stderr> powershell -NoProfile
[14:07:50.834] stderr> 
[14:07:50.835] stderr> Command 'powershell' not found, but can be installed with:
[14:07:50.835] stderr> 
[14:07:50.836] stderr> sudo snap install powershell
[14:07:50.837] stderr> 
[14:07:50.861] stderr> (vikunja-dev) remoteuser@fwk394:~$ 
[14:08:11.257] Terminating local server
[14:08:11.274] Resolver error: Error: Connecting with SSH timed out
	at Function.Timeout (/home/user/.vscode/extensions/ms-vscode-remote.remote-ssh-0.73.2022020915/out/extension.js:1:586043)
	at Timeout._onTimeout (/home/user/.vscode/extensions/ms-vscode-remote.remote-ssh-0.73.2022020915/out/extension.js:1:630685)
	at listOnTimeout (internal/timers.js:554:17)
	at processTimers (internal/timers.js:497:7)
[14:08:11.283] Local server exit: 15
[14:08:11.311] ------

[tanhakabir]

When you connect through just your shell to vsc-v100-alpaka are you able to see your custom bash shell being used?

[SimeonEhrig]

Yes, because the lines also appear, if I log in via interactive shell.

VSC output:

bash: cannot set terminal process group (-1): Inappropriate ioctl for device
[14:07:40.326] stderr> bash: no job control in this shell

interactive shell output:

user@dev-system:~$ ssh vsc-v100-alpaka 
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
(alpaka-dev) user@dev-system:~$ 

[tanhakabir]

@roblourens I believe this is another situation where the shell command in the connection string enabled bash before on a windows system but not anymore now. We could try running probe shell first to actually know which shell they're running and decide the script based on that?

[roblourens]

I don't understand why the platform detection isn't working though. It might have timed out? It looks like the uname response is late

[SimeonEhrig]

I don't understand why the platform detection isn't working though. It might have timed out? It looks like the uname response is late

The script env_init.sh does the following commands:

#!/bin/bash
 
source ~/.bashrc
conda activate vikunja-dev

This takes about 10-15s, because the conda environment also loads some spack modules, which incredible slow. Therefore I increased also the SSH timeout to 30s.

[roblourens]

Ok. I think the platform detection is not included in our configurable ssh timeout. We could make this part of our timeout setting

[eleanorjboyd]

Hello! My name is Eleanor and I am the new maintainer for Remote-SSH.

To summarize for future reference, the ask here is for platform detection to be used as a factor to see if the ssh connection has worked. Since platform detection indicates that ssh did successful connect to the remote machine and began trying to determine its platform, this shows the ssh connection is a success. Therefore the ssh timeout should not be considered any longer and detection script should not timeout based on the ssh timeout settings.

[Oragi]

I had the same issue, however my remote command should not(?) cause a timeout

[18:01:34.972] Spawned 69532
[18:01:35.053] > local-server-1> Running ssh connection command: "-v -T -D 49501 -F C:\\Users\\User\\.ssh\\config vps-linode"
[18:01:35.056] > local-server-1> Spawned ssh, pid=107676
[18:01:35.176] stderr> OpenSSH_9.0p1, OpenSSL 1.1.1q  5 Jul 2022
[18:01:35.524] stderr> debug1: Server host key: ssh-ed25519 SHA256:[--]
[18:01:35.742] stderr> Authenticated to [--] using "publickey".
[18:01:36.789] stderr> bash: cannot set terminal process group (2915064): Inappropriate ioctl for device
[18:01:36.790] stderr> bash: no job control in this shell
[18:01:36.812] stderr> deploy@localhost:/root$ echo 'ready: 77630761f23b'
[18:01:36.812] > ready: 77630761f23b
[18:01:36.813] stderr> 
[18:01:36.813] stderr> deploy@localhost:/root$ 
[18:01:36.875] Platform: windows
[18:01:36.875] Remote command length: 8208/8192 characters
[18:01:36.876] > Linux 5.4.0-109-generic #123-Ubuntu SMP Fri Apr 8 09:10:54 UTC 2022
[18:01:36.876] stderr> deploy@localhost:/root$ 
[18:01:36.949] stderr> powershell -NoProfile
[18:01:37.145] > Please ask your administrator.
[18:01:37.145] stderr> 
[18:01:37.145] stderr> Command 'powershell' not found, but can be installed with:
[18:01:37.145] stderr> 
[18:01:37.145] stderr> snap install powershell
[18:01:37.145] stderr> 
[18:01:37.145] stderr> deploy@localhost:/root$ 
[18:01:51.984] Terminating local server
[18:01:51.986] Resolver error: Error: Connecting with SSH timed out
	at g.Timeout (c:\Users\User\.vscode\extensions\ms-vscode-remote.remote-ssh-0.95.2022122215\out\extension.js:1:584179)
	at Timeout._onTimeout (c:\Users\User\.vscode\extensions\ms-vscode-remote.remote-ssh-0.95.2022122215\out\extension.js:1:629479)
	at listOnTimeout (node:internal/timers:559:17)
	at process.processTimers (node:internal/timers:502:7)
[18:01:51.989] ------

My command was
RemoteCommand sudo -u deploy -s /bin/bash -i
I fixed it by changing to
RemoteCommand sudo -u deploy -s /bin/bash
both commands finish instantly when ran manually from outside vs code
I don't need the -i flag so this solution is fine for me, just posting this in case anyone else has OP's issue with a different cause