How to use ssh if remote host does not support echo command (Hetzner Storage Box)

[phoyer]

Do you have any idea how I can use backintime if rsync and standard file operation commands are available but no echo command which backintime is using to check the connection?

ssh -o PreferredAuthentications=publickey -p 22 -o ServerAliveInterval=240 -o IdentityFile=/home/[user]/.ssh/[id]_rsa backup@[my-local-ip] echo "Hello"

fails with
Command not found. Use 'help' to get a list of available commands.

ssh -o PreferredAuthentications=publickey -p 22 -o ServerAliveInterval=240 -o IdentityFile=/home/[user]/.ssh/[id]_rsa backup@[my-local-ip] ls

works fine.

[buhtz]

Dear @phoyer ,
thank you very much for your report.

First I thought it is a syntax problem because of the ". But I can not reproduce the problem here on my Debian 11.

Please let me ask some questions:

What type of machine is your [my-local-ip]? What operating system runs there? Why isn't echo available?`

Can you login locally into that machine? Please do und try echo "Hello" and tell us the result. Also give us the result of whereis echo.

Also try to login via ssh into a shell. Again do echo "Hello" and whereis echo and show us the result.

Please show the output of backintime --diagnostics. If it doesn't work please tell us the BIT version and how/where do you installed it from.

Note for the @bit-team :
Beside that edge case it is not clear to me why echo is used by BIT.

[phoyer]

The machine is not a real Linux server (or at least you do not have shell access) but it is a storage system "Hetzner Storage Box". Available commands are limited:

uxxx-sub4 /home > echo
Command not found. Use 'help' to get a list of available commands.
uxxx-sub4 /home > help
+-----------------------------------------------------------------------------+
| The following commands are available:                                       |
|   ls                                  list directory content                |
|   tree                                list directory content                |
|   cd                                  change current working directory      |
|   pwd                                 show current working directory        |
|   mkdir                               create new directory                  |
|   rmdir                               delete directory                      |
|   du                                  disk usage of files/directories       |
|   df                                  show disk usage                       |
|   dd                                  read and write files                  |
|   cat                                 output file content                   |
|   touch                               create new file                       |
|   cp                                  copy files/directories                |
|   rm                                  delete files/directories              |
|   unlink                              delete file/directory                 |
|   mv                                  move files/directories                |
|   chmod                               change file/directory permissions     |
|   md5|sha1|sha256|sha512              create hash sum of file               |
|   md5sum|sha1sum|sha256sum|sha512sum  create hash sum of file               |
|   head                                show first lines of file              |
|   tail                                show last lines of file               |
|   grep                                search for specific string in files   |
|   stat                                stat files/directory                  |
|                                                                             |
| Available as server side backend:                                           |
|   borg                                                                      |
|   rsync                                                                     |
|   scp                                                                       |
|   sftp                                                                      |
|   rclone serve restic --stdio                                               |
|                                                                             |
| Please note that this is only a restricted shell which do not               |
| support shell features like redirects or pipes.                             |
|                                                                             |
| You can find more information in our Docs:                                  |
|   https://docs.hetzner.com/robot/storage-box/                               |
+-----------------------------------------------------------------------------+

I suppose, the have used echo as it is the easiest way to just strcmp the response. Maybe using pwd could be an alternative by parsing if a valid path is returned?

I am using backintime 1.2.1 installed through Ubuntu 22.04 apt.

[buhtz]

Interesting use case.

Please note that your BIT version is outdated for years. But it also won't solve your current problem updating to a newer version.

Before modifying anything can you please reproduce the problem while running backintime with the --debug option in a terminal and post me the complete output? Take care of IP and user name in that output.

Can you create aliases on your hetzner shell? It is not a usual bash/sh shell I think. But if aliases work just try alias echo pwd.

Another try would be you modify the code your self. This is the location in your version of the code. Try to modify that line

ssh = self.config.sshCommand(cmd = ['echo', '"Hello"'],

into that

ssh = self.config.sshCommand(cmd = ['pwd'],

Another option for a short term solution/workaround would be to contact Hetzer and ask them to make "echo" available or creating an alias for it. Now sure how many options they have on their site.

[Germar]

You can disable Check if remote host support all necessary commands in Expert Options. This will prevent all pre-checks on remote host. But be aware that it could lead to some wired errors if one of the other necessary commands won't work either.

[buhtz]

Hello Germar,
thanks for the reply. Do you remember any reason why echo was used and not something else? To my understanding it could be something else, right?

@phoyer
In which situation do you got this error message? I assume you tried to setup the snapshot profile, hit the OK button and then got that error message?

Thinking out loud:
I wonder if that function do have to use a shell command to full fill its job. It just checks if the password less login is possible. I have to check that but I assume that removing the shell command (echo "Hello") in the current case would solve this and all other cases. We do have two test cases for that method. So might be easy to fix without to much risk of harm or introducing unwanted behavior.

[Germar]

I just doubble-checked the code, Check if remote host support all necessary commands won't help, sorry. This will only block the check for commands. But while checking the login echo is always in use.

I used echo back in the days because I thought a simple command like this would be available everywhere 🤷

[buhtz]

Maybe the Export Option "Check if remote host is online" will do the job? No, it wont. that option is about ping the remote machine.

I'll think about fixing and improving it.

[phoyer]

Thinking out loud: I wonder if that function do have to use a shell command to full fill its job. It just checks if the password less login is possible. I have to check that but I assume that removing the shell command (echo "Hello") in the current case would solve this and all other cases. We do have two test cases for that method. So might be easy to fix without to much risk of harm or introducing unwanted behavior.

@buhtz when looking at the code in lines 425 and 465 of test_sshtools.py, it does not check at all if there is "Hello" as output of the echo command, but it only checks the command's return value. I have replaced this by cmd=['pwd']. In the command line, this command will (and does in this case) return the current path, so the return value should be 0. For some reason, it runs into an error anyway. I have no idea why this does not work. Maybe someone who is more into Python has an idea?

DEBUG: [common/sshtools.py:350 SSH.unlockSshAgent] Password available: True
DEBUG: [common/sshtools.py:567 SSH.checkKnownHosts] Check known hosts file
DEBUG: [common/sshtools.py:580 SSH.checkKnownHosts] Host u142xxx.your-backup.de was found in known hosts file
DEBUG: [common/sshtools.py:423 SSH.checkLogin] Check login
**ERROR: [qt/settingsdialog.py:1597 SettingsDialog.saveProfile] Password-less authentication for [email protected] failed. Look at 'man backintime' for further instructions.**

DEBUG: [common/sshtools.py:1059 sshCopyId] Call command "ssh-copy-id -i /home/phoyer/.ssh/id_rsa.pub -p 23 [email protected]"
INFO: [common/sshtools.py:1075 sshCopyId] Successfully copied ssh-key "/home/phoyer/.ssh/id_rsa.pub" to "[email protected]"

Testing pwd in command line:

ssh [email protected] -p 23
Last login: Tue Jun 13 10:00:00 2023 from host-79-54-219-144.retail.telecomitalia.it
+------------------------------------------------------------------+
| Welcome to your Storage Box.                                     |
|                                                                  |
| Please note that this is only a restricted shell environment and |
| therefore some shell features like pipes and redirects are not   |
| supported.                                                       |
+------------------------------------------------------------------+
u142xxx-sub4 /home > pwd
/home
u142xxx-sub4 /home > exit
Connection to u142xxx.your-backup.de closed.

If someone likes to try out, I can also provide testing credentials by private message.

[buhtz]

Thanks for checking out and offering your credentials.
I wasn't able to reproduce this on my machine. With pwd the returncode is 0. There might be more hidden things involved.

Can I ask you for the favour to try out this code please?

from subprocess import Popen
p = Popen(['ssh', '[email protected]', '-o', 'PreferredAuthentications=publickey', 'pwd'])
e = p.communicate()
rc = p.returncode

print(f'{e=} {rc=}')

As a first workaround you simply could add return in the first line of the function checkLogin(). Not elegant but might work.

[phoyer]

Your idea has been very useful.

For some reason, using pwd I get:
Command not found. Use 'help' to get a list of available commands. e=(None, None) rc=8

Instead with ls everything is fine:
e=(None, None) rc=0

Now authentication is working perfectly. Unfortunately, this does not help as in further checks the tools uses other bash commands such as test which again is not supported. At this point, the only option will be to mount the ssh drive or to use another service with a "real" shell on the remote end.

Thank you so much for you instant replies anyway and in general it may be useful to change echo into something like ls anyway to see whether you can operate on the file system rather than printing Hello.

[buhtz]

Also thanks for your help. I'll keep this Issue open until it is fixed for all users.

My current proposal is to just use exit as "command". Technically it is ssh-login and logout. So I assume the following code should work with your Hetzner Box also.

from subprocess import Popen
p = Popen([
    'ssh',
    '[email protected]',
    '-o',
    'PreferredAuthentications=publickey',
    'exit'])

e = p.communicate()
rc = p.returncode

print(f'{e=} {rc=}')

EDIT

Dear @phoyer ,
I explained Hetzer the situation and ask them for a test box. But they refused and told me just to sign a usual contract with the "first 14 days free". This is not an option for me.

But it would be OK for me if you could just test the code above and show me the output. I suspect this would work. When you confirm that it works I will integrate it in the code.

[buhtz]

Hello @phoyer ,
can you please check my example code with your Hetzner Box to validate the fix.

Kind
Christian

[buhtz]

I vote to implement this even without feedback from @phoyer .