gitfs broken on Arch

[jamincollins]

Description of Issue/Question

I am trying to use ssh authentication with gitfs to pull git repositories from a personal GitLab installation.

I have double checked my private key for an empty line at the end, like in the release notes. I have successfully downloaded all the git repositories with the key configured.

I've seen previous reports of this being fixed by upgrading libssh to 1.8.1. However, I'm running libssh2 1.8.1 and still seeing the problem.

$ yaourt -Qi libssh2
Name            : libssh2
Version         : 1.8.1-1
Description     : A library implementing the SSH2 protocol as defined by
                  Internet Drafts
Architecture    : x86_64
URL             : https://www.libssh2.org/
Licenses        : BSD
Groups          : None
Provides        : libssh2.so=1-64
Depends On      : openssl  zlib
Optional Deps   : None
Required By     : curl  libgit2  libvirt
Optional For    : None
Conflicts With  : None
Replaces        : None
Installed Size  : 401.00 KiB
Packager        : Levente Polyak <[email protected]>
Build Date      : Mon 18 Mar 2019 10:00:30 PM UTC
Install Date    : Sat 13 Apr 2019 09:10:47 PM UTC
Install Reason  : Installed as a dependency for another package
Install Script  : No
Validated By    : Signature

Setup

$ grep '^[^#]' /etc/salt/master
fileserver_backend:
  - git
gitfs_pubkey: '/etc/salt/ssh/id_rsa.pub'
gitfs_privkey: '/etc/salt/ssh/id_rsa'
gitfs_remotes:
  - git@$host:salt/states.git

Error

[ERROR   ] Error occurred fetching gitfs remote 'git@$host:salt/states.git': Failed to authenticate SSH session: Unable to send userauth-publickey request
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/salt/utils/gitfs.py", line 1712, in _fetch
    fetch_results = origin.fetch(**fetch_kwargs)
  File "/usr/lib/python2.7/site-packages/pygit2/remote.py", line 411, in fetch
    check_error(err)
  File "/usr/lib/python2.7/site-packages/pygit2/errors.py", line 64, in check_error
    raise GitError(message)
GitError: Failed to authenticate SSH session: Unable to send userauth-publickey request

Steps to Reproduce Issue

sudo salt-master -l debug

Versions Report

$ salt --versions-report
Salt Version:
           Salt: 2019.2.0
 
Dependency Versions:
           cffi: 1.12.2
       cherrypy: unknown
       dateutil: Not Installed
      docker-py: Not Installed
          gitdb: 2.0.5
      gitpython: 2.1.11
          ioflo: Not Installed
         Jinja2: 2.10.1
        libgit2: 0.27.8
        libnacl: Not Installed
       M2Crypto: 0.30.1
           Mako: 1.0.8
   msgpack-pure: Not Installed
 msgpack-python: 0.6.1
   mysql-python: Not Installed
      pycparser: 2.19
       pycrypto: 3.8.1
   pycryptodome: Not Installed
         pygit2: 0.27.4
         Python: 2.7.16 (default, Mar 11 2019, 18:59:25)
   python-gnupg: Not Installed
         PyYAML: 3.13
          PyZMQ: 18.0.1
           RAET: Not Installed
          smmap: 2.0.5
        timelib: Not Installed
        Tornado: 5.1.1
            ZMQ: 4.3.1
 
System Versions:
           dist:   
         locale: UTF-8
        machine: x86_64
        release: 5.0.7-arch1-1-ARCH
         system: Linux
        version: Not Installed

[vosscodes]

I've also been having this issue for quite some time, and was unable to get gitfs working again on Arch.

The versions shipping with later 2018 Arch isos are all effected by this. I did not check the releases in between, but I can be sure that the 2018.5 iso worked properly, and that 2018.12 on do not. I also went through the suggested fixes like double checking for newlines, and the keys did work when used directly with git cli.

I'm not sure if other any distros are effected, as eventually I stopped using gitfs entirely due to this issue.

[jamincollins]

I've tried downgrading to older libssh2 releases and upgrading to 1.8.2 (not yet in the Arch repositories), all versions exhibit the same behavior (above error) for me.

[Ch3LL]

i just tested on manjaro which is a fork of arch. I cannot replicate it with libssh version 1.8.0-3. i'm upgrading my laptop really quick to see if i can replicate with the new libssh version.

[Ch3LL]

sure enough upgrading to 1.8.1 i'm able to replicate this:

2019-04-26 15:41:28,193 [salt.utils.gitfs :1734][ERROR   ][20524] Error occurred fetching gitfs remote '[email protected]:Ch3LL/Ch3LLScripts.git': Failed to authenticate SSH session: Unable to send userauth-publickey request
Traceback (most recent call last):
  File "/home/ch3ll/git/salt/salt/utils/gitfs.py", line 1712, in _fetch
    fetch_results = origin.fetch(**fetch_kwargs)
  File "/home/ch3ll/virtual-env/salt-py3/lib/python3.7/site-packages/pygit2/remote.py", line 411, in fetch
    check_error(err)
  File "/home/ch3ll/virtual-env/salt-py3/lib/python3.7/site-packages/pygit2/errors.py", line 64, in check_error
    raise GitError(message)
_pygit2.GitError: Failed to authenticate SSH session: Unable to send userauth-publickey request

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

[digitalkram]

On a fresh Ubuntu (VERSION="20.04.1 LTS (Focal Fossa)") install using "python3-pygit2 1.0.3-1 amd64" this can also be reproduced:
_pygit2.GitError: Failed to authenticate SSH session: Unable to send userauth-publickey request

Steps to reproduce:
curl -L https://bootstrap.saltstack.com | sudo sh -s -- -U -M -A mymasterip
salt-key -Ay

salt * test.ping
apt install -y python3-pygit2

echo "fileserver_backend:
  - git
gitfs_provider: pygit2
gitfs_pubkey: /root/.ssh/id_rsa.pub
gitfs_privkey: /root/.ssh/id_rsa
gitfs_remotes:
  - [email protected]:someorg/somerep.git" > /etc/salt/master.d/master.conf

ssh-keygen -t rsa -N "" -f /root/.ssh/id_rsa

#authorize pub key with repo

systemctl restart salt-master

salt-run cache.clear_git_lock gitfs type=update

salt-run -l debug fileserver.update