Pkg add private repo not using ssh key #1733

xximranxx · 2020-03-24T08:41:24Z

(Unnax) pkg> add [email protected]:aifinancegroup/graph.git
    Cloning git-repo `[email protected]:aifinancegroup/graph.git`
Private key location for '[email protected]' [/Users/imi/.ssh/id_rsa]: 
Private key location for '[email protected]' [/Users/imi/.ssh/id_rsa]: 
Private key location for '[email protected]' [/Users/imi/.ssh/id_rsa]: 
ERROR: failed to clone from [email protected]:aifinancegroup/graph.git, error: GitError(Code:EAUTH, Class:Callback, Aborting, maximum number of prompts reached.)

julia> ENV["SSH_KEY_PATH"]
"/Users/imi/.ssh/id_rsa"

julia> ENV["SSH_PUB_KEY_PATH"]
"/Users/imi/.ssh/id_rsa.pub"

julia> versioninfo()
Julia Version 1.4.0
Commit b8e9a9ecc6 (2020-03-21 16:36 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.6.0)
  CPU: Intel(R) Core(TM) i9-9980HK CPU @ 2.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, skylake)

julia>

The text was updated successfully, but these errors were encountered:

StefanKarpinski · 2020-03-24T14:19:21Z

I think maybe libgit2 just doesn't honor these?

xximranxx · 2020-03-24T14:36:01Z

Maybe --- I do think that this is an important feature to support for non-open-source users to use julia --- Maybe an issue has to be opened for libgit2? Or is it not being passed correctly to libgit2?

DilumAluthge · 2020-03-24T14:51:49Z

See https://discourse.julialang.org/t/how-to-specify-the-ssh-key-for-pkg-to-use/14784/6 and #1516 (comment). The problem is probably not the location of the private key, but some other SSH problem during authentication.

DilumAluthge · 2020-03-24T14:52:52Z

There are plenty of situations in which the regular SSH client works just fine, but libssh (which is what libgit2 uses) does not work.

As long as Julia and Pkg rely on libgit2, these issues will keep cropping up.

imikim · 2020-04-13T16:19:33Z

I see that Pkg.add (and LibGit2 behind it) actually downloads the repo (or at least the .git folder of the repo) but that the refs/heads folder is empty, but HEAD file points to refs/heads/master.

What does this mean?

imikim · 2020-04-14T08:05:34Z

I think another issue is the git@... url, which I suppose denotes a different protocol.
Even if I make the repo public, the git@... url does not work, but the https://... url does work.
And the git@... url gives: invalid git HEAD (reference 'refs/heads/master' not found)

imikim · 2020-04-14T08:33:47Z

libgit2 has released their 1.0 whilst julia uses 0.28.2 ---> maybe an update will help.
It issues is with the git@... protocol (ssh), independent of the repo being public or private.

imikim · 2020-04-14T08:38:42Z

For libssh2, I am confused about which version is being used:

In my julia (1.4) folder, I see the file: libssh2.1.0.1.dylib, implying version 1.0.1
The julia github, I see version set to 1.9.0 (which would be current): https://github.com/JuliaLang/julia/blob/master/deps/libssh2.version

grahamas · 2020-08-08T19:27:13Z

I stumbled onto this again. As @imikim pointed out, even public repos fail in this way, and the workaround is to switch to an https:// link. If there's no fix immediately forthcoming, I'd suggest that there be a warning linked to the private key location request that suggests using an https:// link. It's not an obvious fix based on the current error, and every time I run into this it takes me a while to remember that that's the workaround.

(@v1.5) pkg> dev [email protected]:grahamas/AxisIndices.jl.git
    Cloning git-repo `[email protected]:grahamas/AxisIndices.jl.git`
Private key location for '[email protected]' [/home/graham/.ssh/id_rsa]: 
Private key location for '[email protected]' [/home/graham/.ssh/id_rsa]: 
Private key location for '[email protected]' [/home/graham/.ssh/id_rsa]: 
ERROR: failed to clone from [email protected]:grahamas/AxisIndices.jl.git, error: GitError(Code:EAUTH, Class:Callback, Aborting, maximum number of prompts reached.)

julia> versioninfo()
Julia Version 1.5.0
Commit 96786e22cc (2020-08-01 23:44 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-10710U CPU @ 1.10GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)

edit: can someone change the name so that it doesn't mention private repos? It led me not to look here at first. Something like "Pkg add/dev [email protected]:... fails with private key prompts"

EliseBcm · 2020-11-02T09:57:20Z

Another possible solution :
I had the same error as @xximranxx on macOS after following the whole github documentation to check SSH keys, generate a new one and add it my github account. The problem is that the documentation does not say to add github to known hosts ! It can be done using the following command in the terminal :

$ ssh -T [email protected]
The authenticity of host 'github.com (140.82.121.4)' can't be established.
RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'github.com,140.82.121.4' (RSA) to the list of known hosts.
Hi EliseBcm! You've successfully authenticated, but GitHub does not provide shell access.

I hope it may help someone else. :)

The error I had initially :

(@v1.4) pkg> registry add [email protected]:MyPrivateRegistry.git
   Cloning registry from "[email protected]:MyPrivateRegistry.git"
Private key location for '[email protected]' [/Users/elebreto/.ssh/id_rsa]: 
Private key location for '[email protected]' [/Users/elebreto/.ssh/id_rsa]: 
Private key location for '[email protected]' [/Users/elebreto/.ssh/id_rsa]: 
ERROR: failed to clone from [email protected]:MyPrivateRegistry.git, error: GitError(Code:EAUTH, Class:Callback, Aborting, maximum number of prompts reached.)

julia> versioninfo()
Julia Version 1.4.2
Commit 44fa15b150* (2020-05-23 18:35 UTC)
Platform Info:
  OS: macOS (x86_64-apple-darwin18.7.0)
  CPU: Intel(R) Core(TM) i5-5257U CPU @ 2.70GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-8.0.1 (ORCJIT, broadwell)

StefanKarpinski · 2020-11-02T15:09:19Z

@EliseBcm, thanks for that! I for one did not have any idea that the absence of github.com from the known hosts file was the issue, so adding instructions for that would be good. However, servers with different IP addresses are distinguished by SSH, so it's possible that fixing that once will not be enough — if you end up connecting to a github.com SSH server with a different IP address, you'll have to do it again. Unlike HTTPS, where there's a public key infrastructure — broken and confusing as it is; almost no one currently supports certificate revocation, for example — which means that clients know in advance whether they should trust an HTTPS server or not. SSH relies on you approving the server the first time and then verifying that identity in the future.

One option would be that we could (if possible) configure libgit2 to auto-trust unknown servers the first time one connects to them, but that's pretty iffy as a default policy. If an attacker wants to man-in-the-middle your connection, all they have to do is arrange for your github.com DNS query to return an IP address that they control. Since DNS is often not secured, this is a pretty huge security risk. If we can force DNS-over-HTTPS to a known secure host, that might be acceptable. Of course that will not work for private domains, so it's not a perfect solution either. I'm not sure if there is a perfect solution for this.

At the very least, it would be good to detect this specific failure mode and print a better error message.

xximranxx · 2020-11-02T15:30:43Z

@EliseBcm Your solution unfortunately did not work for me on [email protected]

usmcamp0811 · 2020-11-08T13:02:35Z

I've got problems with adding/cloning git repos using ssh as well. I'm on Arch running Julia 1.5.2. I've tried the following and no joy.

julia> using Pkg
shell> ssh -T [email protected]
Welcome to GitLab, @usmcamp0811!

julia> Pkg.add(Pkg.PackageSpec(url = "[email protected]:usmcamp0811/examplepkg.jl.git"))
    Cloning git-repo `[email protected]:usmcamp0811/examplepkg.jl.git`
ERROR: failed to clone from [email protected]:usmcamp0811/examplepkg.jl.git, error: GitError(Code:ERROR, Class:SSH, error authenticating: no auth sock variable)
Stacktrace:
 [1] top-level scope at REPL[5]:1
 [2] run_repl(::REPL.AbstractREPL, ::Any) at /build/julia/src/julia-1.5.2/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:288
caused by [exception 1]
GitError(Code:ERROR, Class:SSH, error authenticating: no auth sock variable)
Stacktrace:
 [1] top-level scope at REPL[5]:1
 [2] run_repl(::REPL.AbstractREPL, ::Any) at /build/julia/src/julia-1.5.2/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:288

(@v1.5) pkg> add [email protected]:usmcamp0811/examplepkg.jl.git
    Cloning git-repo `[email protected]:usmcamp0811/examplepkg.jl.git`
ERROR: failed to clone from [email protected]:usmcamp0811/examplepkg.jl.git, error: GitError(Code:ERROR, Class:SSH, error authenticating: no auth sock variable)

melonwannajack · 2020-11-20T16:04:05Z

A work around I used for avoiding this behaviour was to eval the ssh-agent inside of the julia terminal. You can open a shell with ;, after this, you can execute the command you wont. In our case, the eval.

; eval $(ssh-agent)

StefanKarpinski · 2021-01-19T22:20:10Z

These issues may be addressed on master with the new NetworkOptions library that is part of Julia 1.6.

GlenHertz · 2021-01-26T13:39:52Z

I had this issue but was able to get it to work with ssh-agent:

eval $(ssh-agent)
ssh-add

I don't think it matters but I also have ForwardAgent yes in my ~/.ssh/config. I haven't tested the workaround on 1.5 but I was having the issue there too.

Also on 1.6-beta1 I had to add " quotes around the URL part:

add "[email protected]:JuliaLang/Statistics.jl.git"#master

I'm on Manjaro Linux. Hopefully that helps some others.

StefanKarpinski · 2021-01-29T15:40:23Z

The original issue here seems to be solved — I'm able to git clone via libgit2 using both encrypted and unencrypted SSH keys. The rest of the issues in this thread seem to be some combination of incorrect ssh-agent setups and needing to adjust the syntax for adding from a specific branch.

fredrikekre · 2021-10-20T15:38:50Z

A colleague had this issue (from the OP) and managed to fix it by converting the private key to PEM format with

ssh-keygen -p -N "" -m PEM -f path/to/private_key

cf #911 (comment) (posting for future me and for any other that might find this issue when searching).

fredrikekre transferred this issue from JuliaLang/julia Mar 24, 2020

frederikgeth mentioned this issue Apr 16, 2020

make sure package can be installed using the package manager frederikgeth/PowerModelsPrivacyPreserving.jl#2

Closed

NHDaly mentioned this issue Jul 30, 2020

Pkg doesn't seem to be able to follow custom Hosts in an ssh config #1933

Open

grahamas mentioned this issue Aug 8, 2020

Can't dev packages with [email protected]:... url #1044

Closed

StefanKarpinski closed this as completed Jan 29, 2021

MatthiasSachs mentioned this issue Jan 4, 2023

changed repo from ssh to https ACEsuit/ACEregistry#1

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pkg add private repo not using ssh key #1733

Pkg add private repo not using ssh key #1733

xximranxx commented Mar 24, 2020

StefanKarpinski commented Mar 24, 2020

xximranxx commented Mar 24, 2020

DilumAluthge commented Mar 24, 2020 •

edited

Loading

DilumAluthge commented Mar 24, 2020

imikim commented Apr 13, 2020

imikim commented Apr 14, 2020

imikim commented Apr 14, 2020

imikim commented Apr 14, 2020 •

edited

Loading

grahamas commented Aug 8, 2020 •

edited

Loading

EliseBcm commented Nov 2, 2020

StefanKarpinski commented Nov 2, 2020 •

edited

Loading

xximranxx commented Nov 2, 2020

usmcamp0811 commented Nov 8, 2020

melonwannajack commented Nov 20, 2020

StefanKarpinski commented Jan 19, 2021

GlenHertz commented Jan 26, 2021 •

edited

Loading

StefanKarpinski commented Jan 29, 2021

fredrikekre commented Oct 20, 2021

Pkg add private repo not using ssh key #1733

Pkg add private repo not using ssh key #1733

Comments

xximranxx commented Mar 24, 2020

StefanKarpinski commented Mar 24, 2020

xximranxx commented Mar 24, 2020

DilumAluthge commented Mar 24, 2020 • edited Loading

DilumAluthge commented Mar 24, 2020

imikim commented Apr 13, 2020

imikim commented Apr 14, 2020

imikim commented Apr 14, 2020

imikim commented Apr 14, 2020 • edited Loading

grahamas commented Aug 8, 2020 • edited Loading

EliseBcm commented Nov 2, 2020

StefanKarpinski commented Nov 2, 2020 • edited Loading

xximranxx commented Nov 2, 2020

usmcamp0811 commented Nov 8, 2020

melonwannajack commented Nov 20, 2020

StefanKarpinski commented Jan 19, 2021

GlenHertz commented Jan 26, 2021 • edited Loading

StefanKarpinski commented Jan 29, 2021

fredrikekre commented Oct 20, 2021

DilumAluthge commented Mar 24, 2020 •

edited

Loading

imikim commented Apr 14, 2020 •

edited

Loading

grahamas commented Aug 8, 2020 •

edited

Loading

StefanKarpinski commented Nov 2, 2020 •

edited

Loading

GlenHertz commented Jan 26, 2021 •

edited

Loading