Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System.DllNotFoundException: Unable to load shared library 'git2-8e0b172' #1125

Closed
s0dyy opened this issue Jun 27, 2019 · 15 comments
Closed
Labels
community-support Issues related to community bring up of distros. up-for-grabs

Comments

@s0dyy
Copy link

s0dyy commented Jun 27, 2019

  fail: Microsoft.DotNet.Darc.Operations.Operation[0]
        Something failed while cloning.
  System.Exception: Something went wrong when cloning repo https://github.com/dotnet/arcade at <default branch> into /home/s0dy/source-build/bin/src/arcade ---> System.TypeInitializationException: The type initializer for 'LibGit2Sharp.Core.NativeMethods' threw an exception. ---> System.DllNotFoundException: Unable to load shared library 'git2-8e0b172' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: libgit2-8e0b172: cannot open shared object file: No such file or directory
     at LibGit2Sharp.Core.NativeMethods.git_libgit2_init()
     at LibGit2Sharp.Core.NativeMethods.LoadNativeLibrary() in C:\projects\libgit2sharp\LibGit2Sharp\Core\NativeMethods.cs:line 47
     at LibGit2Sharp.Core.NativeMethods..cctor() in C:\projects\libgit2sharp\LibGit2Sharp\Core\NativeMethods.cs:line 37
     --- End of inner exception stack trace ---
     at LibGit2Sharp.Core.NativeMethods.git_clone(git_repository*& repo, String origin_url, FilePath workdir_path, GitCloneOptions& opts)
     at LibGit2Sharp.Core.Proxy.git_clone(String url, String workdir, GitCloneOptions& opts) in C:\projects\libgit2sharp\LibGit2Sharp\Core\Proxy.cs:line 354
     at LibGit2Sharp.Repository.Clone(String sourceUrl, String workdirPath, CloneOptions options) in C:\projects\libgit2sharp\LibGit2Sharp\Repository.cs:line 715
     at Microsoft.DotNet.DarcLib.RemoteRepoBase.Clone(String repoUri, String commit, String targetDirectory, ILogger _logger, String pat, String gitDirectory) in /home/s0dy/source-build/tools-local/arcade-services/src/Microsoft.DotNet.Darc/src/DarcLib/RemoteRepoBase.cs:line 176
     --- End of inner exception stack trace ---
     at Microsoft.DotNet.DarcLib.RemoteRepoBase.Clone(String repoUri, String commit, String targetDirectory, ILogger _logger, String pat, String gitDirectory) in /home/s0dy/source-build/tools-local/arcade-services/src/Microsoft.DotNet.Darc/src/DarcLib/RemoteRepoBase.cs:line 210
     at Microsoft.DotNet.DarcLib.GitHubClient.Clone(String repoUri, String commit, String targetDirectory, String gitDirectory) in /home/s0dy/source-build/tools-local/arcade-services/src/Microsoft.DotNet.Darc/src/DarcLib/GitHubClient.cs:line 875
     at Microsoft.DotNet.Darc.Operations.CloneOperation.HandleMasterCopyAndCreateGitDir(RemoteFactory remoteFactory, String repoUrl, String masterGitRepoPath, String masterRepoGitDirPath, String gitDirRedirect, ILogger log) in /home/s0dy/source-build/tools-local/arcade-services/src/Microsoft.DotNet.Darc/src/Darc/Operations/CloneOperation.cs:line 282
     at Microsoft.DotNet.Darc.Operations.CloneOperation.HandleMasterCopyWithGitDirPath(RemoteFactory remoteFactory, String repoUrl, String masterGitRepoPath, String masterRepoGitDirPath, ILogger log) in /home/s0dy/source-build/tools-local/arcade-services/src/Microsoft.DotNet.Darc/src/Darc/Operations/CloneOperation.cs:line 270
     at Microsoft.DotNet.Darc.Operations.CloneOperation.HandleMasterCopy(RemoteFactory remoteFactory, String repoUrl, String masterGitRepoPath, String masterRepoGitDirPath, ILogger log) in /home/s0dy/source-build/tools-local/arcade-services/src/Microsoft.DotNet.Darc/src/Darc/Operations/CloneOperation.cs:line 225
     at Microsoft.DotNet.Darc.Operations.CloneOperation.ExecuteAsync() in /home/s0dy/source-build/tools-local/arcade-services/src/Microsoft.DotNet.Darc/src/Darc/Operations/CloneOperation.cs:line 102
/home/s0dy/source-build/tools-local/init-build.proj(169,5): error MSB3073: The command "/home/s0dy/source-build/Tools/dotnetcli/dotnet /home/s0dy/source-build/tools-local/arcade-services/artifacts/bin/Microsoft.DotNet.Darc/Release/netcoreapp2.1/Microsoft.DotNet.Darc.dll clone --repos-folder /home/s0dy/source-build/bin/src/ --git-dir-folder /home/s0dy/source-build/.git/modules/src/ --include-toolset --ignore-repos ";https://dev.azure.com/dnceng/internal/_git/dotnet-optimization;https://dev.azure.com/devdiv/DevDiv/_git/DotNet-Trusted;https://devdiv.visualstudio.com/DevDiv/_git/DotNet-Trusted;https://[email protected]/dnceng/internal/_git/dotnet-optimization;https://dev.azure.com/dnceng/internal/_git/dotnet-core-setup" --azdev-pat bogus --github-pat bogus --depth 0 --debug" exited with code 42.

I am trying to make a package for Exherbo linux (since dotnet/source-build), I get an error because libgit2-8e0b172.so is linked to openssl-1.0

I realized that some libs were related to openssl-1.0 or 1.1 depending on the system. For example alpine (1.0)

s0dy ~/source-build lddtree packages/libgit2sharp.nativebinaries/1.0.235/runtimes/alpine-x64/native/libgit2-8e0b172.so

libgit2-8e0b172.so => packages/libgit2sharp.nativebinaries/1.0.235/runtimes/alpine-x64/native/libgit2-8e0b172.so (interpreter => none)
    libcurl.so.4 => /usr/host/lib/libcurl.so.4
        libssh2.so.1 => /usr/x86_64-pc-linux-gnu/lib/libssh2.so.1
        libssl.so.1.1 => /usr/x86_64-pc-linux-gnu/lib/libssl.so.1.1
        libcrypto.so.1.1 => /usr/x86_64-pc-linux-gnu/lib/libcrypto.so.1.1
            libdl.so.2 => /usr/host/lib/libdl.so.2
                ld-linux-x86-64.so.2 => /usr/host/lib/ld-linux-x86-64.so.2
        libpthread.so.0 => /usr/x86_64-pc-linux-gnu/lib/libpthread.so.0
        libc.so.6 => /usr/x86_64-pc-linux-gnu/lib/libc.so.6
    libssl.so.1.0.0 => not found
    libcrypto.so.1.0.0 => not found
    libz.so.1 => /usr/host/lib/libz.so.1
    libc.musl-x86_64.so.1 => not found

Debian (1.1)

s0dy ~/source-build ((HEAD detached at v3.0.100-preview6-012264-SDK)) lddtree packages/libgit2sharp.nativebinaries/1.0.235/runtimes/debian.9-x64/native/libgit2-8e0b172.so
libgit2-8e0b172.so => packages/libgit2sharp.nativebinaries/1.0.235/runtimes/debian.9-x64/native/libgit2-8e0b172.so (interpreter => none)
    librt.so.1 => /usr/host/lib/librt.so.1
    libpthread.so.0 => /usr/host/lib/libpthread.so.0
        ld-linux-x86-64.so.2 => /usr/host/lib/ld-linux-x86-64.so.2
    libcurl.so.4 => /usr/host/lib/libcurl.so.4
        libssh2.so.1 => /usr/x86_64-pc-linux-gnu/lib/libssh2.so.1
    libssl.so.1.1 => /usr/host/lib/libssl.so.1.1
    libcrypto.so.1.1 => /usr/host/lib/libcrypto.so.1.1
        libdl.so.2 => /usr/host/lib/libdl.so.2
    libz.so.1 => /usr/host/lib/libz.so.1
    libc.so.6 => /usr/host/lib/libc.so.6

To work around the problem, I copy libgit2-8e0b172.so (from the debian.9-x64 folder) to the linux-x64 folder

Is there a solution to have libgit2-8e0b172.so linked to openssl-1.1 directly via linux-x64?

For older versions, I could do a "git submodule update --init --recursive". How does it work now with Darc? Is it possible to download all the repositories before starting the compilation?

@s0dyy
Copy link
Author

s0dyy commented Jul 3, 2019

Any ideas ?

@markwilkie
Copy link
Member

@dagood or @crummel - is this something you guys could help shed some light on?

@dagood
Copy link
Member

dagood commented Jul 3, 2019

I'm not familiar with the new Darc workflow (I'm not active in source-build), but @crummel and @dseefeld who I'd normally ask about this aren't available to answer right now.

+cc @dleeapho @adaggarwal

You might have luck running build-source-tarball.sh on a different machine where it works, then building the resulting tarball on your target machine. This is the way to do an offline build, and might not even require git (but I don't know the current status).

I'm tentatively transferring this issue to source-build because it sounds more like a question about what the source-build infra supports than a direct bug report against Darc.

@dagood dagood transferred this issue from dotnet/arcade Jul 3, 2019
@omajid
Copy link
Member

omajid commented Jul 3, 2019

#1083 is the same issue, but on Fedora.

@dseefeld
Copy link
Contributor

This may potentially require a Excerbo specific fix analogous to #1083

@omajid
Copy link
Member

omajid commented Aug 22, 2019

@dseefeld The way I understand this issue is that arcade-services wants to do some git operations and makes use of libgit2sharp. libgit2sharp is based on the unmanaged library libgit2 which is obtained and used from nuget. The nuget package (obviously) doesn't support every version of every platform (including all the various linux distributions) on every architecture.

I think source-build/arcade-services should be more flexible here.

I see two possible options that might help:

  1. Let an OS (linux distro, in this case) supply a libgit2 and arcade-services should make use of that (in addition to supporting getting libgit2 from nuget.org for common cases)

  2. arcade-services should do what SourceLink does and work with git directly

What do you think?

@wfurt
Copy link
Member

wfurt commented Aug 22, 2019

There is now version without dependency on libgit2.
I think it would be great to get that in and avoid external dependency.

@omajid
Copy link
Member

omajid commented Aug 22, 2019

@wfurt Wow, that's great news! Do you know where this version is? I still see a dependency on LibGit2Sharp in arcade-services here: https://github.com/dotnet/arcade-services/blob/7bcd17b24aec5101e89723f31a748ed1d2385268/src/Microsoft.DotNet.Darc/src/DarcLib/Microsoft.DotNet.DarcLib.csproj#L14

@wfurt
Copy link
Member

wfurt commented Aug 22, 2019

Take a look at dotnet/sourcelink#288
I don't have any practical experience but I do see a hope.

@omajid
Copy link
Member

omajid commented Aug 22, 2019

@wfurt That's sourcelink, though. It's what I was referring to in option 2 above where I said arcade-services should do what sourcelink does. Do arcade-services and sourcelink share code or codebases? I might take a stab at merging the changes if that's the case.

@wfurt
Copy link
Member

wfurt commented Aug 22, 2019

I don't know. I'm just hoping we can remove dependency on libgit2 instead of trying to fix it for all distributions,

@adaggarwal
Copy link
Member

Related - #1242

@adaggarwal adaggarwal added the community-support Issues related to community bring up of distros. label Sep 27, 2019
@omajid
Copy link
Member

omajid commented Oct 9, 2019

@tmds's fix (libgit2/libgit2sharp#1714) works fine for me.

@omajid
Copy link
Member

omajid commented Nov 1, 2019

dotnet/arcade-services#694 fixes this issue for me. What is the process for bumping our arcade-services submodule ot point to that?

@adaggarwal
Copy link
Member

This is fixed with darc install and update to netcoreapp3.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
community-support Issues related to community bring up of distros. up-for-grabs
Projects
None yet
Development

No branches or pull requests

7 participants