Error in call to get_addable_spec

[eschnett]

When I build openPMD_api, I receive this error message:

┌ Info: Using cached git repository
│   url = "https://github.com/JuliaBinaryWrappers/LibUnwind_jll.jl.git"
└   repo_path = "/Users/eschnett/.julia/packages/BinaryBuilderBase/UqKXf/deps/downloads/clones/LibUnwind_jll.jl.git-d8b39dc8c012241aadec63e3c299a665a34a25040957a23fa92488b6e722d36b"
┌ Info: Using cached git repository
│   url = "https://github.com/JuliaBinaryWrappers/CompilerSupportLibraries_jll.jl.git"
└   repo_path = "/Users/eschnett/.julia/packages/BinaryBuilderBase/UqKXf/deps/downloads/clones/CompilerSupportLibraries_jll.jl.git-01aac636917b99fb66a09babd9a4f76c096f224f98af708089ea3b6adfb00fae"
ERROR: LoadError: MethodError: no method matching get_addable_spec(::String, ::Nothing; verbose=true)
Closest candidates are:
  get_addable_spec(::AbstractString, ::VersionNumber; ctx, verbose) at ~/.julia/packages/BinaryBuilderBase/UqKXf/src/Prefix.jl:426
Stacktrace:
 [1] (::BinaryBuilderBase.var"#88#94"{Bool, Prefix, Vector{Pkg.Types.PackageSpec}, Platform, Vector{String}, Vector{String}})()
   @ BinaryBuilderBase ~/.julia/packages/BinaryBuilderBase/UqKXf/src/Prefix.jl:602
 [2] activate(f::BinaryBuilderBase.var"#88#94"{Bool, Prefix, Vector{Pkg.Types.PackageSpec}, Platform, Vector{String}, Vector{String}}, new_project::String)
   @ Pkg.API ~/julia-1.7/share/julia/stdlib/v1.7/Pkg/src/API.jl:1599
 [3] setup_dependencies(prefix::Prefix, dependencies::Vector{Pkg.Types.PackageSpec}, platform::Platform; verbose::Bool)
   @ BinaryBuilderBase ~/.julia/packages/BinaryBuilderBase/UqKXf/src/Prefix.jl:561
 [4] (::BinaryBuilder.var"#setup_deps#27")(f::typeof(BinaryBuilderBase.is_target_dependency), prefix::Prefix, dependencies::Vector{BinaryBuilderBase.AbstractDependency}, platform::Platform, verbose::Bool)
   @ BinaryBuilder ~/.julia/packages/BinaryBuilder/Ftk2f/src/AutoBuild.jl:754
 [5] autobuild(dir::AbstractString, src_name::AbstractString, src_version::VersionNumber, sources::Vector{<:BinaryBuilderBase.AbstractSource}, script::AbstractString, platforms::Vector, products::Vector{<:Product}, dependencies::Vector{<:BinaryBuilderBase.AbstractDependency}; verbose::Bool, debug::Bool, skip_audit::Bool, ignore_audit_errors::Bool, autofix::Bool, code_dir::Union{Nothing, String}, require_license::Bool, kwargs::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}})
   @ BinaryBuilder ~/.julia/packages/BinaryBuilder/Ftk2f/src/AutoBuild.jl:757
 [6] build_tarballs(ARGS::Any, src_name::Any, src_version::Any, sources::Any, script::Any, platforms::Any, products::Any, dependencies::Any; julia_compat::String, kwargs::Base.Pairs{Symbol, V, Tuple{Vararg{Symbol, N}}, NamedTuple{names, T}} where {V, N, names, T<:Tuple{Vararg{Any, N}}})
   @ BinaryBuilder ~/.julia/packages/BinaryBuilder/Ftk2f/src/AutoBuild.jl:321
 [7] top-level scope
   @ ~/src/Yggdrasil/O/openPMD_api/build_tarballs.jl:99
in expression starting at /Users/eschnett/src/Yggdrasil/O/openPMD_api/build_tarballs.jl:99

My build command is

~/julia-1.7/bin/julia --color=yes build_tarballs.jl --debug --verbose --deploy=local x86_64-apple-darwin-julia_version+1.7.0

I am currently building things locally, with some local changes (updated packages versions), before opening a pull request. Since I am building multiple packages I install them locally, and I'm modifying my dependencies to point to these versions. In particular, the ADIOS2 dependency now is

    Dependency(PackageSpec(name="ADIOS2_jll", path="/Users/eschnett/.julia/dev/ADIOS2_jll");
               platforms=filter(p -> nbits(p) ≠ 32, platforms)),

which points to the result of a previous build_tarballs.jl.

Through trial and error I found that the problem is caused by this ADIOS2 dependency. When I omit this dependency, the error goes away (but obviously the created package would be missing ADIOS2 support).

Looking at the error message, the problem might be that I built ADIOS2 for the architecture x86_64-apple-darwin (without an explicit Julia version number), while openPMD_api is being built for x86_64-apple-darwin-julia_version+1.7.0, with an explicit version number.

[eschnett]

I am building via Julia 1.7, with these package installed:

  [12aac903] BinaryBuilder v0.5.2
  [7f725544] BinaryBuilderBase v1.6.0

[giordano]

I guess that's JuliaLang/Pkg.jl#2942. See

Yggdrasil/L/libcxxwrap_julia/build_tarballs.jl

Lines 5 to 8 in d8235f7

	# See https://github.com/JuliaLang/Pkg.jl/issues/2942
	# Once this Pkg issue is resolved, this must be removed
	uuid = Base.UUID("a83860b7-747b-57cf-bf1f-3e79990d037f")
	delete!(Pkg.Types.get_last_stdlibs(v"1.6.3"), uuid)

[eschnett]

I assume that your comment is just a description of the cause of the error... Or is this a suggestion for a work-around?

[giordano]

That was a suggestion for a workaround, any package requiring libjulia has the same problem.

[eschnett]

I tried it, and it didn't work. The error message doesn't change.

Do I need to use a particular version of Julia to run this, or particular version of BinaryBuilder, or generated code for a particular version of Julia? I'm running with Julia 1.7, building for Julia 1.7, and am using the newest released BinaryBuilder.

[giordano]

Uhm, building the current openPMD_api with

julia build_tarballs.jl --debug --verbose x86_64-apple-darwin-julia_version+1.7.0

actually works for me even without the hack mentioned above. Environment:

(@v1.7) pkg> status BinaryBuilder BinaryBuilderBase
      Status `~/.julia/environments/v1.7/Project.toml`
  [12aac903] BinaryBuilder v0.5.2 `~/.julia/dev/BinaryBuilder`
  [7f725544] BinaryBuilderBase v1.6.0 `~/.julia/dev/BinaryBuilderBase`

Perhaps make sure to use latest version of Yggdrasil?

[eschnett]

After some more experiment I find:

• I can install openPMD_api when I only depend on package in the registry
• Things fail when I depend on a locally installed package.

For example, when I replace the dependency on MPICH_jll by MPItramoline_jll, everything is fine. However, when I replace it by

    Dependency(PackageSpec(name="MPItrampoline_jll", path="/Users/eschnett/.julia/dev/MPItrampoline_jll")),

then I receive the respective error message. (I created the respective package via BinaryBuilder and the --deploy=local option just before.)

[simeonschaub]

I managed to work around this by manually specifying the tree_hash as well. It looks like BinaryBuilder thinks this is a stdlib and then gets confused because it can't find the version number.

[giordano]

So the problem is https://github.com/JuliaPackaging/BinaryBuilderBase.jl/blob/bcd6bbbef57d4eba99de371848476005ddc17773/src/Prefix.jl#L613 right?

[simeonschaub]

Yes, at least as far is I understand it

[giordano]

In the meantime I've got a simpler reproducer which would be a good base for a test once we find a fix:

using BinaryBuilderBase: Dependency, HostPlatform, Prefix, cached_git_clone, getpkg, setup_dependencies
using Pkg: PackageSpec
using LibGit2

mktempdir() do prefix_dir
    mktempdir() do pkgdir
        verbose = true
        prefix = Prefix(prefix_dir)
        cache_dir = cached_git_clone("https://github.com/JuliaBinaryWrappers/HelloWorldC_jll.jl"; verbose)
        LibGit2.with(LibGit2.clone(cache_dir, pkgdir)) do repo
            LibGit2.checkout!(repo, "c7f2e95d9c04e218931c14954ecd31ebde72cca5")
        end
        dependencies = [PackageSpec(; name="HelloWorldC_jll", uuid="dca1746e-5efc-54fc-8249-22745bc95a49", path=pkgdir)]
        setup_dependencies(prefix, dependencies, HostPlatform())
    end
end