Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

try PROJ 7 binaries #41

Merged
merged 1 commit into from
May 3, 2020
Merged

try PROJ 7 binaries #41

merged 1 commit into from
May 3, 2020

Conversation

visr
Copy link
Member

@visr visr commented May 2, 2020

Reading the PROJ 7 breaking changes at https://proj.org/news.html#breaking-changes, it looks like none of them really apply to this package, meaning we could possibly do a non breaking release of this package that allows PROJ 7.

This would probably make migration to a new package based around the new API easier, if they can use the same binary, ref #26 (comment).

Locally on Windows this didn't pass yet due to an issue with the newly added libtiff dependency:

ERROR: LoadError: LoadError: InitError: could not load library "C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll"
The specified module could not be found.
Stacktrace:
 [1] dlopen(::String, ::UInt32; throw_error::Bool) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109
 [2] dlopen at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109 [inlined] (repeats 2 times)
 [3] __init__() at C:\Users\visser_mn\.julia\packages\Libtiff_jll\0kDRi\src\wrappers\x86_64-w64-mingw32.jl:45
 [4] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:697
 [5] _require_search_from_serialized(::Base.PkgId, ::String) at .\loading.jl:781
 [6] _require(::Base.PkgId) at .\loading.jl:1006
 [7] require(::Base.PkgId) at .\loading.jl:927
 [8] require(::Module, ::Symbol) at .\loading.jl:922
 [9] include(::Module, ::String) at .\Base.jl:377
 [10] include(::String) at C:\Users\visser_mn\.julia\packages\PROJ_jll\ZeTfD\src\PROJ_jll.jl:1
 [11] top-level scope at C:\Users\visser_mn\.julia\packages\PROJ_jll\ZeTfD\src\PROJ_jll.jl:50
 [12] include(::Module, ::String) at .\Base.jl:377
 [13] top-level scope at none:2
 [14] eval at .\boot.jl:331 [inlined]
 [15] eval(::Expr) at .\client.jl:449
 [16] top-level scope at .\none:3
during initialization of module Libtiff_jll

But opening this PR as a draft to check the situation on other platforms.

visr added a commit to visr/Yggdrasil that referenced this pull request May 2, 2020
Directly bumps to the latest 4.1.0 release from November 2019.

It seems the current libtiff build is broken on Windows, possibly since JuliaPackaging#961.

```
julia> using Libtiff_jll
ERROR: InitError: could not load library "C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll"
The specified module could not be found.
Stacktrace:
 [1] dlopen(::String, ::UInt32; throw_error::Bool) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109
 [2] dlopen at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109 [inlined] (repeats 2 times)
 [3] __init__() at C:\Users\visser_mn\.julia\packages\Libtiff_jll\0kDRi\src\wrappers\x86_64-w64-mingw32.jl:45
 [4] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:697
 [5] _require_search_from_serialized(::Base.PkgId, ::String) at .\loading.jl:781
 [6] _require(::Base.PkgId) at .\loading.jl:1006
 [7] require(::Base.PkgId) at .\loading.jl:927
 [8] require(::Module, ::Symbol) at .\loading.jl:922
 [9] eval(::Module, ::Any) at .\boot.jl:331
 [10] eval_user_input(::Any, ::REPL.REPLBackend) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:86
 [11] run_backend(::REPL.REPLBackend) at C:\Users\visser_mn\.julia\packages\Revise\WkyNB\src\Revise.jl:1023
 [12] top-level scope at none:0
during initialization of module Libtiff_jll
```

I tried to debug using DependencyWalker (System32 dlls left out below):

```julia
julia> using DependencyWalker
[ Info: Precompiling DependencyWalker [7db386b5-e858-4964-8eb4-2fa66f8df882]

julia> Library(raw"C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll")
◼ C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll
  ◼ C:\Users\visser_mn\.julia\artifacts\217f5fb6408fcad8ec290be23f14646aab4e53b0\bin\libjpeg-62.dll
  ✗ libzstd.dll (NOT FOUND)
  ◼ C:\Users\visser_mn\.julia\artifacts\12dda53f058e2ad8360473e1df8d31d709724a38\bin\libz.dll
```

So it cannot find `libzstd.dll`. It appears to me the reason is that before it was not versioned, but now it is:

```
.julia\artifacts\2c987d0ee933c3900ce3f148b81d7f7c9c704c56\bin\libzstd.dll
BinaryBuilder.jl version: 0.2.2-888d6f2c4e049b9a277e79ddbfb2d50007ffa5a8

.julia\artifacts\eb77caa0e89a4b205fc8393d5aca496bfb13ea4e\bin\libzstd-1.dll
BinaryBuilder.jl version: 0.2.4-tree-e953eddbb5
```

Looking at the log of the latest Libtiff build, it seems to pick it up however:
```
Libtiff is now configured for x86_64-w64-mingw32
  ZSTD support:                       yes
```

But I guess that was built with the old unversioned `libzstd`? So perhaps we simply need to rebuild `libtiff`.
I guess we cannot use the package manager to require `Zstd-v1.4.4+1`.

From https://juliapackages.com/p/zstd_jll it seems the only other user of `libstd` is `Blosc_jll`, so perhaps that needs a rebuild as well.

__Links__
https://github.com/JuliaBinaryWrappers/Libtiff_jll.jl
https://github.com/JuliaBinaryWrappers/Zstd_jll.jl
https://github.com/JuliaBinaryWrappers/Blosc_jll.jl
JuliaGeo/Proj.jl#41
http://www.libtiff.org/build.html
giordano pushed a commit to JuliaPackaging/Yggdrasil that referenced this pull request May 2, 2020
Directly bumps to the latest 4.1.0 release from November 2019.

It seems the current libtiff build is broken on Windows, possibly since #961.

```
julia> using Libtiff_jll
ERROR: InitError: could not load library "C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll"
The specified module could not be found.
Stacktrace:
 [1] dlopen(::String, ::UInt32; throw_error::Bool) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109
 [2] dlopen at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Libdl\src\Libdl.jl:109 [inlined] (repeats 2 times)
 [3] __init__() at C:\Users\visser_mn\.julia\packages\Libtiff_jll\0kDRi\src\wrappers\x86_64-w64-mingw32.jl:45
 [4] _include_from_serialized(::String, ::Array{Any,1}) at .\loading.jl:697
 [5] _require_search_from_serialized(::Base.PkgId, ::String) at .\loading.jl:781
 [6] _require(::Base.PkgId) at .\loading.jl:1006
 [7] require(::Base.PkgId) at .\loading.jl:927
 [8] require(::Module, ::Symbol) at .\loading.jl:922
 [9] eval(::Module, ::Any) at .\boot.jl:331
 [10] eval_user_input(::Any, ::REPL.REPLBackend) at D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\REPL\src\REPL.jl:86
 [11] run_backend(::REPL.REPLBackend) at C:\Users\visser_mn\.julia\packages\Revise\WkyNB\src\Revise.jl:1023
 [12] top-level scope at none:0
during initialization of module Libtiff_jll
```

I tried to debug using DependencyWalker (System32 dlls left out below):

```julia
julia> using DependencyWalker
[ Info: Precompiling DependencyWalker [7db386b5-e858-4964-8eb4-2fa66f8df882]

julia> Library(raw"C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll")
◼ C:\Users\visser_mn\.julia\artifacts\f98adb5b4de5c29a8d77def412b792beaec6180b\bin\libtiff-5.dll
  ◼ C:\Users\visser_mn\.julia\artifacts\217f5fb6408fcad8ec290be23f14646aab4e53b0\bin\libjpeg-62.dll
  ✗ libzstd.dll (NOT FOUND)
  ◼ C:\Users\visser_mn\.julia\artifacts\12dda53f058e2ad8360473e1df8d31d709724a38\bin\libz.dll
```

So it cannot find `libzstd.dll`. It appears to me the reason is that before it was not versioned, but now it is:

```
.julia\artifacts\2c987d0ee933c3900ce3f148b81d7f7c9c704c56\bin\libzstd.dll
BinaryBuilder.jl version: 0.2.2-888d6f2c4e049b9a277e79ddbfb2d50007ffa5a8

.julia\artifacts\eb77caa0e89a4b205fc8393d5aca496bfb13ea4e\bin\libzstd-1.dll
BinaryBuilder.jl version: 0.2.4-tree-e953eddbb5
```

Looking at the log of the latest Libtiff build, it seems to pick it up however:
```
Libtiff is now configured for x86_64-w64-mingw32
  ZSTD support:                       yes
```

But I guess that was built with the old unversioned `libzstd`? So perhaps we simply need to rebuild `libtiff`.
I guess we cannot use the package manager to require `Zstd-v1.4.4+1`.

From https://juliapackages.com/p/zstd_jll it seems the only other user of `libstd` is `Blosc_jll`, so perhaps that needs a rebuild as well.

__Links__
https://github.com/JuliaBinaryWrappers/Libtiff_jll.jl
https://github.com/JuliaBinaryWrappers/Zstd_jll.jl
https://github.com/JuliaBinaryWrappers/Blosc_jll.jl
JuliaGeo/Proj.jl#41
http://www.libtiff.org/build.html
@visr
Copy link
Member Author

visr commented May 2, 2020

This is now working after JuliaPackaging/Yggdrasil#970. I would propose merging this into this package, since it does not appear to be breaking for us.

@visr visr marked this pull request as ready for review May 2, 2020 16:18
@c42f
Copy link
Member

c42f commented May 3, 2020

Looks good to me.

@visr visr merged commit bb485bb into master May 3, 2020
@visr visr deleted the proj7 branch May 4, 2020 12:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants