proposing a version bump of libcurl to 7.81

[ngam]

I actually find it quite counter productive to bring up curl on this issue. A more productive conversation would be had in a separate issue requesting or proposing a version bump to of libcurl to 7.81, where the matter could be tracked and discovered by others.

Originally posted by @mkitti in #43666 (comment)

[ngam]

Updating to latest libcurl leads to a segfault error. See issues discovered downstream (in unsupported builds) as soon as updating libcurl. This is both a heads-up and a proposal to update libcurl to the latest version and resolving these issues. Thank you all for the good work!

• pacman julia crashes with segmentation fault on ]up #43682
• cleanup v1.6 LTS conda-forge/julia-feedstock#159 (comment)
• Incompatibility between latest julia=1.7.1 and latest openlibm=0.8.0 on conda-forge conda-forge/julia-feedstock#169 (comment)

[mkitti]

I just bisected the segmentation fault to curl/curl@2b3dd01

git bisect start 'curl-7_81_0' 'curl-7_80_0'
# bad: [45c6448e68da77d9d553bc221c449dcb3b4ad49f] test1556: adjust for hyper
git bisect bad 45c6448e68da77d9d553bc221c449dcb3b4ad49f
# good: [45bcb2eaa78c790966f678c17482e70cca6593db] urlapi: cleanup scheme parsing
git bisect good 45bcb2eaa78c790966f678c17482e70cca6593db
# good: [470d546048e3cf88158a522204397956f3ee26d8] docs: Update the Reducing Size section
git bisect good 470d546048e3cf88158a522204397956f3ee26d8
# good: [a82fe5d2d06fed20873ecdc2b9eaff85b8a917c6] tool_listhelp: sync
git bisect good a82fe5d2d06fed20873ecdc2b9eaff85b8a917c6
# good: [b3f9c1aa09cbcdf6350d627f79f7d0e86cb178d3] http2:set_transfer_url() return early on OOM
git bisect good b3f9c1aa09cbcdf6350d627f79f7d0e86cb178d3
# bad: [05f899c842a4c7f0b2367ecdda0c3ca0694c8eba] TODO: 5.7 Require HTTP version X or higher
git bisect bad 05f899c842a4c7f0b2367ecdda0c3ca0694c8eba
# bad: [1a4c14cbac9a29e7a943846cfa59cb404556961e] CURLOPT_STDERR.3: does not work with libcurl as a win32 DLL
git bisect bad 1a4c14cbac9a29e7a943846cfa59cb404556961e
# bad: [2b3dd01b779e4eff623a735792a915cde914b74a] multi: handle errors returned from socket/timer callbacks
git bisect bad 2b3dd01b779e4eff623a735792a915cde914b74a
# first bad commit: [2b3dd01b779e4eff623a735792a915cde914b74a] multi: handle errors returned from socket/timer callbacks

[mkitti]

This is when configuring CURL as follows, which differs from julia's supported configuration.

./configure     --prefix=${PREFIX}     --disable-ldap     --with-ca-bundle=${PREFIX}/ssl/cacert.pem     --with-ssl=${PREFIX}     --with-zlib=${PREFIX}     --with-gssapi=${PREFIX}     --with-libssh2=${PREFIX} --enable-debug

$ julia -E 'using LibCURL; unsafe_string(LibCURL.curl_version())'
"libcurl/7.81.0-DEV OpenSSL/3.0.0 zlib/1.2.11 libssh2/1.10.0 nghttp2/1.43.0"

 julia -E 'download("https://curl.se")'

signal (11): Segmentation fault
in expression starting at none:1
prune_dead_connections at /home/mkitti/src/curl/lib/url.c:1081
create_conn at /home/mkitti/src/curl/lib/url.c:3859
Curl_connect at /home/mkitti/src/curl/lib/url.c:4119
multi_runsingle at /home/mkitti/src/curl/lib/multi.c:1844
multi_socket at /home/mkitti/src/curl/lib/multi.c:3115
curl_multi_socket_action at /home/mkitti/src/curl/lib/multi.c:3236
curl_multi_socket_action at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/LibCURL/src/lC_curl_h.jl:230 [inlined]
curl_multi_socket_action at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/utils.jl:91 [inlined]
macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/utils.jl:35 [inlined]
#47 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/Multi.jl:147
lock at ./lock.jl:190
timer_callback at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/Multi.jl:146
jfptr_timer_callback_19000.clone_1 at /home/mkitti/anaconda3/envs/curltest/lib/julia/sys.so (unknown line)
_jl_invoke at /usr/local/src/conda/julia-1.7.1/src/gf.c:2247 [inlined]
jl_apply_generic at /usr/local/src/conda/julia-1.7.1/src/gf.c:2429
jlcapi_timer_callback_18861.clone_1 at /home/mkitti/anaconda3/envs/curltest/lib/julia/sys.so (unknown line)
Curl_update_timer at /home/mkitti/src/curl/lib/multi.c:3349
curl_multi_add_handle at /home/mkitti/src/curl/lib/multi.c:489
curl_multi_add_handle at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/LibCURL/src/lC_curl_h.jl:194 [inlined]
macro expansion at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/utils.jl:35 [inlined]
#27 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/Multi.jl:51
lock at ./lock.jl:190
add_handle at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/Multi.jl:44 [inlined]
#9 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:345
with_handle at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Curl/Curl.jl:64
#8 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:311 [inlined]
arg_write at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/ArgTools/src/ArgTools.jl:112
#7 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:310 [inlined]
arg_read at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/ArgTools/src/ArgTools.jl:61 [inlined]
#request#5 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:309
request##kw at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:293 [inlined]
#3 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:222 [inlined]
arg_write at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/ArgTools/src/ArgTools.jl:101
#download#2 at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:221 [inlined]
download at /home/conda/feedstock_root/build_artifacts/julia_1641439672340/work/usr/share/julia/stdlib/v1.7/Downloads/src/Downloads.jl:221
_jl_invoke at /usr/local/src/conda/julia-1.7.1/src/gf.c:2247 [inlined]
jl_apply_generic at /usr/local/src/conda/julia-1.7.1/src/gf.c:2429
jl_apply at /usr/local/src/conda/julia-1.7.1/src/julia.h:1788 [inlined]
jl_f__call_latest at /usr/local/src/conda/julia-1.7.1/src/builtins.c:757
#invokelatest#2 at ./essentials.jl:716 [inlined]
invokelatest at ./essentials.jl:714 [inlined]
do_download at ./download.jl:24 [inlined]
download at ./download.jl:20
_jl_invoke at /usr/local/src/conda/julia-1.7.1/src/gf.c:2247 [inlined]
jl_apply_generic at /usr/local/src/conda/julia-1.7.1/src/gf.c:2429
jl_apply at /usr/local/src/conda/julia-1.7.1/src/julia.h:1788 [inlined]
do_call at /usr/local/src/conda/julia-1.7.1/src/interpreter.c:126
eval_value at /usr/local/src/conda/julia-1.7.1/src/interpreter.c:215
eval_stmt_value at /usr/local/src/conda/julia-1.7.1/src/interpreter.c:166 [inlined]
eval_body at /usr/local/src/conda/julia-1.7.1/src/interpreter.c:601
jl_interpret_toplevel_thunk at /usr/local/src/conda/julia-1.7.1/src/interpreter.c:731
jl_toplevel_eval_flex at /usr/local/src/conda/julia-1.7.1/src/toplevel.c:885
jl_toplevel_eval_flex at /usr/local/src/conda/julia-1.7.1/src/toplevel.c:830
jl_toplevel_eval_in at /usr/local/src/conda/julia-1.7.1/src/toplevel.c:944
eval at ./boot.jl:373 [inlined]
exec_options at ./client.jl:270
_start at ./client.jl:495
jfptr__start_40346.clone_1 at /home/mkitti/anaconda3/envs/curltest/lib/julia/sys.so (unknown line)
_jl_invoke at /usr/local/src/conda/julia-1.7.1/src/gf.c:2247 [inlined]
jl_apply_generic at /usr/local/src/conda/julia-1.7.1/src/gf.c:2429
jl_apply at /usr/local/src/conda/julia-1.7.1/src/julia.h:1788 [inlined]
true_main at /usr/local/src/conda/julia-1.7.1/src/jlapi.c:559
jl_repl_entrypoint at /usr/local/src/conda/julia-1.7.1/src/jlapi.c:701
main at julia (unknown line)
__libc_start_main at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x55c7ea807098)
Allocations: 2721 (Pool: 2712; Big: 9); GC: 0
Segmentation fault (core dumped)

[ngam]

differs from julia's supported configuration

only with curl 7.81.0 or are there other differences?

[ngam]

Do you have any idea how this commit curl/curl@2b3dd01 is causing the segmentation fault here?

[mkitti]

I believe JuliaLang/Downloads.jl#164 may have resolved it.