Fix support for outbound SSL w/ SSL1.1.x

[andrewbenton]

Without this patch, there are compilation errors with undefined
references when trying to seen HTTPS connections when using
OpenSSL1.1.x. You can duplicate this issue by trying
requestHTTP("https://google.com").toString.writeln;

I've seen this issue occur and be fixed with this patch on both
Fedora 26 and 27.

[s-ludwig]

Is this actually a fix against master/0.8.2-rc.1 or against 0.8.1? The mangling of both signatures should be the same, so it's currently still unclear to me why this would fix a linker error. 0.8.1 on the other hand had these functions defined as aliases, which indeed resulted in unresolved symbol errors.

[andrewbenton]

I see the same failure signature for both 0.8.2 and 0.8.2-rc.1

[s-ludwig]

Hm, interesting. Can you post the error message? It would be interesting to know what exactly differs from the linker's point of view (I don't have a Fedora installation available ATM, so I can't reproduce it).

Thanks for the fix. Hopefully this will finally get all current systems to work (and not break some other system). In the long run we should set up a Docker based Fedora/CentOS test on Travis-CI to better catch these distribution specific differences.

[andrewbenton]

Using 0.8.1-rc.1 (locally tagged master as 0.8.2):

../.dub/build/vibe-d-0.8.2/openssl-debug-linux.posix-x86_64-ldc_2075-B5F8B19F330CD9B9E7E9C85D76249E48/libvibe-d_tls.a(vibe.stream.openssl.o): In function `_D4vibe6stream7openssl14OpenSSLContext6__ctorMFNfE4vibe6stream3tls14TLSContextKindE4vibe6stream3tls10TLSVersionZ9__lambda3MFNbNeZv':
/tmp/hsandbox-6DXfNv/../../home/andrew/proj/vibe.d/tls/vibe/stream/openssl.d:467: undefined reference to `SSLv23_client_method'
/tmp/hsandbox-6DXfNv/../../home/andrew/proj/vibe.d/tls/vibe/stream/openssl.d:468: undefined reference to `SSLv23_client_method'
/tmp/hsandbox-6DXfNv/../../home/andrew/proj/vibe.d/tls/vibe/stream/openssl.d:472: undefined reference to `SSLv23_client_method'
/tmp/hsandbox-6DXfNv/../../home/andrew/proj/vibe.d/tls/vibe/stream/openssl.d:473: undefined reference to `SSLv23_client_method'
/tmp/hsandbox-6DXfNv/../../home/andrew/proj/vibe.d/tls/vibe/stream/openssl.d:480: undefined reference to `SSLv23_server_method'
/tmp/hsandbox-6DXfNv/../../home/andrew/proj/vibe.d/tls/vibe/stream/openssl.d:481: undefined reference to `SSLv23_server_method'
/tmp/hsandbox-6DXfNv/../../home/andrew/proj/vibe.d/tls/vibe/stream/openssl.d:483: undefined reference to `SSLv23_server_method'
/tmp/hsandbox-6DXfNv/../../home/andrew/proj/vibe.d/tls/vibe/stream/openssl.d:484: undefined reference to `SSLv23_server_method'
../.dub/build/vibe-d-0.8.2/openssl-debug-linux.posix-x86_64-ldc_2075-B5F8B19F330CD9B9E7E9C85D76249E48/libvibe-d_tls.a(vibe.stream.openssl.o): In function `_D4vibe6stream7openssl14OpenSSLContext11setDHParamsMFNeAyaZv':
/tmp/hsandbox-6DXfNv/../../home/andrew/proj/vibe.d/tls/vibe/stream/openssl.d:722: undefined reference to `get_rfc3526_prime_2048'
../.dub/build/vibe-d-0.8.2/openssl-debug-linux.posix-x86_64-ldc_2075-B5F8B19F330CD9B9E7E9C85D76249E48/libvibe-d_tls.a(vibe.stream.openssl.o): In function `_D4vibe6stream7openssl18_sharedStaticCtor1FZv':
/tmp/hsandbox-6DXfNv/../../home/andrew/proj/vibe.d/tls/vibe/stream/openssl.d:911: undefined reference to `SSL_load_error_strings'
/tmp/hsandbox-6DXfNv/../../home/andrew/proj/vibe.d/tls/vibe/stream/openssl.d:912: undefined reference to `SSL_library_init'
/tmp/hsandbox-6DXfNv/../../home/andrew/proj/vibe.d/tls/vibe/stream/openssl.d:914: undefined reference to `CRYPTO_num_locks'
/tmp/hsandbox-6DXfNv/../../home/andrew/proj/vibe.d/tls/vibe/stream/openssl.d:922: undefined reference to `CRYPTO_set_id_callback'
/tmp/hsandbox-6DXfNv/../../home/andrew/proj/vibe.d/tls/vibe/stream/openssl.d:923: undefined reference to `CRYPTO_set_locking_callback'
/tmp/hsandbox-6DXfNv/../../home/andrew/proj/vibe.d/tls/vibe/stream/openssl.d:928: undefined reference to `SSL_get_ex_new_index'
../.dub/build/vibe-d-0.8.2/openssl-debug-linux.posix-x86_64-ldc_2075-B5F8B19F330CD9B9E7E9C85D76249E48/libvibe-d_tls.a(vibe.stream.openssl.o): In function `_D6deimos7openssl9safestack57__T10SKM_sk_numTS6deimos7openssl6x509v315GENERAL_NAME_stZ16__T10SKM_sk_numZ10SKM_sk_numFNbPS6deimos7openssl9safestack54__T8STACK_OFTS6deimos7openssl6x509v315GENERAL_NAME_stZ8STACK_OFZi':
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/openssl-1.1.6_1.0.1g/openssl/deimos/openssl/safestack.d:140: undefined reference to `sk_num'
../.dub/build/vibe-d-0.8.2/openssl-debug-linux.posix-x86_64-ldc_2075-B5F8B19F330CD9B9E7E9C85D76249E48/libvibe-d_tls.a(vibe.stream.openssl.o): In function `_D6deimos7openssl9safestack59__T12SKM_sk_valueTS6deimos7openssl6x509v315GENERAL_NAME_stZ18__T12SKM_sk_valueZ12SKM_sk_valueFNbPS6deimos7openssl9safestack54__T8STACK_OFTS6deimos7openssl6x509v315GENERAL_NAME_stZ8STACK_OFiZPS6deimos7openssl6x509v315GENERAL_NAME_st':
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/openssl-1.1.6_1.0.1g/openssl/deimos/openssl/safestack.d:142: undefined reference to `sk_value'
collect2: error: ld returned 1 exit status
Error: /usr/bin/gcc failed with status: 1
ldc2 failed with exit code 1.

using 0.8.1:

../.dub/build/vibe-d-0.8.1/openssl-debug-linux.posix-x86_64-ldc_2075-B5F8B19F330CD9B9E7E9C85D76249E48/libvibe-d_tls.a(vibe.stream.openssl.o): In function `_D4vibe6stream7openssl14OpenSSLContext6__ctorMFNfE4vibe6stream3tls14TLSContextKindE4vibe6stream3tls10TLSVersionZ9__lambda3MFNbNeZv':
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/vibe-d-0.8.1/vibe-d/tls/vibe/stream/openssl.d:480: undefined reference to `SSLv23_client_method'
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/vibe-d-0.8.1/vibe-d/tls/vibe/stream/openssl.d:481: undefined reference to `SSLv23_client_method'
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/vibe-d-0.8.1/vibe-d/tls/vibe/stream/openssl.d:485: undefined reference to `SSLv23_client_method'
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/vibe-d-0.8.1/vibe-d/tls/vibe/stream/openssl.d:486: undefined reference to `SSLv23_client_method'
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/vibe-d-0.8.1/vibe-d/tls/vibe/stream/openssl.d:493: undefined reference to `SSLv23_server_method'
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/vibe-d-0.8.1/vibe-d/tls/vibe/stream/openssl.d:494: undefined reference to `SSLv23_server_method'
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/vibe-d-0.8.1/vibe-d/tls/vibe/stream/openssl.d:496: undefined reference to `SSLv23_server_method'
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/vibe-d-0.8.1/vibe-d/tls/vibe/stream/openssl.d:497: undefined reference to `SSLv23_server_method'
../.dub/build/vibe-d-0.8.1/openssl-debug-linux.posix-x86_64-ldc_2075-B5F8B19F330CD9B9E7E9C85D76249E48/libvibe-d_tls.a(vibe.stream.openssl.o): In function `_D4vibe6stream7openssl14OpenSSLContext11setDHParamsMFNeAyaZv':
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/vibe-d-0.8.1/vibe-d/tls/vibe/stream/openssl.d:735: undefined reference to `get_rfc3526_prime_2048'
../.dub/build/vibe-d-0.8.1/openssl-debug-linux.posix-x86_64-ldc_2075-B5F8B19F330CD9B9E7E9C85D76249E48/libvibe-d_tls.a(vibe.stream.openssl.o): In function `_D4vibe6stream7openssl18_sharedStaticCtor1FZv':
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/vibe-d-0.8.1/vibe-d/tls/vibe/stream/openssl.d:924: undefined reference to `SSL_load_error_strings'
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/vibe-d-0.8.1/vibe-d/tls/vibe/stream/openssl.d:925: undefined reference to `SSL_library_init'
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/vibe-d-0.8.1/vibe-d/tls/vibe/stream/openssl.d:927: undefined reference to `CRYPTO_num_locks'
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/vibe-d-0.8.1/vibe-d/tls/vibe/stream/openssl.d:935: undefined reference to `CRYPTO_set_id_callback'
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/vibe-d-0.8.1/vibe-d/tls/vibe/stream/openssl.d:936: undefined reference to `CRYPTO_set_locking_callback'
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/vibe-d-0.8.1/vibe-d/tls/vibe/stream/openssl.d:941: undefined reference to `SSL_get_ex_new_index'
../.dub/build/vibe-d-0.8.1/openssl-debug-linux.posix-x86_64-ldc_2075-B5F8B19F330CD9B9E7E9C85D76249E48/libvibe-d_tls.a(vibe.stream.openssl.o): In function `_D6deimos7openssl9safestack57__T10SKM_sk_numTS6deimos7openssl6x509v315GENERAL_NAME_stZ16__T10SKM_sk_numZ10SKM_sk_numFNbPS6deimos7openssl9safestack54__T8STACK_OFTS6deimos7openssl6x509v315GENERAL_NAME_stZ8STACK_OFZi':
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/openssl-1.1.6_1.0.1g/openssl/deimos/openssl/safestack.d:140: undefined reference to `sk_num'
../.dub/build/vibe-d-0.8.1/openssl-debug-linux.posix-x86_64-ldc_2075-B5F8B19F330CD9B9E7E9C85D76249E48/libvibe-d_tls.a(vibe.stream.openssl.o): In function `_D6deimos7openssl9safestack59__T12SKM_sk_valueTS6deimos7openssl6x509v315GENERAL_NAME_stZ18__T12SKM_sk_valueZ12SKM_sk_valueFNbPS6deimos7openssl9safestack54__T8STACK_OFTS6deimos7openssl6x509v315GENERAL_NAME_stZ8STACK_OFiZPS6deimos7openssl6x509v315GENERAL_NAME_st':
/tmp/hsandbox-6DXfNv/../../home/andrew/.dub/packages/openssl-1.1.6_1.0.1g/openssl/deimos/openssl/safestack.d:142: undefined reference to `sk_value'
collect2: error: ld returned 1 exit status
Error: /usr/bin/gcc failed with status: 1
ldc2 failed with exit code 1.

[s-ludwig]

There are some undefined symbols, such as SSLv23_client_method, which should be unrelated to this fix. Was the VibeUseOpenSSL11 version defined when compiling with master (building with --override-config vibe-d:tls/openssl-1.1 should work, too)?

[andrewbenton]

You are absolutely correct. That was my mistake. Here are the correct errors:

w/ 0.8.1

core.exception.AssertError@../../home/andrew/.dub/packages/vibe-d-0.8.1/vibe-d/tls/vibe/stream/openssl.d(863): Assertion failure
----------------
??:? [0x5f02ee]
??:? [0x5f5992]
??:? [0x5efa1d]
openssl.d:863 [0x5209a1]
??:? [0x349c0119]
??:? X509_verify_cert [0x349c0725]
??:? [0x34ca5d96]
??:? [0x34cb88a2]
??:? [0x34cb639c]
??:? SSL_do_handshake [0x34cae780]
openssl.d:180 [0x51db41]
openssl.d:180 [0x51ccbf]
openssl.d:682 [0x52102d]
tls.d:144 [0x408317]
client.d:621 [0x495905]
client.d:528 [0x493b24]
client.d:500 [0x492408]
client.d:101 [0x491a67]
client.d:77 [0x49149b]
sandbox.d:19 [0x406040]
??:? [0x5fb4ee]
??:? [0x5fb4ad]
??:? [0x5fb3a8]
__entrypoint.d:8 [0x4081a4]
??:? __libc_start_main [0x33685039]
??:? [0x405ea9]
Program exited with code 1

w/ 0.8.2-rc.1

../../home/andrew/.dub/packages/openssl-1.1.6_1.0.1g/openssl/deimos/openssl/stack.d(144,9): Error: Function type does not match previously declared function with the same mangled name: sk_num
ldc2 failed with exit code 1.

With this patch, the request succeeds.

[s-ludwig]

Ah, okay, that makes sense. Maybe it's simply DMD that didn't catch it.