Feature probes incorrectly fail on AL2 due to zlib issues

[toidiu]

Problem:

Some feature probes (eg. S2N_LIBCRYPTO_SUPPORTS_EVP_MD5_SHA1_HASH) fail with a zlib error in the s2nUnitAl2Arm and s2nUnitAl2Openssl111 CI test. This means we are not correctly testing some the feature probes on AL2 platforms.

I was able to see the error if I uncomment the message line in the cmake file.

zlib Failure:

Run Build Command(s):/usr/bin/gmake cmTC_ff5a3/fast && /usr/bin/gmake  -f CMakeFiles/cmTC_ff5a3.dir/build.make CMakeFiles/cmTC_ff5a3.dir/build
gmake[1]: Entering directory `/codebuild/output/src2287290689/src/github.com/aws/s2n-tls/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_ff5a3.dir/S2N_LIBCRYPTO_SUPPORTS_FLAG_NO_CHECK_TIME.c.o
/usr/bin/cc   -I /codebuild/output/src2287290689/src/github.com/aws/s2n-tls -include /codebuild/output/src2287290689/src/github.com/aws/s2n-tls/utils/s2n_prelude.h -c -Werror-implicit-function-declaration -Wno-unused-variable -pthread -o CMakeFiles/cmTC_ff5a3.dir/S2N_LIBCRYPTO_SUPPORTS_FLAG_NO_CHECK_TIME.c.o   -c /codebuild/output/src2287290689/src/github.com/aws/s2n-tls/tests/features/S2N_LIBCRYPTO_SUPPORTS_FLAG_NO_CHECK_TIME.c
Linking C executable cmTC_ff5a3
/usr/bin/cmake3 -E cmake_link_script CMakeFiles/cmTC_ff5a3.dir/link.txt --verbose=1
/usr/bin/cc      -rdynamic CMakeFiles/cmTC_ff5a3.dir/S2N_LIBCRYPTO_SUPPORTS_FLAG_NO_CHECK_TIME.c.o  -o cmTC_ff5a3  /usr/lib64/libcrypto.a -ldl -lrt -pthread 
/usr/lib64/libcrypto.a(c_zlib.o): In function `zlib_stateful_expand_block':
(.text+0x4c): undefined reference to `inflate'
/usr/lib64/libcrypto.a(c_zlib.o): In function `zlib_stateful_compress_block':
(.text+0xc8): undefined reference to `deflate'
/usr/lib64/libcrypto.a(c_zlib.o): In function `zlib_stateful_finish':
(.text+0x114): undefined reference to `inflateEnd'
(.text+0x11c): undefined reference to `deflateEnd'
/usr/lib64/libcrypto.a(c_zlib.o): In function `zlib_stateful_init':
(.text+0x1e8): undefined reference to `inflateInit_'
(.text+0x23c): undefined reference to `deflateInit_'
/usr/lib64/libcrypto.a(c_zlib.o): In function `bio_zlib_ctrl':
(.text+0x414): undefined reference to `deflate'
(.text+0x5cc): undefined reference to `zError'
/usr/lib64/libcrypto.a(c_zlib.o): In function `bio_zlib_write':
(.text+0x6e8): undefined reference to `deflate'
(.text+0x770): undefined reference to `zError'
(.text+0x7fc): undefined reference to `deflateInit_'
/usr/lib64/libcrypto.a(c_zlib.o): In function `bio_zlib_read':
(.text+0x904): undefined reference to `inflate'
(.text+0x968): undefined reference to `zError'
(.text+0x9ec): undefined reference to `inflateInit_'
/usr/lib64/libcrypto.a(c_zlib.o): In function `bio_zlib_free':
(.text+0xa70): undefined reference to `inflateEnd'
(.text+0xa90): undefined reference to `deflateEnd'
collect2: error: ld returned 1 exit status
gmake[1]: *** [cmTC_ff5a3] Error 1
gmake[1]: Leaving directory `/codebuild/output/src2287290689/src/github.com/aws/s2n-tls/build/CMakeFiles/CMakeTmp'
gmake: *** [cmTC_ff5a3/fast] Error 2

Investigation:

The s2nUnitAL2, even though it is AL2, does not have zlib errors. One difference I detected was which libcrypto we linked against.

# Fails: s2nUnitAl2Arm and s2nUnitAl2Openssl111
S2N_LIBCRYPTO=openssl-1.1.1

# Passes: s2nUnitAL2
S2N_LIBCRYPTO=default

• Is AL2 missing zlib? Can we add zlib?
• Does openssl 1.1.1 require zlib? (can we build it without zlib)?
• What libcrypto do we use in s2nUnitAL2, when it is default. Why doesnt it error on zlib?

Resolution:

• We should fix the zlib issue for AL2.

Detect silent feature probe failures:

Is there a way we could have detected this automatically? Maybe we can assert that the feature probe fails with an error we expect (capture the feature probe error and assert that it matches a known error).

For example we add the file S2N_LIBCRYPTO_SUPPORTS_EC_KEY_CHECK_FIPS.error to the features dir. If the S2N_LIBCRYPTO_SUPPORTS_EC_KEY_CHECK_FIPS feature probe fails, we assert that the failure matches the following exactly:

/codebuild/output/src3724912470/src/github.com/aws/s2n-tls/tests/features/S2N_LIBCRYPTO_SUPPORTS_EC_KEY_CHECK_FIPS.c: In function 'main':
/codebuild/output/src3724912470/src/github.com/aws/s2n-tls/tests/features/S2N_LIBCRYPTO_SUPPORTS_EC_KEY_CHECK_FIPS.c:21:5: error: implicit declaration of function 'EC_KEY_check_fips'; did you mean 'EC_KEY_check_key'? [-Werror=implicit-function-declaration]
     EC_KEY_check_fips(ec_key);
     ^~~~~~~~~~~~~~~~~
     EC_KEY_check_key

[dougch]

Is AL2 missing zlib ?

Zlib, including headers are installed here: https://github.com/aws/s2n-tls/blob/main/codebuild/bin/install_al_dependencies.sh#L51

What libcrypto is default
AL2 shipped with openssl102

Might these BUILD_FLAGS be part of the issue?

    # AL2 case; Linker flags are a workaround for system openssl
    if [[ ${VERSION_ID} == '2' ]]; then
       BUILD_FLAGS=$(echo -e '-DCMAKE_EXE_LINKER_FLAGS="-lcrypto -lz" \
         -DCMAKE_EXPORT_COMPILE_COMMANDS=ON')
    fi