Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes dynamic loading bug #4024

Merged
merged 16 commits into from
Jun 2, 2023
1 change: 1 addition & 0 deletions codebuild/bin/s2n_codebuild.sh
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,7 @@ if [[ "$TESTS" == "ALL" || "$TESTS" == "asan" ]]; then make clean; S2N_ADDRESS_S
if [[ "$TESTS" == "ALL" || "$TESTS" == "integrationv2" ]]; then run_integration_v2_tests; fi
if [[ "$TESTS" == "ALL" || "$TESTS" == "crt" ]]; then ./codebuild/bin/build_aws_crt_cpp.sh $(mktemp -d) $(mktemp -d); fi
if [[ "$TESTS" == "ALL" || "$TESTS" == "sharedandstatic" ]]; then ./codebuild/bin/test_install_shared_and_static.sh $(mktemp -d); fi
if [[ "$TESTS" == "ALL" || "$TESTS" == "dynamicload" ]]; then ./codebuild/bin/test_dynamic_load.sh $(mktemp -d); fi
if [[ "$TESTS" == "ALL" || "$TESTS" == "fuzz" ]]; then (make clean && make fuzz) ; fi
if [[ "$TESTS" == "ALL" || "$TESTS" == "benchmark" ]]; then (make clean && make benchmark) ; fi
if [[ "$TESTS" == "sawHMAC" ]] && [[ "$OS_NAME" == "linux" ]]; then make -C tests/saw/ tmp/verify_HMAC.log ; fi
Expand Down
79 changes: 79 additions & 0 deletions codebuild/bin/s2n_dynamic_load_test.c
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
/*
* Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License").
* You may not use this file except in compliance with the License.
* A copy of the License is located at
*
* http://aws.amazon.com/apache2.0
*
* or in the "license" file accompanying this file. This file is distributed
* on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
* express or implied. See the License for the specific language governing
* permissions and limitations under the License.
*/

#include <dlfcn.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>

static void *s2n_load_dynamic_lib(void *ctx)
{
const char *s2n_so_path = ctx;

void *s2n_so = dlopen(s2n_so_path, RTLD_NOW);
if (!s2n_so) {
exit(1);
}

int (*s2n_init_dl)(void) = NULL;
*(void **) (&s2n_init_dl) = dlsym(s2n_so, "s2n_init");
if (dlerror()) {
exit(1);
}

int (*s2n_cleanup_dl)(void) = NULL;
*(void **) (&s2n_cleanup_dl) = dlsym(s2n_so, "s2n_cleanup");
if (dlerror()) {
exit(1);
}

if ((*s2n_init_dl)()) {
exit(1);
}
if ((*s2n_cleanup_dl)()) {
exit(1);
}
Comment on lines +45 to +47
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also add a test case scenario where we dont call cleanup and verify that this doesnt break

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you don't call cleanup this will blow up because the dangling destructor will get invoked after unload. If you want a solution where that doesn't happen then you'd need to completely rethink how thread locals and cleanup are managed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dlclose call will technically calls s2n_cleanup because of our atexit handler. So there wouldn't be a dangling destructor in this case.

https://linux.die.net/man/3/dlopen
Since glibc 2.2.3, atexit can be used to register an exit handler that is automatically called when a library is unloaded.

Although it is dependent on the glibc version I guess.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right I expect this to not fail if an application fails to all s2n_cleanup since the atexit callback will be invoked. Do we want to test that case here as well to confirm that


if (dlclose(s2n_so)) {
exit(1);
}

return NULL;
}

int main(int argc, char *argv[])
{
if (argc != 2) {
printf("Usage: s2n_dynamic_load_test <path to s2n shared object>\n");
exit(1);
}

/* s2n-tls library can be dynamically loaded and cleaned up safely
*
* We can't use any s2n test macros because then the compiler gets
* confused about whether or not to link the s2n functions.
maddeleine marked this conversation as resolved.
Show resolved Hide resolved
*/
{
pthread_t thread_id = { 0 };
if (pthread_create(&thread_id, NULL, &s2n_load_dynamic_lib, argv[1])) {
exit(1);
}
if (pthread_join(thread_id, NULL)) {
exit(1);
}
}

return 0;
}
56 changes: 56 additions & 0 deletions codebuild/bin/test_dynamic_load.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
#!/usr/bin/env bash
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License").
# You may not use this file except in compliance with the License.
# A copy of the License is located at
#
# http://aws.amazon.com/apache2.0
#
# or in the "license" file accompanying this file. This file is distributed
# on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either
# express or implied. See the License for the specific language governing
# permissions and limitations under the License.
#

# This script compiles s2n-tls as a shared library and compiles a test
# without linking to the library. This enables us to test behavior when
# s2n-tls is dynamically loaded.

WORK_DIR=$1

if [ ! -z "$NIX_STORE" ]; then
OPENSSL=$(which openssl)
LIBCRYPTO_ROOT=$(nix-store --query $OPENSSL)
else
source codebuild/bin/s2n_setup_env.sh
fi

S2N_BUILD_ARGS=(-H. -DCMAKE_PREFIX_PATH=$LIBCRYPTO_ROOT -DBUILD_TESTING=OFF)

# create installation dir with libs2n.so
if [ ! -d $WORK_DIR/s2n-install-shared ]; then
(set -x; cmake -B$WORK_DIR/s2n-build-shared -DCMAKE_INSTALL_PREFIX=$WORK_DIR/s2n-install-shared -DBUILD_SHARED_LIBS=ON ${S2N_BUILD_ARGS[@]})
(set -x; cmake --build $WORK_DIR/s2n-build-shared --target install -- -j $(nproc))
fi

# Compile the test file
$CC -Wl,-rpath $LIBCRYPTO_ROOT -o s2n_dynamic_load_test codebuild/bin/s2n_dynamic_load_test.c -ldl -lpthread

LDD_OUTPUT=$(ldd s2n_dynamic_load_test)

# Confirm executable doesn't have libs2n.so loaded
if echo "$LDD_OUTPUT" | grep -q libs2n; then
echo "test failure: libs2n should not appear in ldd output"
exit 1
fi

# Run the test with the path to libs2n
echo "Running s2n_dynamic_load_test"
LD_LIBRARY_PATH=$LIBCRYPTO_ROOT/lib ./s2n_dynamic_load_test $WORK_DIR/s2n-install-shared/lib/libs2n.so
returncode=$?
if [ $returncode -ne 0 ]; then
echo "test failure: s2n_dynamic_load_test did not succeed"
exit 1
fi
echo "Passed s2n_dynamic_load_test"
6 changes: 3 additions & 3 deletions nix/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ In the context of s2n-tls, we're using it to ease the setup of development envir

- `sudo bash -c “mkdir /nix && chmod 755 /nix && chown -R $USERNAME /nix”`
- Run the single-user command from `https://nixos.org/download.html#nix-install-linux`
- Enable flakes: `mkdir ~/.config/nix; echo "experimental-features = nix-command flakes" > ~/.config/nix.conf`
- Enable flakes: `mkdir ~/.config/nix; echo "experimental-features = nix-command flakes" > ~/.config/nix/nix.conf`
maddeleine marked this conversation as resolved.
Show resolved Hide resolved
- `cd s2n-tls`

#### What is this doing?
Expand Down Expand Up @@ -39,7 +39,7 @@ The first time this is run, it might take a while to build everything.
From inside the devShell after configuring and build finish, run `unit <test name>`, or with no test name for all of the tests.
For example, to run the stuffer_test use: `unit stuffer_test`, or `unit stuffer` to run all of tests with stuffer in the name.

The CI does this in one shot with: `nix develop --max-jobs auto --ignore-environnment --command bash -c "source ./nix/shell.sh; configure;build;unit" `.
The CI does this in one shot with: `nix develop --max-jobs auto --ignore-environment --command bash -c "source ./nix/shell.sh; configure;build;unit" `.

What is this doing?

Expand All @@ -66,7 +66,7 @@ In its simplest form, the `nix copy` command can be used to stash a specific pac

By using inputDerivation, we can create a meta-package that contains all the packages in our devShell.

As an exmample, this copy will stash the s2n-tls devShell:
As an example, this copy will stash the s2n-tls devShell:
maddeleine marked this conversation as resolved.
Show resolved Hide resolved

```
nix copy --to 's3://my-nix-chache-bucket?region=us-west-2' .#devShell
Expand Down
5 changes: 5 additions & 0 deletions nix/shell.sh
Original file line number Diff line number Diff line change
Expand Up @@ -131,3 +131,8 @@ function test_toolchain_counts {
echo -e "python nassl:\t $(pip freeze|grep -c 'nassl')"
}

function test_nonstandard_compilation {
maddeleine marked this conversation as resolved.
Show resolved Hide resolved
# Any script that needs to compile s2n in a non-standard way can run here
./codebuild/bin/test_dynamic_load.sh $(mktemp -d)
}

3 changes: 3 additions & 0 deletions utils/s2n_random.c
Original file line number Diff line number Diff line change
Expand Up @@ -530,6 +530,9 @@ S2N_RESULT s2n_rand_cleanup_thread(void)

s2n_per_thread_rand_state.drbgs_initialized = false;

/* Unset the thread-local destructor */
pthread_setspecific(s2n_per_thread_rand_state_key, NULL);
maddeleine marked this conversation as resolved.
Show resolved Hide resolved

return S2N_RESULT_OK;
}

Expand Down