Works under cargo test, but not when harness is executed directly?

[lilith]

I'm seeing different behavior between cargo test and when I run one of my own test harnesses directly.

(Ssl(Failure(MidHandshakeSslStream { stream: SslStream { stream: HttpStream(_), ssl: Ssl { state: "error", verify_result: X509VerifyError { code: 20, error: "unable to get local issuer certificate" } } }, error: Ssl(ErrorStack([Error { code: 336134278, library: "SSL routines", function: "ssl3_get_server_certificate", reason: "certificate verify failed" }])) })))

Any environmental differences for me to look at?

[sfackler]

Cargo does set some environment variables - LD_LIBRARY_PATH among them. Could be that it ends up pulling a different OpenSSL when run directly?

There might also be places in the tests that are dependent on the working directory the test is run from, but I think I fixed all of those?

[sfackler]

I just checked, and there are definitely working directory issues but the errors caused there differ from what you're seeing.

[lilith]

Are any working directory issues related to the certificate chain?

[lilith]

Actually, the working dir should be the same in both cases. What about env vars?

Also, "directly" is incorrect. The integration harness starts a server. The server (as a child process) is reporting the SSL failure when connecting to https://raw.github.com.

But 'cargo test' makes the tests pass, and SSL work!

[sfackler]

Oh sorry, I was thinking you were talking about the tests in this crate.

My best guess is that this is a case where the dynamic linker tracks down different copies of libssl in the two cases. OpenSSL contains a default certificate store path that's set at build time, and it could differ between the two copies. I'd probably add a bit of code to the integration test to dump the environment and look at what the differences are.

[lilith]

From the harness, the differences when running under Cargo are:

LD_LIBRARY_PATH=/home/n/.conan/data/OpenSSL/1.0.2j/lasote/stable/package/8dc0071dc4241826d308f93b7589b40fa9c51286/lib:/home/n/Documents/imazen/imageflow/target/debug/build/miniz-sys-18005000ddedadf4/out:/home/n/.conan/data/libjpeg-turbo/1.5.1/imazen/testing/package/5dfb956a1439c84606389c92be7ec45b23c2e716/lib:/home/n/.conan/data/littlecms/2.7/lasote/stable/package/3a5f72c8cd50641b8efa6ed13e6914c6ced2747c/lib:/home/n/.conan/data/imageflow_c/0.1.0/imazen/testing/package/9590d048412ceec2ab01993c362139477de0da0d/lib:/home/n/.conan/data/zlib/1.2.8/lasote/stable/package/755fc07adbed235a7d3eadec4b6882d4912f09bc/lib:/home/n/.conan/data/libpng/1.6.21/lasote/stable/package/4bed5bcecebb11c5b891863880f3ef10411fb2da/lib:/home/n/.conan/data/OpenSSL/1.0.2j/lasote/stable/package/8dc0071dc4241826d308f93b7589b40fa9c51286/lib:/home/n/Documents/imazen/imageflow/target/debug:/home/n/Documents/imazen/imageflow/target/debug/deps:/home/n/.rustup/toolchains/nightly-2017-02-05-x86_64-unknown-linux-gnu/lib:/home/n/.local/share/umake/rust/rust-lang/rustc/lib
_=/home/n/.cargo/bin/cargo
CARGO_HOME=/home/n/.cargo
RUST_RECURSION_COUNT=1
RUSTUP_TOOLCHAIN=nightly-2017-02-05-x86_64-unknown-linux-gnu
RUSTUP_HOME=/home/n/.rustup
SSL_CERT_FILE=/usr/lib/ssl/certs/ca-certificates.crt
SSL_CERT_DIR=/usr/lib/ssl/certs
CARGO_PKG_HOMEPAGE=
CARGO_PKG_VERSION=0.1.0
CARGO_MANIFEST_DIR=/home/n/Documents/imazen/imageflow/imageflow_server
CARGO_PKG_DESCRIPTION=
CARGO_PKG_NAME=imageflow_server
CARGO_PKG_VERSION_MAJOR=0
CARGO_PKG_VERSION_PRE=
CARGO_PKG_AUTHORS=Nathanael Jones <[email protected]>
CARGO_PKG_VERSION_MINOR=1
CARGO_PKG_VERSION_PATCH=0

Is OpenSSL on linux dynamically linked by default?

[lilith]

Seems related: rust-lang/cargo#2888

[sfackler]

We always prefer to dynamically link rather than statically link when possible, yeah.

[sfackler]

You can set OPENSSL_STATIC=1 to statically link, FYI.

[lilith]

Thanks!

Any idea where SSL_CERT_DIR and SSL_CERT_FILE might be set?

[sfackler]

Assuming they aren't inherited from the shell, they could be leaking out of Cargo, which seems like a bug on its part - it could be setting them for libcurl?

[lilith]

Setting them makes all the difference. SSL_CERT_FILE=/usr/lib/ssl/certs/ca-certificates.crt SSL_CERT_DIR=/usr/lib/ssl/certs /path/to/harness works great.

[sfackler]

Awesome! I filed a bug on Cargo about it leaking those variables to subprocesses: rust-lang/cargo#3676

[lilith]

Thanks. Any idea where to acquire good values for SSL_CERT_FILE and SSL_CERT_DIR, since it appears they are required? Or does system OpenSSL not need them?

If system OpenSSL doesn't require them, has #565 landed in a release yet? If the link path issue is resolved I can go back to a system dynamic link (although I'd rather not).

[sfackler]

If those aren't set, OpenSSL will use defaults that were set when it was compiled. The system OpenSSL normally points the defaults to the right place, but presumably conan's version doesn't. Cargo uses https://crates.io/crates/openssl-probe to look in a bunch of common locations to track down the right paths.

Ah yeah, I will cut a release today.

[cssivision]

i have the same issue, but use https://crates.io/crates/openssl-probe in my application to set the SSL_CERT_FILE and SSL_CERT_DIR env vars fix this.