Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cargo:rustc-link-args not applied to doctest builds #9895

Closed
davepacheco opened this issue Sep 9, 2021 · 4 comments · Fixed by #9916
Closed

cargo:rustc-link-args not applied to doctest builds #9895

davepacheco opened this issue Sep 9, 2021 · 4 comments · Fixed by #9916
Assignees
@chansuke
Labels
A-build-scripts Area: build.rs scripts A-doctests Area: rustdoc --test C-bug Category: bug E-easy Experience: Easy

Comments

@davepacheco
Copy link
Contributor

davepacheco commented Sep 9, 2021 •
edited by rustbot
Loading

I'm not sure if this is a bug or not, but it doesn't seem like desired behavior.

The docs for cargo:rustc-link-arg say:

The rustc-link-arg instruction tells Cargo to pass the -C link-arg=FLAG option to the compiler, but only when building supported targets (benchmarks, binaries, cdylib crates, examples, and tests).

Are doctest binaries intended to be supported or not? They seem to fall under "tests". Empirically, it looks like the requested linker flags are not passed to the compiler when building doctests. The rest of this comment shows why I think this.

To show this, I created this repo. This is a library crate that depends on pq-sys (which just happens to be the crate I discovered this with). This test crate has one function that uses something in the native libpq, wrapped by pq-sys. There's a unit test and a doctest that use this function.

My libpq is installed in "/opt/ooce/pgsql-13/lib/amd64", which is not on ld.so's default search path, so I need to build with -R/opt/ooce/pgsql-13/lib/amd64, which means passing -Wl,-R/opt/ooce/pgsql-13/lib/amd64 to gcc. To achieve this, my test crate has a build.rs that uses cargo:rustc-link-arg. To make it easier to show the problem, the build.rs script looks at the environment variable TEST_LIBPQ_DIR. If it's set, it uses cargo:rustc-link-arg to set the corresponding linker flag. If the environment variable is not set, the build.rs does nothing.

I'm using:

$ cargo +nightly version
cargo 1.56.0-nightly (18751dd3f 2021-09-01)

If I just run cargo +nightly test, the unit test fails because ld.so can't find libpq:

$ cargo +nightly test
   Compiling pq-sys v0.4.6
   Compiling link-args-doctest v0.1.0 (/home/dap/link-args-doctest)
warning: not specifying libpq dir
    Finished test [unoptimized + debuginfo] target(s) in 3.19s
     Running unittests (target/debug/deps/link_args_doctest-1d3464329f24b2d7)
ld.so.1: link_args_doctest-1d3464329f24b2d7: fatal: libpq.so.5: open failed: No such file or directory
error: test failed, to rerun pass '--lib'

Caused by:
  process didn't exit successfully: `/home/dap/link-args-doctest/target/debug/deps/link_args_doctest-1d3464329f24b2d7` (signal: 9, SIGKILL: kill)

This means our linker flag was not passed -- that's expected.

Now, if I use RUSTFLAGS and RUSTDOCFLAGS to specify the linker args, everything works:

$ RUSTFLAGS="-Clink-args=-R/opt/ooce/pgsql-13/lib/amd64" RUSTDOCFLAGS="-Clink-args=-R/opt/ooce/pgsql-13/lib/amd64" cargo +nightly test
warning: not specifying libpq dir
    Finished test [unoptimized + debuginfo] target(s) in 0.01s
     Running unittests (target/debug/deps/link_args_doctest-1d3464329f24b2d7)

running 1 test
test test_foo ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests link-args-doctest

running 1 test
test src/lib.rs - foo (line 1) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.33s

This is also expected -- we forced rustc and rustdoc to pass gcc the right linker arg so that it would find the library.

What's surprising is that if I set my environment variable so that build.rs uses cargo:rustc-link-args, then the unit test works, but the doctest doesn't:

$ TEST_LIBPQ_DIR=/opt/ooce/pgsql-13/lib/amd64 cargo +nightly test
   Compiling pq-sys v0.4.6
   Compiling link-args-doctest v0.1.0 (/home/dap/link-args-doctest)
warning: using libpq dir from environment: /opt/ooce/pgsql-13/lib/amd64
    Finished test [unoptimized + debuginfo] target(s) in 2.02s
     Running unittests (target/debug/deps/link_args_doctest-1d3464329f24b2d7)

running 1 test
test test_foo ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests link-args-doctest

running 1 test
test src/lib.rs - foo (line 1) ... FAILED

failures:

---- src/lib.rs - foo (line 1) stdout ----
Test executable failed (terminated by signal).

stderr:
ld.so.1: rust_out: fatal: libpq.so.5: open failed: No such file or directory



failures:
    src/lib.rs - foo (line 1)

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.47s

error: test failed, to rerun pass '--doc'

Since the unit test passed, we know the linker arg was passed correctly for that. We can also see that the test binary has the right directory on the RUNPATH:

$ readelf -d /home/dap/link-args-doctest/target/debug/deps/link_args_doctest-1d3464329f24b2d7 | grep RUNPATH
 0x000000000000001d (RUNPATH)            Library runpath: [/usr/gcc/10/lib/amd64:/opt/ooce/pgsql-13/lib/amd64]

But the doctest build doesn't.

To be really sure, here's the very verbose output:

$ TEST_LIBPQ_DIR=/opt/ooce/pgsql-13/lib/amd64 cargo +nightly test -vv
   Compiling pq-sys v0.4.6
   Compiling link-args-doctest v0.1.0 (/home/dap/link-args-doctest)
     Running `CARGO=/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/bin/cargo CARGO_CRATE_NAME=build_script_build CARGO_MANIFEST_DIR=/home/dap/.cargo/registry/src/github.com-1ecc6299db9ec823/pq-sys-0.4.6 CARGO_PKG_AUTHORS='Sean Griffin <[email protected]>' CARGO_PKG_DESCRIPTION='Auto-generated rust bindings for libpq' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=pq-sys CARGO_PKG_REPOSITORY='https://github.com/sgrif/pq-sys' CARGO_PKG_VERSION=0.4.6 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=4 CARGO_PKG_VERSION_PATCH=6 CARGO_PKG_VERSION_PRE='' LD_LIBRARY_PATH='/home/dap/link-args-doctest/target/debug/deps:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib' rustc --crate-name build_script_build /home/dap/.cargo/registry/src/github.com-1ecc6299db9ec823/pq-sys-0.4.6/build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=066bbe521cca295f -C extra-filename=-066bbe521cca295f --out-dir /home/dap/link-args-doctest/target/debug/build/pq-sys-066bbe521cca295f -L dependency=/home/dap/link-args-doctest/target/debug/deps --cap-lints warn`
     Running `CARGO=/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/bin/cargo CARGO_CRATE_NAME=build_script_build CARGO_MANIFEST_DIR=/home/dap/link-args-doctest CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=link-args-doctest CARGO_PKG_REPOSITORY='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH='/home/dap/link-args-doctest/target/debug/deps:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib' rustc --crate-name build_script_build --edition=2018 build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=ae900bc3a3724b07 -C extra-filename=-ae900bc3a3724b07 --out-dir /home/dap/link-args-doctest/target/debug/build/link-args-doctest-ae900bc3a3724b07 -C incremental=/home/dap/link-args-doctest/target/debug/incremental -L dependency=/home/dap/link-args-doctest/target/debug/deps`
     Running `/home/dap/link-args-doctest/target/debug/build/pq-sys-066bbe521cca295f/build-script-build`
[pq-sys 0.4.6] cargo:rerun-if-env-changed=PQ_LIB_DIR
[pq-sys 0.4.6] cargo:rerun-if-env-changed=PQ_LIB_STATIC
[pq-sys 0.4.6] cargo:rerun-if-env-changed=TARGET
[pq-sys 0.4.6] cargo:rerun-if-env-changed=PG_CONFIG_X86_64_UNKNOWN_ILLUMOS
[pq-sys 0.4.6] cargo:rustc-link-search=native=/opt/ooce/pgsql-13/lib/amd64
[pq-sys 0.4.6] cargo:rerun-if-env-changed=PQ_LIB_STATIC_X86_64_UNKNOWN_ILLUMOS
[pq-sys 0.4.6] cargo:rustc-link-lib=pq
     Running `/home/dap/link-args-doctest/target/debug/build/link-args-doctest-ae900bc3a3724b07/build-script-build`
     Running `CARGO=/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/bin/cargo CARGO_CRATE_NAME=pq_sys CARGO_MANIFEST_DIR=/home/dap/.cargo/registry/src/github.com-1ecc6299db9ec823/pq-sys-0.4.6 CARGO_PKG_AUTHORS='Sean Griffin <[email protected]>' CARGO_PKG_DESCRIPTION='Auto-generated rust bindings for libpq' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=pq-sys CARGO_PKG_REPOSITORY='https://github.com/sgrif/pq-sys' CARGO_PKG_VERSION=0.4.6 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=4 CARGO_PKG_VERSION_PATCH=6 CARGO_PKG_VERSION_PRE='' LD_LIBRARY_PATH='/home/dap/link-args-doctest/target/debug/deps:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib' OUT_DIR=/home/dap/link-args-doctest/target/debug/build/pq-sys-3988470aaaa1871a/out rustc --crate-name pq_sys /home/dap/.cargo/registry/src/github.com-1ecc6299db9ec823/pq-sys-0.4.6/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debuginfo=2 -C metadata=92e5a3b5ec0e85a5 -C extra-filename=-92e5a3b5ec0e85a5 --out-dir /home/dap/link-args-doctest/target/debug/deps -L dependency=/home/dap/link-args-doctest/target/debug/deps --cap-lints warn -L native=/opt/ooce/pgsql-13/lib/amd64 -l pq`
[link-args-doctest 0.1.0] cargo:rerun-if-env-changed=TEST_LIBPQ_DIR
[link-args-doctest 0.1.0] cargo:warning=using libpq dir from environment: /opt/ooce/pgsql-13/lib/amd64
[link-args-doctest 0.1.0] cargo:rustc-link-arg=-Wl,-R/opt/ooce/pgsql-13/lib/amd64
warning: using libpq dir from environment: /opt/ooce/pgsql-13/lib/amd64
     Running `CARGO=/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/bin/cargo CARGO_CRATE_NAME=link_args_doctest CARGO_MANIFEST_DIR=/home/dap/link-args-doctest CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=link-args-doctest CARGO_PKG_REPOSITORY='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH='/home/dap/link-args-doctest/target/debug/deps:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib' OUT_DIR=/home/dap/link-args-doctest/target/debug/build/link-args-doctest-d6beefd3d9f74ce9/out rustc --crate-name link_args_doctest --edition=2018 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debuginfo=2 -C metadata=5f3b72e47d45b795 -C extra-filename=-5f3b72e47d45b795 --out-dir /home/dap/link-args-doctest/target/debug/deps -C incremental=/home/dap/link-args-doctest/target/debug/incremental -L dependency=/home/dap/link-args-doctest/target/debug/deps --extern pq_sys=/home/dap/link-args-doctest/target/debug/deps/libpq_sys-92e5a3b5ec0e85a5.rmeta -C link-arg=-Wl,-R/opt/ooce/pgsql-13/lib/amd64 -L native=/opt/ooce/pgsql-13/lib/amd64`
     Running `CARGO=/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/bin/cargo CARGO_CRATE_NAME=link_args_doctest CARGO_MANIFEST_DIR=/home/dap/link-args-doctest CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=link-args-doctest CARGO_PKG_REPOSITORY='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH='/home/dap/link-args-doctest/target/debug/deps:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib:/home/dap/.rustup/toolchains/nightly-x86_64-unknown-illumos/lib' OUT_DIR=/home/dap/link-args-doctest/target/debug/build/link-args-doctest-d6beefd3d9f74ce9/out rustc --crate-name link_args_doctest --edition=2018 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --test -C metadata=1d3464329f24b2d7 -C extra-filename=-1d3464329f24b2d7 --out-dir /home/dap/link-args-doctest/target/debug/deps -C incremental=/home/dap/link-args-doctest/target/debug/incremental -L dependency=/home/dap/link-args-doctest/target/debug/deps --extern pq_sys=/home/dap/link-args-doctest/target/debug/deps/libpq_sys-92e5a3b5ec0e85a5.rlib -C link-arg=-Wl,-R/opt/ooce/pgsql-13/lib/amd64 -L native=/opt/ooce/pgsql-13/lib/amd64`
    Finished test [unoptimized + debuginfo] target(s) in 3.17s
     Running `/home/dap/link-args-doctest/target/debug/deps/link_args_doctest-1d3464329f24b2d7`

running 1 test
test test_foo ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

   Doc-tests link-args-doctest
     Running `rustdoc --edition=2018 --crate-type lib --crate-name link_args_doctest --test /home/dap/link-args-doctest/src/lib.rs -L dependency=/home/dap/link-args-doctest/target/debug/deps -L dependency=/home/dap/link-args-doctest/target/debug/deps -L native=/opt/ooce/pgsql-13/lib/amd64 --extern link_args_doctest=/home/dap/link-args-doctest/target/debug/deps/liblink_args_doctest-5f3b72e47d45b795.rlib --extern pq_sys=/home/dap/link-args-doctest/target/debug/deps/libpq_sys-92e5a3b5ec0e85a5.rlib -C embed-bitcode=no --error-format human`

running 1 test
test src/lib.rs - foo (line 1) ... FAILED

failures:

---- src/lib.rs - foo (line 1) stdout ----
Test executable failed (terminated by signal).

stderr:
ld.so.1: rust_out: fatal: libpq.so.5: open failed: No such file or directory



failures:
    src/lib.rs - foo (line 1)

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.48s

error: test failed, to rerun pass '--doc'

The rustc invocation for building the unit test includes -C link-arg=-Wl,-R/opt/ooce/pgsql-13/lib/amd64 but the one for rustdoc does not. If I take the rustdoc invocation, add "+nightly" and -C link-arg=-Wl,-R/opt/ooce/pgsql-13/lib/amd64 by hand, then the rustdoc test works:

$ rustdoc +nightly --edition=2018 --crate-type lib --crate-name link_args_doctest --test /home/dap/link-args-doctest/src/lib.rs -L dependency=/home/dap/link-args-doctest/target/debug/deps -L dependency=/home/dap/link-args-doctest/target/debug/deps -L native=/opt/ooce/pgsql-13/lib/amd64 --extern link_args_doctest=/home/dap/link-args-doctest/target/debug/deps/liblink_args_doctest-5f3b72e47d45b795.rlib --extern pq_sys=/home/dap/link-args-doctest/target/debug/deps/libpq_sys-92e5a3b5ec0e85a5.rlib -C embed-bitcode=no --error-format human -C link-arg=-Wl,-R/opt/ooce/pgsql-13/lib/amd64

running 1 test
test src/lib.rs - foo (line 1) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.33s

Anyway, I expected that the link args from "cargo:rustc-link-args" would apply to rustdoc tests, but they don't seem to be. Is this a bug?
@davepacheco davepacheco added the C-bug Category: bug label Sep 9, 2021
This was referenced Sep 9, 2021
Closed
Open
@ehuss
Copy link
Contributor

ehuss commented Sep 11, 2021

Oh my, doc tests do tend to get left in the cold when new flags are added. I'm not sure if that can be solved generally, but to solve this problem specifically, I think roughly the following changes would be needed:

Add the appropriate -C link-arg arguments to the Doctest.args around here. I believe output.linker_args contains the args, and it needs to check if they "apply" (see here).

@ehuss ehuss added A-build-scripts Area: build.rs scripts A-doctests Area: rustdoc --test E-easy Experience: Easy labels Sep 11, 2021
@chansuke
Copy link
Contributor

@rustbot claim

@davepacheco davepacheco mentioned this issue Sep 15, 2021
Merged
@chansuke
Copy link
Contributor

@ehuss Thank you for your comment. In order to reproduce this case in tests, I think /opt/ooce/pgsql-13/lib/amd64 needs to be created at cargo test but I don't know how to achieve this...

@chansuke chansuke mentioned this issue Sep 16, 2021
Merged
@ehuss
Copy link
Contributor

ehuss commented Sep 16, 2021

There are several tests in build_script_extra_link_arg.rs where it just passes a bogus argument, and then checks that it is passed with the -v flag. So roughly:

  • Have a lib.rs with a doc test.
  • Have a build.rs which passes the bogus flag.
  • Run cargo test --doc -v and check that it passes the flag.

@bors bors closed this as completed in bd6a512 Oct 14, 2021
@davepacheco davepacheco mentioned this issue Oct 20, 2021
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@chansuke chansuke

Labels
A-build-scripts Area: build.rs scripts A-doctests Area: rustdoc --test C-bug Category: bug E-easy Experience: Easy
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add rustc-link-args to doctest build chansuke/cargo
3 participants
@ehuss @chansuke @davepacheco