build fails due to missing libpq #213
Comments
|
re: the static linking error about pqsignal, I asked on their IRC channel (#postgresql on LiberaChat). The suggestion I got was to try passing (*) not even sure this would be a good option as I believe even though both functions are named the same they have different semantics. |
|
Thinking about this more: I do think advising people to set RUSTFLAGS is less fraught than using LD_LIBRARY_PATH, but it's still pretty sucky because there's no point where we'd be able to remove this workaround until Rust provides a solution for |
|
This change to pq-sys appears to do the right thing: If I run this with nightly, I get the right behavior: If I run it with stable cargo (1.54), I get a warning related to trying to use an unstable feature, and it appears to ignore it, resulting in the broken test binary and failed test: That's as expected, and no worse than what happens today without the change. I wanted to make sure this wouldn't totally break people using stable cargo that's so old that it doesn't even know about this flag. So I changed the key to something else: On both stable and nightly, this produces no warnings, a broken binary, and a failed test: I think this is the best we can hope for. I was a little surprised it worked this way, but the Cargo docs do say that any unrecognized key is treated as metadata. So this will create confusing, useless metadata, but I believe the only consumers are people trying to consume a particular metadata key, so I don't think this is a problem. @luqmana What do you think of this? If this seems reasonable, I think I'll create a PR upstream. In parallel I may create our own fork and update Omicron to use that with a "patch" section in Cargo.toml, just because upstream doesn't seem to be actively maintained. We can remove the "patch" and fork if/when upstream does land it. |
|
I think that's a pretty targeted and reasonable approach 👍🏾 One thing is, should we use |
|
This is a good question. I'm inclined to test this on Mac, illumos, and Linux, then add an appropriate |
|
Yea, that sounds good. Just wanted to make sure we don't introduce a "unknown linker flag" sorta error on major platform it already works on. |
|
I've filed an issue upstream on pq-sys: sgrif/pq-sys#36 Since upstream does not seem actively maintained, I've created our own fork here: I will put together a PR for Omicron that points us at that fork. |
|
Well, I think that approach is a bust. See #220 for details. I think at this point the remaining options are:
|
|
I tried the approach where we add a build.rs to omicron that consumes metadata from the pq-sys build and emits the linker instruction rather than putting the linker instruction into pq-sys. Here's the change to pq-sys (from its "master") to emit the metadata: However, I found that it did not appear in the omicron build script's environment. And then I found this quote in the docs:
But omicron is not an immediate dependent of pq-sys; diesel is. |
|
While testing my latest approach, I found the issue fixed by #224. The good news is that the above works if I add a dependency from the omicron crates directly on pq-sys. But it does not address the issue found under #224 because the cargo:link_args instruction is not used for rustdoc tests. I have not determined if that's intentional or a bug. |
|
I've put up #225 -- but for reasons explained there, I'm putting this approach aside. Next steps:
|
|
Filed rust-lang/cargo#9895 about the surprising cargo:rustc-link-arg behavior. |
|
I spent a bit of time understanding the static linking problem. tl;dr: this looks like a PostgreSQL bug plus some undesirable behavior from Rust that doesn't appear to be configurable. The problem looks like this: This is easily reproduced. I tried it on:
In alll cases, I'm building sgrif/pq-sys with sgrif/pq-sys#28 applied, which is necessary to statically link against PostgreSQL 12 or later. There was some discussion that this might relate to gcc 10's change to Assuming it's true that we need to link against libpgcommon and libpgport (which we understand from sgrif/pq-sys#28 and the related PostgreSQL mailing list posts), it seems like a bug in PostgreSQL that these libraries include overlapping symbols. And they really are different functions with the same name: the one coming from legacy-pqsignal.c is documented to exist solely for old libpq clients and is frozen to match the behavior for those old clients -- it's explicitly different than the one in pqsignal.c, which is used elsewhere. So why are we hitting this issue when other PostgreSQL users aren't reporting it? @jclulow guessed it: when statically linking a native library, Rust uses the --whole-archive linker flag, which causes all symbols in the .a file to be included in the resulting binary. The more common behavior (which presumably other consumers of libpq.a are using) only includes referenced symbols. At least one of these two copies is not exported, and the other is likely not widely used, so they likely wouldn't be included if --whole-archive weren't set. Why does Rust do this? This behavior was introduced in rust-lang/rust#16110. rust-lang/rust#85144 started to remove it, but is still a work in progress. I have not found a way around Rust's behavior here. Although there's recently support for requesting the --whole-archive behavior in other cases, I don't see any way to disable it. |
|
On the PostgreSQL front: here's what I've learned. First, about the three libraries:
Now, according to the Makefiles, libpgport and libpgcommon generate three different .a files: one for client applications (libpgport.a), one for client shared libraries (libpgport_shlib.a), and one for the backend (libpgport_srv.a). I infer that the first one is for inclusion in applications (like those shipped within PostgreSQL), the second is for inclusion in libpq (or other external libraries), and the last is used when building the PostgreSQL backend. The third one is irrelevant for us (it contains different contents). The only difference I see between the first two is that one is built with As far as I can tell:
This explains why libpq and libpgport both have Most consumers presumably don't use the undocumented @luqmana found something I think is a bit better, which is to link these libraries with Rust's "nobundle" option. The documentation is not very detailed, but it looks like this records the library's name rather than its contents and the final link happens without --whole-archive at the end of the build. (Presumably, this is also recording the path of the library, since otherwise I don't see how this would work.) This seems to work and I'll put up a PR for this shortly. |
tl;dr: On Helios, to build "main" after #192, you should:
pkg install library/postgresql-13/opt/ooce/binis on your PATHRUSTFLAGS="-Clink-args=-R$(pg_config --libdir)"in the environmentBelow are my notes from debugging this. Much thanks to @rmustacc @wesolows @jclulow @luqmana and @bnaecker for help with debugging this and finding possible workarounds. At the end there's a discussion of different approaches we could take, but my conclusion for now is we should just tell people to set
RUSTFLAGS.Debugging the problem
On Helios, a full build-and-test from a clean clone currently fails.
Here's an example from @bnaecker:
Since this came in with the recent Diesel change, and libpq is a PostgreSQL-related library, I figured there's probably some new package that's pulling that in. So I looked at
cargo tree:dap@helios omicron $ cargo tree omicron-bootstrap-agent v0.1.0 (/home/dap/omicron/omicron-bootstrap-agent) ├── dropshot v0.5.2-dev (https://github.com/oxidecomputer/dropshot?branch=main#1c6f4d15) │ ├── async-trait v0.1.51 (proc-macro) │ │ ├── proc-macro2 v1.0.28 │ │ │ └── unicode-xid v0.2.2 │ │ ├── quote v1.0.9 │ │ │ └── proc-macro2 v1.0.28 (*) │ │ └── syn v1.0.74 │ │ ├── proc-macro2 v1.0.28 (*) │ │ ├── quote v1.0.9 (*) │ │ └── unicode-xid v0.2.2 │ ├── base64 v0.13.0 │ ├── bytes v1.0.1 │ │ └── serde v1.0.127 │ │ └── serde_derive v1.0.127 (proc-macro) │ │ ├── proc-macro2 v1.0.28 (*) │ │ ├── quote v1.0.9 (*) │ │ └── syn v1.0.74 (*) │ ├── chrono v0.4.19 │ │ ├── libc v0.2.98 │ │ ├── num-integer v0.1.44 │ │ │ └── num-traits v0.2.14 │ │ │ [build-dependencies] │ │ │ └── autocfg v1.0.1 │ │ │ [build-dependencies] │ │ │ └── autocfg v1.0.1 │ │ ├── num-traits v0.2.14 (*) │ │ ├── serde v1.0.127 (*) │ │ └── time v0.1.44 │ │ └── libc v0.2.98 │ ├── dropshot_endpoint v0.5.2-dev (proc-macro) (https://github.com/oxidecomputer/dropshot?branch=main#1c6f4d15) │ │ ├── proc-macro2 v1.0.28 (*) │ │ ├── quote v1.0.9 (*) │ │ ├── serde v1.0.127 │ │ │ └── serde_derive v1.0.127 (proc-macro) (*) │ │ ├── serde_tokenstream v0.1.2 │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ ├── serde v1.0.127 (*) │ │ │ └── syn v1.0.74 (*) │ │ └── syn v1.0.74 (*) │ ├── futures v0.3.15 │ │ ├── futures-channel v0.3.15 │ │ │ ├── futures-core v0.3.15 │ │ │ └── futures-sink v0.3.15 │ │ ├── futures-core v0.3.15 │ │ ├── futures-executor v0.3.15 │ │ │ ├── futures-core v0.3.15 │ │ │ ├── futures-task v0.3.15 │ │ │ └── futures-util v0.3.15 │ │ │ ├── futures-channel v0.3.15 (*) │ │ │ ├── futures-core v0.3.15 │ │ │ ├── futures-io v0.3.15 │ │ │ ├── futures-macro v0.3.15 (proc-macro) │ │ │ │ ├── proc-macro-hack v0.5.19 (proc-macro) │ │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ │ ├── quote v1.0.9 (*) │ │ │ │ └── syn v1.0.74 (*) │ │ │ │ [build-dependencies] │ │ │ │ └── autocfg v1.0.1 │ │ │ ├── futures-sink v0.3.15 │ │ │ ├── futures-task v0.3.15 │ │ │ ├── memchr v2.4.0 │ │ │ ├── pin-project-lite v0.2.6 │ │ │ ├── pin-utils v0.1.0 │ │ │ ├── proc-macro-hack v0.5.19 (proc-macro) │ │ │ ├── proc-macro-nested v0.1.7 │ │ │ └── slab v0.4.3 │ │ │ [build-dependencies] │ │ │ └── autocfg v1.0.1 │ │ ├── futures-io v0.3.15 │ │ ├── futures-sink v0.3.15 │ │ ├── futures-task v0.3.15 │ │ └── futures-util v0.3.15 (*) │ ├── hostname v0.3.1 │ │ ├── libc v0.2.98 │ │ └── match_cfg v0.1.0 │ ├── http v0.2.4 │ │ ├── bytes v1.0.1 (*) │ │ ├── fnv v1.0.7 │ │ └── itoa v0.4.7 │ ├── hyper v0.14.10 │ │ ├── bytes v1.0.1 (*) │ │ ├── futures-channel v0.3.15 (*) │ │ ├── futures-core v0.3.15 │ │ ├── futures-util v0.3.15 (*) │ │ ├── h2 v0.3.3 │ │ │ ├── bytes v1.0.1 (*) │ │ │ ├── fnv v1.0.7 │ │ │ ├── futures-core v0.3.15 │ │ │ ├── futures-sink v0.3.15 │ │ │ ├── futures-util v0.3.15 (*) │ │ │ ├── http v0.2.4 (*) │ │ │ ├── indexmap v1.6.2 │ │ │ │ ├── hashbrown v0.9.1 │ │ │ │ └── serde v1.0.127 (*) │ │ │ │ [build-dependencies] │ │ │ │ └── autocfg v1.0.1 │ │ │ ├── slab v0.4.3 │ │ │ ├── tokio v1.10.0 │ │ │ │ ├── bytes v1.0.1 (*) │ │ │ │ ├── libc v0.2.98 │ │ │ │ ├── memchr v2.4.0 │ │ │ │ ├── mio v0.7.11 │ │ │ │ │ ├── libc v0.2.98 │ │ │ │ │ └── log v0.4.14 │ │ │ │ │ └── cfg-if v1.0.0 │ │ │ │ ├── num_cpus v1.13.0 │ │ │ │ │ └── libc v0.2.98 │ │ │ │ ├── once_cell v1.7.2 │ │ │ │ ├── parking_lot v0.11.1 │ │ │ │ │ ├── instant v0.1.9 │ │ │ │ │ │ └── cfg-if v1.0.0 │ │ │ │ │ ├── lock_api v0.4.4 │ │ │ │ │ │ └── scopeguard v1.1.0 │ │ │ │ │ └── parking_lot_core v0.8.3 │ │ │ │ │ ├── cfg-if v1.0.0 │ │ │ │ │ ├── instant v0.1.9 (*) │ │ │ │ │ ├── libc v0.2.98 │ │ │ │ │ └── smallvec v1.6.1 │ │ │ │ ├── pin-project-lite v0.2.6 │ │ │ │ ├── signal-hook-registry v1.4.0 │ │ │ │ │ └── libc v0.2.98 │ │ │ │ └── tokio-macros v1.2.0 (proc-macro) │ │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ │ ├── quote v1.0.9 (*) │ │ │ │ └── syn v1.0.74 (*) │ │ │ │ [build-dependencies] │ │ │ │ └── autocfg v1.0.1 │ │ │ ├── tokio-util v0.6.7 │ │ │ │ ├── bytes v1.0.1 (*) │ │ │ │ ├── futures-core v0.3.15 │ │ │ │ ├── futures-sink v0.3.15 │ │ │ │ ├── log v0.4.14 (*) │ │ │ │ ├── pin-project-lite v0.2.6 │ │ │ │ └── tokio v1.10.0 (*) │ │ │ └── tracing v0.1.26 │ │ │ ├── cfg-if v1.0.0 │ │ │ ├── pin-project-lite v0.2.6 │ │ │ └── tracing-core v0.1.18 │ │ │ └── lazy_static v1.4.0 │ │ ├── http v0.2.4 (*) │ │ ├── http-body v0.4.2 │ │ │ ├── bytes v1.0.1 (*) │ │ │ ├── http v0.2.4 (*) │ │ │ └── pin-project-lite v0.2.6 │ │ ├── httparse v1.4.1 │ │ ├── httpdate v1.0.1 │ │ ├── itoa v0.4.7 │ │ ├── pin-project-lite v0.2.6 │ │ ├── socket2 v0.4.0 │ │ │ └── libc v0.2.98 │ │ ├── tokio v1.10.0 (*) │ │ ├── tower-service v0.3.1 │ │ ├── tracing v0.1.26 (*) │ │ └── want v0.3.0 │ │ ├── log v0.4.14 (*) │ │ └── try-lock v0.2.3 │ ├── indexmap v1.6.2 (*) │ ├── openapiv3 v0.5.0 │ │ ├── indexmap v1.6.2 (*) │ │ ├── serde v1.0.127 (*) │ │ ├── serde_json v1.0.64 │ │ │ ├── itoa v0.4.7 │ │ │ ├── ryu v1.0.5 │ │ │ └── serde v1.0.127 (*) │ │ └── serde_yaml v0.8.17 │ │ ├── dtoa v0.4.8 │ │ ├── linked-hash-map v0.5.4 │ │ ├── serde v1.0.127 (*) │ │ └── yaml-rust v0.4.5 │ │ └── linked-hash-map v0.5.4 │ ├── paste v1.0.5 (proc-macro) │ ├── percent-encoding v2.1.0 │ ├── proc-macro2 v1.0.28 (*) │ ├── schemars v0.8.3 │ │ ├── bytes v1.0.1 (*) │ │ ├── chrono v0.4.19 (*) │ │ ├── dyn-clone v1.0.4 │ │ ├── schemars_derive v0.8.3 (proc-macro) │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ ├── quote v1.0.9 (*) │ │ │ ├── serde_derive_internals v0.25.0 │ │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ │ ├── quote v1.0.9 (*) │ │ │ │ └── syn v1.0.74 (*) │ │ │ └── syn v1.0.74 (*) │ │ ├── serde v1.0.127 (*) │ │ ├── serde_json v1.0.64 (*) │ │ └── uuid v0.8.2 │ │ ├── getrandom v0.2.3 │ │ │ ├── cfg-if v1.0.0 │ │ │ └── libc v0.2.98 │ │ └── serde v1.0.127 (*) │ ├── serde v1.0.127 (*) │ ├── serde_json v1.0.64 (*) │ ├── serde_urlencoded v0.7.0 │ │ ├── form_urlencoded v1.0.1 │ │ │ ├── matches v0.1.8 │ │ │ └── percent-encoding v2.1.0 │ │ ├── itoa v0.4.7 │ │ ├── ryu v1.0.5 │ │ └── serde v1.0.127 (*) │ ├── slog v2.7.0 │ ├── slog-async v2.6.0 │ │ ├── crossbeam-channel v0.5.1 │ │ │ ├── cfg-if v1.0.0 │ │ │ └── crossbeam-utils v0.8.5 │ │ │ ├── cfg-if v1.0.0 │ │ │ └── lazy_static v1.4.0 │ │ ├── slog v2.7.0 │ │ ├── take_mut v0.2.2 │ │ └── thread_local v1.1.3 │ │ └── once_cell v1.7.2 │ ├── slog-bunyan v2.3.0 │ │ ├── chrono v0.4.19 (*) │ │ ├── hostname v0.3.1 (*) │ │ ├── slog v2.7.0 │ │ └── slog-json v2.3.0 │ │ ├── chrono v0.4.19 (*) │ │ ├── serde v1.0.127 (*) │ │ ├── serde_json v1.0.64 (*) │ │ └── slog v2.7.0 │ ├── slog-json v2.3.0 (*) │ ├── slog-term v2.8.0 │ │ ├── atty v0.2.14 │ │ │ └── libc v0.2.98 │ │ ├── chrono v0.4.19 (*) │ │ ├── slog v2.7.0 │ │ ├── term v0.7.0 │ │ │ └── dirs-next v2.0.0 │ │ │ ├── cfg-if v1.0.0 │ │ │ └── dirs-sys-next v0.1.2 │ │ │ └── libc v0.2.98 │ │ └── thread_local v1.1.3 (*) │ ├── syn v1.0.74 │ │ ├── proc-macro2 v1.0.28 (*) │ │ ├── quote v1.0.9 (*) │ │ └── unicode-xid v0.2.2 │ ├── tokio v1.10.0 (*) │ ├── toml v0.5.8 │ │ └── serde v1.0.127 (*) │ └── uuid v0.8.2 (*) ├── http v0.2.4 (*) ├── hyper v0.14.10 (*) ├── omicron-common v0.1.0 (/home/dap/omicron/omicron-common) │ ├── anyhow v1.0.42 │ ├── api_identity v0.1.0 (proc-macro) (/home/dap/omicron/api_identity) │ │ ├── proc-macro2 v1.0.28 (*) │ │ ├── quote v1.0.9 (*) │ │ └── syn v1.0.74 (*) │ ├── async-trait v0.1.51 (proc-macro) (*) │ ├── backoff v0.3.0 │ │ ├── futures-core v0.3.15 │ │ ├── getrandom v0.2.3 (*) │ │ ├── instant v0.1.9 (*) │ │ ├── pin-project v1.0.7 │ │ │ └── pin-project-internal v1.0.7 (proc-macro) │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ ├── quote v1.0.9 (*) │ │ │ └── syn v1.0.74 (*) │ │ ├── rand v0.8.3 │ │ │ ├── libc v0.2.98 │ │ │ ├── rand_chacha v0.3.0 │ │ │ │ ├── ppv-lite86 v0.2.10 │ │ │ │ └── rand_core v0.6.2 │ │ │ │ └── getrandom v0.2.3 (*) │ │ │ └── rand_core v0.6.2 (*) │ │ └── tokio v1.10.0 (*) │ ├── chrono v0.4.19 (*) │ ├── diesel v2.0.0 (https://github.com/diesel-rs/diesel?rev=a39dd2e#a39dd2eb) │ │ ├── bitflags v1.2.1 │ │ ├── byteorder v1.4.3 │ │ ├── chrono v0.4.19 (*) │ │ ├── diesel_derives v2.0.0 (proc-macro) (https://github.com/diesel-rs/diesel?rev=a39dd2e#a39dd2eb) │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ ├── quote v1.0.9 (*) │ │ │ └── syn v1.0.74 (*) │ │ ├── ipnetwork v0.18.0 │ │ │ └── serde v1.0.127 (*) │ │ ├── itoa v0.4.7 │ │ ├── libc v0.2.98 │ │ ├── pq-sys v0.4.6 │ │ ├── r2d2 v0.8.9 │ │ │ ├── log v0.4.14 (*) │ │ │ ├── parking_lot v0.11.1 (*) │ │ │ └── scheduled-thread-pool v0.2.5 │ │ │ └── parking_lot v0.11.1 (*) │ │ ├── serde_json v1.0.64 (*) │ │ └── uuid v0.8.2 (*) │ ├── dropshot v0.5.2-dev (https://github.com/oxidecomputer/dropshot?branch=main#1c6f4d15) (*) │ ├── futures v0.3.15 (*) │ ├── http v0.2.4 (*) │ ├── hyper v0.14.10 (*) │ ├── ipnet v2.3.1 │ │ └── serde v1.0.127 (*) │ ├── libc v0.2.98 │ ├── macaddr v1.0.1 │ │ └── serde v1.0.127 (*) │ ├── postgres-protocol v0.6.1 │ │ ├── base64 v0.13.0 │ │ ├── byteorder v1.4.3 │ │ ├── bytes v1.0.1 (*) │ │ ├── fallible-iterator v0.2.0 │ │ ├── hmac v0.10.1 │ │ │ ├── crypto-mac v0.10.0 │ │ │ │ ├── generic-array v0.14.4 │ │ │ │ │ └── typenum v1.13.0 │ │ │ │ │ [build-dependencies] │ │ │ │ │ └── version_check v0.9.3 │ │ │ │ └── subtle v2.4.0 │ │ │ └── digest v0.9.0 │ │ │ └── generic-array v0.14.4 (*) │ │ ├── md-5 v0.9.1 │ │ │ ├── block-buffer v0.9.0 │ │ │ │ └── generic-array v0.14.4 (*) │ │ │ ├── digest v0.9.0 (*) │ │ │ └── opaque-debug v0.3.0 │ │ ├── memchr v2.4.0 │ │ ├── rand v0.8.3 (*) │ │ ├── sha2 v0.9.5 │ │ │ ├── block-buffer v0.9.0 (*) │ │ │ ├── cfg-if v1.0.0 │ │ │ ├── cpufeatures v0.1.4 │ │ │ ├── digest v0.9.0 (*) │ │ │ └── opaque-debug v0.3.0 │ │ └── stringprep v0.1.2 │ │ ├── unicode-bidi v0.3.5 │ │ │ └── matches v0.1.8 │ │ └── unicode-normalization v0.1.19 │ │ └── tinyvec v1.2.0 │ │ └── tinyvec_macros v0.1.0 │ ├── propolis-server v0.1.0 (https://github.com/oxidecomputer/propolis?rev=b6da043d#b6da043d) │ │ ├── anyhow v1.0.42 │ │ ├── dropshot v0.5.2-dev (https://github.com/oxidecomputer/dropshot?branch=main#1c6f4d15) (*) │ │ ├── futures v0.3.15 (*) │ │ ├── hyper v0.14.10 (*) │ │ ├── propolis v0.1.0 (https://github.com/oxidecomputer/propolis?rev=b6da043d#b6da043d) │ │ │ ├── bhyve_api v0.1.0 (https://github.com/oxidecomputer/propolis?rev=b6da043d#b6da043d) │ │ │ │ ├── libc v0.2.98 │ │ │ │ └── num_enum v0.5.1 │ │ │ │ ├── derivative v2.2.0 (proc-macro) │ │ │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ │ │ ├── quote v1.0.9 (*) │ │ │ │ │ └── syn v1.0.74 (*) │ │ │ │ └── num_enum_derive v0.5.1 (proc-macro) │ │ │ │ ├── proc-macro-crate v0.1.5 │ │ │ │ │ └── toml v0.5.8 (*) │ │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ │ ├── quote v1.0.9 (*) │ │ │ │ └── syn v1.0.74 (*) │ │ │ ├── bitflags v1.2.1 │ │ │ ├── byteorder v1.4.3 │ │ │ ├── dladm v0.1.0 (https://github.com/oxidecomputer/propolis?rev=b6da043d#b6da043d) │ │ │ │ ├── libc v0.2.98 │ │ │ │ └── num_enum v0.5.1 (*) │ │ │ ├── lazy_static v1.4.0 │ │ │ ├── libc v0.2.98 │ │ │ ├── num_enum v0.5.1 (*) │ │ │ ├── thiserror v1.0.26 │ │ │ │ └── thiserror-impl v1.0.26 (proc-macro) │ │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ │ ├── quote v1.0.9 (*) │ │ │ │ └── syn v1.0.74 (*) │ │ │ ├── usdt v0.1.13 │ │ │ │ ├── dtrace-parser v0.1.9 │ │ │ │ │ ├── pest v2.1.3 │ │ │ │ │ │ └── ucd-trie v0.1.3 │ │ │ │ │ ├── pest_derive v2.1.0 (proc-macro) │ │ │ │ │ │ ├── pest v2.1.3 (*) │ │ │ │ │ │ └── pest_generator v2.1.3 │ │ │ │ │ │ ├── pest v2.1.3 (*) │ │ │ │ │ │ ├── pest_meta v2.1.3 │ │ │ │ │ │ │ ├── maplit v1.0.2 │ │ │ │ │ │ │ └── pest v2.1.3 (*) │ │ │ │ │ │ │ [build-dependencies] │ │ │ │ │ │ │ └── sha-1 v0.8.2 │ │ │ │ │ │ │ ├── block-buffer v0.7.3 │ │ │ │ │ │ │ │ ├── block-padding v0.1.5 │ │ │ │ │ │ │ │ │ └── byte-tools v0.3.1 │ │ │ │ │ │ │ │ ├── byte-tools v0.3.1 │ │ │ │ │ │ │ │ ├── byteorder v1.4.3 │ │ │ │ │ │ │ │ └── generic-array v0.12.4 │ │ │ │ │ │ │ │ └── typenum v1.13.0 │ │ │ │ │ │ │ ├── digest v0.8.1 │ │ │ │ │ │ │ │ └── generic-array v0.12.4 (*) │ │ │ │ │ │ │ ├── fake-simd v0.1.2 │ │ │ │ │ │ │ └── opaque-debug v0.2.3 │ │ │ │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ │ │ │ ├── quote v1.0.9 (*) │ │ │ │ │ │ └── syn v1.0.74 (*) │ │ │ │ │ └── thiserror v1.0.26 (*) │ │ │ │ ├── usdt-impl v0.1.8 │ │ │ │ │ ├── byteorder v1.4.3 │ │ │ │ │ ├── dof v0.1.5 │ │ │ │ │ │ ├── thiserror v1.0.26 (*) │ │ │ │ │ │ └── zerocopy v0.3.0 │ │ │ │ │ │ ├── byteorder v1.4.3 │ │ │ │ │ │ └── zerocopy-derive v0.2.0 (proc-macro) │ │ │ │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ │ │ │ ├── syn v1.0.74 (*) │ │ │ │ │ │ └── synstructure v0.12.4 │ │ │ │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ │ │ │ ├── quote v1.0.9 (*) │ │ │ │ │ │ ├── syn v1.0.74 (*) │ │ │ │ │ │ └── unicode-xid v0.2.2 │ │ │ │ │ ├── dtrace-parser v0.1.9 (*) │ │ │ │ │ ├── libc v0.2.98 │ │ │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ │ │ ├── quote v1.0.9 (*) │ │ │ │ │ ├── serde v1.0.127 (*) │ │ │ │ │ ├── syn v1.0.74 (*) │ │ │ │ │ └── thiserror v1.0.26 (*) │ │ │ │ └── usdt-macro v0.1.9 (proc-macro) │ │ │ │ ├── dtrace-parser v0.1.9 (*) │ │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ │ ├── quote v1.0.9 (*) │ │ │ │ ├── serde_tokenstream v0.1.2 (*) │ │ │ │ ├── syn v1.0.74 (*) │ │ │ │ └── usdt-impl v0.1.8 │ │ │ │ ├── byteorder v1.4.3 │ │ │ │ ├── dof v0.1.5 (*) │ │ │ │ ├── dtrace-parser v0.1.9 (*) │ │ │ │ ├── libc v0.2.98 │ │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ │ ├── quote v1.0.9 (*) │ │ │ │ ├── serde v1.0.127 (*) │ │ │ │ ├── syn v1.0.74 (*) │ │ │ │ └── thiserror v1.0.26 (*) │ │ │ └── viona_api v0.1.0 (https://github.com/oxidecomputer/propolis?rev=b6da043d#b6da043d) │ │ ├── propolis-client v0.1.0 (https://github.com/oxidecomputer/propolis?rev=b6da043d#b6da043d) │ │ │ ├── reqwest v0.11.4 │ │ │ │ ├── base64 v0.13.0 │ │ │ │ ├── bytes v1.0.1 (*) │ │ │ │ ├── encoding_rs v0.8.28 │ │ │ │ │ └── cfg-if v1.0.0 │ │ │ │ ├── futures-core v0.3.15 │ │ │ │ ├── futures-util v0.3.15 (*) │ │ │ │ ├── http v0.2.4 (*) │ │ │ │ ├── http-body v0.4.2 (*) │ │ │ │ ├── hyper v0.14.10 (*) │ │ │ │ ├── hyper-rustls v0.22.1 │ │ │ │ │ ├── futures-util v0.3.15 (*) │ │ │ │ │ ├── hyper v0.14.10 (*) │ │ │ │ │ ├── log v0.4.14 (*) │ │ │ │ │ ├── rustls v0.19.1 │ │ │ │ │ │ ├── base64 v0.13.0 │ │ │ │ │ │ ├── log v0.4.14 (*) │ │ │ │ │ │ ├── ring v0.16.20 │ │ │ │ │ │ │ ├── once_cell v1.7.2 │ │ │ │ │ │ │ ├── spin v0.5.2 │ │ │ │ │ │ │ └── untrusted v0.7.1 │ │ │ │ │ │ │ [build-dependencies] │ │ │ │ │ │ │ └── cc v1.0.68 │ │ │ │ │ │ ├── sct v0.6.1 │ │ │ │ │ │ │ ├── ring v0.16.20 (*) │ │ │ │ │ │ │ └── untrusted v0.7.1 │ │ │ │ │ │ └── webpki v0.21.4 │ │ │ │ │ │ ├── ring v0.16.20 (*) │ │ │ │ │ │ └── untrusted v0.7.1 │ │ │ │ │ ├── tokio v1.10.0 (*) │ │ │ │ │ ├── tokio-rustls v0.22.0 │ │ │ │ │ │ ├── rustls v0.19.1 (*) │ │ │ │ │ │ ├── tokio v1.10.0 (*) │ │ │ │ │ │ └── webpki v0.21.4 (*) │ │ │ │ │ └── webpki v0.21.4 (*) │ │ │ │ ├── hyper-tls v0.5.0 │ │ │ │ │ ├── bytes v1.0.1 (*) │ │ │ │ │ ├── hyper v0.14.10 (*) │ │ │ │ │ ├── native-tls v0.2.7 │ │ │ │ │ │ ├── log v0.4.14 (*) │ │ │ │ │ │ ├── openssl v0.10.34 │ │ │ │ │ │ │ ├── bitflags v1.2.1 │ │ │ │ │ │ │ ├── cfg-if v1.0.0 │ │ │ │ │ │ │ ├── foreign-types v0.3.2 │ │ │ │ │ │ │ │ └── foreign-types-shared v0.1.1 │ │ │ │ │ │ │ ├── libc v0.2.98 │ │ │ │ │ │ │ ├── once_cell v1.7.2 │ │ │ │ │ │ │ └── openssl-sys v0.9.63 │ │ │ │ │ │ │ └── libc v0.2.98 │ │ │ │ │ │ │ [build-dependencies] │ │ │ │ │ │ │ ├── autocfg v1.0.1 │ │ │ │ │ │ │ ├── cc v1.0.68 │ │ │ │ │ │ │ └── pkg-config v0.3.19 │ │ │ │ │ │ ├── openssl-probe v0.1.4 │ │ │ │ │ │ └── openssl-sys v0.9.63 (*) │ │ │ │ │ ├── tokio v1.10.0 (*) │ │ │ │ │ └── tokio-native-tls v0.3.0 │ │ │ │ │ ├── native-tls v0.2.7 (*) │ │ │ │ │ └── tokio v1.10.0 (*) │ │ │ │ ├── ipnet v2.3.1 (*) │ │ │ │ ├── lazy_static v1.4.0 │ │ │ │ ├── log v0.4.14 (*) │ │ │ │ ├── mime v0.3.16 │ │ │ │ ├── native-tls v0.2.7 (*) │ │ │ │ ├── percent-encoding v2.1.0 │ │ │ │ ├── pin-project-lite v0.2.6 │ │ │ │ ├── rustls v0.19.1 (*) │ │ │ │ ├── serde v1.0.127 (*) │ │ │ │ ├── serde_json v1.0.64 (*) │ │ │ │ ├── serde_urlencoded v0.7.0 (*) │ │ │ │ ├── tokio v1.10.0 (*) │ │ │ │ ├── tokio-native-tls v0.3.0 (*) │ │ │ │ ├── tokio-rustls v0.22.0 (*) │ │ │ │ ├── url v2.2.2 │ │ │ │ │ ├── form_urlencoded v1.0.1 (*) │ │ │ │ │ ├── idna v0.2.3 │ │ │ │ │ │ ├── matches v0.1.8 │ │ │ │ │ │ ├── unicode-bidi v0.3.5 (*) │ │ │ │ │ │ └── unicode-normalization v0.1.19 (*) │ │ │ │ │ ├── matches v0.1.8 │ │ │ │ │ └── percent-encoding v2.1.0 │ │ │ │ └── webpki-roots v0.21.1 │ │ │ │ └── webpki v0.21.4 (*) │ │ │ ├── ring v0.16.20 (*) │ │ │ ├── schemars v0.8.3 (*) │ │ │ ├── serde v1.0.127 (*) │ │ │ ├── serde_json v1.0.64 (*) │ │ │ ├── slog v2.7.0 │ │ │ ├── structopt v0.3.22 │ │ │ │ ├── clap v2.33.3 │ │ │ │ │ ├── ansi_term v0.11.0 │ │ │ │ │ ├── atty v0.2.14 (*) │ │ │ │ │ ├── bitflags v1.2.1 │ │ │ │ │ ├── strsim v0.8.0 │ │ │ │ │ ├── textwrap v0.11.0 │ │ │ │ │ │ └── unicode-width v0.1.8 │ │ │ │ │ ├── unicode-width v0.1.8 │ │ │ │ │ └── vec_map v0.8.2 │ │ │ │ ├── lazy_static v1.4.0 │ │ │ │ └── structopt-derive v0.4.15 (proc-macro) │ │ │ │ ├── heck v0.3.3 │ │ │ │ │ └── unicode-segmentation v1.7.1 │ │ │ │ ├── proc-macro-error v1.0.4 │ │ │ │ │ ├── proc-macro-error-attr v1.0.4 (proc-macro) │ │ │ │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ │ │ │ └── quote v1.0.9 (*) │ │ │ │ │ │ [build-dependencies] │ │ │ │ │ │ └── version_check v0.9.3 │ │ │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ │ │ ├── quote v1.0.9 (*) │ │ │ │ │ └── syn v1.0.74 (*) │ │ │ │ │ [build-dependencies] │ │ │ │ │ └── version_check v0.9.3 │ │ │ │ ├── proc-macro2 v1.0.28 (*) │ │ │ │ ├── quote v1.0.9 (*) │ │ │ │ └── syn v1.0.74 (*) │ │ │ ├── thiserror v1.0.26 (*) │ │ │ └── uuid v0.8.2 (*) │ │ ├── serde v1.0.127 (*) │ │ ├── serde_derive v1.0.127 (proc-macro) (*) │ │ ├── slog v2.7.0 │ │ ├── structopt v0.3.22 (*) │ │ ├── thiserror v1.0.26 (*) │ │ ├── tokio v1.10.0 (*) │ │ ├── tokio-tungstenite v0.14.0 │ │ │ ├── futures-util v0.3.15 (*) │ │ │ ├── log v0.4.14 (*) │ │ │ ├── pin-project v1.0.7 (*) │ │ │ ├── tokio v1.10.0 (*) │ │ │ └── tungstenite v0.13.0 │ │ │ ├── base64 v0.13.0 │ │ │ ├── byteorder v1.4.3 │ │ │ ├── bytes v1.0.1 (*) │ │ │ ├── http v0.2.4 (*) │ │ │ ├── httparse v1.4.1 │ │ │ ├── input_buffer v0.4.0 │ │ │ │ └── bytes v1.0.1 (*) │ │ │ ├── log v0.4.14 (*) │ │ │ ├── rand v0.8.3 (*) │ │ │ ├── sha-1 v0.9.6 │ │ │ │ ├── block-buffer v0.9.0 (*) │ │ │ │ ├── cfg-if v1.0.0 │ │ │ │ ├── cpufeatures v0.1.4 │ │ │ │ ├── digest v0.9.0 (*) │ │ │ │ └── opaque-debug v0.3.0 │ │ │ ├── thiserror v1.0.26 (*) │ │ │ ├── url v2.2.2 (*) │ │ │ └── utf-8 v0.7.6 │ │ ├── toml v0.5.8 (*) │ │ └── uuid v0.8.2 (*) │ ├── rayon v1.5.1 │ │ ├── crossbeam-deque v0.8.0 │ │ │ ├── cfg-if v1.0.0 │ │ │ ├── crossbeam-epoch v0.9.5 │ │ │ │ ├── cfg-if v1.0.0 │ │ │ │ ├── crossbeam-utils v0.8.5 (*) │ │ │ │ ├── lazy_static v1.4.0 │ │ │ │ ├── memoffset v0.6.4 │ │ │ │ │ [build-dependencies] │ │ │ │ │ └── autocfg v1.0.1 │ │ │ │ └── scopeguard v1.1.0 │ │ │ └── crossbeam-utils v0.8.5 (*) │ │ ├── either v1.6.1 │ │ └── rayon-core v1.9.1 │ │ ├── crossbeam-channel v0.5.1 (*) │ │ ├── crossbeam-deque v0.8.0 (*) │ │ ├── crossbeam-utils v0.8.5 (*) │ │ ├── lazy_static v1.4.0 │ │ └── num_cpus v1.13.0 (*) │ │ [build-dependencies] │ │ └── autocfg v1.0.1 │ ├── reqwest v0.11.4 (*) │ ├── ring v0.16.20 (*) │ ├── schemars v0.8.3 (*) │ ├── serde v1.0.127 (*) │ ├── serde_derive v1.0.127 (proc-macro) (*) │ ├── serde_json v1.0.64 (*) │ ├── signal-hook v0.3.9 │ │ ├── libc v0.2.98 │ │ └── signal-hook-registry v1.4.0 (*) │ ├── signal-hook-tokio v0.3.0 │ │ ├── futures-core v0.3.15 │ │ ├── libc v0.2.98 │ │ ├── signal-hook v0.3.9 (*) │ │ └── tokio v1.10.0 (*) │ ├── slog v2.7.0 │ ├── smf v0.2.1 │ │ └── thiserror v1.0.26 (*) │ ├── steno v0.1.0 (https://github.com/oxidecomputer/steno?branch=main#8da3eda3) │ │ ├── anyhow v1.0.42 │ │ ├── async-trait v0.1.51 (proc-macro) (*) │ │ ├── chrono v0.4.19 (*) │ │ ├── futures v0.3.15 (*) │ │ ├── newtype_derive v0.1.6 │ │ │ [build-dependencies] │ │ │ └── rustc_version v0.1.7 │ │ │ └── semver v0.1.20 │ │ ├── petgraph v0.5.1 │ │ │ ├── fixedbitset v0.2.0 │ │ │ └── indexmap v1.6.2 (*) │ │ ├── schemars v0.8.3 (*) │ │ ├── serde v1.0.127 (*) │ │ ├── serde_json v1.0.64 (*) │ │ ├── slog v2.7.0 │ │ ├── thiserror v1.0.26 (*) │ │ ├── tokio v1.10.0 (*) │ │ └── uuid v0.8.2 (*) │ ├── structopt v0.3.22 (*) │ ├── subprocess v0.2.7 │ │ └── libc v0.2.98 │ ├── tar v0.4.35 │ │ ├── filetime v0.2.14 │ │ │ ├── cfg-if v1.0.0 │ │ │ └── libc v0.2.98 │ │ ├── libc v0.2.98 │ │ └── xattr v0.2.2 │ │ └── libc v0.2.98 │ ├── tempfile v3.2.0 │ │ ├── cfg-if v1.0.0 │ │ ├── libc v0.2.98 │ │ ├── rand v0.8.3 (*) │ │ └── remove_dir_all v0.5.3 │ ├── thiserror v1.0.26 (*) │ ├── tokio v1.10.0 (*) │ ├── tokio-postgres v0.7.2 │ │ ├── async-trait v0.1.51 (proc-macro) (*) │ │ ├── byteorder v1.4.3 │ │ ├── bytes v1.0.1 (*) │ │ ├── fallible-iterator v0.2.0 │ │ ├── futures v0.3.15 (*) │ │ ├── log v0.4.14 (*) │ │ ├── parking_lot v0.11.1 (*) │ │ ├── percent-encoding v2.1.0 │ │ ├── phf v0.8.0 │ │ │ └── phf_shared v0.8.0 │ │ │ └── siphasher v0.3.5 │ │ ├── pin-project-lite v0.2.6 │ │ ├── postgres-protocol v0.6.1 (*) │ │ ├── postgres-types v0.2.1 │ │ │ ├── bytes v1.0.1 (*) │ │ │ ├── chrono v0.4.19 (*) │ │ │ ├── fallible-iterator v0.2.0 │ │ │ ├── postgres-protocol v0.6.1 (*) │ │ │ ├── serde v1.0.127 (*) │ │ │ ├── serde_json v1.0.64 (*) │ │ │ └── uuid v0.8.2 (*) │ │ ├── socket2 v0.4.0 (*) │ │ ├── tokio v1.10.0 (*) │ │ └── tokio-util v0.6.7 (*) │ ├── toml v0.5.8 (*) │ ├── uuid v0.8.2 (*) │ └── walkdir v2.3.2 │ └── same-file v1.0.6 │ [dev-dependencies] │ ├── expectorate v1.0.3 │ │ ├── difference v2.0.0 │ │ └── newline-converter v0.1.0 │ ├── serde_urlencoded v0.7.0 (*) │ └── tokio v1.10.0 (*) ├── serde v1.0.127 (*) ├── serde_json v1.0.64 (*) ├── slog v2.7.0 ├── smf v0.2.1 (*) ├── structopt v0.3.22 (*) ├── tar v0.4.35 (*) ├── thiserror v1.0.26 (*) ├── tokio v1.10.0 (*) ├── toml v0.5.8 (*) └── uuid v0.8.2 (*)One of the crates there is
pq-sys, depended-on by the recently-added "diesel", so that's promising:That's this crate, source here. It's a bindgen-generated binding for libpq. As expected, its Cargo.toml includes
"links = "pq":https://github.com/sgrif/pq-sys/blob/3e367d53019a2740054d5dc6946e07931f1fb70b/Cargo.toml#L8
And build.rs uses the rustc-link-lib instruction like you'd expect. Now, the README for pq-sys has notes:
Now on Helios we'd expect the library itself to be delivered by pkg:/ooce/library/postgresql-13. Sure enough, that delivers pg_config as well:
It puts it into /opt/ooce/pgsql-13/bin. @jclulow found with
pkg search link:basename:pg_configthat it also installs a mediated link into /opt/ooce/bin. One of these directories needs to be in your PATH at build time for the pq-sys build to find it. If we do set PATH accordingly, then the library is correctly found at build time and the build succeeds. But the binaries fail at runtime instead:Usually if ld.so can't find a library like this, the library is NEEDED (e.g.,
lddwould show it) but not found in the standard locations or the RUNPATH. Sure enough, ldd shows that it's needed and not found:elfdump -dshows details:We can see that the RUNPATH/RPATH does not include the directory containing libpq. What directory is this? The build script gets it with:
It seems like the cargo build did not update the RUNPATH. It seems like it should do that, right?
I've spent quite a while over the last two days trying to figure out how this is supposed to work in Cargo and my conclusion is that it doesn't. The clearest evidence is rust-lang/cargo#5077. It's pretty clear that a lot of people have wanted to be able to do this for some time, but it hasn't been implemented.
It should be noted that although this only affects Oxide employees running on illumos, the problem is not illumos-specific at all -- it only has to do with libraries installed in a directory that's not on ld.so.1's default search path.
Failed solutions / workarounds
Codegen option "rpath"
There's a Rust codgen option
-C rpath, which is documented to "enable rpath". It's not clear what this means. I did rebuild withRUSTFLAGS='-C rpath=yes', and I got the same failure mode:though the runpath was different (arguably more broken):
It's not clear why these directories got added, but they're certainly not right for us here. We abandoned this approach.
Cargo's rustc-link-arg instruction
Keith pointed out that Cargo has grown a feature that could help us:
We could use this to pass
-Rto the linker. pq-sys could emitcargo:rustc-link-arg=-Wl,R,/the/library/pathor maybe we do this in Omicron. (The-Wl,prefix is what you pass to the compiler driver to pass the rest of it to the linker.) This is somewhat platform-specific (it probably won't work on Windows, for example).The docs imply that this will be in 1.56. Current stable is 1.54. So this won't be available in stable Cargo for a few months (!). Now, Omicron is on "nightly", so we could use it there. But where would we put this directive? I think it would have to go into pq-sys, whose consumers are not necessarily on "nightly". I abandoned this path at this point, though we could create our own fork of pq-sys or else try to upstream a change that uses this flag conditional on it being available.
Statically linking libpq
pq-sys can statically link against libpq. That would obviate the need to get the RPATH of the binary right. To test this out, I switched over to a build of "pq-sys" directly, rather than trying to do this inside "omicron".
If we build with PQ_LIB_STATIC, we get a whole bunch of linker errors for undefined symbols:
dap@helios pq-sys $ PQ_LIB_STATIC=yes cargo test Compiling pq-sys v0.4.6 (/home/dap/pq-sys) error: linking with `gcc` failed: exit status: 1 | = note: "gcc" "-m64" "-std=c99" "/home/dap/pq-sys/target/debug/deps/pq_sys-f0d6d12067370f8b.28zc43ngthik1xi0.rcgu.o" "/home/dap/pq-sys/target/debug/deps/pq_sys-f0d6d12067370f8b.2ae9uey3x8spvh2b.rcgu.o" "/home/dap/pq-sys/target/debug/deps/pq_sys-f0d6d12067370f8b.37ak78lfd7i8lgrk.rcgu.o" "/home/dap/pq-sys/target/debug/deps/pq_sys-f0d6d12067370f8b.3ly6jh3378tbmq2z.rcgu.o" "/home/dap/pq-sys/target/debug/deps/pq_sys-f0d6d12067370f8b.3pvg9mos4au2z44z.rcgu.o" "/home/dap/pq-sys/target/debug/deps/pq_sys-f0d6d12067370f8b.3q6bskioxmtlfjp9.rcgu.o" "/home/dap/pq-sys/target/debug/deps/pq_sys-f0d6d12067370f8b.3t9wgvj2kqufeobn.rcgu.o" "/home/dap/pq-sys/target/debug/deps/pq_sys-f0d6d12067370f8b.4fjphvztobxrfil.rcgu.o" "/home/dap/pq-sys/target/debug/deps/pq_sys-f0d6d12067370f8b.4ga1rzu3lsuqgy9o.rcgu.o" "/home/dap/pq-sys/target/debug/deps/pq_sys-f0d6d12067370f8b.5ghmmeuvdumo07ij.rcgu.o" "/home/dap/pq-sys/target/debug/deps/pq_sys-f0d6d12067370f8b.kycxi9ya29lqvrp.rcgu.o" "/home/dap/pq-sys/target/debug/deps/pq_sys-f0d6d12067370f8b.pj0395ry7x85v9d.rcgu.o" "/home/dap/pq-sys/target/debug/deps/pq_sys-f0d6d12067370f8b.sepdlklc4pe8hnh.rcgu.o" "/home/dap/pq-sys/target/debug/deps/pq_sys-f0d6d12067370f8b.153ebt2aq433ew1r.rcgu.o" "-Wl,-z" "-Wl,ignore" "-L" "/home/dap/pq-sys/target/debug/deps" "-L" "/opt/ooce/pgsql-13/lib/amd64" "-L" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib" "-Wl,-Bstatic" "-Wl,--whole-archive" "-lpq" "-Wl,--no-whole-archive" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/libtest-535dee8e4f3c883b.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/libterm-be39c4177d57d29f.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/libgetopts-47c81854ba90f851.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/libunicode_width-b8d4734d4eda0e2a.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/librustc_std_workspace_std-67f67341054cb595.rlib" "-Wl,--start-group" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/libstd-50bed294e78c95c2.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/libpanic_unwind-068f7a96e5aa9c3e.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/libminiz_oxide-79be3b483094292f.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/libadler-24b6fed8d1040d6b.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/libobject-cc9fa5d3172400da.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/libaddr2line-5b1c430684c7f47b.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/libgimli-2d01878f44f162f6.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/libstd_detect-0a4773aa77217f2e.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/librustc_demangle-b4f6a6a789a585b0.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/libhashbrown-eda90539ad5b3632.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/librustc_std_workspace_alloc-168e3556278a7b7d.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/libunwind-23043d223378f39c.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/libcfg_if-389ca2ce1f2d8f07.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/liblibc-04d9200ebe50adaf.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/liballoc-30bf7d00f2682790.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/librustc_std_workspace_core-9bad5249b1907a01.rlib" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/libcore-5d546fb1aabd96b7.rlib" "-Wl,--end-group" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib/libcompiler_builtins-5d2b8a810efe2040.rlib" "-Wl,-Bdynamic" "-lsocket" "-lposix4" "-lpthread" "-lresolv" "-lnsl" "-lumem" "-lgcc_s" "-lm" "-lrt" "-lpthread" "-lc" "-lssp" "-L" "/home/dap/.rustup/toolchains/stable-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib" "-o" "/home/dap/pq-sys/target/debug/deps/pq_sys-f0d6d12067370f8b" "-nodefaultlibs" = note: Undefined first referenced symbol in file ERR_clear_error /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_CTX_set_default_passwd_cb /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) OPENSSL_sk_pop_free /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_CTX_set_options /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_use_PrivateKey /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_get_peer_certificate /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) BIO_set_flags /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_CIPHER_get_bits /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_CIPHER_get_name /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pg_link_canary_is_frontend /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-connect.o) SSL_use_PrivateKey_file /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) scram_ServerKey /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-auth-scram.o) scram_HMAC_final /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-auth-scram.o) BIO_int_ctrl /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pg_vsnprintf /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-exec.o) OPENSSL_sk_num /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_write /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_CTX_set_default_passwd_cb_userdata /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) BIO_meth_set_write /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) BIO_get_new_index /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_get_error /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) ENGINE_init /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) ENGINE_free /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) BIO_clear_flags /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) BIO_s_socket /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pg_b64_encode /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-auth-scram.o) pg_b64_decode /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-auth-scram.o) X509_get_ext_d2i /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) ENGINE_load_private_key /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) X509_free /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_set_options /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) ERR_get_error /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) ERR_put_error /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) EVP_get_digestbyname /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) scram_build_secret /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-auth-scram.o) OPENSSL_sk_value /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pg_snprintf /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-connect.o) BIO_meth_set_callback_ctrl /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) scram_HMAC_update /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-auth-scram.o) BIO_meth_get_callback_ctrl /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) X509_digest /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) X509_STORE_set_flags /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) scram_SaltedPassword /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-auth-scram.o) TLS_method /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) OBJ_nid2sn /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_CTX_use_certificate_chain_file /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pg_char_to_encoding /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-exec.o) pg_strip_crlf /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-connect.o) EVP_sha256 /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) getpeereid /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-connect.o) BIO_new /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_pending /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pg_b64_enc_len /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-auth-scram.o) pg_md5_encrypt /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-auth.o) SSL_CTX_get_cert_store /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) scram_ClientKey /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-auth-scram.o) GENERAL_NAME_free /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) X509_NAME_get_entry /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pg_b64_dec_len /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-auth-scram.o) SSL_free /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_read /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) X509_STORE_load_locations /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) OBJ_find_sigid_algs /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_new /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) OPENSSL_init_ssl /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) BIO_meth_new /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_CTX_new /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pg_saslprep /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-auth-scram.o) X509_NAME_ENTRY_get_data /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) BIO_get_data /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) BIO_set_data /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_CTX_load_verify_locations /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pg_freeaddrinfo_all /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-connect.o) SSL_set_bio /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) scram_H /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-auth-scram.o) pg_encoding_mblen /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-exec.o) pg_getaddrinfo_all /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-connect.o) BIO_meth_get_create /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) BIO_meth_set_create /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) X509_NAME_get_index_by_NID /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_get_current_compression /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) ASN1_STRING_get0_data /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_connect /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) X509_get_signature_nid /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pg_set_noblock /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-connect.o) SSL_shutdown /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pg_encoding_to_char /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-connect.o) pg_strerror_r /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-connect.o) pg_sprintf /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-connect.o) pg_encoding_dsplen /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-misc.o) ASN1_STRING_length /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pg_getnameinfo_all /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-connect.o) pg_tolower /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-exec.o) ERR_reason_error_string /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_check_private_key /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_clear_options /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pg_get_encoding_from_locale /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-connect.o) BIO_meth_free /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pg_strong_random /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-auth-scram.o) SSL_CTX_ctrl /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_CTX_free /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pqGetpwuid /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-connect.o) SSL_get_current_cipher /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pg_inet_net_ntop /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-connect.o) scram_HMAC_init /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-auth-scram.o) pg_fprintf /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-connect.o) SSL_get_version /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pg_encoding_max_length /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-protocol3.o) BIO_meth_get_destroy /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_set_ex_data /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) SSL_set_verify /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) BIO_meth_set_destroy /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) BIO_meth_get_gets /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) BIO_meth_set_gets /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) BIO_meth_set_ctrl /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) BIO_meth_get_ctrl /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) BIO_meth_get_puts /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) BIO_meth_set_puts /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) BIO_meth_set_read /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) pg_strcasecmp /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-connect.o) X509_get_subject_name /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) ENGINE_finish /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) ENGINE_by_id /opt/ooce/pgsql-13/lib/amd64/libpq.a(fe-secure-openssl.o) ld: fatal: symbol referencing errors. No output written to /home/dap/pq-sys/target/debug/deps/pq_sys-f0d6d12067370f8b collect2: error: ld returned 1 exit statusThese symbols appear to be in a combination of OpenSSL and libpqcommon. In the static build case, pq-sys only picks up libpq, not libpqcommon:
https://github.com/sgrif/pq-sys/blob/master/build.rs#L51-L54
It was around this time I decided to see if there were known issues here in pq-sys. Sure enough, there are two PRs that seem relevant:
opensslsomehow. sgrif/pq-sys#25, PR Add feature for including openssl-sys sgrif/pq-sys#29I have a local clone of pq-sys where I've pulled in both of these changes, but now it's failing with:
The symbol "pqsignal" appears to be in both static objects and the linker's unhappy. I have not dug further yet.
Solutions and workarounds that work
Regardless of anything else, you need to have libpq installed and
pg_configmust be on your PATH. On OmniOS and Helios, this is provided by "pkg:/ooce/library/postgresql-13" and you'll want/opt/ooce/binon your PATH.Then you can do any of these things:
RUSTFLAGS="-Clink-args=-R$(pg_config --libdir)"in the environment. This assumes that pq-sys will usepg_configto find PostgreSQL. If you've got pkg-config set up, or you've overridden PQ_LIB_DIR or something like that, this will definitely do the wrong thing.LD_LIBRARY_PATH=/opt/ooce/pgsql-13/lib/amd64. It's generally not recommended to set LD_LIBRARY_PATH as a matter of course, partly because it affects everything. Needing to do this should be considered a bug.elfeditto update the RPATH of the produced binaries. This breaks the usualcargo runandcargo testworkflow -- you'd have to always build first, then use elfedit, thencargo runorcargo test.Explicitly setting RUSTFLAGS in the environment seems like the best option to me, so my plan will be to put that in the README.
The only other avenues would be:
The text was updated successfully, but these errors were encountered: