[2.2.0 regression] mysqlclient-sys does not provide bundled bindings for libmysqlclient 21. Consider using the buildtime_bindgen feature or contribute bindings to the crate

[yurivict]

Setup

Regular build.

Versions

• Rust: rust-1.78.0
• Diesel: 2.2.0
• Operating System FreeBSD 14.0

Problem Description

[mysqlclient-sys 0.3.1] thread 'main' panicked at /wrkdirs/usr/ports/databases/diesel/work/diesel-2.2.0/cargo-crates/mysqlclient-sys-0.3.1/build.rs:160:9:
[mysqlclient-sys 0.3.1] mysqlclient-sys does not provide bundled bindings for libmysqlclient 21. Consider using the `buildtime_bindgen` feature or contribute bindings to the crate
[mysqlclient-sys 0.3.1] stack backtrace:
[mysqlclient-sys 0.3.1]    0: rust_begin_unwind
[mysqlclient-sys 0.3.1]    1: core::panicking::panic_fmt
[mysqlclient-sys 0.3.1]    2: build_script_build::parse_version
[mysqlclient-sys 0.3.1]    3: build_script_build::main
[mysqlclient-sys 0.3.1]    4: core::ops::function::FnOnce::call_once
[mysqlclient-sys 0.3.1] note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
error: failed to run custom build command for `mysqlclient-sys v0.3.1`
note: To improve backtraces for build dependencies, set the CARGO_PROFILE_RELEASE_BUILD_OVERRIDE_DEBUG=true environment variable to enable debug information generation.

Caused by:
  process didn't exit successfully: `CARGO=/usr/local/bin/cargo CARGO_CFG_PANIC=unwind CARGO_CFG_TARGET_ABI='' CARGO_CFG_TARGET_ARCH=x86_64 CARGO_CFG_TARGET_ENDIAN=little CARGO_CFG_TARGET_ENV='' CARGO_CFG_TARGET_FAMILY=unix CARGO_CFG_TARGET_FEATURE=fxsr,sse,sse2 CARGO_CFG_TARGET_HAS_ATOMIC=16,32,64,8,ptr CARGO_CFG_TARGET_OS=freebsd CARGO_CFG_TARGET_POINTER_WIDTH=64 CARGO_CFG_TARGET_VENDOR=unknown CARGO_CFG_UNIX='' CARGO_ENCODED_RUSTFLAGS='-Clink-arg=-L/usr/local/lib-Clink-arg=-fstack-protector-strong' CARGO_FEATURE_DEFAULT=1 CARGO_MANIFEST_DIR=/wrkdirs/usr/ports/databases/diesel/work/diesel-2.2.0/cargo-crates/mysqlclient-sys-0.3.1 CARGO_MANIFEST_LINKS=mysqlclient CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='Auto-generated rust bindings for libmysqlclient' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=mysqlclient-sys CARGO_PKG_README=README.md CARGO_PKG_REPOSITORY='https://github.com/sgrif/mysqlclient-sys' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.3.1 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=3 CARGO_PKG_VERSION_PATCH=1 CARGO_PKG_VERSION_PRE='' DEBUG=false HOST=x86_64-unknown-freebsd LD_LIBRARY_PATH='/wrkdirs/usr/ports/databases/diesel/work/target/release/deps:/wrkdirs/usr/ports/databases/diesel/work/target/release:/usr/local/lib/rustlib/x86_64-unknown-freebsd/lib' NUM_JOBS=6 OPT_LEVEL=2 OUT_DIR=/wrkdirs/usr/ports/databases/diesel/work/target/x86_64-unknown-freebsd/release/build/mysqlclient-sys-61f26f1caf885244/out PROFILE=release RUSTC=/usr/local/bin/rustc RUSTC_LINKER=cc RUSTDOC=/usr/local/bin/rustdoc TARGET=x86_64-unknown-freebsd /wrkdirs/usr/ports/databases/diesel/work/target/release/build/mysqlclient-sys-6bcf5f36c20e6c65/build-script-build` (exit status: 101)
  --- stdout

What are you trying to accomplish?

What is the expected output?

What is the actual output?

Are you seeing any additional errors?

Steps to reproduce

[weiznich]

Thanks for reporting. This is currently somewhat expected behaviour. The provided bindings only cover systems we can actively test on. As neither any contributor nor GitHub actions provide an easy way to test on NetBsd we unfortunately cannot provide official support for this platform. That means we cannot guarantee that anything works there or continues to work. We accept contributions that fixes problems for these platforms as long as that does not affect supported platforms. Otherwise we cannot really work on solutions for such problems due to missing access to such systems.

That all written: The error message con certainly be better an should point out that the target is the problem, not the version. That's the reason why I keep this issue open for now.

You can likely workaround that problem by enabling the buildtime_bindgen feature for the mysqlclient-system crate. This will generate bindings based on your local libmysqlclient version.

[chenrui333]

seeing the same with homebrew, not sure if we can just add a macos/brew build for regression testing?

---

also I tried this patch, it does not work for me

$ git diff
diff --git a/diesel_cli/Cargo.toml b/diesel_cli/Cargo.toml
index 78dc0e5..eb1e5a1 100644
--- a/diesel_cli/Cargo.toml
+++ b/diesel_cli/Cargo.toml
@@ -47,7 +47,7 @@ url = { version = "2.2.2" }
 libsqlite3-sys = { version = ">=0.17.2, <0.29.0", optional = true }
 pq-sys = { version = ">=0.4, <0.7.0", optional = true }
 openssl-sys = { version = "0.9.93", features = ["vendored"], optional = true }
-mysqlclient-sys = { version = ">=0.2.5, <0.4.0", optional = true }
+mysqlclient-sys = { version = ">=0.2.5, <0.4.0", optional = true, features = ["buildtime_bindgen"] }
 diffy = "0.3.0"
 regex = "1.0.6"
 serde_regex = "1.1"

[weiznich]

@chenrui333 We already have several CI setups running testing the macOS with homebrew setup here and also for mysqlclient-sys itself. It works for these runners. (This includes the buildtime_bindings feature)

I personally do not have access to macOS so I cannot provide further debugging.

I fear as currently written you report is unfortunately unactionable for anyone as it is missing a lot of relevant information such as:

• The exact build output
• Information about your development environment (rustc version, os version, ..)
• Information about which package versions are installed exactly
• Information about which environment variables are set

[weiznich]

@chenrui333 I assume you reported this due to Homebrew/homebrew-core#173312?
I just had a look at potential differences between our CI setup and what you are doing there and it boils down to:

• We are using libmariadb instead of libmysqlclient in our macos setup
• For that systems where we use libmysqlclient for testing pgk-config returns a semver version, that seems not to be the case for the brew packages. So they just do not match the expected version specifier. I'm not yet sure if I would call that a bug or if there is a good way for us to allow this kind of versions as well without running into problems on other platforms. (I would really like to avoid having to deal with a different version scheme per platform. At that point I will likely just require that you always need to specify the version via an environment variable instead as I don't see a way to maintain that mess than anymore).

(So to summarize that: I'm not sure if I agree with your classification as "upstream bug" as that's more a mismatch between what you implicitly expected us to provide and between what we expected you to provide)

see here for jobs that passed recently.

That all written: As you build a system package I think you should prefer to use the buildtime_bindgen feature flag anyway to ensure that the bindings match the version you provide. This is especially important for mysqlclient as they seem to change the header in incompatible ways between versions. See #3929 for example for how such an issue would look like. (This is a general advice for all people packaging any rust crate that depends on native libraries!!)

I just locally tested the following install command: cargo install diesel_cli --features "mysqlclient-sys/buildtime_bindgen pq-sys/buildtime_bindgen" and that seemed to generated the bindings on build time.

[chenrui333]

@chenrui333 I assume you reported this due to Homebrew/homebrew-core#173312? I just had a look at potential differences between our CI setup and what you are doing there and it boils down to:

yeah, that is the PR I am working on

• We are using libmariadb instead of libmysqlclient in our macos setup
• For that systems where we use libmysqlclient for testing pgk-config returns a semver version, that seems not to be the case for the brew packages. So they just do not match the expected version specifier. I'm not yet sure if I would call that a bug or if there is a good way for us to allow this kind of versions as well without running into problems on other platforms. (I would really like to avoid having to deal with a different version scheme per platform. At that point I will likely just require that you always need to specify the version via an environment variable instead as I don't see a way to maintain that mess than anymore).

(So to summarize that: I'm not sure if I agree with your classification as "upstream bug" as that's more a mismatch between what you implicitly expected us to provide and between what we expected you to provide)

see here for jobs that passed recently.

That all written: As you build a system package I think you should prefer to use the buildtime_bindgen feature flag anyway to ensure that the bindings match the version you provide. This is especially important for mysqlclient as they seem to change the header in incompatible ways between versions. See #3929 for example for how such an issue would look like. (This is a general advice for all people packaging any rust crate that depends on native libraries!!)

I just locally tested the following install command: cargo install diesel_cli --features "mysqlclient-sys/buildtime_bindgen pq-sys/buildtime_bindgen" and that seemed to generated the bindings on build time.

Cool, I can give this command a shot, thanks!

[chenrui333]

it works for me, just pushed a change, thanks!

[chenrui333]

while it works for macos CI jobs, but it does not work linux CI, any idea?

    Unable to generate bindings: ClangDiagnostic("bindings/wrapper.h:4:10: fatal error: 'mysql/mysql.h' file not found\n")

linux ci run log, https://github.com/Homebrew/homebrew-core/actions/runs/9339158533/job/25703533805?pr=173312

[weiznich]

Sounds need to somehow provide the mentioned header for bindgen to work. I'm not sure if it's installed while your job runs? If it's installed the relevant path is likely not in the default include path. In this case you can use the BINDGEN_EXTRA_CLANG_ARGS variable to add the relevant path to the search path.

[chenrui333]

I will try that.

[altmannmarcelo]

I can reproduce it with:

$ cat Dockerfile.test
FROM public.ecr.aws/docker/library/rust:1.78

RUN \
        apt-get update && \
        apt-get install -y libmariadb-dev

RUN cargo install diesel_cli --locked --no-default-features --features mysql

$ docker build - < Dockerfile.test

It fails with :

6.137   --- stderr
6.137   thread 'main' panicked at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/mysqlclient-sys-0.3.0/build.rs:158:9:
6.137   mysqlclient-sys does not provide bundled bindings for libmysqlclient 3.3.8. Consider using the `buildtime_bindgen` feature or contribute bindings to the crate
6.137   note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
6.137 warning: build failed, waiting for other jobs to finish...
8.234 error: failed to compile `diesel_cli v2.2.0`, intermediate artifacts can be found at `/tmp/cargo-installgZEThb`.
8.234 To reuse those artifacts with a future compilation, set the environment variable `CARGO_TARGET_DIR` to that path.

[weiznich]

@altmannmarcelo That's expected as --locked does not pull in any fixed done at a later point. Using the same install command without --locked does result in a successful build.

[chrismoos]

Sounds need to somehow provide the mentioned header for bindgen to work. I'm not sure if it's installed while your job runs? If it's installed the relevant path is likely not in the default include path. In this case you can use the BINDGEN_EXTRA_CLANG_ARGS variable to add the relevant path to the search path.

Thanks for this, I was able to get it going on my M1 Mac with:

$ BINDGEN_EXTRA_CLANG_ARGS="-I/opt/homebrew/Cellar/mysql/8.3.0_1/include" cargo install diesel_cli --no-default-features --features "mysql,mysqlclient-sys/buildtime_bindgen"

[Banyc]

Quick fix for anyone wants to get things compiled asap:

diesel = { version = "=2.1.6" }

tho not ideal, this solved my breakage