Build failure on FreeBSD #980
Comments
|
if you want FreeBSD native you might be able to get by using https://github.com/mobilecoinofficial/full-service/blob/main/tools/build-fs.sh The error your getting is because the enclave css(not cascading style sheets) environment variables are not specified via:
When these aren't specified then the build tries to build the enclaves to generate the css files. The you could also use the docker image. The |
|
@nick-mobilecoin Thanks a lot for your swift reply! However I was actually using the build-fs.sh script, here is the full output of the terminal session (but you helped me find out the actual issue): root@mobilecoinfullservicetest:/home/mobilecoin/full-service/tools # ./build-fs.sh test
/root/.cargo/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
/bin/csh
Setting 'test' SGX, IAS and enclave values
IAS_MODE: PROD
SGX_MODE: HW
CONSENSUS_ENCLAVE_CSS: /home/mobilecoin/full-service/target/release/consensus-enclave.css
INGEST_ENCLAVE_CSS: /home/mobilecoin/full-service/target/release/ingest-enclave.css
building full service...
Compiling typenum v1.17.0
Compiling clang-sys v1.7.0
Compiling protobuf v2.28.0
Compiling mc-util-build-info v6.0.0 (/home/mobilecoin/full-service/mobilecoin/util/build/info)
Compiling num-bigint v0.4.4
Compiling crunchy v0.2.2
Compiling hidapi v2.4.1
error: failed to run custom build command for `hidapi v2.4.1`
Caused by:
process didn't exit successfully: `/home/mobilecoin/full-service/target/release/build/hidapi-0f1b6d5217db2a16/build-script-build` (exit status: 101)
--- stdout
cargo:rerun-if-env-changed=HIDAPI_NO_PKG_CONFIG
cargo:rerun-if-env-changed=PKG_CONFIG_x86_64-unknown-freebsd
cargo:rerun-if-env-changed=PKG_CONFIG_x86_64_unknown_freebsd
cargo:rerun-if-env-changed=HOST_PKG_CONFIG
cargo:rerun-if-env-changed=PKG_CONFIG
cargo:rerun-if-env-changed=HIDAPI_STATIC
cargo:rerun-if-env-changed=HIDAPI_DYNAMIC
cargo:rerun-if-env-changed=PKG_CONFIG_ALL_STATIC
cargo:rerun-if-env-changed=PKG_CONFIG_ALL_DYNAMIC
cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-unknown-freebsd
cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_unknown_freebsd
cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
cargo:rerun-if-env-changed=PKG_CONFIG_PATH
cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-unknown-freebsd
cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_unknown_freebsd
cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-unknown-freebsd
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_unknown_freebsd
cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
cargo:rerun-if-env-changed=HIDAPI_STATIC
cargo:rerun-if-env-changed=HIDAPI_DYNAMIC
cargo:rerun-if-env-changed=PKG_CONFIG_ALL_STATIC
cargo:rerun-if-env-changed=PKG_CONFIG_ALL_DYNAMIC
cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64-unknown-freebsd
cargo:rerun-if-env-changed=PKG_CONFIG_PATH_x86_64_unknown_freebsd
cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
cargo:rerun-if-env-changed=PKG_CONFIG_PATH
cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64-unknown-freebsd
cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_x86_64_unknown_freebsd
cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64-unknown-freebsd
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_x86_64_unknown_freebsd
cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
--- stderr
thread 'main' panicked at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hidapi-2.4.1/build.rs:131:41:
Unable to find hidapi: Could not run `PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1 pkg-config --libs --cflags hidapi`
The pkg-config command could not be found.
Most likely, you need to install a pkg-config package for your OS.
Try `apt install pkg-config`, or `yum install pkg-config`,
or `pkg install pkg-config`, or `apk add pkgconfig` depending on your distribution.
If you've already installed it, ensure the pkg-config command is one of the
directories in the PATH environment variable.
If you did not expect this build to link to a pre-installed system library,
then check documentation of the hidapi crate for an option to
build the library from source, or disable features or dependencies
that require pkg-config.
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...Well, when posting this message, I noticed that the actual error is that I hadn't installed the pkg-config package (pkgconf on FreeBSD) and the hidapi package. But I am now hitting this build error when building the package 820ish out of 924 (nearly there!!!): Compiling mc-util-parse v6.0.0 (/home/mobilecoin/full-service/mobilecoin/util/parse)
error[E0432]: unresolved import `crate::platform::PeripheralId`
--> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/btleplug-0.10.5/src/api/mod.rs:44:5
|
44 | use crate::platform::PeripheralId;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `PeripheralId` in `platform`
|
note: found an item that was configured out
--> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/btleplug-0.10.5/src/platform.rs:6:77
|
6 | adapter::Adapter, manager::Manager, peripheral::Peripheral, peripheral::PeripheralId,
| ^^^^^^^^^^^^
= note: the item is gated behind the `linux` feature
note: found an item that was configured out
--> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/btleplug-0.10.5/src/platform.rs:10:77
|
10 | adapter::Adapter, manager::Manager, peripheral::Peripheral, peripheral::PeripheralId,
| ^^^^^^^^^^^^
note: found an item that was configured out
--> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/btleplug-0.10.5/src/platform.rs:14:83
|
14 | adapter::Adapter, init, manager::Manager, peripheral::Peripheral, peripheral::PeripheralId,
| ^^^^^^^^^^^^
= note: the item is gated behind the `android` feature
note: found an item that was configured out
--> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/btleplug-0.10.5/src/platform.rs:18:77
|
18 | adapter::Adapter, manager::Manager, peripheral::Peripheral, peripheral::PeripheralId,
| ^^^^^^^^^^^^
= note: the item is gated behind the `windows` feature
error[E0432]: unresolved import `crate::platform::PeripheralId`
--> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/btleplug-0.10.5/src/common/adapter_manager.rs:15:5
|
15 | use crate::platform::PeripheralId;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ no `PeripheralId` in `platform`
|
note: found an item that was configured out
--> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/btleplug-0.10.5/src/platform.rs:6:77
|
6 | adapter::Adapter, manager::Manager, peripheral::Peripheral, peripheral::PeripheralId,
| ^^^^^^^^^^^^
= note: the item is gated behind the `linux` feature
note: found an item that was configured out
--> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/btleplug-0.10.5/src/platform.rs:10:77
|
10 | adapter::Adapter, manager::Manager, peripheral::Peripheral, peripheral::PeripheralId,
| ^^^^^^^^^^^^
note: found an item that was configured out
--> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/btleplug-0.10.5/src/platform.rs:14:83
|
14 | adapter::Adapter, init, manager::Manager, peripheral::Peripheral, peripheral::PeripheralId,
| ^^^^^^^^^^^^
= note: the item is gated behind the `android` feature
note: found an item that was configured out
--> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/btleplug-0.10.5/src/platform.rs:18:77
|
18 | adapter::Adapter, manager::Manager, peripheral::Peripheral, peripheral::PeripheralId,
| ^^^^^^^^^^^^
= note: the item is gated behind the `windows` feature
error[E0412]: cannot find type `Adapter` in this scope
--> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/btleplug-0.10.5/src/platform.rs:29:18
|
29 | assert_impl_all!(Adapter: Central, Clone, Debug, Send, Sized, Sync);
| ^^^^^^^ not found in this scope
error[E0412]: cannot find type `Manager` in this scope
--> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/btleplug-0.10.5/src/platform.rs:30:18
|
30 | assert_impl_all!(Manager: api::Manager, Clone, Debug, Send, Sized, Sync);
| ^^^^^^^ not found in this scope
|
help: consider importing this trait
|
21 + use crate::api::Manager;
|
error[E0412]: cannot find type `Peripheral` in this scope
--> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/btleplug-0.10.5/src/platform.rs:31:18
|
31 | assert_impl_all!(Peripheral: api::Peripheral, Clone, Debug, Send, Sized, Sync);
| ^^^^^^^^^^ not found in this scope
|
help: consider importing this trait
|
21 + use crate::api::Peripheral;
|
error[E0412]: cannot find type `PeripheralId` in this scope
--> /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/btleplug-0.10.5/src/platform.rs:33:5
|
33 | PeripheralId: Clone,
| ^^^^^^^^^^^^ not found in this scope
PS: Docker is not portable to FreeBSD so building from source is the only option we've got I'm afraid. |
|
Here is the output of I am not a rust developer so I know very little about this ecosystem and what could cause the issue, but my guess is that the toolchain i use may be lacking capabilities. |
nkosi23
changed the title
|
Okay the issue is apparently in the platform.rs file of btleplug, there is no definition for the FreeBSD OS. I will try to submit a PR: #[cfg(target_os = "linux")]
pub use crate::bluez::{
adapter::Adapter, manager::Manager, peripheral::Peripheral, peripheral::PeripheralId,
};
#[cfg(any(target_os = "macos", target_os = "ios"))]
pub use crate::corebluetooth::{
adapter::Adapter, manager::Manager, peripheral::Peripheral, peripheral::PeripheralId,
};
#[cfg(target_os = "android")]
pub use crate::droidplug::{
adapter::Adapter, init, manager::Manager, peripheral::Peripheral, peripheral::PeripheralId,
};
#[cfg(target_os = "windows")]
pub use crate::winrtble::{
adapter::Adapter, manager::Manager, peripheral::Peripheral, peripheral::PeripheralId,
}; |
|
I have just submitted a PR containing a quick monkey patch to add a dummy FreeBSD definition reusing the crate for linux, which is guaranteed to cause a runtime error since FreeBSD's bluetooth stack is totally different. But I'm crossing my fingers that no bluetooth capability is actually used, because this begs the question, why in the world does the full service API has a dependency on btleplug to start with? 😕 There are very little chances that my ridiculous PR is going to be approved upstream now that I think about it 😆 But applying this patch downstream is not going to be trivial as the rust toolchain downloads and builds crates automatically, I'd have to fiddle quite a bit to get going with the rust toolchain (edit: eventually came across The alternative and right approach would be to write proper upstream adapters for FreeBSD for btleplug, but this would require me to first learn the bluetooth spec, and then learn the bluetooth stack of FreeBSD. Oh and crash learn rust programming basics. One month of work at least 😢 Hopefully someone will reply that this package is being imported by mistake 😄 |
|
Okay, progressing as I get my feet wet on the rust toolchain, the culprit is apparently ledger-lib, as per And the btleplug dependency looks quite central to what this library is doing 😞 :
Why is this library needed by the full service API? Any chance to put it behind a platform-specific compile flag (using the [cfg(target_os = "linux")] attribute)? Maybe the build of ledger-mob itself could be put behind an OS flag since apparently it is specific to Ledger's hardware so I am not sure why including it in the full-service API is even relevant:
|
I did a little digging around, it looks like the ledger logic is spread around a bit in the code base. My vote would be that it goes behind a feature flag maybe something like |
|
Oh sad to hear that. As a gentle feedback, working on adding mobilecoin support to our non-crypto-focused product has been constantly pushed back due to perceived and now real cans of worms. Initially, based on the documentation, we assumed that it was not possible to build the full-service API without having an intel processor and the SGX libraries installed. Since we exclusively use FreeBSD for our workloads, and AMD processors, needless to say that we weren't very eager to get into this 🥲 I was surprised when it turned out that the build actually worked fine without SGX for the reason you explained. Now this ledger thing will be the actual show stopper. I guess the "easiest" course of action for us would be to provision a linux VM somewhere, which is not totally trivial as it also has implications on things like sysadmin (we'll now have 2 operating systems to run, update, secure, backup etc...). In our case, we can't use bhyve (FreeBSD's virtualizer allowing one to run VM's as easily as jails hosted by a FreeBSD host) which would have made things easier but our FreeBSD hosts are already running virtualized under a KVM managed by our hosting partner. |
I am not in a position to commit to working on it right now but to have visibility: would you welcome a PR isolating the ledger code, or would you prefer the core team to handle it itself (for example because there'd be too many implications on design, architecture etc...) |
|
as a quick experiment, would you see if the experiment/no-hardware-wallet branch builds on freebsd? It's building on MacOS sans the ledger-mob submodule, though I haven't really tested it. As Nick said, it would be hard for us to prioritize refactoring things right now, so just gathering data... |
|
@holtzman Many thanks for the experiment, I am hitting a panic when building the grpcio-sys crate so I have opened an issue in their repo. However since this happens when building the crate 851 out 890, I have a sense that the rest of the build will work fine once the issue with grpc is addressed upstream. The way you isolated ledger seems to work, but I'll need to run it to confirm this once the build issue is fixed. |
|
@nkosi23 I have removed the bug label as full-service is not supported for freebsd. |
|
@nkosi23, fyi, I created a freebsd 14 instance on EC2 and it built grpcio-sys. Perhaps there are packages in freebsd 13 that could be upgraded to get you past that? Now I'm stuck at linking. There appear to be encryption-related symbols that sqlite3 is using but the linker is not finding. Full-service will optionally encrypt the wallet-db... |
|
@holtzman Mmh have you installed the openssl package? Apparently the issue I was hitting was due to the need to build gprcio against master because of an issue with the crate I have been able to build the grpcio-sys dependency successfully using the latest nightly rust toolchain, but now I am hitting the following build error near the end of the build process : |
|
My hunch is this
Those kind of errors, "expect name, got (same) name", are usually two different versions of the crate. I'd be curious how you did the following
We may be calling out grpcio inconsistently and only one place was updated |
|
Thanks a lot for your help! Indeed the Cargo.lock file contained multiple versions of grpcio and grpcio-sys but the issue is 100% from me. Initially I wasn't aware that So, in light of your pointer, starting from the experiment branch, here are all the changes i eventually made:
And I am glad to report that the build completed succesfully 🎉 🎉 🎆 The content of the build was put inside |
|
Okay, after adding the symlink:
Everything works like a charm, the ledger is currently syncing :) I will test that the main features needed by our app work and will keep you posted. Thanks a lot for your help with this |
|
Oh I have a question, since we'll likely be stuck with the version available in the experiment branch for a while, are releases usually backward compatible? I mean at a protocol level, how likely is it that at some point the transactions created by our node become rejected? The spirit of my question is: is it guaranteed to happen with every update - for example due to SGX attestation - or are such breaking changes relatively rare. |
|
Good question! TL; DR: full-service has few mandatory updates and they are far between. One is imminent, but we forked the experimental branch off main after the mandatory updates had landed, so that branch is already good to go. When full-service attests the enclaves it uses, it does not require that their individual build measurements are on a whitelist compiled into full-service. This is exactly so that the same version of full-service can continue to work across enclave-side updates, as long as there are no breaking protocol changes. It does validate that the enclave was signed by the trusted authority, has the expected product id, and meets a minimum version requirement before trusting it. In contrast, Sentz and Signal do validate that each enclave's build measurements are on a compiled-in whitelist. The environments of use, ease of updating, and education / contextual awareness level of the operator of each of these are very different and thus different policies are appropriate. On occasion (3 times so far) we have had to make breaking changes in the protocol that have required full-service to be updated. The first was when we went multi-asset (~10/22); the second was phase 2 of multi-asset which introduced signed-contingent-inputs, allowing for atomic swaps between assets (~3/2023); and, the third will be to switch from SGX EPID attestation to SGX DCAP due to Intel phasing out EPID support (7/2024). The timing of this branch is very good for that 3rd one as we are mid-way through the deployment process, and full-service is already updated and compatible with both the current and soon-to-be-deployed enclaves. In fact, testnet is already upgraded to v6.0, which is scheduled to be deployed to mainnet the week of July 8. |
|
Inspired by the topic, I merged main back into this branch as we've done a bunch of feature updates since the branch forked. I built and tested on MacOS, including a scripted client test that end-to-end exercises a running full-service via its api. |
|
Hi gentlemen! I am running the full-service API version 2.10.2 |
|
Thanks for the heads up, looks like testnet node1 has been having some issues since about Oct 8th. |
Describe the bug
When building the full service API on FreeBSD, everything builds smoothly until the 798th package out of 924, the build error is:
To Reproduce
Steps to reproduce the behavior:
Install rustup on FreeBSD as explained in: https://gist.github.com/fabiotatsuo/e88b025baf3dfa47683419948723706f
Install the build dependencies with the command:
pkg install cmake protobuf llvm llvm-devel sqlite3 sqlcipher git bash
Expected behavior
The build completes without error. Rusts binaries are usually easily portable to FreeBSD, it looks like this issue could just be a matter of adding a few config files to support FreeBSD.
Environment (please complete the following information):
Logs
And the build eventually ends with these logs.
Additional context
Add any other context about the problem here.
The text was updated successfully, but these errors were encountered: