error: Could not compile snips-nlu-resources-packed

[warent]

I'm not terribly experienced with Rust, so I apologize if the information here is lacking.

I have a project with the following Cargo.toml

[dependencies]
snips-nlu-lib = { git = "https://github.com/talkative-ai/snips-nlu-rs", branch = "master" }
redis = "0.8.0"
hyper = "0.11"
futures = "0.1.14"

When I execute cargo run --verbose eventually the compilation fails:

error: Could not compile `snips-nlu-resources-packed`

The information on the error is as follows:

Caused by: process didn't exit successfully: `rustc --crate-name snips_nlu_resources_packed /usr/local/cargo/git/checkouts/snips-nlu-rs-c3854c7c83110319/85787f9/snips-nlu-resources-packed/src/lib.rs --crate-type lib --emit=dep-info,link -C debuginfo=2 -C metadata=cdca8d510a223171 -C extra-filename=-cdca8d510a223171 --out-dir /tmp/project/target/debug/deps -L dependency=/tmp/project/target/debug/deps --extern snips_nlu_ontology=/tmp/project/target/debug/deps/libsnips_nlu_ontology-64e186b4f4ebb163.rlib --extern phf=/tmp/project/target/debug/deps/libphf-f0c00edcacda1116.rlib --extern failure=/tmp/project/target/debug/deps/libfailure-4e47e324962fe3f2.rlib --cap-lints allow -L native=/tmp/project/target/debug/build/backtrace-sys-7833d5b313c46d32/out/.libs`

rustc and cargo info:
rustc 1.23.0 (766bd11c8 2018-01-01)
cargo 0.24.0 (45043115c 2017-12-05)

Any ideas why this is happening?

[warent]

I suspect this is happening due to me using the wrong linker/compiler settings. I had something similar on a different machine but was able to resolve it with a series of steps, partially helped by https://chr4.org/blog/2017/03/15/cross-compile-and-link-a-static-binary-on-macos-for-linux-with-cargo-and-rust/

[klefevre]

hi @warent . This error occurs generally when you compile snips-nlu-lib from source (from this repository) when submodules aren't initialized/updated because snips-nlu-resources-packed relies on a submodule...

But I see that you added snips-nlu-lib as dependency so cargo should clone submodules as well...

I can't reprocuce the error so it's a bit difficult to know what is your problem but the link you sent talks about cross compilation from macOS to Linux.

• have you tried to compile for your host on a tiny project ?
• have you succesfully compiled https://github.com/snipsco/snips-nlu-rs from sources ?

Also I suggest you to update your rust version because you're out-of-date (as this moment 1.25.0 is the latest version).

[klefevre]

As we can't reproduce the bug and no feedbacks I close this issue.

[ballwood]

@klefevre Hi, also having this issue. Trying to make some bindings for NodeJS with Neon so I can use it on aws lambda.

Built on an amazonlinux:latest docker container. Dockerfile I'm using is below, should be reproducible.

FROM amazonlinux:latest

WORKDIR /work
ADD . /work

RUN curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain stable && \
    PATH="/root/.cargo/bin:$PATH" rustup install stable && \
    PATH="/root/.cargo/bin:$PATH" rustup install nightly
RUN curl -sL https://rpm.nodesource.com/setup_8.x | bash -
RUN yum install -y bzip2-devel gcc gcc-c++ clang libgcc cmake wget git openssl-devel readline-devel sqlite-devel zip nodejs \
 && yum clean all

RUN node -v
ENV PATH $PATH:/root/.cargo/bin

RUN git clone https://github.com/ballwood/snips-nlu-js.git
CMD ["/bin/bash"]

cd into snips-nlu-js and run npm install. If it doesn't build, run npm build. Neon hides the full error from cargo build and I don't think you can pass it --verbose but returns "Could not compile snips-nlu-resources-packed".

Seems to be a linux/docker issue as works correctly on native OSX. Haven't tried windows

[adrienball]

Hey @ballwood,
I managed to build without error using your Dockerfile so I suspect it might be a memory issue. The compiling of snips-nlu-rs is quite memory intensive at the moment (we are working on improving that), so my guess is that your container runs out of memory.
Can you try to increase the allocated memory of the container and see if you still have the issue ?
Thanks!

[ballwood]

@adrienball yep perfect. upped the container to use 4gb and worked fine, might wanna put a notice in the readme.

Thanks for the help, really appreciate it. Great work on the library btw guys, its super fast!

[adrienball]

I'm closing this issue because the way resources are managed has been completely redesigned in the latest release (0.58.0).
Language resources are now loaded dynamically from the trained engine directory, instead of being statically hardcoded (which BTW reduces the binary size by 30Mb!). Hence, the crate snips-nlu-resources-packed has been removed.

Cheers!