Error running create_most_targets.py (last rust-mos installation step)

[U007D]

Transferred from email thread:

Hi, @mrk-its,

I was delighted to see your rust-mos​ crate.

I followed the installation instructions and am getting a failure on the very last line. (So close!)

I am wondering if you recognize the issue and can give me a clue on how to resolve it.

First, I did the following:

llvm-mos

(Note the change to -DLIBXML2_LIBRARY​, as I am on Mac OS X.) Build succeeded​:

git clone https://github.com/llvm-mos/llvm-mos

cd llvm-mos
cmake -C clang/cmake/caches/MOS.cmake -G "Ninja" -S llvm -B build \
   -DLLVM_INSTALL_TOOLCHAIN_ONLY=OFF \
   -DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_LINK_LLVM_DYLIB=ON \
   -DLLVM_INSTALL_UTILS=ON -DLLVM_BUILD_UTILS=ON -DLLVM_TOOLCHAIN_UTILITIES=FileCheck \
   -DLLVM_TOOLCHAIN_TOOLS="llvm-addr2line;llvm-ar;llvm-cxxfilt;llvm-dwarfdump;llvm-mc;llvm-nm;llvm-objcopy;llvm-objdump;llvm-ranlib;llvm-readelf;llvm-readobj;llvm-size;llvm-strings;llvm-strip;llvm-symbolizer;llvm-config;llc" \
   -DLIBXML2_LIBRARY=/usr/local/opt/libxml2/lib/libxml2.dylib \
   -DLLVM_TARGETS_TO_BUILD="MOS;X86" \
   -DLLVM_ENABLE_PROJECTS="clang;lld;lldb"
cmake --build build -t install

llvm-mos-sdk​

(No changes to your instructions.) Build succeeded:

git clone https://github.com/llvm-mos/llvm-mos-sdk
cd llvm-mos-sdk
cmake -G "Ninja" -B build
cmake --build build -t install

rust-mos​

(Build succeeded, but create_mos_targets.py​ step is failing, with both default and custom $RUST_TARGET_PATH​):

export RUST_TARGET_PATH=/Users/sergibli/Development/3rd_party/llvm-mos/rust-mos/bin
cp config.toml.example config.toml
# in config.toml adjust path to llvm-config
# if llvm-mos is installed to other than /usr/local prefix
  # I uncommented and adjusted `prefix = "/Users/sergibli/Development/3rd_party/llvm-mos/rust-mos/"`,
  # as I was unsure if `sudo` permissions were a problem for installation (probably not?)
./x.py build -i --stage 0 src/tools/cargo
./x.py build -i && (
    ln -s ../../stage0-tools-bin/cargo build/x86_64-apple-darwin/stage1/bin/cargo
    rustup toolchain link mos build/x86_64-apple-darwin/stage1
    rustup default mos
    mkdir -p $RUST_TARGET_PATH
    python3 create_mos_targets.py $RUST_TARGET_PATH
)

gives:

   ...
   Compiling rustdoc v0.0.0 (/Users/sergibli/Development/3rd_party/llvm-mos/rust-mos/src/librustdoc)
   Compiling rustdoc-tool v0.0.0 (/Users/sergibli/Development/3rd_party/llvm-mos/rust-mos/src/tools/rustdoc)
    Finished release [optimized] target(s) in 1m 35s
Build completed successfully in 0:03:13
info: default toolchain set to 'mos'
Traceback (most recent call last):
  File "/Users/sergibli/Development/3rd_party/llvm-mos/rust-mos/create_mos_targets.py", line 15, in 
    target_spec = json.loads(
  File "/Users/sergibli/.pyenv/versions/3.9.6/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/Users/sergibli/.pyenv/versions/3.9.6/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/Users/sergibli/.pyenv/versions/3.9.6/lib/python3.9/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Examining create_mos_targets.py​ shows that l.15 shells out to rustup​. Running that line directly gives:

▶ rustup run mos rustc --target mos-unknown-none -Z unstable-options --print target-spec-json
rustc -Cllvm-args="..." with: Unknown command line argument '--align-large-globals=false'.  Try: 'rustc -Cllvm-args="..." with --help'
rustc -Cllvm-args="..." with: Did you mean '--asan-opt-globals=false'?

--target mos-unknown-none​ seems to be the culprit, even though the target does exist:

▶ rustup run mos rustc -Z unstable-options --print target-spec-json
{
  "abi-return-struct-as-int": true,
  "arch": "x86_64",
  "archive-format": "darwin",
  "cpu": "core2",
  "data-layout": "e-m:o-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128",
  "dll-suffix": ".dylib",
  ...

The rustup​ mos​ toolchain is present:

▶ rustup toolchain list
stable-x86_64-apple-darwin
nightly-2020-07-27-x86_64-apple-darwin
nightly-2021-06-15-x86_64-apple-darwin
nightly-x86_64-apple-darwin
1.55-x86_64-apple-darwin
mos (default)
1.56.0-x86_64-apple-darwin

and mos​ links to the stage1 llvm-mos​ build:

sergibli@HephaestusMobi:~/.rustup/toolchains
▶ l
total 0
drwxr-xr-x  9 sergibli  staff  288 Apr 24 14:50 .
drwxr-xr-x  7 sergibli  staff  224 Jun 15  2021 ..
drwxr-xr-x  7 sergibli  staff  224 Feb  4 12:16 1.55-x86_64-apple-darwin
drwxr-xr-x  7 sergibli  staff  224 Feb  1 07:29 1.56.0-x86_64-apple-darwin
lrwxr-xr-x  1 sergibli  staff   88 Apr 24 14:50 mos -> /Users/sergibli/Development/3rd_party/llvm-mos/rust-mos/build/x86_64-apple-darwin/stage1
drwxr-xr-x  6 sergibli  staff  192 Dec 24  2020 nightly-2020-07-27-x86_64-apple-darwin
drwxr-xr-x  7 sergibli  staff  224 Sep 22  2021 nightly-2021-06-15-x86_64-apple-darwin
drwxr-xr-x  7 sergibli  staff  224 Apr 20 21:09 nightly-x86_64-apple-darwin
drwxr-xr-x  7 sergibli  staff  224 Apr  8 18:26 stable-x86_64-apple-darwin

The target triple is also listed:

▶ rustc --print target-list
aarch64-apple-darwin
...
mipsel-sony-psp
mipsisa64r6el-unknown-linux-gnuabi64
mos-unknown-none
msp430-none-elf
nvptx64-nvidia-cuda
...

Have you seen anything like the errors I am encountering before? Any educated guesses on where I might want to dig?

Thanks so much for your efforts with this crate!

All the best,
U007D

[U007D]

From @mrk-its:

I followed the installation instructions and am getting a failure on the very last line. (So close!)

Indeed, you are really close. This latest step creates separate sub-targets for each supported platform. It does by modifying a bit target spec json for 'mos-unknown-none' - that's why it needs to be retrieved with rustc --print target-spec-json

Examining create_mos_targets.py shows that l.15 shells out to rustup. Running that line directly gives:
▶ rustup run mos rustc --target mos-unknown-none -Z unstable-options --print target-spec-json rustc -Cllvm-args="..." with: Unknown command line argument '--align-large-globals=false'. Try: 'rustc -Cllvm-args="..." with --help' rustc -Cllvm-args="..." with: Did you mean '--asan-opt-globals=false'?

It is really strange, it looks like for some reason it verifies returned json and it doesn't like llvm-args there:
{
...
"llvm-args": [
"--force-precise-rotation-cost",
"--jump-inst-cost=6",
"--force-loop-cold-block",
"--phi-node-folding-threshold=0",
"--two-entry-phi-node-folding-threshold=0",
"--align-large-globals=false",
"--disable-spill-hoist"
],
...
}

It is possible that rust-mos stopped working with the latest version of llvm-mos (I see upstream llvm changes were merged recently). I'll try to rebuild it and let you know.
In the meantime you can remove this --align-large-globals=false from list of the llvm-args here:

https://github.com/mrk-its/rust-mos/blob/master/compiler/rustc_target/src/spec/mos_unknown_none.rs#L20

But I just checked clang still sets this arg:

https://github.com/llvm-mos/llvm-mos/blob/main/clang/lib/Driver/ToolChains/CommonArgs.cpp#L1666

Ok, I'll try to reproduce it locally and let you know

[U007D]

For @mrk-its or others who may read this in the future, @mrk-its' hint of changes in llvm-mos possibly being the culprit prompted me to try the version of llvm-mos that was current when this version of rust-mos commit SHA 65009f shipped (Mar. 30, 2022).

Through some trial and error, building llvm-mos commit SHA 84a2c0a (Mar. 14, 2022) along with llvm-mos-sdk commit SHA cfba7b0 (Mar. 15, 2022) yielded a successful (if not fully current) build. 🎉

(Note no other commit of llvm-mos-sdk I tried worked with llvm-mos@84a2c0a.)

[Serentty]

Even with the most recent commits (more recent than this issue), I am also getting this.

[U007D]

I tried again this evening with clean, latest source code builds of llvm-mos, llvm-mos-sdk and rust-mos, but got the same error in the end. Both before and after removing --align-large-globals=false from mos-unknown-none.rs, I got the following build error:

▶ export RUST_TARGET_PATH=/usr/local/rust-mos-targets/
▶ export TARGET_ARCH=x86_64-apple-darwin #for MacOS
▶ cp config.toml.example config.toml

Note config.toml:683 changed from:
llvm-has-rust-patches = if llvm-config { false } else { true } to
llvm-has-rust-patches = true #if llvm-config { false } else { true }
(more on this below)

▶ ./x.py build -i --stage 0 src/tools/cargo
extracting /Users/Shared/Development/3rd_party/rust-mos/build/cache/2022-09-20/rust-std-beta-x86_64-apple-darwin.tar.xz
extracting /Users/Shared/Development/3rd_party/rust-mos/build/cache/2022-09-20/rustc-beta-x86_64-apple-darwin.tar.xz
extracting /Users/Shared/Development/3rd_party/rust-mos/build/cache/2022-09-20/cargo-beta-x86_64-apple-darwin.tar.xz
Building rustbuild
  Compiling libc v0.2.126
...
  Compiling cargo-credential-1password v0.1.0 (/Users/Shared/Development/3rd_party/rust-mos/src/tools/cargo/crates/credential/cargo-credential-1password)
   Finished release [optimized] target(s) in 9.63s
Build completed successfully in 1:06:27
▶ ./x.py build -i && (
  ln -s ../../stage0-tools-bin/cargo build/${TARGET_ARCH}/stage1/bin/
  cargo
     rustup toolchain link mos build/${TARGET_ARCH}/stage1
     rustup default mos
     mkdir -p $RUST_TARGET_PATH
     python3 create_mos_targets.py $RUST_TARGET_PATH
  )
Building rustbuild
...
Copying stage1 std from stage1 (x86_64-apple-darwin -> x86_64-apple-darwin / x86_64-apple-darwin)
Building rustdoc for stage1 (x86_64-apple-darwin)
  Compiling rustdoc v0.0.0 (/Users/Shared/Development/3rd_party/rust-mos/src/librustdoc)
  Compiling rustdoc-tool v0.0.0 (/Users/Shared/Development/3rd_party/rust-mos/src/tools/rustdoc)
   Finished release [optimized] target(s) in 8.99s
Build completed successfully in 0:09:59
Rust's package manager

Usage: cargo [+toolchain] [OPTIONS] [COMMAND]

Options:
 -V, --version             Print version info and exit
     --list                List installed commands
     --explain <CODE>      Run `rustc --explain CODE`
 -v, --verbose...          Use verbose output (-vv very verbose/build.rs output)
 -q, --quiet               Do not print cargo log messages
     --color <WHEN>        Coloring: auto, always, never
     --frozen              Require Cargo.lock and cache are up to date
     --locked              Require Cargo.lock is up to date
     --offline             Run without accessing the network
     --config <KEY=VALUE>  Override a configuration value
 -Z <FLAG>                 Unstable (nightly-only) flags to Cargo, see 'cargo -Z help' for details
 -h, --help                Print help information

Some common cargo commands are (see all commands with --list):
   build, b    Compile the current package
   check, c    Analyze the current package and report errors, but don't build object files
   clean       Remove the target directory
   doc, d      Build this package's and its dependencies' documentation
   new         Create a new cargo package
   init        Create a new cargo package in an existing directory
   add         Add dependencies to a manifest file
   run, r      Run a binary or example of the local package
   test, t     Run the tests
   bench       Run the benchmarks
   update      Update dependencies listed in Cargo.lock
   search      Search registry for crates
   publish     Package and upload this package to the registry
   install     Install a Rust binary. Default location is $HOME/.cargo/bin
   uninstall   Uninstall a Rust binary

See 'cargo help <command>' for more information on a specific command.
info: default toolchain set to 'mos'
Traceback (most recent call last):
 File "/Users/Shared/Development/3rd_party/rust-mos/create_mos_targets.py", line 15, in <module>
   target_spec = json.loads(
 File "/Users/sergibli/.pyenv/versions/3.9.6/lib/python3.9/json/__init__.py", line 346, in loads
   return _default_decoder.decode(s)
 File "/Users/sergibli/.pyenv/versions/3.9.6/lib/python3.9/json/decoder.py", line 337, in decode
   obj, end = self.raw_decode(s, idx=_w(s, 0).end())
 File "/Users/sergibli/.pyenv/versions/3.9.6/lib/python3.9/json/decoder.py", line 355, in raw_decode
   raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

I'm still stuck on this. Any thoughts on how to move forward?

P.S. During the build process, I also ran into an issue with config.toml (from config.toml.example) where l.683 reads:

llvm-has-rust-patches = if llvm-config { false } else { true }

which I don't think is legal TOML? This line gave a build error, so I temporarily hard-coded this to true to proceed.