Native M1 Support

[natestemen]

Hey there,

I'm unable to install bqskitrs due to the following error

$ pip install bqskitrs
ERROR: Could not find a version that satisfies the requirement bqskitrs (from versions: none)
ERROR: No matching distribution found for bqskitrs

I first ran into this while trying to install the pre-release of bqskit via pip install --pre 'bqskit[ext]', but that failed with the following.

ERROR: Cannot install bqskit[ext]==0.1.1, bqskit[ext]==0.2.0, bqskit[ext]==0.2.2, bqskit[ext]==0.2.3, bqskit[ext]==0.3.0, bqskit[ext]==0.4.0, bqskit[ext]==0.4.1, bqskit[ext]==0.5.0, bqskit[ext]==0.5.1, bqskit[ext]==0.5.2, bqskit[ext]==1.0.0a1 and bqskit[ext]==1.0.0a2 because these package versions have conflicting dependencies.

The conflict is caused by:
    bqskit[ext] 1.0.0a2 depends on bqskitrs>=0.2.3
    bqskit[ext] 1.0.0a1 depends on bqskitrs>=0.2.3
    bqskit[ext] 0.5.2 depends on bqskitrs>=0.2.0
    bqskit[ext] 0.5.1 depends on bqskitrs>=0.2.0
    bqskit[ext] 0.5.0 depends on bqskitrs>=0.2.0
    bqskit[ext] 0.4.1 depends on bqskitrs==0.1.0_beta.2
    bqskit[ext] 0.4.0 depends on bqskitrs==0.1.0_beta.2
    bqskit[ext] 0.3.0 depends on bqskitrs==0.1.0_beta.2
    bqskit[ext] 0.2.3 depends on bqskitrs==0.1.0_beta.2
    bqskit[ext] 0.2.2 depends on bqskitrs==0.1.0_beta.2
    bqskit[ext] 0.2.0 depends on bqskitrs==0.1.0_beta.2
    bqskit[ext] 0.1.1 depends on bqskitrs==0.1.0_beta.2

[natestemen]

It seems this issue is also causing pip install bqskit to fail as well due to pip being unaware of bqskitrs's version number.

[WolfLink]

What operating system and python version are you using?

[natestemen]

I'm on an M1 mac running Monterey (12.5.1) with Python 3.9.13.

[WolfLink]

Ahh interesting. We have support for Intel Macs but haven’t tried with an M1 Mac yet.

As a workaround, you could try building bqskitrs from source (the build instructions are in the bqskitrs README).

You might also be able to get it to work if you can find a way to run an x86_64 version of Python. I’ve heard M1 Macs have a way of running x86 executables but I haven’t gotten a chance to try one yet.

[wlav]

If you use an intel mac conda install, x86 binaries work fine through rosetta.

[natestemen]

I'm trying to build bqskitrs from source, but running into a problem.

In a recent release of maturin both --cargo-extra-args and --no-sdist have both been removed. Following their recommendations, I then ran

maturin build --no-default-features --features python,accelerate,ceres/static,mimalloc/local_dynamic_tls --release --sdist

as that seems to be the new way of doing things. That ultimately failed with

...
   Compiling derive_more v0.99.17
   Compiling pyo3-macros v0.16.6
   Compiling ceres-sys v0.1.0 (/Users/nate/code/bqskitrs/ceres/ceres-sys)
   Compiling ndarray_einsum_beta v0.7.0
   Compiling lax v0.2.0
error: failed to run custom build command for `ceres-sys v0.1.0 (/Users/nate/code/bqskitrs/ceres/ceres-sys)`

Caused by:
  process didn't exit successfully: `/Users/nate/code/bqskitrs/target/release/build/ceres-sys-ca1cd0ecf5a63e2b/build-script-build` (exit status: 101)
  --- stdout
  CMAKE_TOOLCHAIN_FILE_aarch64-apple-darwin = None
  CMAKE_TOOLCHAIN_FILE_aarch64_apple_darwin = None
  HOST_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_aarch64-apple-darwin = None
  CMAKE_GENERATOR_aarch64_apple_darwin = None
  HOST_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_aarch64-apple-darwin = None
  CMAKE_PREFIX_PATH_aarch64_apple_darwin = None
  HOST_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_aarch64-apple-darwin = None
  CMAKE_aarch64_apple_darwin = None
  HOST_CMAKE = None
  CMAKE = None
  running: "cmake" "/Users/nate/code/bqskitrs/ceres/ceres-sys/ceres-solver" "-DEXPORT_BUILD_DIR=ON" "-DCXX_THREADS=ON" "-DBUILD_TESTING=OFF" "-DBUILD_BENCHMARKS=OFF" "-DMINIGLOG=ON" "-DLAPACK=OFF" "-DCUSTOM_BLAS=OFF" "-DSCHUR_SPECIALIZATIONS=OFF" "-DBUILD_EXAMPLES=OFF" "-DLIB_SUFFIX=" "-DSUITESPARSE=OFF" "-DCXSPARSE=OFF" "-DCMAKE_INSTALL_PREFIX=/Users/nate/code/bqskitrs/target/release/build/ceres-sys-714e6f483bf54721/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -arch arm64" "-DCMAKE_C_COMPILER=/usr/bin/cc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -arch arm64" "-DCMAKE_CXX_COMPILER=/usr/bin/c++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -arch arm64" "-DCMAKE_ASM_COMPILER=/usr/bin/cc" "-DCMAKE_BUILD_TYPE=RelWithDebInfo"

  --- stderr
  CMake Error: The source directory "/Users/nate/code/bqskitrs/ceres/ceres-sys/ceres-solver" does not appear to contain CMakeLists.txt.
  Specify --help for usage, or press the help button on the CMake GUI.
  thread 'main' panicked at '
  command did not execute successfully, got: exit status: 1

  build script failed, must exit now', /Users/nate/.cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.48/src/lib.rs:975:5
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
💥 maturin failed
  Caused by: Failed to build a native library through cargo
  Caused by: Cargo build finished with "exit status: 101": `cargo rustc --manifest-path Cargo.toml --message-format json --no-default-features --features python,accelerate,ceres/static,mimalloc/local_dynamic_tls --release --lib -- -C link-arg=-undefined -C link-arg=dynamic_lookup -C link-args=-Wl,-install_name,@rpath/bqskitrs.cpython-38-darwin.so`

I've not used maturin so I'm not quite sure what to do from here. FWIW I also tried to run the original command with maturin==0.12.20 (i.e. before the release where they removed the arguments), but I hit the same error.

[edyounis]

I recommend installing using intel conda x86 with rosetta. You can see instructions in our tutorial series here. We are working on supporting native M1, but it is not currently ready.

[natestemen]

Thanks for the help! I've been able to do some development on another machine for now. It would be much appreciated if you could ping me when bqskit support M1 natively (if that's planned/being worked on).

[edyounis]

@natestemen I have been able to build the macOS wheels for both x86 and arm through GitHub actions. You can see and download them here. I don't have an effective means of testing if these built correctly, can you see if the wheels install correctly on your system? You should be able to run python -c 'import bqskitrs' without errors after installing the wheel.

[natestemen]

It works! I was able to download and install the wheel, and run python -c 'import bqskitrs' without errors. I figured that may allow me to download bqskit as well, but it's failed with

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
jupyter-client 7.4.2 requires tornado>=6.2, but you have tornado 6.1 which is incompatible.

That said, this is not an issue with bqskitrs, so happy to close this. Should I create a new issue for the above in the bqskit repo?

[edyounis]

Awesome! Thanks for the help! Before we close, can you double-check that the x86 wheel installs and imports correctly?

That error comes from having BQSKit and Jupyter installed which has a package version issue somewhere down the dependency chain of both. The relevant issue is with Dask and can be found here and is actively being addressed by their team. I have seen this error and ignored it without repricussion. Is it actually preventing you from installing bqskit?

[natestemen]

check that the x86 wheel installs and imports correctly

Yup looks like it does on my other mac machine! (I made sure to uninstall the 0.2.3 version of bqskitrs prior to testing as well).

That error comes from having BQSKit and Jupyter installed

Gotcha. Thanks for the info. Confirming I can also run bqskit code locally now.