rust-android-gradle: investigate parallel Rust builds #27
Comments
|
Pending issue 8176 on rust-android-gradle We can likely implement |
|
When compiling rust my device CPU utlilzation jumps to 100% and the fans stars kiking. So I don't know if parallizing it would increase the speed. |
dae
added a commit
to ankitects/Anki-Android-Backend
that referenced
this issue
Nov 29, 2022
The 2.1.55 desktop release switched away from Bazel to a new build system, which required some changes to this repo. Making the changes was a bit complicated due to the complexity of the current build system, and I ended up shaving some yaks to make things simpler while I was working on the changes. Changes: - Building for the current architecture instead of all architectures is now the default, so getting started is easier, and a build in Android Studio no longer requires special flags. - The build-current.sh script has been split into build-aar and build-robo for the Android and Robolectric parts, and can be used for both single-arch and multi-arch builds. - On arm64 Macs, the build scripts now create arm binaries - In a multi-arch build, both x86 and arm64 Mac libs are built, and they're merged into a single library. This can be done in CI, so there is no manual step required for M1 machines anymore. - The build now uses protobuf and python binaries/libs that the desktop build downloads, so they don't need to be installed separately. - The pinned Rust version and Rust targets are automatically installed as required. - The per-platform CI builds now build in debug mode and are faster. - Updated the docs to explain how the NDK can be installed via Android Studio, instead of via separate command-line tools. - The cross/docker stuff has been stripped out, as it's of limited use as it can't target macOS legally. Easier to use GH actions for the multi-arch builds, and keep things simple for local development. - Fix lint not being run in CI; caught an API 23 reference. Bumps ankidroid#179 (builds on M1 already work, so this may be simpler than expected?) Bumps ankidroid#174 (a bunch of the doctor stuff is obsolete; updated HOWTO.md and GH actions should be consulted) Bumps ankidroid#27 (I recommend closing this; single-platform is the default for local builds, and CI runners don't have any extra compute available) Tentatively closes ankidroid#109 (didn't see the flake when I was updating the actions) Closes ankidroid#235 (translation submodules now automatically synced with anki submodule) Closes ankidroid#213 (path based on script now) Closes ankidroid#211 (builds for Arm Mac on Arm Macs) Closes ankidroid#197 (single arch is the default) Closes ankidroid#196 (desktop venv is used) Closes ankidroid#195 (can be done via the GUI, and does not require separate cli download) Closes ankidroid#168 (latest Rust; easier changing via rust-toolchain.toml) Closes ankidroid#164 (universal dylib) Closes ankidroid#127 (no docker) Closes ankidroid#106 (most of those scripts obsolete; some commands moved into build scripts) Closes ankidroid#99 (no docker) Closes ankidroid#98 (no docker) Closes ankidroid#97 (submodule automatically updated) Closes ankidroid#96 (build will fail if commit unavailable) Closes ankidroid#53 (no docker) Closes ankidroid#40 (DEBUG=1 option documented) Closes ankidroid#9 (simpler OOTB experience, and updated docs)
dae
added a commit
to ankitects/Anki-Android-Backend
that referenced
this issue
Nov 29, 2022
The 2.1.55 desktop release switched away from Bazel to a new build system, which required some changes to this repo. Making the changes was a bit complicated due to the complexity of the current build system, and I ended up shaving some yaks to make things simpler while I was working on the changes. Changes: - Building for the current architecture instead of all architectures is now the default, so getting started is easier, and a build in Android Studio no longer requires special flags. - The build-current.sh script has been split into build-aar and build-robo for the Android and Robolectric parts, and can be used for both single-arch and multi-arch builds. - On arm64 Macs, the build scripts now create arm binaries - In a multi-arch build, both x86 and arm64 Mac libs are built, and they're merged into a single library. This can be done in CI, so there is no manual step required for M1 machines anymore. - The build now uses protobuf and python binaries/libs that the desktop build downloads, so they don't need to be installed separately. - The pinned Rust version and Rust targets are automatically installed as required. - The per-platform CI builds now build in debug mode and are faster. - Updated the docs to explain how the NDK can be installed via Android Studio, instead of via separate command-line tools. - The cross/docker stuff has been stripped out, as it's of limited use as it can't target macOS legally. Easier to use GH actions for the multi-arch builds, and keep things simple for local development. - Fix lint not being run in CI; caught an API 23 reference. Bumps ankidroid#179 (builds on M1 already work, so this may be simpler than expected?) Bumps ankidroid#174 (a bunch of the doctor stuff is obsolete; updated HOWTO.md and GH actions should be consulted) Bumps ankidroid#27 (I recommend closing this; single-platform is the default for local builds, and CI runners don't have any extra compute available) Tentatively closes ankidroid#109 (didn't see the flake when I was updating the actions) Closes ankidroid#235 (translation submodules now automatically synced with anki submodule) Closes ankidroid#213 (path based on script now) Closes ankidroid#211 (builds for Arm Mac on Arm Macs) Closes ankidroid#197 (single arch is the default) Closes ankidroid#196 (desktop venv is used) Closes ankidroid#195 (can be done via the GUI, and does not require separate cli download) Closes ankidroid#168 (latest Rust; easier changing via rust-toolchain.toml) Closes ankidroid#164 (universal dylib) Closes ankidroid#127 (no docker) Closes ankidroid#106 (most of those scripts obsolete; some commands moved into build scripts) Closes ankidroid#99 (no docker) Closes ankidroid#98 (no docker) Closes ankidroid#97 (submodule automatically updated) Closes ankidroid#96 (build will fail if commit unavailable) Closes ankidroid#53 (no docker) Closes ankidroid#40 (DEBUG=1 option documented) Closes ankidroid#9 (simpler OOTB experience, and updated docs)
dae
added a commit
to ankitects/Anki-Android-Backend
that referenced
this issue
Nov 29, 2022
The 2.1.55 desktop release switched away from Bazel to a new build system, which required some changes to this repo. Making the changes was a bit complicated due to the complexity of the current build system, and I ended up shaving some yaks to make things simpler while I was working on the changes. Changes: - Building for the current architecture instead of all architectures is now the default, so getting started is easier, and a build in Android Studio no longer requires special flags. - The build-current.sh script has been split into build-aar and build-robo for the Android and Robolectric parts, and can be used for both single-arch and multi-arch builds. - On arm64 Macs, the build scripts now create arm binaries - In a multi-arch build, both x86 and arm64 Mac libs are built, and they're merged into a single library. This can be done in CI, so there is no manual step required for M1 machines anymore. - The build now uses protobuf and python binaries/libs that the desktop build downloads, so they don't need to be installed separately. - The pinned Rust version and Rust targets are automatically installed as required. - The per-platform CI builds now build in debug mode and are faster. - Updated the docs to explain how the NDK can be installed via Android Studio, instead of via separate command-line tools. - The cross/docker stuff has been stripped out, as it's of limited use as it can't target macOS legally. Easier to use GH actions for the multi-arch builds, and keep things simple for local development. - Fix lint not being run in CI; caught an API 23 reference. Bumps ankidroid#179 (builds on M1 already work, so this may be simpler than expected?) Bumps ankidroid#174 (a bunch of the doctor stuff is obsolete; updated HOWTO.md and GH actions should be consulted) Bumps ankidroid#27 (I recommend closing this; single-platform is the default for local builds, and CI runners don't have any extra compute available) Tentatively closes ankidroid#109 (didn't see the flake when I was updating the actions) Closes ankidroid#235 (translation submodules now automatically synced with anki submodule) Closes ankidroid#213 (path based on script now) Closes ankidroid#211 (builds for Arm Mac on Arm Macs) Closes ankidroid#197 (single arch is the default) Closes ankidroid#196 (desktop venv is used) Closes ankidroid#195 (can be done via the GUI, and does not require separate cli download) Closes ankidroid#168 (latest Rust; easier changing via rust-toolchain.toml) Closes ankidroid#164 (universal dylib) Closes ankidroid#127 (no docker) Closes ankidroid#106 (most of those scripts obsolete; some commands moved into build scripts) Closes ankidroid#99 (no docker) Closes ankidroid#98 (no docker) Closes ankidroid#97 (submodule automatically updated) Closes ankidroid#96 (build will fail if commit unavailable) Closes ankidroid#53 (no docker) Closes ankidroid#40 (DEBUG=1 option documented) Closes ankidroid#9 (simpler OOTB experience, and updated docs)
mikehardy
added a commit
that referenced
this issue
Dec 16, 2022
* Move anki submodule to top level It's not only used to build the bridge, but also to generate TS files, so top-level makes more sense. It will also fit more nicely into a cargo workspace. * Add a missing drop() that recent Rust versions complain about * Update to 2.1.55 + ninja build * Refactor the build system The 2.1.55 desktop release switched away from Bazel to a new build system, which required some changes to this repo. Making the changes was a bit complicated due to the complexity of the current build system, and I ended up shaving some yaks to make things simpler while I was working on the changes. Changes: - Building for the current architecture instead of all architectures is now the default, so getting started is easier, and a build in Android Studio no longer requires special flags. - The build-current.sh script has been split into build-aar and build-robo for the Android and Robolectric parts, and can be used for both single-arch and multi-arch builds. - On arm64 Macs, the build scripts now create arm binaries - In a multi-arch build, both x86 and arm64 Mac libs are built, and they're merged into a single library. This can be done in CI, so there is no manual step required for M1 machines anymore. - The build now uses protobuf and python binaries/libs that the desktop build downloads, so they don't need to be installed separately. - The pinned Rust version and Rust targets are automatically installed as required. - The per-platform CI builds now build in debug mode and are faster. - Updated the docs to explain how the NDK can be installed via Android Studio, instead of via separate command-line tools. - The cross/docker stuff has been stripped out, as it's of limited use as it can't target macOS legally. Easier to use GH actions for the multi-arch builds, and keep things simple for local development. - Fix lint not being run in CI; caught an API 23 reference. Bumps #179 (builds on M1 already work, so this may be simpler than expected?) Bumps #174 (a bunch of the doctor stuff is obsolete; updated HOWTO.md and GH actions should be consulted) Bumps #27 (I recommend closing this; single-platform is the default for local builds, and CI runners don't have any extra compute available) Tentatively closes #109 (didn't see the flake when I was updating the actions) Closes #235 (translation submodules now automatically synced with anki submodule) Closes #213 (path based on script now) Closes #211 (builds for Arm Mac on Arm Macs) Closes #197 (single arch is the default) Closes #196 (desktop venv is used) Closes #195 (can be done via the GUI, and does not require separate cli download) Closes #168 (latest Rust; easier changing via rust-toolchain.toml) Closes #164 (universal dylib) Closes #127 (no docker) Closes #106 (most of those scripts obsolete; some commands moved into build scripts) Closes #99 (no docker) Closes #98 (no docker) Closes #97 (submodule automatically updated) Closes #96 (build will fail if commit unavailable) Closes #53 (no docker) Closes #40 (DEBUG=1 option documented) Closes #9 (simpler OOTB experience, and updated docs) * Tweak print statements in build.gradle Opted to match the actual variable name rather than make it human readable; the print statements are just there to help debug issues when the user forgets to shut down gradle first. * Elaborate on prerequisites * Integrate HOWTO inside README; remove more obsolete instructions * Remove plantuml files * Add platform guard to multi-arch build * Fix clippy lints; check in CI * Include lint in check, and remind users they have to use build-aar first * Custom path no longer required for rsdroid-testing * Reduce verbosity of cargo build in aar * Mention python3 env * Strip some obsolete stuff from doctor.sh * Increase timeout to 10 seconds #246 (comment) * Mention msys2 * Apply build-all.yml suggestions from Mike * Increase timeout of multi-arch build * Tweak cache keys * Confirm aar/jar output * Change Mani's PS script to a .bat file Avoids the need to change the execution policy, and the new script checks execution success. * Fix incorrect build target in build-aar.bat * Use desktop build's bundled Python on Windows Avoids the need to install it separately * Add os suffix to artifacts * Apply suggestions from code review Co-authored-by: Mike Hardy <[email protected]> * Only hash lockfiles https://github.com/ankidroid/Anki-Android-Backend/pull/246/files#r1038175214 * Update to stable release commit Co-authored-by: Mike Hardy <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Unsure if this is feasible, but we could potentially get a x4 speedup on aspects of the Android builds, and x3 on the macOS builds
The text was updated successfully, but these errors were encountered: