Skip to content

Commit

Permalink
Refactor the build system
Browse files Browse the repository at this point in the history
The 2.1.55 desktop release switched away from Bazel to a new build system,
which required some changes to this repo. Making the changes was a bit complicated
due to the complexity of the current build system, and I ended up shaving some yaks
to make things simpler while I was working on the changes.

Changes:

- Building for the current architecture instead of all architectures is now the
default, so getting started is easier, and a build in Android Studio no longer
requires special flags.
- The build-current.sh script has been split into build-aar and build-robo
for the Android and Robolectric parts, and can be used for both single-arch
and multi-arch builds.
- On arm64 Macs, the build scripts now create arm binaries
- In a multi-arch build, both x86 and arm64 Mac libs are built, and they're
merged into a single library. This can be done in CI, so there is no manual step
required for M1 machines anymore.
- The build now uses protobuf and python binaries/libs that the desktop build
downloads, so they don't need to be installed separately.
- The pinned Rust version and Rust targets are automatically installed as
required.
- The per-platform CI builds now build in debug mode and are faster.
- Updated the docs to explain how the NDK can be installed via Android Studio,
instead of via separate command-line tools.
- The cross/docker stuff has been stripped out, as it's of limited use as it
can't target macOS legally. Easier to use GH actions for the multi-arch
builds, and keep things simple for local development.
- Fix lint not being run in CI; caught an API 23 reference.

Bumps ankidroid#179 (builds on M1 already work, so this may be simpler than expected?)
Bumps ankidroid#174 (a bunch of the doctor stuff is obsolete; updated HOWTO.md and GH actions should be consulted)
Bumps ankidroid#27 (I recommend closing this; single-platform is the default for local builds, and
CI runners don't have any extra compute available)

Tentatively closes ankidroid#109 (didn't see the flake when I was updating the actions)

Closes ankidroid#235 (translation submodules now automatically synced with anki submodule)
Closes ankidroid#213 (path based on script now)
Closes ankidroid#211 (builds for Arm Mac on Arm Macs)
Closes ankidroid#197 (single arch is the default)
Closes ankidroid#196 (desktop venv is used)
Closes ankidroid#195 (can be done via the GUI, and does not require separate cli download)
Closes ankidroid#168 (latest Rust; easier changing via rust-toolchain.toml)
Closes ankidroid#164 (universal dylib)
Closes ankidroid#127 (no docker)
Closes ankidroid#106 (most of those scripts obsolete; some commands moved into build scripts)
Closes ankidroid#99 (no docker)
Closes ankidroid#98 (no docker)
Closes ankidroid#97 (submodule automatically updated)
Closes ankidroid#96 (build will fail if commit unavailable)
Closes ankidroid#53 (no docker)
Closes ankidroid#40 (DEBUG=1 option documented)
Closes ankidroid#9 (simpler OOTB experience, and updated docs)
  • Loading branch information
dae committed Nov 29, 2022
1 parent d8f2a0b commit d0acb5a
Show file tree
Hide file tree
Showing 50 changed files with 496 additions and 4,391 deletions.
4 changes: 4 additions & 0 deletions .cargo/config.toml
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
[build]
# override any global customizations, as the gradle plugin requires a specific path
target-dir = "target"

[env]
STRINGS_JSON = { value = "anki/out/strings.json", relative = true }
PROTOC = { value = "anki/out/extracted/protoc/bin/protoc", relative = true }
4 changes: 0 additions & 4 deletions .github/scripts/install_rust_robolectric_targets.sh

This file was deleted.

9 changes: 0 additions & 9 deletions .github/scripts/install_rust_targets.sh

This file was deleted.

7 changes: 0 additions & 7 deletions .github/scripts/linux_install_macos_cross_compile.sh

This file was deleted.

15 changes: 0 additions & 15 deletions .github/scripts/linux_install_protobuf.sh

This file was deleted.

2 changes: 0 additions & 2 deletions .github/scripts/linux_install_x86_64-unknown-linux-gnu-gcc.sh

This file was deleted.

29 changes: 0 additions & 29 deletions .github/scripts/macos_install_protobuf_compiler.sh

This file was deleted.

7 changes: 0 additions & 7 deletions .github/scripts/macos_install_pyenv.sh

This file was deleted.

5 changes: 0 additions & 5 deletions .github/scripts/protoc_gen_deps.py

This file was deleted.

134 changes: 134 additions & 0 deletions .github/workflows/build-all.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
name: Build AAR and Robo (all platforms)
on:
workflow_dispatch:
push:
schedule:
- cron: "0 0 * * *"
pull_request:

env:
ALL_ARCHS: 1

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}-release
cancel-in-progress: true

jobs:
build:
runs-on: macos-latest
timeout-minutes: 80
steps:
- uses: actions/checkout@v2

- name: Fetch submodules
run: git submodule update --init

- name: Setup Ninja
uses: ashutoshvarma/setup-ninja@master
with:
version: 1.10.0

# The action puts a relative path on the PATH 🙄
- name: Make path absolute
run: echo $(pwd)/ninja_bin >> $GITHUB_PATH

- name: Configure JDK 1.11
uses: actions/setup-java@v3
with:
distribution: "adopt"
java-version: "11" # minimum for Android API31

- name: Install Android Command Line Tools
uses: android-actions/setup-android@v2

- name: Install Windows cross compiler
run: brew install mingw-w64 && x86_64-w64-mingw32-gcc -v

- name: Install Linux cross compiler
run: |
brew tap SergioBenitez/osxct
brew install x86_64-unknown-linux-gnu
x86_64-unknown-linux-gnu-gcc -v
- name: Install NDK
run: .github/scripts/install_ndk.sh 22.0.7026061

- name: Rust Cache
uses: actions/cache@v3
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: ${{ runner.os }}-rust-release-v2-${{ hashFiles('rslib-bridge/**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-rust-release-v2
${{ runner.os }}-rust
- name: Anki cache
uses: actions/cache@v3
with:
path: |
anki/out/node_modules
anki/out/rust
anki/out/extracted
key: ${{ runner.os }}-anki-v4-${{ hashFiles('anki/package.json') }}
restore-keys: |
${{ runner.os }}-anki-v4
- name: Build Android/All
run: ./build-aar.sh

- name: Build Robolectric/All
run: ./build-robo.sh

- name: Check Compiled Libraries
run: >
cd rsdroid-testing/assets &&
../../.github/scripts/check_robolectric_assets.sh
- name: Upload rsdroid AAR as artifact
uses: actions/upload-artifact@v2
with:
name: rsdroid-aar
if-no-files-found: error
path: rsdroid/build/outputs/aar

- name: Upload rsdroid-robo JAR as artifact
uses: actions/upload-artifact@v2
with:
name: rsdroid-robo
if-no-files-found: error
path: rsdroid-testing/build/libs

# following steps only run on workflow dispatch

- name: Publish AAR to Maven
if: github.event_name == "workflow_dispatch"
env:
ORG_GRADLE_PROJECT_SIGNING_PRIVATE_KEY: ${{ secrets.SIGNING_PRIVATE_KEY }}
ORG_GRADLE_PROJECT_SIGNING_PASSWORD: ${{ secrets.SIGNING_PASSWORD }}
SONATYPE_NEXUS_USERNAME: david-allison-1
SONATYPE_NEXUS_PASSWORD: ${{ secrets.SONATYPE_NEXUS_PASSWORD }}
run: |
./gradlew rsdroid:uploadArchives -DtestBuildType=release -Dorg.gradle.daemon=false -Dorg.gradle.console=plain
- name: Publish JAR to Maven
if: github.event_name == "workflow_dispatch"
env:
ORG_GRADLE_PROJECT_SIGNING_PRIVATE_KEY: ${{ secrets.SIGNING_PRIVATE_KEY }}
ORG_GRADLE_PROJECT_SIGNING_PASSWORD: ${{ secrets.SIGNING_PASSWORD }}
SONATYPE_NEXUS_USERNAME: david-allison-1
SONATYPE_NEXUS_PASSWORD: ${{ secrets.SONATYPE_NEXUS_PASSWORD }}
run: |
export ANKIDROID_LINUX_CC=x86_64-unknown-linux-gnu-gcc
export ANKIDROID_MACOS_CC=cc
export RUST_DEBUG=1
export RUST_BACKTRACE=1
export RUST_LOG=trace
export NO_CROSS=true
./gradlew rsdroid-testing:uploadArchives -Dorg.gradle.project.macCC=$ANKIDROID_MACOS_CC -DtestBuildType=debug -Dorg.gradle.daemon=false -Dorg.gradle.console=plain
- name: ℹ️Additional Release Instructions (requires human interaction)
if: github.event_name == "workflow_dispatch"
run: echo "Sign in to https://oss.sonatype.org/#stagingRepositories , close the repository, then release it"
108 changes: 108 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
name: Build AAR and Robo (single platform)
on:
workflow_dispatch:
push:
schedule:
- cron: "0 0 * * *"
pull_request:

env:
DEBUG: 1

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
build:
strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
fail-fast: false
runs-on: ${{ matrix.os }}
timeout-minutes: 80
steps:
- uses: actions/checkout@v2

- name: Fetch submodules
run: git submodule update --init

- name: Setup Ninja
uses: ashutoshvarma/setup-ninja@master
with:
version: 1.10.0

# The action puts a relative path on the PATH 🙄
- name: Make path absolute
run: echo $(pwd)/ninja_bin >> $GITHUB_PATH

- name: Configure JDK 1.11
uses: actions/setup-java@v3
with:
distribution: "adopt"
java-version: "11" # minimum for Android API31

- name: Install Android Command Line Tools
uses: android-actions/setup-android@v2

- name: Install NDK
if: matrix.os != 'windows-latest'
run: .github/scripts/install_ndk.sh 22.0.7026061

- name: Install NDK (Windows)
if: matrix.os == 'windows-latest'
run: |
Write-Host "NDK Install Started"
(. sdkmanager.bat --install "ndk;22.0.7026061" --sdk_root="$Env:ANDROID_SDK_ROOT") | out-null
Write-Host "NDK Install Completed"
- name: Rust Cache
uses: actions/cache@v3
with:
path: |
~/.cargo/registry
~/.cargo/git
target
key: ${{ runner.os }}-rust-debug-v4-${{ hashFiles('rslib-bridge/**/Cargo.lock') }}
restore-keys: |
${{ runner.os }}-rust-debug-v4
${{ runner.os }}-rust-debug
- name: Anki cache
uses: actions/cache@v3
with:
path: |
anki/out/node_modules
anki/out/rust
anki/out/extracted
key: ${{ runner.os }}-anki-v4-${{ hashFiles('anki/package.json') }}
restore-keys: |
${{ runner.os }}-anki-v4
${{ runner.os }}-anki
- name: Build Android/X86_64
if: matrix.os != 'windows-latest'
run: ./build-aar.sh

- name: Build Android/X86_64 (Win)
if: matrix.os == 'windows-latest'
run: ./build-aar.bat

- name: Build Robolectric/X86_64
if: matrix.os != 'windows-latest'
run: ./build-robo.sh

- name: Build Robolectric/X86_64 (Win)
if: matrix.os == 'windows-latest'
run: ./build-robo.bat

- name: Run tests (Mac)
uses: reactivecircus/android-emulator-runner@v2
if: matrix.os == 'macos-latest'
timeout-minutes: 30
with:
api-level: 21
target: default
arch: x86_64
profile: Nexus 6
script: ./gradlew rsdroid:lint rsdroid-instrumented:connectedCheck
Loading

0 comments on commit d0acb5a

Please sign in to comment.