Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] Perf: Unpin old Python Version #106

Closed
david-allison opened this issue May 26, 2021 · 4 comments · Fixed by #246
Closed

[CI] Perf: Unpin old Python Version #106

david-allison opened this issue May 26, 2021 · 4 comments · Fixed by #246
Labels
performance

Comments

@david-allison
Copy link
Member

for discussion: We can likely remove the need for an older version of Python now we're on the latest version of protobuf-compiler

If we're using (and checking) the native version of Python for the machine, this should save a few minutes.
@dae
Copy link
Contributor

dae commented Jun 21, 2022

I don't see any pins - can this be closed?

@mikehardy
Copy link
Member

A quick glance reveals this for me?

Anki-Android-Backend/.github/scripts/linux_install_protobuf.sh

Lines 8 to 12 in eeb5b29

echo "Installing Python 3.7.9"
pyenv install 3.7.9
pyenv shell 3.7.9
pyenv global 3.7.9
python --version

Anki-Android-Backend/.github/scripts/macos_install_protobuf_compiler.sh

Lines 20 to 23 in eeb5b29

pyenv install 3.7.9
pyenv shell 3.7.9
pyenv global 3.7.9
python3 -v

I have not inspected deeply but those appear to be old versions.
No idea why a) we need older versions or b) we're doing this in semi-opaque sub-scripts vs using a setup-python action and just including the needed logic in a workflow step (I generally don't like sub-scripts in CI if they can be avoided...)

@dae
Copy link
Contributor

dae commented Jun 21, 2022

Ah, sorry, I only grepped through the workflow files, expecting something like a setup-python call.

@mikehardy
Copy link
Member

Ah, sorry, I only grepped through the workflow files, expecting something like a setup-python call.

Exactly! Thus my distaste for what I think of as "out-of-band" CI work (little sub-scripts doing who knows what instead of having everything in the yaml...)

dae added a commit to ankitects/Anki-Android-Backend that referenced this issue Nov 29, 2022
@dae
Refactor the build system
dc222ab 
The 2.1.55 desktop release switched away from Bazel to a new build system,
which required some changes to this repo. Making the changes was a bit complicated
due to the complexity of the current build system, and I ended up shaving some yaks
to make things simpler while I was working on the changes.

Changes:

- Building for the current architecture instead of all architectures is now the
default, so getting started is easier, and a build in Android Studio no longer
requires special flags.
- The build-current.sh script has been split into build-aar and build-robo
for the Android and Robolectric parts, and can be used for both single-arch
and multi-arch builds.
- On arm64 Macs, the build scripts now create arm binaries
- In a multi-arch build, both x86 and arm64 Mac libs are built, and they're
merged into a single library. This can be done in CI, so there is no manual step
required for M1 machines anymore.
- The build now uses protobuf and python binaries/libs that the desktop build
downloads, so they don't need to be installed separately.
- The pinned Rust version and Rust targets are automatically installed as
required.
- The per-platform CI builds now build in debug mode and are faster.
- Updated the docs to explain how the NDK can be installed via Android Studio,
instead of via separate command-line tools.
- The cross/docker stuff has been stripped out, as it's of limited use as it
can't target macOS legally. Easier to use GH actions for the multi-arch
builds, and keep things simple for local development.
- Fix lint not being run in CI; caught an API 23 reference.

Bumps ankidroid#179 (builds on M1 already work, so this may be simpler than expected?)
Bumps ankidroid#174 (a bunch of the doctor stuff is obsolete; updated HOWTO.md and GH actions should be consulted)
Bumps ankidroid#27 (I recommend closing this; single-platform is the default for local builds, and
CI runners don't have any extra compute available)

Tentatively closes ankidroid#109 (didn't see the flake when I was updating the actions)

Closes ankidroid#235 (translation submodules now automatically synced with anki submodule)
Closes ankidroid#213 (path based on script now)
Closes ankidroid#211 (builds for Arm Mac on Arm Macs)
Closes ankidroid#197 (single arch is the default)
Closes ankidroid#196 (desktop venv is used)
Closes ankidroid#195 (can be done via the GUI, and does not require separate cli download)
Closes ankidroid#168 (latest Rust; easier changing via rust-toolchain.toml)
Closes ankidroid#164 (universal dylib)
Closes ankidroid#127 (no docker)
Closes ankidroid#106 (most of those scripts obsolete; some commands moved into build scripts)
Closes ankidroid#99 (no docker)
Closes ankidroid#98 (no docker)
Closes ankidroid#97 (submodule automatically updated)
Closes ankidroid#96 (build will fail if commit unavailable)
Closes ankidroid#53 (no docker)
Closes ankidroid#40 (DEBUG=1 option documented)
Closes ankidroid#9 (simpler OOTB experience, and updated docs)
dae added a commit to ankitects/Anki-Android-Backend that referenced this issue Nov 29, 2022
@dae
Refactor the build system
d0acb5a 
The 2.1.55 desktop release switched away from Bazel to a new build system,
which required some changes to this repo. Making the changes was a bit complicated
due to the complexity of the current build system, and I ended up shaving some yaks
to make things simpler while I was working on the changes.

Changes:

- Building for the current architecture instead of all architectures is now the
default, so getting started is easier, and a build in Android Studio no longer
requires special flags.
- The build-current.sh script has been split into build-aar and build-robo
for the Android and Robolectric parts, and can be used for both single-arch
and multi-arch builds.
- On arm64 Macs, the build scripts now create arm binaries
- In a multi-arch build, both x86 and arm64 Mac libs are built, and they're
merged into a single library. This can be done in CI, so there is no manual step
required for M1 machines anymore.
- The build now uses protobuf and python binaries/libs that the desktop build
downloads, so they don't need to be installed separately.
- The pinned Rust version and Rust targets are automatically installed as
required.
- The per-platform CI builds now build in debug mode and are faster.
- Updated the docs to explain how the NDK can be installed via Android Studio,
instead of via separate command-line tools.
- The cross/docker stuff has been stripped out, as it's of limited use as it
can't target macOS legally. Easier to use GH actions for the multi-arch
builds, and keep things simple for local development.
- Fix lint not being run in CI; caught an API 23 reference.

Bumps ankidroid#179 (builds on M1 already work, so this may be simpler than expected?)
Bumps ankidroid#174 (a bunch of the doctor stuff is obsolete; updated HOWTO.md and GH actions should be consulted)
Bumps ankidroid#27 (I recommend closing this; single-platform is the default for local builds, and
CI runners don't have any extra compute available)

Tentatively closes ankidroid#109 (didn't see the flake when I was updating the actions)

Closes ankidroid#235 (translation submodules now automatically synced with anki submodule)
Closes ankidroid#213 (path based on script now)
Closes ankidroid#211 (builds for Arm Mac on Arm Macs)
Closes ankidroid#197 (single arch is the default)
Closes ankidroid#196 (desktop venv is used)
Closes ankidroid#195 (can be done via the GUI, and does not require separate cli download)
Closes ankidroid#168 (latest Rust; easier changing via rust-toolchain.toml)
Closes ankidroid#164 (universal dylib)
Closes ankidroid#127 (no docker)
Closes ankidroid#106 (most of those scripts obsolete; some commands moved into build scripts)
Closes ankidroid#99 (no docker)
Closes ankidroid#98 (no docker)
Closes ankidroid#97 (submodule automatically updated)
Closes ankidroid#96 (build will fail if commit unavailable)
Closes ankidroid#53 (no docker)
Closes ankidroid#40 (DEBUG=1 option documented)
Closes ankidroid#9 (simpler OOTB experience, and updated docs)
dae added a commit to ankitects/Anki-Android-Backend that referenced this issue Nov 29, 2022
@dae
Refactor the build system
7223936 
The 2.1.55 desktop release switched away from Bazel to a new build system,
which required some changes to this repo. Making the changes was a bit complicated
due to the complexity of the current build system, and I ended up shaving some yaks
to make things simpler while I was working on the changes.

Changes:

- Building for the current architecture instead of all architectures is now the
default, so getting started is easier, and a build in Android Studio no longer
requires special flags.
- The build-current.sh script has been split into build-aar and build-robo
for the Android and Robolectric parts, and can be used for both single-arch
and multi-arch builds.
- On arm64 Macs, the build scripts now create arm binaries
- In a multi-arch build, both x86 and arm64 Mac libs are built, and they're
merged into a single library. This can be done in CI, so there is no manual step
required for M1 machines anymore.
- The build now uses protobuf and python binaries/libs that the desktop build
downloads, so they don't need to be installed separately.
- The pinned Rust version and Rust targets are automatically installed as
required.
- The per-platform CI builds now build in debug mode and are faster.
- Updated the docs to explain how the NDK can be installed via Android Studio,
instead of via separate command-line tools.
- The cross/docker stuff has been stripped out, as it's of limited use as it
can't target macOS legally. Easier to use GH actions for the multi-arch
builds, and keep things simple for local development.
- Fix lint not being run in CI; caught an API 23 reference.

Bumps ankidroid#179 (builds on M1 already work, so this may be simpler than expected?)
Bumps ankidroid#174 (a bunch of the doctor stuff is obsolete; updated HOWTO.md and GH actions should be consulted)
Bumps ankidroid#27 (I recommend closing this; single-platform is the default for local builds, and
CI runners don't have any extra compute available)

Tentatively closes ankidroid#109 (didn't see the flake when I was updating the actions)

Closes ankidroid#235 (translation submodules now automatically synced with anki submodule)
Closes ankidroid#213 (path based on script now)
Closes ankidroid#211 (builds for Arm Mac on Arm Macs)
Closes ankidroid#197 (single arch is the default)
Closes ankidroid#196 (desktop venv is used)
Closes ankidroid#195 (can be done via the GUI, and does not require separate cli download)
Closes ankidroid#168 (latest Rust; easier changing via rust-toolchain.toml)
Closes ankidroid#164 (universal dylib)
Closes ankidroid#127 (no docker)
Closes ankidroid#106 (most of those scripts obsolete; some commands moved into build scripts)
Closes ankidroid#99 (no docker)
Closes ankidroid#98 (no docker)
Closes ankidroid#97 (submodule automatically updated)
Closes ankidroid#96 (build will fail if commit unavailable)
Closes ankidroid#53 (no docker)
Closes ankidroid#40 (DEBUG=1 option documented)
Closes ankidroid#9 (simpler OOTB experience, and updated docs)
@dae dae mentioned this issue Nov 29, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
performance
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update to 2.1.55 and refactor the build system ankitects/Anki-Android-Backend
3 participants
@dae @mikehardy @david-allison