Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Determine if/how to update upstream translations in Anki-Android-Backend #235

Closed
mikehardy opened this issue Sep 19, 2022 · 3 comments · Fixed by #246
Closed

Determine if/how to update upstream translations in Anki-Android-Backend #235

mikehardy opened this issue Sep 19, 2022 · 3 comments · Fixed by #246
Labels
good first issue Good for newcomers help wanted Extra attention is needed

Comments

@mikehardy
Copy link
Member

We need to work on frequency/freshness of the Pontoon translations which are at this point pretty static (we have not ingested changes from that system in months, and...have no plan to do so right now which is not good, I want a plan we just don't have one). I'm not sure if we can ingest just translations easily or if we must necessarily update the whole backend or not, the question bears investigating

Originally posted by @mikehardy in ankidroid/Anki-Android#12448 (comment)
@mikehardy mikehardy added help wanted Extra attention is needed good first issue Good for newcomers labels Sep 19, 2022
@mikehardy mikehardy mentioned this issue Sep 19, 2022
Merged
5 tasks
@dae
Copy link
Contributor

dae commented Sep 20, 2022

Each desktop release pins the strings at a particular point, since newer code may depend on new strings, and obsolete strings may be dropped. You thus can't update the translations separately from the anki module without risk of things falling out of sync. 2.1.55 is hopefully not more than a month away, and when it comes out and a new backend release is made, all the accumulated contributions will be in it. AD doesn't necessarily have to stick to stable releases either - it can update the backend before then if it wishes.

@mikehardy
Copy link
Member Author

Sounds like the same process we have - code and strings need to match (for same reason - string add/drops).
So the process for ingestion of new strings will be to "re-sync" the backend to some specific commit - either a stable tag or some interesting commit, no matter from a sync perspective, it's a commit hash at the end.

A finer question: is there any risk in any specific commits? For example, we typically commit a PR that does string add/drops then we do an i18n sync with crowdin that populates the other languages and commit that. So from an i18n perspective there is a brief moment where strings are out of sync for non-english locales. I don't think that actually has a risk of crashing in Android (it should fall back to the default locale for missing strings?) but maybe it's different for backend?

That would inform whether commits to sync backend need any care when they are chosen or really if any commit would work (assuming it was a "good" commit to choose from a functionality/error perspective)

@dae
Copy link
Contributor

dae commented Sep 21, 2022

Generally any commit on main is safe, though there have been a few instances in the past where a bug was introduced and then fixed in a follow-up commit after it was discovered.

dae added a commit to ankitects/Anki-Android-Backend that referenced this issue Nov 29, 2022
@dae
Refactor the build system
dc222ab 
The 2.1.55 desktop release switched away from Bazel to a new build system,
which required some changes to this repo. Making the changes was a bit complicated
due to the complexity of the current build system, and I ended up shaving some yaks
to make things simpler while I was working on the changes.

Changes:

- Building for the current architecture instead of all architectures is now the
default, so getting started is easier, and a build in Android Studio no longer
requires special flags.
- The build-current.sh script has been split into build-aar and build-robo
for the Android and Robolectric parts, and can be used for both single-arch
and multi-arch builds.
- On arm64 Macs, the build scripts now create arm binaries
- In a multi-arch build, both x86 and arm64 Mac libs are built, and they're
merged into a single library. This can be done in CI, so there is no manual step
required for M1 machines anymore.
- The build now uses protobuf and python binaries/libs that the desktop build
downloads, so they don't need to be installed separately.
- The pinned Rust version and Rust targets are automatically installed as
required.
- The per-platform CI builds now build in debug mode and are faster.
- Updated the docs to explain how the NDK can be installed via Android Studio,
instead of via separate command-line tools.
- The cross/docker stuff has been stripped out, as it's of limited use as it
can't target macOS legally. Easier to use GH actions for the multi-arch
builds, and keep things simple for local development.
- Fix lint not being run in CI; caught an API 23 reference.

Bumps ankidroid#179 (builds on M1 already work, so this may be simpler than expected?)
Bumps ankidroid#174 (a bunch of the doctor stuff is obsolete; updated HOWTO.md and GH actions should be consulted)
Bumps ankidroid#27 (I recommend closing this; single-platform is the default for local builds, and
CI runners don't have any extra compute available)

Tentatively closes ankidroid#109 (didn't see the flake when I was updating the actions)

Closes ankidroid#235 (translation submodules now automatically synced with anki submodule)
Closes ankidroid#213 (path based on script now)
Closes ankidroid#211 (builds for Arm Mac on Arm Macs)
Closes ankidroid#197 (single arch is the default)
Closes ankidroid#196 (desktop venv is used)
Closes ankidroid#195 (can be done via the GUI, and does not require separate cli download)
Closes ankidroid#168 (latest Rust; easier changing via rust-toolchain.toml)
Closes ankidroid#164 (universal dylib)
Closes ankidroid#127 (no docker)
Closes ankidroid#106 (most of those scripts obsolete; some commands moved into build scripts)
Closes ankidroid#99 (no docker)
Closes ankidroid#98 (no docker)
Closes ankidroid#97 (submodule automatically updated)
Closes ankidroid#96 (build will fail if commit unavailable)
Closes ankidroid#53 (no docker)
Closes ankidroid#40 (DEBUG=1 option documented)
Closes ankidroid#9 (simpler OOTB experience, and updated docs)
dae added a commit to ankitects/Anki-Android-Backend that referenced this issue Nov 29, 2022
@dae
Refactor the build system
d0acb5a 
The 2.1.55 desktop release switched away from Bazel to a new build system,
which required some changes to this repo. Making the changes was a bit complicated
due to the complexity of the current build system, and I ended up shaving some yaks
to make things simpler while I was working on the changes.

Changes:

- Building for the current architecture instead of all architectures is now the
default, so getting started is easier, and a build in Android Studio no longer
requires special flags.
- The build-current.sh script has been split into build-aar and build-robo
for the Android and Robolectric parts, and can be used for both single-arch
and multi-arch builds.
- On arm64 Macs, the build scripts now create arm binaries
- In a multi-arch build, both x86 and arm64 Mac libs are built, and they're
merged into a single library. This can be done in CI, so there is no manual step
required for M1 machines anymore.
- The build now uses protobuf and python binaries/libs that the desktop build
downloads, so they don't need to be installed separately.
- The pinned Rust version and Rust targets are automatically installed as
required.
- The per-platform CI builds now build in debug mode and are faster.
- Updated the docs to explain how the NDK can be installed via Android Studio,
instead of via separate command-line tools.
- The cross/docker stuff has been stripped out, as it's of limited use as it
can't target macOS legally. Easier to use GH actions for the multi-arch
builds, and keep things simple for local development.
- Fix lint not being run in CI; caught an API 23 reference.

Bumps ankidroid#179 (builds on M1 already work, so this may be simpler than expected?)
Bumps ankidroid#174 (a bunch of the doctor stuff is obsolete; updated HOWTO.md and GH actions should be consulted)
Bumps ankidroid#27 (I recommend closing this; single-platform is the default for local builds, and
CI runners don't have any extra compute available)

Tentatively closes ankidroid#109 (didn't see the flake when I was updating the actions)

Closes ankidroid#235 (translation submodules now automatically synced with anki submodule)
Closes ankidroid#213 (path based on script now)
Closes ankidroid#211 (builds for Arm Mac on Arm Macs)
Closes ankidroid#197 (single arch is the default)
Closes ankidroid#196 (desktop venv is used)
Closes ankidroid#195 (can be done via the GUI, and does not require separate cli download)
Closes ankidroid#168 (latest Rust; easier changing via rust-toolchain.toml)
Closes ankidroid#164 (universal dylib)
Closes ankidroid#127 (no docker)
Closes ankidroid#106 (most of those scripts obsolete; some commands moved into build scripts)
Closes ankidroid#99 (no docker)
Closes ankidroid#98 (no docker)
Closes ankidroid#97 (submodule automatically updated)
Closes ankidroid#96 (build will fail if commit unavailable)
Closes ankidroid#53 (no docker)
Closes ankidroid#40 (DEBUG=1 option documented)
Closes ankidroid#9 (simpler OOTB experience, and updated docs)
dae added a commit to ankitects/Anki-Android-Backend that referenced this issue Nov 29, 2022
@dae
Refactor the build system
7223936 
The 2.1.55 desktop release switched away from Bazel to a new build system,
which required some changes to this repo. Making the changes was a bit complicated
due to the complexity of the current build system, and I ended up shaving some yaks
to make things simpler while I was working on the changes.

Changes:

- Building for the current architecture instead of all architectures is now the
default, so getting started is easier, and a build in Android Studio no longer
requires special flags.
- The build-current.sh script has been split into build-aar and build-robo
for the Android and Robolectric parts, and can be used for both single-arch
and multi-arch builds.
- On arm64 Macs, the build scripts now create arm binaries
- In a multi-arch build, both x86 and arm64 Mac libs are built, and they're
merged into a single library. This can be done in CI, so there is no manual step
required for M1 machines anymore.
- The build now uses protobuf and python binaries/libs that the desktop build
downloads, so they don't need to be installed separately.
- The pinned Rust version and Rust targets are automatically installed as
required.
- The per-platform CI builds now build in debug mode and are faster.
- Updated the docs to explain how the NDK can be installed via Android Studio,
instead of via separate command-line tools.
- The cross/docker stuff has been stripped out, as it's of limited use as it
can't target macOS legally. Easier to use GH actions for the multi-arch
builds, and keep things simple for local development.
- Fix lint not being run in CI; caught an API 23 reference.

Bumps ankidroid#179 (builds on M1 already work, so this may be simpler than expected?)
Bumps ankidroid#174 (a bunch of the doctor stuff is obsolete; updated HOWTO.md and GH actions should be consulted)
Bumps ankidroid#27 (I recommend closing this; single-platform is the default for local builds, and
CI runners don't have any extra compute available)

Tentatively closes ankidroid#109 (didn't see the flake when I was updating the actions)

Closes ankidroid#235 (translation submodules now automatically synced with anki submodule)
Closes ankidroid#213 (path based on script now)
Closes ankidroid#211 (builds for Arm Mac on Arm Macs)
Closes ankidroid#197 (single arch is the default)
Closes ankidroid#196 (desktop venv is used)
Closes ankidroid#195 (can be done via the GUI, and does not require separate cli download)
Closes ankidroid#168 (latest Rust; easier changing via rust-toolchain.toml)
Closes ankidroid#164 (universal dylib)
Closes ankidroid#127 (no docker)
Closes ankidroid#106 (most of those scripts obsolete; some commands moved into build scripts)
Closes ankidroid#99 (no docker)
Closes ankidroid#98 (no docker)
Closes ankidroid#97 (submodule automatically updated)
Closes ankidroid#96 (build will fail if commit unavailable)
Closes ankidroid#53 (no docker)
Closes ankidroid#40 (DEBUG=1 option documented)
Closes ankidroid#9 (simpler OOTB experience, and updated docs)
@dae dae mentioned this issue Nov 29, 2022
Merged
mikehardy added a commit that referenced this issue Dec 16, 2022
@dae @mikehardy
Update to 2.1.55 and refactor the build system (#246)
98a9f86 
* Move anki submodule to top level

It's not only used to build the bridge, but also to generate TS files,
so top-level makes more sense. It will also fit more nicely into a
cargo workspace.

* Add a missing drop() that recent Rust versions complain about

* Update to 2.1.55 + ninja build

* Refactor the build system

The 2.1.55 desktop release switched away from Bazel to a new build system,
which required some changes to this repo. Making the changes was a bit complicated
due to the complexity of the current build system, and I ended up shaving some yaks
to make things simpler while I was working on the changes.

Changes:

- Building for the current architecture instead of all architectures is now the
default, so getting started is easier, and a build in Android Studio no longer
requires special flags.
- The build-current.sh script has been split into build-aar and build-robo
for the Android and Robolectric parts, and can be used for both single-arch
and multi-arch builds.
- On arm64 Macs, the build scripts now create arm binaries
- In a multi-arch build, both x86 and arm64 Mac libs are built, and they're
merged into a single library. This can be done in CI, so there is no manual step
required for M1 machines anymore.
- The build now uses protobuf and python binaries/libs that the desktop build
downloads, so they don't need to be installed separately.
- The pinned Rust version and Rust targets are automatically installed as
required.
- The per-platform CI builds now build in debug mode and are faster.
- Updated the docs to explain how the NDK can be installed via Android Studio,
instead of via separate command-line tools.
- The cross/docker stuff has been stripped out, as it's of limited use as it
can't target macOS legally. Easier to use GH actions for the multi-arch
builds, and keep things simple for local development.
- Fix lint not being run in CI; caught an API 23 reference.

Bumps #179 (builds on M1 already work, so this may be simpler than expected?)
Bumps #174 (a bunch of the doctor stuff is obsolete; updated HOWTO.md and GH actions should be consulted)
Bumps #27 (I recommend closing this; single-platform is the default for local builds, and
CI runners don't have any extra compute available)

Tentatively closes #109 (didn't see the flake when I was updating the actions)

Closes #235 (translation submodules now automatically synced with anki submodule)
Closes #213 (path based on script now)
Closes #211 (builds for Arm Mac on Arm Macs)
Closes #197 (single arch is the default)
Closes #196 (desktop venv is used)
Closes #195 (can be done via the GUI, and does not require separate cli download)
Closes #168 (latest Rust; easier changing via rust-toolchain.toml)
Closes #164 (universal dylib)
Closes #127 (no docker)
Closes #106 (most of those scripts obsolete; some commands moved into build scripts)
Closes #99 (no docker)
Closes #98 (no docker)
Closes #97 (submodule automatically updated)
Closes #96 (build will fail if commit unavailable)
Closes #53 (no docker)
Closes #40 (DEBUG=1 option documented)
Closes #9 (simpler OOTB experience, and updated docs)

* Tweak print statements in build.gradle

Opted to match the actual variable name rather than make it human
readable; the print statements are just there to help debug issues
when the user forgets to shut down gradle first.

* Elaborate on prerequisites

* Integrate HOWTO inside README; remove more obsolete instructions

* Remove plantuml files

* Add platform guard to multi-arch build

* Fix clippy lints; check in CI

* Include lint in check, and remind users they have to use build-aar first

* Custom path no longer required for rsdroid-testing

* Reduce verbosity of cargo build in aar

* Mention python3 env

* Strip some obsolete stuff from doctor.sh

* Increase timeout to 10 seconds

#246 (comment)

* Mention msys2

* Apply build-all.yml suggestions from Mike

* Increase timeout of multi-arch build

* Tweak cache keys

* Confirm aar/jar output

* Change Mani's PS script to a .bat file

Avoids the need to change the execution policy, and the new script
checks execution success.

* Fix incorrect build target in build-aar.bat

* Use desktop build's bundled Python on Windows

Avoids the need to install it separately

* Add os suffix to artifacts

* Apply suggestions from code review

Co-authored-by: Mike Hardy <[email protected]>

* Only hash lockfiles

https://github.com/ankidroid/Anki-Android-Backend/pull/246/files#r1038175214

* Update to stable release commit

Co-authored-by: Mike Hardy <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
good first issue Good for newcomers help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update to 2.1.55 and refactor the build system ankitects/Anki-Android-Backend
2 participants
@dae @mikehardy