Alternative Packaging/Fusing Method

[MikuAuahDark]

Not sure if this really fits in GitHub issue but it's related to love-android game packaging.

As per Google announcements, submitting updates (or new apps) in Play Store starting on Q2 2021 requires Android Application Bundle. This renders the previously preferred method obsolete.

Recently I realize that there's 2 ways to fix this, both with their downsides:

1. Tell user to complie from source. This offers user the highest flexibility, including modification to the Java side (or even C++ side if user also needs to embed external Lua C modules) of the app. The downside is user needs to download GBs of SDKs and NDKs and additional build-time dependency, which may not suitable for some users due to slow internet, low disk space (those things are huge), and/or slow PC/laptop.
2. Perform modification to Android Application Budle. This requires knowledge on the bundle format (it's just zip file with specifically structured files inside), especially how the AndroidManifest.xml is encoded. Also GitHub Actions already provides AAB as the artifacts so someone can simply grab those AABs. The downside is someone needs to contribute to the script to manipulate AABs, which can means never. I also don't know if this method is even allowed by Google. Relevant tweet.

Any suggestion and pointer is welcome.

Cross-posted to LÖVE forums.

[IsometricShahil]

I would personally prefer compiling from source rather than any hacky AAB modification and I think if it is not already disallowed then Google will probably disallow it in the future anyway.
The higher flexibility offered by this would be great.
I don't see a way to avoid the massive SDK,NDK downloads unfortunately, Android dev is too hacky without those imo but one can (ab)use github actions.

[flamendless]

Same as miku, I'm very limited to the size I can download due to ISP and machine (very low storage).

I have experience compiling the apk and using APKTool. For me, APKTool is way easier as we can make scripts to automate it. Building would take longer, also for people inexperienced with Android environment, a lot of configuration will pose confusion such as correct ndk/sdk versions to download, etc etv

[MikuAuahDark]

Generally the version of which SDK and NDK version to download is clear, written both in the repository wiki and in the readme. The main problem however is downloading those SDK and NDK. It takes space and lots of internet bandwidth.

[bazilxp]

Generally the version of which SDK and NDK version to download is clear, written both in the repository wiki and in the readme. The main problem however is downloading those SDK and NDK. It takes space and lots of internet bandwidth.

I think about the method of fusing there are main challenge, to be specific APK about gradle which version to take ,it's require juggle on the MacOS jdk to make it happen for me :) Compile is straight forward :)

[bazilxp]

Same as miku, I'm very limited to the size I can download due to ISP and machine (very low storage).

I have experience compiling the apk and using APKTool. For me, APKTool is way easier as we can make scripts to automate it. Building would take longer, also for people inexperienced with Android environment, a lot of configuration will pose confusion such as correct ndk/sdk versions to download, etc etv

NDK/SDK One thing , love x86 apk is not provided , so if you later want try android x86 simulator , need to compile apk.

[flamendless]

For making small changes and fixes in the love file, compiling takes longer than the apktool method. For me, the only benefit of compiling is when implementing android features and using java api or something. I guess both will be good depending on the use case ofc

[MikuAuahDark]

I think about the method of fusing there are main challenge, to be specific APK about gradle which version to take ,it's require juggle on the MacOS jdk to make it happen for me :) Compile is straight forward :)

JDK is out of scope here. You'll always need JDK regardless of packaging methods. If you choose to compile from source, rest assured you've setup JDK correctly so it can run Gradle. The current master branch uses Gradle 6.6.1 which should support Java 14.

NDK/SDK One thing , love x86 apk is not provided , so if you later want try android x86 simulator , need to compile apk.

x86 (and x86-64) is not provided due to build time and space considerations. Having only ARMv7 and ARMv8 binaries alone gives the APK size of 8MB with 20 minutes of build time in GitHub Actions. Double that up for x86 and x86-64 binaries too. That's overkill for a game framework like this compared to Windows binaries which is just 4MB. However I may consider compiling x86 and x86-64 binaries for LOVE 12.0 (with separate APKS for x86 and x86-64 for 11.4 probably) for Chromebook support, but I don't know how worth it is.

[bazilxp]

I think about the method of fusing there are main challenge, to be specific APK about gradle which version to take ,it's require juggle on the MacOS jdk to make it happen for me :) Compile is straight forward :)

JDK is out of scope here. You'll always need JDK regardless of packaging methods. If you choose to compile from source, rest assured you've setup JDK correctly so it can run Gradle. The current master branch uses Gradle 6.6.1 which should support Java 14.

NDK/SDK One thing , love x86 apk is not provided , so if you later want try android x86 simulator , need to compile apk.

x86 (and x86-64) is not provided due to build time and space considerations. Having only ARMv7 and ARMv8 binaries alone gives the APK size of 8MB with 20 minutes of build time in GitHub Actions. Double that up for x86 and x86-64 binaries too. That's overkill for a game framework like this compared to Windows binaries which is just 4MB. However I may consider compiling x86 and x86-64 binaries for LOVE 12.0 (with separate APKS for x86 and x86-64 for 11.4 probably) for Chromebook support, but I don't know how worth it is.

The x86 , check mpg123 , disable the "SSE" there only one thing which is not supported by CLANG , the reset should be fine.

[MikuAuahDark]

I was thinking of refactoring part of the packaging method, mainly the part where you should put game.love to assets folder inside the APK. Instead of putting your game.love into the assets folder, you simply need to put your unpackaged game instead. So the APK will have something like this folder structure

+ assets
-+ main.lua // this is the game entry point
-+ conf.lua // this is the game configuration
-+ (other files/folders for your game)
+ lib
-+ lib/arm64-v8a
-+ lib/armeabi-v7a
+ res
+ classes.dex
+ (other files)

And for AAB

+ base
-+ assets
--+ main.lua // this is the game entry point
--+ conf.lua // this is the game configuration
--+ (other files/folders for your game)
-+ lib
--+ lib/arm64-v8a
--+ lib/armeabi-v7a
-+ (other files)
+ BundleConfig.pb

I have plan to add this as experimental in 11.4 (game.love will be looked up first) and prefer it in 12.0 (main.lua will be looked up first).

Let me know what do you think.

[MikuAuahDark]

Yes they are. It will contain all the .so needed. On unrelated note, I was thinking of adding some kind of support to dynamic "require" by automatically pointing the C require path to the architecture-specific folder, but there are 2 problems. 1. If useNativeLibs is set to false (Android allows loading .so inside ZIP directly as long as it's page-aligned and compressed) then there's no path to set. 2. LuaJIT needs to be linked dynamically So in the end I decided not to bother. Pada tanggal Kam, 4 Feb 2021 00.19, FlamingArr <[email protected]> menulis:

…

Will lib/arm.. folders contain shared libraries, i.e. .so files? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#201 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABZHFFVIQ24JS4DF2G4CFDDS5FZSFANCNFSM4WE4ZDJQ> .

[bjornbytes]

lovr also puts game files unzipped in assets, it simply mounts the apk as the game archive with assets as the archive root (can use PHYSFS_setRoot for that).

For libraries, the lovr C require path on android will resolve a module to <apk>!lib/arm64-v8a/<module>.so to load libraries from the apk (we can force extractNativeLibs to false, but this may not be possible on all the android versions love needs to support).

[MikuAuahDark]

Looks like PHYSFS_setRoot is PhysFS 3.1.0 pre-release which is available in LOVE forked version, but not recent enough to have full Android support, while the documentation I read is 3.0.2. Thanks, will use that.

For libraries, I think that's clever, but some Lua modules have multiple shared objects and they need to be placed in subdirectory (require("something.something") goes to lib/arm64-v8a/something/something.so), but Android seems doesn't allow you to have subdirectory inside the lib/arch/ folder.

[MikuAuahDark]

It looks like that AAsset handles so thing called "split APKs whilst PHYSFS_setRoot only operates on the base APK, so I decided to implement AAsset virtual archive and use that.

[boswelja]

Hey everyone. I'm not too familiar with this project, so please correct me if I make any incorrect assumptions. With that out of the way, I propose a solution to the packaging issues.

I strongly agree with @FlamingArr, it's definitely not a good idea to use hacky AAB/APK modification in the first place, but I understand some people are under certain constraints and it's not feasible to set up the Android SDK on their machine.

This is where my proposed solution(s) comes in. I'm confident using the below together should be enough of a "middle ground" for everyone.

The big change:

Game code gets provided at compile time, via a (possibly configurable) folder. Now, this has some advantages and disadvantages.

1. This requires compiling engine along with the rest of your code.
2. We can throw an error at compile time if anything is amiss.
3. More "proper" way of doing this, i.e. no hacky modifications to APKs/AABs

Now, since 1 is a blocking compromise for quite a few people, I propose moving workflows to your own repos and set up GitHub Actions. The end result is a system similar to before, where you only need to edit your own LUA scripts locally. The main drawback is you then become limited by your CI build speed. You can probably mitigate a good chunk of that by distributing as much of the engine as possible as prebuilt binaries, but it will cause more delays than hacky APK/AAB modification. The other caveat here is you're effectively required to have some sort of CI/CD setup, but if you're storing code in GitHub then you're half way there already.

To summarise:

Provide game code at compile time.

Use a CI/CD-friendly architecture

If needed, offload AAB/APK building to CI/CD

Thoughts?

[MikuAuahDark]

Compiling the game framework along with game code that user wrote is already supported (point 1 of the original comment) but their drawbacks hinder the users from doing this.

Telling user to fork this repository and use GitHub Actions to package the game is also a possible solution, but we may get into trouble with GitHub if they found out that (ab)using Actions to package game comes from us telling them to. Furthermore some users who doesn't like putting their game code online may found this as a drawback too. Furthermore, we can't tell them to put their code online because our game framework doesn't use (L)GPL for this reason.

A possible way would be having a template repository that users can fork then change the package name prior triggering the workflow, but if user doesn't want to publish their game code online then they have to modify the AAB by putting their game code inside anyway.

[boswelja]

I don't see how you'd "get in trouble with GitHub", since it's ultimately up to developers whether they use Workflows to build their games. I personally use Workflows to handle building and publishing most, if not all of my projects.

Your comment about publishing the game code online is invalid, since GitHub, along with many other hosting solutions, offer private repositories. Private repos do have some limitations around running workflows, namely (without paying extra) you're limited by the maximum amount of time workflows can run for in any given month. According to their plan pricing that limit is currently 2000 minutes per month.

I still don't see any issues with this. It is a compromise, it will always be a compromise, but no production app should ever be modifying APKs before publishing. This seems like the clear solution.

[MikuAuahDark]

I see, so setting up a "template" repository where users would able to put their games inside (and modify the icon) would be a solution.

[boswelja]

Definitely seems like the way forward, since you should also be able to provide basic workflow files as well. Would be great to get some thoughts from devs affected by this, in case there's any major issues we haven't covered

[IsometricShahil]

Yeah, +1 from me as well, setting up a template repo seems to bring the best of both worlds. The flexibility of compiling from source and the ease of hacky modification.

[yusufmalikul]

I can't find the link to download NDK 21.3.6528147 or r21d. So, this is also a problem for new users. Not just internet speed but also the download link.

even this official link doesn't have it: https://github.com/android/ndk/wiki/Unsupported-Downloads

edit: I craft the download link manually https://dl.google.com/android/repository/android-ndk-r21d-windows-x86_64.zip (1GBs)

[MikuAuahDark]

Looks like they changed some things. We can't upgrade to recent NDK because it drops support for Android 4.2.

21.3 correspond to r21d (21.0 is r21, 21.1 is r21b, ...), so if you change "r21e" in https://dl.google.com/android/repository/android-ndk-r21e-linux-x86_64.zip to "r21d", you'll get 21.3.x version.

[ravener]

By the way I don't know if you heard of this but google maintains the bundletool here https://github.com/google/bundletool that could probably be used to create android app bundles. Would be nice to keep the old way instead of all this android studio/sdk/ndk/gradle stuff.

[IoriBranford]

I agree with a bundletool-based solution somewhere down the line. In the meantime, I have been working on a Github Action and Docker images to eliminate setup and compile time. https://github.com/ioribranford/docker-build-love-android

[MikuAuahDark]

A bundletool solution would be great, but there are no guarantee if Google will allow such modified bundle to Play Store.

[ravener]

I still believe the apktool method shouldn't be called "obsolete". Not everyone has the goal of uploading to playstore and it helps some people to quickly upload demo apks and whatnot.

[MikuAuahDark]

I don't think so either. Recently Android made changes where application ID and package name are no longer interchangeable. Changing the package name in APKTool may result in app crashing due to missing class resources (although I haven't really tested this in LOVE, but can confirm when modifying other APK) because APKTool still treat application ID and package name as one thing.

iBotPeaches/Apktool#3011