Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] upgrade reanimated to v3 #15769

Closed
wants to merge 14 commits into from
Closed

[WIP] upgrade reanimated to v3 #15769

wants to merge 14 commits into from

Conversation

OmarBasem
Copy link
Contributor

@OmarBasem OmarBasem commented Apr 27, 2023

fixes: #15764

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm something went wrong over here, we should revert this file to the way it was

@status-im-auto
Copy link
Member

status-im-auto commented Apr 27, 2023

Jenkins Builds

Click to see older builds (41)
Commit #️⃣ Finished (UTC) Duration Platform Result
697e939 #1 2023-04-27 13:06:04 ~2 min android 📄log
697e939 #1 2023-04-27 13:06:08 ~2 min android-e2e 📄log
697e939 #1 2023-04-27 13:10:01 ~6 min tests 📄log
fb5caa7 #2 2023-04-27 13:12:39 ~31 sec android-e2e 📄log
fb5caa7 #2 2023-04-27 13:12:59 ~52 sec android 📄log
fb5caa7 #2 2023-04-27 13:18:13 ~6 min tests 📄log
✔️ fb5caa7 #2 2023-04-27 13:47:37 ~35 min ios 📱ipa 📲
0b1248f #3 2023-04-27 16:57:06 ~23 sec android 📄log
0b1248f #3 2023-04-27 16:57:20 ~37 sec android-e2e 📄log
d3d44ce #4 2023-04-27 16:57:54 ~20 sec android-e2e 📄log
d3d44ce #4 2023-04-27 16:57:58 ~21 sec android 📄log
d3d44ce #4 2023-04-27 17:03:24 ~5 min tests 📄log
✔️ d3d44ce #4 2023-04-27 17:04:04 ~6 min ios 📱ipa 📲
e7f4f6c #5 2023-04-28 14:00:32 ~4 min android 📄log
e7f4f6c #5 2023-04-28 14:00:39 ~4 min tests 📄log
e7f4f6c #5 2023-04-28 14:01:20 ~5 min android-e2e 📄log
✔️ e7f4f6c #5 2023-04-28 14:04:39 ~8 min ios 📱ipa 📲
a3829381 #6 2023-04-28 15:21:43 ~1 min android-e2e 📄log
a3829381 #6 2023-04-28 15:24:36 ~4 min android 📄log
a3829381 #6 2023-04-28 15:24:52 ~4 min tests 📄log
✔️ a3829381 #6 2023-04-28 15:30:00 ~9 min ios 📱ipa 📲
d7a90c4 #7 2023-05-02 14:01:37 ~1 min android-e2e 📄log
d7a90c4 #7 2023-05-02 14:05:42 ~5 min tests 📄log
d7a90c4 #7 2023-05-02 14:06:00 ~5 min android 📄log
✔️ d7a90c4 #7 2023-05-02 14:06:35 ~6 min ios 📱ipa 📲
fa219f7 #8 2023-05-04 16:59:57 ~1 min android 📄log
fa219f7 #8 2023-05-04 16:59:57 ~1 min android-e2e 📄log
fa219f7 #8 2023-05-04 17:02:52 ~4 min tests 📄log
✔️ fa219f7 #8 2023-05-04 17:04:40 ~6 min ios 📱ipa 📲
d2c56b6 #9 2023-05-05 08:39:48 ~17 sec android-e2e 📄log
d2c56b6 #9 2023-05-05 08:39:51 ~16 sec android 📄log
d2c56b6 #9 2023-05-05 08:45:55 ~6 min tests 📄log
✔️ d2c56b6 #9 2023-05-05 08:47:24 ~7 min ios 📱ipa 📲
62f9aff #10 2023-05-05 09:39:35 ~2 min android-e2e 📄log
62f9aff #10 2023-05-05 09:40:58 ~3 min android 📄log
62f9aff #10 2023-05-05 09:44:09 ~6 min tests 📄log
✔️ 62f9aff #10 2023-05-05 09:45:55 ~8 min ios 📱ipa 📲
22d0338 #11 2023-05-05 10:06:37 ~4 min tests 📄log
22d0338 #11 2023-05-05 10:07:53 ~5 min android 📄log
22d0338 #11 2023-05-05 10:07:57 ~5 min android-e2e 📄log
✔️ 22d0338 #11 2023-05-05 10:08:51 ~6 min ios 📱ipa 📲
Commit #️⃣ Finished (UTC) Duration Platform Result
38d6f70 #12 2023-05-05 11:34:44 ~17 sec android 📄log
38d6f70 #12 2023-05-05 11:34:57 ~31 sec android-e2e 📄log
38d6f70 #12 2023-05-05 11:39:37 ~5 min tests 📄log
✔️ 38d6f70 #12 2023-05-05 11:40:58 ~6 min ios 📱ipa 📲
e62218e #13 2023-05-05 11:49:24 ~3 min android 📄log
e62218e #13 2023-05-05 11:49:26 ~3 min android-e2e 📄log
e62218e #13 2023-05-05 11:50:45 ~4 min tests 📄log
✔️ e62218e #13 2023-05-05 11:52:15 ~6 min ios 📱ipa 📲

@OmarBasem
Copy link
Contributor Author

OmarBasem commented Apr 27, 2023

When running make run-android I get the following error:

java.lang.RuntimeException: com.android.builder.sdk.InstallFailedException: Failed to install the following SDK components: patcher;v4 SDK Patch Applier v4 ndk;23.1.7779620 NDK (Side by side) 23.1.7779620

Upgrading NDK to version 23.1.7779620 in nix/pkgs/android-sdk/compose.nix gives the following error:

/nix/store/m4h36k1cbmhavv177mgmyg4q8vjri7b3-gomobile-unstable-2022-05-18/bin/gomobile: no usable NDK in /nix/store/n4zz2v9vadj706rbpbpa6f870ix5vl7j-androidsdk-mod-sdk: open /nix/store/n4zz2v9vadj706rbpbpa6f870ix5vl7j-androidsdk-mod-sdk/ndk/23.1.7779620/meta/platforms.json: no such file or directory, open /nix/store/n4zz2v9vadj706rbpbpa6f870ix5vl7j-androidsdk-mod-sdk/ndk-bundle/meta/platforms.json: no such file or directory

Also, running make android-clean gives the following error:

Configure project :react-native-reanimated No AAR for react-native-reanimated found. Attempting to build from source. Android gradle plugin: 7.3.1 Gradle: 7.5 WARNING:Using flatDir should be avoided because it doesn't support any meta-data formats. WARNING:Software Components will not be created automatically for Maven publishing from Android Gradle Plugin 8.0. To opt-in to the future behavior, set the Gradle property android.disableAutomaticComponentCreation=true in thegradle.propertiesfile or use the new publishing DSL.

@OmarBasem
Copy link
Contributor Author

Hi @jakubgs,
I am trying to upgrade a library (react-native-reanimated). To upgrade it, I had to upgrade gradle version and grade build tools version. I am still running into some errors on Android. Could you please take a look.

@OmarBasem OmarBasem requested a review from jakubgs April 27, 2023 13:27
@jakubgs
Copy link
Member

jakubgs commented Apr 27, 2023

It doesn't seem like you changed the SDK version:

ndkVersion = "22.1.7171670";

If it even is available in our version of nixpkgs.

@OmarBasem
Copy link
Contributor Author

@jakubgs I tried changing it, and then reverted it back

@OmarBasem
Copy link
Contributor Author

OmarBasem commented Apr 27, 2023

I have just pushed a commit with the updated version to see if it will be any different on CI. (and it seems there is no difference)

Btw, I see the NDK version in /nix/pkgs/android-sdk/compose.nix do not match the ones in /android/build.gradle

@jakubgs
Copy link
Member

jakubgs commented Apr 27, 2023

The AAPT2 build issue is known, and was fixed in one of the RN update PRs with this: d69277ed

@jakubgs
Copy link
Member

jakubgs commented Apr 27, 2023

Yes, value in android/build.gradle should normally reflect the version we are using from Nix.

Also, please don't add a root slash to relative paths.
It's android/build.gradle, not /android/build.gradle, pretty sure it's not at the root of your filesystem.

@OmarBasem
Copy link
Contributor Author

The AAPT2 build issue is known, and was fixed in one of the RN update PRs with this: d69277ed

@jakubgs after doing that update to default.nix I am getting the following error:
error: 1 dependencies of derivation '/nix/store/vhknay6drrb228a7hz0hpy3s4kjb0jv0-status-mobile-maven-deps.drv' failed to build error: build of '/nix/store/1wvicqsq8p6kiqidibd36gd4pn25ajyf-status-go-0.148.3-8608aec-android.drv', '/nix/store/vhknay6drrb228a7hz0hpy3s4kjb0jv0-status-mobile-maven-deps.drv' failed make: *** [run-android] Error 1

@jakubgs
Copy link
Member

jakubgs commented Apr 28, 2023

No, that's not the error. That's just what is printed after the actual error.
Your builds are failing due to missing gradle deps:

> Could not resolve all files for configuration ':classpath'.
   > Could not find commons-codec:commons-codec:1.11.
     Searched in the following locations:
       - file:/build/android/libs/commons-codec-1.11.jar
       - file:/build/android/libs/commons-codec.jar
       - file:/build/android/app/libs/commons-codec-1.11.jar
       - file:/build/android/app/libs/commons-codec.jar
       - file:/nix/store/csq035dx5q64pxjnxdn8x4h0ap2b14s3-status-mobile-maven-deps/commons-codec/commons-codec/1.11/commons-codec-1.11.pom
     Required by:
         project : > com.android.tools.build:gradle:7.3.1 > com.android.tools.build:builder:7.3.1

Are you sure you updated them?

@jakubgs
Copy link
Member

jakubgs commented Apr 28, 2023

Oh, I see, running make nix-update-gradle is failing with:

* What went wrong:
The Android Gradle plugin supports only Kotlin Gradle plugin version 1.5.20 and higher.
The following dependencies do not satisfy the required version:
root project 'StatusIm' -> org.jetbrains.kotlin:kotlin-gradle-plugin:1.4.31

That's what the nix/deps/gradle/add_package.sh script was created for:

 > make shell TARGET=gradle
Configuring Nix shell for target 'gradle'...
Yarn modules changed, copying new version over
Copying node_modules from Nix store:
 - /nix/store/zbrx7vqcz1y3r60md3qricgv2bp7x72b-status-mobile-node-deps-1.22.0
Done in: 12.136s

[nix-shell:~/work/status-mobile]$ nix/deps/gradle/add_package.sh org.jetbrains.kotlin:kotlin-gradle-plugin:1.5.20
Changes made:
 nix/deps/gradle/deps.urls | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

Regenerating Nix files...
Successfully added: org.jetbrains.kotlin:kotlin-gradle-plugin:1.5.20

NOTICE: Running 'make nix-update-gradle' in a new shell is recommended.

I've pushed a commit with updated dependencies.
For more details see: https://github.com/status-im/status-mobile/blob/develop/nix/deps/gradle/README.md#known-issues

@jakubgs
Copy link
Member

jakubgs commented Apr 28, 2023

But I see this is more complex than just that. It needs more work.

@OmarBasem
Copy link
Contributor Author

OmarBasem commented May 2, 2023

@jakubgs Trying to run make shell TARGET=gradle fails with error:

patching sources
installing
error: builder for '/nix/store/wz30h0nd7vf31hjq36jqj68bdn1x8ifc-aapt2-7.3.0-8691043.drv' failed with exit code 12;
       last 8 log lines:
       > unpacking sources
       > Archive:  /nix/store/7nyznw5199fg5j1j6yawn1z17ks5kwmz-aapt2-7.3.0-8691043/aapt2-7.3.0-8691043-osx.jar
       >    creating: /private/tmp/nix-build-aapt2-7.3.0-8691043.drv-0/tmp.i9ieU6xMHh/META-INF/
       >   inflating: /private/tmp/nix-build-aapt2-7.3.0-8691043.drv-0/tmp.i9ieU6xMHh/META-INF/MANIFEST.MF
       >   inflating: /private/tmp/nix-build-aapt2-7.3.0-8691043.drv-0/tmp.i9ieU6xMHh/aapt2
       >   inflating: /private/tmp/nix-build-aapt2-7.3.0-8691043.drv-0/tmp.i9ieU6xMHh/NOTICE
       > patching sources
       > installing
       For full logs, run 'nix log /nix/store/wz30h0nd7vf31hjq36jqj68bdn1x8ifc-aapt2-7.3.0-8691043.drv'.
error: 1 dependencies of derivation '/nix/store/7fmq4lxn2sca1jz7wlc5gxa6knj6f3di-status-mobile-maven-deps.drv' failed to build
make: *** [shell] Error 1

@OmarBasem
Copy link
Contributor Author

But I see this is more complex than just that. It needs more work.

So what does it need :)

@OmarBasem OmarBasem marked this pull request as ready for review May 2, 2023 14:00
@jakubgs
Copy link
Member

jakubgs commented May 4, 2023

It means it needs more work. Also, please use triple backticks for large logs, otherwise the formatting is disgusting:
https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/creating-and-highlighting-code-blocks#fenced-code-blocks

jakubgs added 2 commits May 4, 2023 18:57
Signed-off-by: Jakub Sokołowski <[email protected]>
@jakubgs
Copy link
Member

jakubgs commented May 4, 2023

Pushed some of my changes, but the fundamental issue is in the sub-projects that specify 1.4.31 as their Kotlin plugin version.

@jakubgs
Copy link
Member

jakubgs commented May 5, 2023

@siddarthkay already created a tagged release of react-native-hole-view here at v2.1.3-status:
https://github.com/status-im/react-native-hole-view/tree/v2.1.3-status

Which does make it possible to override the Kotlin Gradle Plugin version, as seen here:

    def kotlin_version = rootProject.ext.has('RNHV_kotlinVersion') ? rootProject.ext.get('RNHV_kotlinVersion') : project.properties['RNHV_kotlinVersion']
    def gradle_version = rootProject.ext.has('RNHV_gradleVersion') ? rootProject.ext.get('RNHV_gradleVersion') : project.properties['RNHV_gradleVersion']

https://github.com/status-im/react-native-hole-view/blob/v2.1.3-status/android/build.gradle#L3-L4
Which in theory should work.

@jakubgs
Copy link
Member

jakubgs commented May 5, 2023

Using v2.1.3-status release does work, but now we fail on a missing Gradle dependency:

Regenerating Nix files...
Found 40 sub-projects...
Found 699 direct dependencies...
Found 1045 dependency URLs...
 ! Failed to fetch: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-bom/9.5/asm-bom-9.5.jar

Or specifically, a missing JAR/AAR for the POM:
https://repo.maven.apache.org/maven2/org/ow2/asm/asm-bom/9.5/

That's an edge case that we should already be handing but we do not:

# Some deps are just POMs, in which case there is no JAR to fetch.
[[ "${OBJ_TYPE}" == "" ]] && fetch_and_template_file "${PKG_NAME}.jar"

```
* What went wrong:
Could not determine the dependencies of task ':app:lintVitalReportPr'.
> Failed to find Build Tools revision 30.0.3
```

Signed-off-by: Jakub Sokołowski <[email protected]>
@jakubgs
Copy link
Member

jakubgs commented May 5, 2023

This is an issue that we already saw in the RN upgrade PR(#15203 (comment)):

* What went wrong:
Execution failed for task ':react-native-dialogs:extractReleaseAnnotations'.
> Could not resolve all files for configuration ':react-native-dialogs:detachedConfiguration1'.
   > Could not resolve com.android.tools.lint:lint-gradle:30.3.1.
     Required by:
         project :react-native-dialogs
      > No cached version of com.android.tools.lint:lint-gradle:30.3.1 available for offline mode.
      > No cached version of com.android.tools.lint:lint-gradle:30.3.1 available for offline mode.

For now I can add it manually, but it will disappear at next use of make nix-update-gradle.

@jakubgs
Copy link
Member

jakubgs commented May 5, 2023

Wht the fuck is this:

Could not evaluate onlyIf predicate for task ':react-native-reanimated:downloadBoost'.
 > Unable to download file 'boost_1_63_0.tar.gz' in offline mode.

@OmarBasem
Copy link
Contributor Author

You must get rid of all v1 usage in code first and move that to v2 and only then you should upgrade this library :)

@siddarthkay we can get rid of v1 code immediately no problem. I don't think we are using v1 code in status-im2

@OmarBasem
Copy link
Contributor Author

OmarBasem commented May 8, 2023

Could not evaluate onlyIf predicate for task ':react-native-reanimated:downloadBoost'.

Unable to download file 'boost_1_63_0.tar.gz' in offline mode.

@jakubgs possibly related issue: software-mansion/react-native-reanimated#3638

@jakubgs
Copy link
Member

jakubgs commented May 8, 2023

Well, that issue says to upgrade to v2, which is probably not what we want:

547766 newfstatat(AT_FDCWD, "/home/jakubgs/work/status-mobile/node_modules/react-native-reanimated/android/build/downloads/boost_1_63_0.tar.gz", 0x7fcdf81e6f00, 0) = -1 ENOENT (No such file or directory)

This can be found in the strace log from Gradle. It's looking for it in:

node_modules/react-native-reanimated/android/build/downloads/boost_1_63_0.tar.gz

Wtf is this and why it's trying to download it in some custom way? No idea.

@jakubgs
Copy link
Member

jakubgs commented May 8, 2023

Apparently this is only necessary when using RN older than 71:

if (REACT_NATIVE_MINOR_VERSION < 71) {
    // You need to have following folders in this directory:
    //   - boost_1_63_0
    //   - double-conversion-1.1.6
    //   - folly-deprecate-dynamic-initializer
    //   - glog-0.3.5
    def dependenciesPath = System.getenv("REACT_NATIVE_DEPENDENCIES")

    // The Boost library is a very large download (>100MB).
    // If Boost is already present on your system, define the REACT_NATIVE_BOOST_PATH env variable
    // and the build will use that.
    def boostPath = dependenciesPath ?: System.getenv("REACT_NATIVE_BOOST_PATH")

https://github.com/software-mansion/react-native-reanimated/blob/3.1.0/android/build.gradle#L705-L716

Now that's some dumb shit. Do they just expect people to download those manually?

@jakubgs
Copy link
Member

jakubgs commented May 8, 2023

They actually download them:

    if (REACT_NATIVE_MINOR_VERSION < 71) {
        extractAARHeaders.dependsOn(prepareThirdPartyNdkHeaders)
        extractSOFiles.dependsOn(prepareThirdPartyNdkHeaders)

        nativeBuildDependsOn(prepareThirdPartyNdkHeaders)
    task prepareThirdPartyNdkHeaders(dependsOn:[
        downloadNdkBuildDependencies,
    task downloadBoost(dependsOn: resolveBoost, type: Download) {
        def transformedVersion = BOOST_VERSION.replace("_", ".")
        def artifactLocalName = "boost_${BOOST_VERSION}.tar.gz"
        def srcUrl = "https://boostorg.jfrog.io/artifactory/main/release/${transformedVersion}/source/boost_${BOOST_VERSION}.tar.gz"
        if (REACT_NATIVE_MINOR_VERSION < 69) {
            srcUrl = "https://github.com/react-native-community/boost-for-react-native/releases/download/v${transformedVersion}-0/boost_${BOOST_VERSION}.tar.gz"
        }

But since the Nix build is pure there is no network for it to do that.

Why do they re-invent the wheel and do their own dependency management? Who thought this was a good idea?
Now we might have to write some custom Nix derivations to download this and then patch the module to make this work.

Or we could actually do the React Native upgrade...

@siddarthkay
Copy link
Contributor

@jakubgs : I am resuming work on the upgrade this week, should be fun, will keep you posted.

@OmarBasem
Copy link
Contributor Author

if (REACT_NATIVE_MINOR_VERSION < 71) {

Oh I see..

@OmarBasem
Copy link
Contributor Author

Fine we can do the RN upgrade then

@OmarBasem
Copy link
Contributor Author

Dependency: #15203

@flexsurfer flexsurfer changed the title upgrade reanimated to v3 [WIP] upgrade reanimated to v3 May 16, 2023
@flexsurfer
Copy link
Member

feel free to reopen when needed, or create a new one

@flexsurfer flexsurfer closed this Jun 1, 2023
@jakubgs jakubgs deleted the reanimated-3 branch August 23, 2024 07:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
No open projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Upgrade Reanimated to V3
6 participants