[RN][Release Testing] Download artifacts from CI to speed up testing

[cipolleschi]

Summary:

Testing releases takes a lot of time because we have to build locally several configurations.
However, the artifacts that we build locally are also built in CI.

The goal of this PR is to implement a mechanism to download those artifacts from the CI instead of build locally, so that testing the release locally can take much less time.

As an example, the full test cycle can take more than 2 hours given that we need to repackage and rebuilt the app from the template.

My plan is to add a table with the time saved once the PR is done

TODO:

• Download Hermes tarball for RNTester iOS
• Download Hermes APK for RNTester Android
• Download JSC APK for RNTester Android
• Download Packaged version of React Native to create a new app
• Use the downloaded React Native to initialize an app from the template
• Download Maven Local prebuilt in CI and use it for Template Android app

Time Savings

Setup	Before [s]	After [s]	Notes
iOS RNTester Hermes	339.68	194.86	Time saved by downloading Hermes rather then building it
iOS RNTester JSC	129.80	123.35	Not significant, expected as this workflow did not change
Android RNTester Hermes	1188.82	5.28	Huge improvement: we download the APK rather then build
Android RNTester JSC	103.10	6.28	Huge improvement: we download the APK rather then build
Creating the RNTestProject	2074.82	191.16	We download Maven, the packaged version of RN and Hermes instead of building from scratch

Changelog:

[Internal] - Speed up Release testing by downloading the CircleCI artifacts

Test Plan:

• Tested the script locally

[facebook-github-bot]

@cipolleschi has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[facebook-github-bot]

@cipolleschi has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[analysis-bot]

Platform	Engine	Arch	Size (bytes)	Diff
android	hermes	arm64-v8a	8,843,819	-1
android	hermes	armeabi-v7a	8,152,946	+2
android	hermes	x86	9,349,590	+2
android	hermes	x86_64	9,192,339	+3
android	jsc	arm64-v8a	9,455,587	+2
android	jsc	armeabi-v7a	8,636,726	+5
android	jsc	x86	9,538,676	+2
android	jsc	x86_64	9,781,974	+2

Base commit: 5705661
Branch: main

[kelset]

I've written already 6 comments so I'll stop here from now. Some are questions, some are feedbacks.

I'll give it another pass once they are addressed - thanks for kicking off the work in this area, the improvements for the "standard/golden path" flow seems massive!

Let's just make sure that there's a way to fallback to generating things manually.

[kelset]

still a few fixes needed

[cipolleschi]

/rebase

[kelset]

Test matrix (for testing once CI has finished post-rebasing):

local branch (cipolleschi/download_prebuilt_ci)

old/local flow:
RNTester + iOS + Hermes: ✅ 6m 29s
RNTester + iOS + JSC: ✅ 1m 41s
RNTester + Android + Hermes: ✅ 7m 4s
RNTester + Android + JSC: ✅ 24s
RNTestProject + iOS + Hermes: ✅ (generating RNTestProject: 20m 22s)
RNTestProject + iOS + JSC: ✅ (generating RNTestProject: 20m 22s)
RNTestProject + Android + Hermes: ❌(generating RNTestProject: 20m 22s)
RNTestProject + Android + JSC: ❌(generating RNTestProject: 20m 22s)
(the Android failure is most likely unrelated to these changes but because I'm on top of main)

new/token flow:
RNTester + iOS + Hermes: ✅ 2m 36s
RNTester + iOS + JSC: ✅ 1m 33s
RNTester + Android + Hermes: ✅ 12s 😱🤯
RNTester + Android + JSC: ✅ 8s
RNTestProject + iOS + Hermes: ✅ (generating RNTestProject: 3m 34s)
RNTestProject + iOS + JSC: ✅ (generating RNTestProject: 3m 34s)
RNTestProject + Android + Hermes: ❌(generating RNTestProject: 3m 34s)
RNTestProject + Android + JSC: ❌(generating RNTestProject: 3m 34s)
(the android failure is probably for the same reason as above)

There is no significant difference between testing in this branch VS main so I'm only testing the branch. I'll delay the testing in release branches such as 71 or 72 to when we'll do the dedicated PRs to backport this there, since we'll have to do some other changes to ensure the whole thing works correctly.

[kelset]

tested locally once again (check out my previous comment).

Overall I think we are ready to merge, the improvements are MASSIVE. Of course take my benchmarks with a pinch of salt 'cause I have an M1 Max and a 1Gbps fiber broadband, but I think we can easily agree that this will make the release crew life much easier with less time spend in testing a release.

Overall we can probably keep iterating on it and improving it, but I'd rather do that as follow up PRs and get this in, then focus on getting it on the 72 and 71 branches, then do improvements and tweaks afterwards so that they are easily cherry-picked across all the various branches 👍

[facebook-github-bot]

@cipolleschi has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[facebook-github-bot]

@cipolleschi has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[github-actions]

This pull request was successfully merged by Riccardo Cipolleschi in f6197cd.

^{When will my fix make it into a release? | Upcoming Releases}