[eas-cli] [ENG-12650] add fingerprint sources to build metadata #2422

kadikraman · 2024-06-11T16:56:36Z

Related PRs

Why

It's very difficult to debug why the local and remote fingerprint are different.

Currently the only way to do it would be manual: add step to print out the remote sources, calculate local fingerprint, diff the two. Ideally we'd like to show the diff in the build step instead.

How

Save the local fingerprint in a google cloud bucket and show a diff if the fingerprints are different.

Test Plan

Ran turtle and www locally with changes from the these packages manually linked to turtle.

wschurman

I think the build-tools PR will need to be deployed first (and maybe even the www one) but I'm not super familiar with the push safety between these repos as far as the build job validation goes.

kadikraman · 2024-06-12T08:36:06Z

@wschurman you're right, this is what the package dependencies look like:

It seems like actually I might have to break the @expo/eas-build-job into a separate PR so it could be merged separately since the dependency is used in @expo/build-tools.

So the order would be:

@expo/eas-build-job -> merge and publish
@expo/build-tools -> add dependency (@expo/eas-build-job), merge and release
turtle worker -> add dependencies (@expo/eas-build-job and @expo/build-tools), merge and deploy
www -> merge and deploy
eas-cli -> add dependency (@expo/eas-build-job), merge and release

szdziedzic

It seems like actually I might have to break the @expo/eas-build-job into a separate PR so it could be merged separately since the dependency is used in @expo/build-tools.
So the order would be:
@expo/eas-build-job -> merge and publish
@expo/build-tools -> add dependency (@expo/eas-build-job), merge and release
turtle worker -> add dependencies (@expo/eas-build-job and @expo/build-tools), merge and deploy
www -> merge and deploy
eas-cli -> add dependency (@expo/eas-build-job), merge and release

Re 1 and 2: The eas-build repo is set up in a way that packages from eas-build use local versions of other packages from the same repo by default, so you just need to rebuild the @expo/eas-build-job package to be able to use it in the @expo/build-tools. You don't even need to split the @expo/eas-build-job and @expo/build-tools changes into 2 PRs. You can just merge it all together and once you run yarn release our release script will publish correct versions in correct order and bump everything in package.json.

szdziedzic

Oh, looking at https://github.com/expo/universe/pull/15517#pullrequestreview-2113659688 I believe that the fingerprint file upload to GCS should happen in CLI (same as we do for project archive and build metadata) and we should just pass a data necessary to link uploaded file with build to mutation

szdziedzic · 2024-06-13T12:18:49Z

Also one more question: What about builds triggered with GH trigger?

We don't install node modules nor run expo updates CLI before starting the GH build. We run EAS CLI inside the build process to resolve and backfill metadata at the beginning of the build process in the Resolve build config phase.

Will it be compatible with your changes? I mean in that case both "before" and "after" fingerprints will come from the build worker then (but from different build phases).

kadikraman · 2024-06-20T14:31:04Z

Updated the PR to store fingerprints in Google cloud instead.

Regarding your question on builds triggered via the GH @szdziedzic - I don't know for sure, but if the project runtime is currently calculated correctly on GH builds for projects that use fingerprint then this should work as well.

wschurman

One minor change to naming and a question, otherwise looks good.

packages/eas-cli/src/build/build.ts

wschurman

Code looks good. Double check CI failures before landing. And also ensure the other PRs land ahead of this.

szdziedzic

If comments from expo/eas-build#407 (review) end up being applicable we will probably also need to change metadata structure here

packages/eas-cli/src/build/build.ts

github-actions · 2024-06-27T14:16:31Z

Size Change: -1 kB (0%)

Total Size: 52 MB

Filename	Size	Change
`./packages/eas-cli/dist/eas-linux-x64.tar.gz`	52 MB	-1 kB (0%)

_{compressed-size-action}

codecov · 2024-06-27T14:16:59Z

Codecov Report

Attention: Patch coverage is 38.88889% with 33 lines in your changes missing coverage. Please review.

Project coverage is 53.44%. Comparing base (1c61465) to head (005c038).

Files	Patch %	Lines
packages/eas-cli/src/build/build.ts	13.80%	22 Missing and 3 partials ⚠️
.../eas-cli/src/project/resolveRuntimeVersionAsync.ts	0.00%	5 Missing ⚠️
packages/eas-cli/src/build/metadata.ts	50.00%	1 Missing ⚠️
packages/eas-cli/src/build/utils/repository.ts	0.00%	1 Missing ⚠️
...kages/eas-cli/src/rollout/actions/CreateRollout.ts	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2422      +/-   ##
==========================================
- Coverage   53.47%   53.44%   -0.02%     
==========================================
  Files         530      530              
  Lines       19508    19552      +44     
  Branches     3958     3968      +10     
==========================================
+ Hits        10430    10448      +18     
- Misses       8327     8350      +23     
- Partials      751      754       +3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

packages/eas-cli/src/build/build.ts

packages/eas-cli/src/build/graphql.ts

packages/eas-cli/src/build/metadata.ts

linear · 2024-06-28T13:29:20Z

ENG-12650 Save or upload local fingeprint sources during eas build

kadikraman · 2024-06-28T14:54:34Z

I think this should do it @szdziedzic ! Tested both iOS and Android on both --local and EAS 👀

On EAS

Android	iOS

With `--local`

Android	iOS

github-actions · 2024-06-28T14:57:47Z

✅ Thank you for adding the changelog entry!

szdziedzic

awesome! good job! 🙌 🚀

giorgiofellipe · 2024-07-03T20:20:28Z

@kadikraman Hi Kadi, thanks for this amazing work.
I've just updated my eas-cli to the latest version to check the diff but it is not showing to me.

I still see the old message:

[CONFIGURE_EXPO_UPDATES] Runtime version from the app config evaluated on your local machine (5b9cea733cdb7a2a2a4e25163b2b6f0a8b15923b) does not match the one resolved here (3868961bbb19877ecdf3ebae7abb910226a65157).
[CONFIGURE_EXPO_UPDATES] If you're using conditional app configs, e.g. depending on an environment variable, make sure to set the variable in eas.json or configure it with EAS Secret.

My env info:

expo-env-info 1.2.0 environment info:
    System:
      OS: macOS 13.5.1
      Shell: 5.9 - /bin/zsh
    Binaries:
      Node: 22.2.0 - /opt/homebrew/bin/node
      Yarn: 1.22.19 - /usr/local/bin/yarn
      npm: 10.7.0 - /opt/homebrew/bin/npm
    Managers:
      CocoaPods: 1.15.2 - /opt/homebrew/bin/pod
    SDKs:
      iOS SDK:
        Platforms: DriverKit 23.2, iOS 17.2, macOS 14.2, tvOS 17.2, visionOS 1.0, watchOS 10.2
      Android SDK:
        API Levels: 26, 29, 30, 31, 32, 33, 34
        Build Tools: 29.0.2, 29.0.3, 30.0.2, 30.0.3, 31.0.0, 32.0.0, 33.0.0, 33.0.1, 33.0.2, 34.0.0
        System Images: android-30 | Google APIs ARM 64 v8a, android-33 | Google APIs ARM 64 v8a, android-33 | Google Play ARM 64 v8a, android-34 | Google APIs ARM 64 v8a
    IDEs:
      Android Studio: 2022.3 AI-223.8836.35.2231.10671973
      Xcode: 15.2/15C500b - /usr/bin/xcodebuild
    npmPackages:
      babel-preset-expo: ^11.0.8 => 11.0.8 
      expo: ~51.0.17 => 51.0.17 
      expo-router: ^3.5.15 => 3.5.15 
      react: 18.2.0 => 18.2.0 
      react-dom: 18.2.0 => 18.2.0 
      react-native: 0.74.1 => 0.74.1 
      react-native-web: ~0.19.6 => 0.19.12 
    npmGlobalPackages:
      eas-cli: 10.1.0
    Expo Workflow: managed

Am I missing something?
Thank you!

wschurman · 2024-07-03T22:00:43Z

@giorgiofellipe - please file a new issue with steps to reproduce the issue you're seeing. Fingerprint is still in beta (appreciate you testing it out!), so we're still working out the kinks, and a step-by-step set of repro steps is the easiest way to communicate the issue and get to a fix.

tfcornerstone · 2024-08-07T15:22:01Z

@kadikraman Doesn't work with --non-interactive?

wschurman · 2024-08-07T15:52:45Z

@tfcornerstone - please file a new issue. See #2422 (comment)

Add fingerprint sources to build metadata

a0c173a

kadikraman requested review from Kudo and wschurman June 11, 2024 16:56

kadikraman requested review from szdziedzic, khamilowicz and sjchmiela as code owners June 11, 2024 16:56

kadikraman mentioned this pull request Jun 11, 2024

[build-tools] [ENG-12652] display fingerprint sources diff is not the same expo/eas-build#407

Merged

wschurman approved these changes Jun 12, 2024

View reviewed changes

Kudo approved these changes Jun 12, 2024

View reviewed changes

szdziedzic approved these changes Jun 13, 2024

View reviewed changes

szdziedzic requested changes Jun 13, 2024

View reviewed changes

kadikraman added 2 commits June 19, 2024 12:45

Merge branch 'main' into @kadikraman/diff-fingerprint

9cb7c4a

Upload fingerprint to google cloud

48cc686

kadikraman requested review from szdziedzic, wschurman and Kudo June 20, 2024 14:31

wschurman requested changes Jun 20, 2024

View reviewed changes

packages/eas-cli/src/build/build.ts Show resolved Hide resolved

packages/eas-cli/src/build/build.ts Outdated Show resolved Hide resolved

Rename fingerprintGCSLocation -> fingerprintGCSBucketKey

c413b41

kadikraman requested a review from wschurman June 21, 2024 12:12

wschurman approved these changes Jun 21, 2024

View reviewed changes

szdziedzic requested changes Jun 24, 2024

View reviewed changes

Add fingerprint source in an object

740064c

Kudo approved these changes Jun 24, 2024

View reviewed changes

packages/eas-cli/src/build/build.ts Show resolved Hide resolved

kadikraman mentioned this pull request Jun 25, 2024

[eas-build-job] [ENG-12593] add fingerprint sources to metadata expo/eas-build#421

Merged

kadikraman added 3 commits June 27, 2024 15:08

Merge branch 'main' into @kadikraman/diff-fingerprint

7596385

Update dependencies

cd52a23

Add comment about fingerprintSources

fe0fb1a

Add a changelog entry

8a0b7b6

kadikraman requested a review from szdziedzic June 27, 2024 14:20

szdziedzic requested changes Jun 28, 2024

View reviewed changes

packages/eas-cli/src/build/build.ts Show resolved Hide resolved

packages/eas-cli/src/build/graphql.ts Show resolved Hide resolved

packages/eas-cli/src/build/metadata.ts Outdated Show resolved Hide resolved

kadikraman changed the title ~~[eas-cli] add fingerprint sources to build metadata~~ [eas-cli] [ENG-12650] add fingerprint sources to build metadata Jun 28, 2024

Add support for local build plugin

fdc6205

Add safety check that a temp directory exists

005c038

kadikraman requested a review from szdziedzic June 28, 2024 14:58

szdziedzic approved these changes Jul 1, 2024

View reviewed changes

kadikraman merged commit 2b23c11 into main Jul 2, 2024
9 checks passed

kadikraman deleted the @kadikraman/diff-fingerprint branch July 2, 2024 11:58

giorgiofellipe mentioned this pull request Jul 4, 2024

Always generating different fingerprints for my builds, also not showing the fingerprint diff #2448

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[eas-cli] [ENG-12650] add fingerprint sources to build metadata #2422

[eas-cli] [ENG-12650] add fingerprint sources to build metadata #2422

kadikraman commented Jun 11, 2024 •

edited

Loading

wschurman left a comment

kadikraman commented Jun 12, 2024

szdziedzic left a comment

szdziedzic left a comment

szdziedzic commented Jun 13, 2024 •

edited

Loading

kadikraman commented Jun 20, 2024

wschurman left a comment

wschurman left a comment

szdziedzic left a comment

github-actions bot commented Jun 27, 2024 •

edited

Loading

codecov bot commented Jun 27, 2024 •

edited

Loading

linear bot commented Jun 28, 2024

kadikraman commented Jun 28, 2024

github-actions bot commented Jun 28, 2024

szdziedzic left a comment

giorgiofellipe commented Jul 3, 2024 •

edited

Loading

wschurman commented Jul 3, 2024 •

edited

Loading

tfcornerstone commented Aug 7, 2024

wschurman commented Aug 7, 2024

[eas-cli] [ENG-12650] add fingerprint sources to build metadata #2422

[eas-cli] [ENG-12650] add fingerprint sources to build metadata #2422

Conversation

kadikraman commented Jun 11, 2024 • edited Loading

Related PRs

Why

How

Test Plan

wschurman left a comment

Choose a reason for hiding this comment

kadikraman commented Jun 12, 2024

szdziedzic left a comment

Choose a reason for hiding this comment

szdziedzic left a comment

Choose a reason for hiding this comment

szdziedzic commented Jun 13, 2024 • edited Loading

kadikraman commented Jun 20, 2024

wschurman left a comment

Choose a reason for hiding this comment

wschurman left a comment

Choose a reason for hiding this comment

szdziedzic left a comment

Choose a reason for hiding this comment

github-actions bot commented Jun 27, 2024 • edited Loading

codecov bot commented Jun 27, 2024 • edited Loading

Codecov Report

linear bot commented Jun 28, 2024

kadikraman commented Jun 28, 2024

On EAS

With --local

github-actions bot commented Jun 28, 2024

szdziedzic left a comment

Choose a reason for hiding this comment

giorgiofellipe commented Jul 3, 2024 • edited Loading

wschurman commented Jul 3, 2024 • edited Loading

tfcornerstone commented Aug 7, 2024

wschurman commented Aug 7, 2024

kadikraman commented Jun 11, 2024 •

edited

Loading

szdziedzic commented Jun 13, 2024 •

edited

Loading

github-actions bot commented Jun 27, 2024 •

edited

Loading

codecov bot commented Jun 27, 2024 •

edited

Loading

With `--local`

giorgiofellipe commented Jul 3, 2024 •

edited

Loading

wschurman commented Jul 3, 2024 •

edited

Loading