Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: Extends the Android metrics diffMin similar to iOS #4159

Merged
merged 1 commit into from
Oct 10, 2024
Merged

chore: Extends the Android metrics diffMin similar to iOS #4159

merged 1 commit into from
Oct 10, 2024

Conversation

antonis
Copy link
Collaborator

@antonis antonis commented Oct 9, 2024 •
edited
Loading

📢 Type of change

  • Bugfix
  • New feature
  • Enhancement
  • Refactoring

📜 Description

Extends the Android metrics diffMin similar to iOS to increase failure tolerance. This should prevent most failures due to negative results in the check indicating that the app with Sentry starts faster that the app without the SDK. Note that the root cause of the issue is not covered by this PR.

💡 Motivation and Context

Partially #3413

💚 How did you test it?

CI

📝 Checklist

  • I reviewed submitted code
  • I added tests to verify changes
  • No new PII added or SDK only sends newly added PII if sendDefaultPII is enabled
  • All tests passing
  • No breaking changes

🔮 Next steps

#skip-changelog

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Oct 9, 2024

Android (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 498.78 ms 511.28 ms 12.50 ms
Size 17.73 MiB 20.11 MiB 2.37 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
27ef4ee 317.40 ms 321.70 ms 4.30 ms
22e31b6 396.48 ms 419.64 ms 23.16 ms
728164b 414.34 ms 449.22 ms 34.88 ms
d361d38 354.10 ms 381.69 ms 27.59 ms
e2b64fe 316.88 ms 330.23 ms 13.35 ms
2ec71da 438.14 ms 460.46 ms 22.32 ms
3853f43 329.68 ms 346.32 ms 16.64 ms
e5c9b8b 409.02 ms 426.66 ms 17.64 ms
34aba08 328.10 ms 342.84 ms 14.74 ms
1d86dd6 405.14 ms 411.06 ms 5.92 ms

App size

Revision Plain With Sentry Diff
27ef4ee 17.73 MiB 19.82 MiB 2.08 MiB
22e31b6 17.73 MiB 19.84 MiB 2.10 MiB
728164b 17.73 MiB 19.85 MiB 2.12 MiB
d361d38 17.73 MiB 19.81 MiB 2.08 MiB
e2b64fe 17.73 MiB 19.80 MiB 2.07 MiB
2ec71da 17.73 MiB 20.10 MiB 2.37 MiB
3853f43 17.73 MiB 19.81 MiB 2.08 MiB
e5c9b8b 17.73 MiB 19.83 MiB 2.10 MiB
34aba08 17.73 MiB 19.80 MiB 2.07 MiB
1d86dd6 17.73 MiB 19.86 MiB 2.12 MiB

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Oct 9, 2024

Android (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 427.31 ms 477.31 ms 50.00 ms
Size 7.15 MiB 8.38 MiB 1.23 MiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
575f9da+dirty 337.15 ms 370.47 ms 33.32 ms
70e6261+dirty 395.08 ms 408.12 ms 13.04 ms
c398f67+dirty 315.08 ms 345.60 ms 30.52 ms
d0bf494+dirty 253.73 ms 308.23 ms 54.49 ms
70caa60+dirty 308.83 ms 393.06 ms 84.23 ms
d43a46b+dirty 417.65 ms 472.98 ms 55.33 ms
c2a4e9b+dirty 392.94 ms 474.55 ms 81.61 ms
dadc233+dirty 363.19 ms 370.37 ms 7.18 ms
5bb8d5f+dirty 356.71 ms 389.65 ms 32.94 ms
5571a20+dirty 359.52 ms 389.80 ms 30.28 ms

App size

Revision Plain With Sentry Diff
575f9da+dirty 7.15 MiB 8.10 MiB 979.68 KiB
70e6261+dirty 7.15 MiB 8.21 MiB 1.07 MiB
c398f67+dirty 7.15 MiB 8.21 MiB 1.07 MiB
d0bf494+dirty 7.15 MiB 8.04 MiB 910.85 KiB
70caa60+dirty 7.15 MiB 8.03 MiB 901.79 KiB
d43a46b+dirty 7.15 MiB 8.34 MiB 1.19 MiB
c2a4e9b+dirty 7.15 MiB 8.34 MiB 1.19 MiB
dadc233+dirty 7.15 MiB 8.04 MiB 910.84 KiB
5bb8d5f+dirty 7.15 MiB 8.21 MiB 1.06 MiB
5571a20+dirty 7.15 MiB 8.20 MiB 1.05 MiB

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Oct 9, 2024

iOS (legacy) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1213.75 ms 1228.73 ms 14.98 ms
Size 2.36 MiB 3.14 MiB 793.32 KiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
575f9da+dirty 1266.22 ms 1274.84 ms 8.62 ms
1c65324+dirty 1235.17 ms 1235.08 ms -0.09 ms
baa882f+dirty 1218.00 ms 1227.04 ms 9.04 ms
457e29f+dirty 1253.94 ms 1269.18 ms 15.24 ms
148f924+dirty 1214.76 ms 1215.73 ms 0.97 ms
abb7058+dirty 1255.42 ms 1268.86 ms 13.44 ms
728164b+dirty 1256.10 ms 1259.08 ms 2.98 ms
27ef4ee+dirty 1293.52 ms 1296.08 ms 2.56 ms
700cbf4+dirty 1234.59 ms 1227.71 ms -6.88 ms
3ffcddd+dirty 1244.47 ms 1264.14 ms 19.67 ms

App size

Revision Plain With Sentry Diff
575f9da+dirty 2.36 MiB 2.87 MiB 520.20 KiB
1c65324+dirty 2.36 MiB 3.04 MiB 698.64 KiB
baa882f+dirty 2.36 MiB 3.08 MiB 731.91 KiB
457e29f+dirty 2.36 MiB 2.87 MiB 520.67 KiB
148f924+dirty 2.36 MiB 3.04 MiB 696.25 KiB
abb7058+dirty 2.36 MiB 2.87 MiB 520.42 KiB
728164b+dirty 2.36 MiB 2.88 MiB 530.38 KiB
27ef4ee+dirty 2.36 MiB 2.85 MiB 500.03 KiB
700cbf4+dirty 2.36 MiB 3.08 MiB 734.22 KiB
3ffcddd+dirty 2.36 MiB 2.84 MiB 489.60 KiB

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Oct 9, 2024

iOS (new) Performance metrics 🚀

  Plain With Sentry Diff
Startup time 1240.88 ms 1240.69 ms -0.18 ms
Size 2.92 MiB 3.69 MiB 794.16 KiB

Baseline results on branch: main

Startup times

Revision Plain With Sentry Diff
575f9da+dirty 1272.00 ms 1284.38 ms 12.38 ms
1c65324+dirty 1239.71 ms 1239.86 ms 0.15 ms
baa882f+dirty 1235.48 ms 1229.02 ms -6.46 ms
457e29f+dirty 1256.71 ms 1258.50 ms 1.79 ms
148f924+dirty 1220.72 ms 1221.30 ms 0.58 ms
abb7058+dirty 1260.28 ms 1266.56 ms 6.28 ms
728164b+dirty 1280.06 ms 1285.26 ms 5.20 ms
27ef4ee+dirty 1236.41 ms 1244.90 ms 8.49 ms
700cbf4+dirty 1233.96 ms 1228.27 ms -5.69 ms
3ffcddd+dirty 1272.22 ms 1273.98 ms 1.76 ms

App size

Revision Plain With Sentry Diff
575f9da+dirty 2.92 MiB 3.43 MiB 524.26 KiB
1c65324+dirty 2.92 MiB 3.61 MiB 705.56 KiB
baa882f+dirty 2.92 MiB 3.64 MiB 738.56 KiB
457e29f+dirty 2.92 MiB 3.43 MiB 524.75 KiB
148f924+dirty 2.92 MiB 3.60 MiB 701.88 KiB
abb7058+dirty 2.92 MiB 3.43 MiB 524.53 KiB
728164b+dirty 2.92 MiB 3.44 MiB 533.26 KiB
27ef4ee+dirty 2.92 MiB 3.41 MiB 503.72 KiB
700cbf4+dirty 2.92 MiB 3.64 MiB 740.57 KiB
3ffcddd+dirty 2.92 MiB 3.40 MiB 494.39 KiB

@antonis antonis marked this pull request as ready for review October 9, 2024 13:31
lucas-zimerman
lucas-zimerman approved these changes Oct 10, 2024
View reviewed changes
Copy link
Collaborator

@lucas-zimerman lucas-zimerman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@krystofwoldrich krystofwoldrich merged commit e540498 into main Oct 10, 2024
66 of 71 checks passed
@krystofwoldrich krystofwoldrich deleted the antonis/android-metrics-tollerance branch October 10, 2024 09:43
antonis added a commit that referenced this pull request Nov 19, 2024
@antonis
Reset diffMin workaround introduced with #4159
0a810f0
@antonis antonis mentioned this pull request Nov 19, 2024
Draft
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@krystofwoldrich krystofwoldrich krystofwoldrich approved these changes

@lucas-zimerman lucas-zimerman lucas-zimerman approved these changes

Assignees
No one assigned
Labels
Platform: Android
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@antonis @lucas-zimerman @krystofwoldrich