feat: add result upload #703

anhappdev · 2023-05-03T12:53:49Z

Part of #670

User can upload benchmark result to MLCommons cloud storage via the app.
The Share button now has 2 options:

Share with others: Share via phone file sharing feature.
Share with MLCommons: Upload result file to MLCommons cloud storage.

github-actions · 2023-05-03T12:54:02Z

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

com.android.builder.dexing.DexArchiveMergerException: Error while merging dex archives: The number of method references in a .dex file cannot exceed 64K.

anhappdev · 2023-05-03T15:42:45Z

flutter/android/app/build.gradle

@@ -43,7 +43,7 @@ android {

    defaultConfig {
        applicationId "org.mlcommons.android.mlperfbench"
-        minSdkVersion 16
+        minSdkVersion 21


I need to increase the minSdkVersion to 21 to fix a build error described here https://developer.android.com/build/multidex

Per this screenshot (from Android Studio) this should not affect much the number of Android phones which can run this app.

freedomtan · 2023-05-23T05:42:36Z

@anhappdev suggested we have to as Nathan to give read access to other guys who want to test this (let one who wants to test this be a user of that firebase project).

anhappdev · 2023-05-24T10:57:54Z

@freedomtan @AhmedTElthakeb @mohitmundhragithub
Please use this APK to test the upload. You should already have access to the Firebase Storage.
2023-05-24_mlperfbench-ec030a7-t.apk

freedomtan · 2023-05-25T08:41:22Z

@freedomtan @AhmedTElthakeb @mohitmundhragithub Please use this APK to test the upload. You should already have access to the Firebase Storage. 2023-05-24_mlperfbench-ec030a7-t.apk

Yes, I can see "mobile-app-build" project in the Firebase Console.

I can see data I uploaded in "Storage"
I cannot see them in "Firebase Database". I assume this is right for now.

anhappdev · 2023-05-25T08:45:18Z

I cannot see them in "Firebase Database". I assume this is right for now.

Yes. It's correct. The app uploads the result as a JSON file to Storage. We don't use "Firebase Database".

freedomtan · 2023-06-06T05:47:00Z

@anhappdev please add uploading feature to all the backends by building an apk with all the backends

freedomtan · 2023-06-20T07:39:37Z

@anhappdev as @AhmedTElthakeb reported, the app crashed for each task. The attached log was captured when running the mobilenet edgetpu. It crashed because of null pointer

06-20 15:34:08.106 20345 20345 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
06-20 15:34:08.106 20345 20345 F DEBUG   : Build fingerprint: 'samsung/g0sxeea/g0s:13/TP1A.220624.014/S906BXXU5CWEA:user/release-keys'
06-20 15:34:08.106 20345 20345 F DEBUG   : Revision: '28'
06-20 15:34:08.106 20345 20345 F DEBUG   : ABI: 'arm64'
06-20 15:34:08.106 20345 20345 F DEBUG   : Processor: '5'
06-20 15:34:08.106 20345 20345 F DEBUG   : Timestamp: 2023-06-20 15:34:07.900482968+0800
06-20 15:34:08.106 20345 20345 F DEBUG   : Process uptime: 23s
06-20 15:34:08.106 20345 20345 F DEBUG   : Cmdline: org.mlcommons.android.mlperfbench
06-20 15:34:08.106 20345 20345 F DEBUG   : pid: 20177, tid: 20297, name: DartWorker  >>> org.mlcommons.android.mlperfbench <<<
06-20 15:34:08.106 20345 20345 F DEBUG   : uid: 10382
06-20 15:34:08.106 20345 20345 F DEBUG   : tagged_addr_ctrl: 0000000000000001 (PR_TAGGED_ADDR_ENABLE)
06-20 15:34:08.106 20345 20345 F DEBUG   : pac_enabled_keys: 000000000000000f (PR_PAC_APIAKEY, PR_PAC_APIBKEY, PR_PAC_APDAKEY, PR_PAC_APDBKEY)
06-20 15:34:08.106 20345 20345 F DEBUG   : signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0000000000000040
06-20 15:34:08.106 20345 20345 F DEBUG   : Cause: null pointer dereference
06-20 15:34:08.106 20345 20345 F DEBUG   :     x0  000000798682777c  x1  0000000000000000  x2  0000000000000001  x3  000000000000005b
06-20 15:34:08.106 20345 20345 F DEBUG   :     x4  0000000000000021  x5  0000000000027eda  x6  636e6c5e67626073  x7  7f7f7f7f7f7f7f7f
06-20 15:34:08.106 20345 20345 F DEBUG   :     x8  0000000000000000  x9  346f474788b54376  x10 00000000000003e8  x11 346f474788b54376
06-20 15:34:08.106 20345 20345 F DEBUG   :     x12 0000007986c00fe0  x13 0000000000000052  x14 0000000000000000  x15 0000000215f208ce
06-20 15:34:08.106 20345 20345 F DEBUG   :     x16 0000007cb2d6a430  x17 0000007cb2d47f60  x18 0000007985ef6000  x19 b400007b05194090
06-20 15:34:08.106 20345 20345 F DEBUG   :     x20 0000007986c03a68  x21 0000007986c03a70  x22 0000007986c03940  x23 0000007986c03a70
06-20 15:34:08.106 20345 20345 F DEBUG   :     x24 b400007b05194210  x25 0000007986c03970  x26 b400007a351993d0  x27 00000000000000b0
06-20 15:34:08.106 20345 20345 F DEBUG   :     x28 0000007986c06000  x29 0000007986c03930
06-20 15:34:08.106 20345 20345 F DEBUG   :     lr  0000007904319adc  sp  0000007986c02890  pc  0000007986827784  pst 0000000060001000
06-20 15:34:08.106 20345 20345 F DEBUG   : backtrace:
06-20 15:34:08.106 20345 20345 F DEBUG   :       #00 pc 0000000000011784  /data/app/~~ecO03F6aRzgmubX9396QEA==/org.mlcommons.android.mlperfbench-9UL9TVeXWv52p95Wkotvcw==/lib/arm64/libmbe2200_core.so (backend_get_input_count+8) (BuildId: ea7836a581c6d255620c5edf16b00c1736169338)
...
...

log:
foobar.txt

anhappdev · 2023-06-20T08:47:08Z

@anhappdev as @AhmedTElthakeb reported, the app crashed for each task. The attached log was captured when running the mobilenet edgetpu. It crashed because of null pointer

It looks like the error is somewhere in the mobile_back_samsung code. @AhmedTElthakeb Can you please check if the checksum of libmbe2200_core.so is up-to-date?

Anyway, I don't think the error is related to this pull request, maybe it's already there for some time. We may need to fix it in another pull request.
@freedomtan @AhmedTElthakeb Can you please test this APK if we still have that crash?
https://github.com/mlcommons/mobile_app_open/actions/runs/5319375937?pr=735

AhmedTElthakeb · 2023-06-21T14:37:35Z

Anyway, I don't think the error is related to this pull request, maybe it's already there for some time. We may need to fix it in another pull request. @freedomtan @AhmedTElthakeb Can you please test this APK if we still have that crash? https://github.com/mlcommons/mobile_app_open/actions/runs/5319375937?pr=735

@anhappdev
Yes this one crashes same way.

It could be due to some incompatible recent commits to the master branch we have not accounted for yet. We are preparing a pull request to catch up with master.

In the meantime, is it possible to try this "add result upload" feature on top of v3.0 branch instead of master?

anhappdev · 2023-06-21T15:19:32Z

@anhappdev Yes this one crashes same way.

It could be due to some incompatible recent commits to the master branch we have not accounted for yet. We are preparing a pull request to catch up with master.

In the meantime, is it possible to try this "add result upload" feature on top of v3.0 branch instead of master?

The v3.0 branch now differs greatly from the master branch, so it would require significant effort to make this PR work on the v3.0 branch.

Is it critical to test this PR with the Samsung backend? I mean that the upload feature works independently of any backends.

mohitmundhragithub · 2023-06-26T10:43:16Z

Here is a new APK with all backends: 2023-06-06_mlperfbench-2e482f8-qsmgt.apk

this app shows just white screen on qualcomm device.

@anhappdev as @AhmedTElthakeb reported, the app crashed for each task. The attached log was captured when running the mobilenet edgetpu. It crashed because of null pointer

It looks like the error is somewhere in the mobile_back_samsung code. @AhmedTElthakeb Can you please check if the checksum of libmbe2200_core.so is up-to-date?

Anyway, I don't think the error is related to this pull request, maybe it's already there for some time. We may need to fix it in another pull request. @freedomtan @AhmedTElthakeb Can you please test this APK if we still have that crash? https://github.com/mlcommons/mobile_app_open/actions/runs/5319375937?pr=735

this app doesn't show option to share results to mlcommon's firebase.

mohitmundhragithub · 2023-06-26T10:46:23Z

Here is a new APK with all backends: 2023-06-06_mlperfbench-2e482f8-qsmgt.apk

I tested this APK on couple devices, and the app crashes on both devices.

06-24 18:58:38.963 2787 3363 D CoreBackPreview: Window{8dd7795 u0 Splash Screen org.mlcommons.android.mlperfbench EXITING}: Setting back callback null
06-24 18:58:38.963 2787 3365 W InputManager-JNI: Input channel object '8dd7795 Splash Screen org.mlcommons.android.mlperfbench (client)' was disposed without first being removed with the input manager!

I see these errors in logcat logs.

anhappdev · 2023-06-26T11:39:22Z

this app doesn't show option to share results to mlcommon's firebase.

Yes, because it's based on the master branch, to test if the error with Samsung backend was introduced with this PR or it's already in the master for some time.

this app shows just white screen on qualcomm device.

That is interesting. I just merged the master branch into this PR to have the CI automatically build the APK. Can you test again with this new build and then upload a full log (as .txt file) here for debugging?
The CI build can be downloaded by clicking on "Details" of App Build / build-android-apk (pull_request) then "Summary".

freedomtan · 2023-06-27T03:09:23Z

Here is a new APK with all backends: 2023-06-06_mlperfbench-2e482f8-qsmgt.apk

this app shows just white screen on qualcomm device.

@anhappdev as @AhmedTElthakeb reported, the app crashed for each task. The attached log was captured when running the mobilenet edgetpu. It crashed because of null pointer

It looks like the error is somewhere in the mobile_back_samsung code. @AhmedTElthakeb Can you please check if the checksum of libmbe2200_core.so is up-to-date?
Anyway, I don't think the error is related to this pull request, maybe it's already there for some time. We may need to fix it in another pull request. @freedomtan @AhmedTElthakeb Can you please test this APK if we still have that crash? https://github.com/mlcommons/mobile_app_open/actions/runs/5319375937?pr=735

this app doesn't show option to share results to mlcommon's firebase.

FYR. This 2023-06-06_mlperfbench-2e482f8-qsmgt.apk works for a colleague's Xiaomi 13 Pro (a 8 Gen 2 device). Uploading also works.

mohitmundhragithub · 2023-06-27T04:59:10Z

this app doesn't show option to share results to mlcommon's firebase.

Yes, because it's based on the master branch, to test if the error with Samsung backend was introduced with this PR or it's already in the master for some time.

this app shows just white screen on qualcomm device.

That is interesting. I just merged the master branch into this PR to have the CI automatically build the APK. Can you test again with this new build and then upload a full log (as .txt file) here for debugging? The CI build can be downloaded by clicking on "Details" of App Build / build-android-apk (pull_request) then "Summary".

thanks. this apk works. But unable to upload. Basically, nothing happens on clicking "share with mlcommons"

anhappdev · 2023-06-27T05:03:19Z

thanks. this apk works. But unable to upload. Basically, nothing happens on clicking "share with mlcommons"

I fixed this recently so you will need the APK with build number > 49.

anhappdev · 2023-06-29T07:07:37Z

@mohitmundhragithub Did you have a chance to test again with a newer build?

mohitmundhragithub · 2023-07-03T09:49:51Z

@mohitmundhragithub Did you have a chance to test again with a newer build?

I tried with build "android-apks-50" again. But seeing white screen with this.

07-01 17:58:44.532 2787 10195 D CoreBackPreview: Window{4085017 u0 Splash Screen org.mlcommons.android.mlperfbench EXITING}: Setting back callback null
07-01 17:58:44.532 2787 12730 W InputManager-JNI: Input channel object '4085017 Splash Screen org.mlcommons.android.mlperfbench (client)' was disposed without first being removed with the input manager!

This seems to be the culprit log.

mohitmundhragithub · 2023-07-03T09:51:30Z

Update: after waiting for sometime, I see this error:

Wanted to check, is firebase not available from china machines?

anhappdev · 2023-07-03T10:14:29Z

It looks like a connection issue. A quick Google search shows indeed that Firebase does not work in China.

mohitmundhragithub · 2023-07-03T10:29:14Z

It looks like a connection issue. A quick Google search shows indeed that Firebase does not work in China.

actually i tried with a device in india as well, but it didn't work :(
Will try with some more devices and update

anhappdev · 2023-07-03T10:42:51Z

actually i tried with a device in india as well, but it didn't work :(

Is the device behind a proxy or has limited internet connection?

mohitmundhragithub · 2023-07-03T11:00:34Z

actually i tried with a device in india as well, but it didn't work :(

Is the device behind a proxy or has limited internet connection?

Its connected to office wifi open for all personal devices... so no proxy.
I will check in my home network on some other device later in the day and update.

anhappdev · 2023-07-05T13:36:48Z

Its connected to office wifi open for all personal devices... so no proxy. I will check in my home network on some other device later in the day and update.

@mohitmundhragithub Any update on this?

mohitmundhragithub · 2023-07-06T13:35:59Z

I just tried out on a personal device, and it's working fine.
I am able to see the results on firebasestorage as well.

Just one comment that having the results to be sorted or having some meaningful name for the folders may help in searching the results on the portal.

anhappdev · 2023-07-07T04:43:52Z

Just one comment that having the results to be sorted or having some meaningful name for the folders may help in searching the results on the portal.

Thank you for your feedback, but the Firebase website is not meant to be used as a frontend. Searching or viewing the results will be done in the app.

sonarqubecloud · 2023-07-07T05:17:46Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

anhappdev added 8 commits May 3, 2023 10:18

Add Firebase SDK

58b705e

Add docs to configure Firebase

6b98496

Add FirebaseService to handle Firebase related tasks

21ccd85

Add Flutter's multidex support

16e278e

Add CI user

d2b784a

Set env for android-build.yaml

9371491

Refactor Firebase services

72cd13c

Add ShareButton with 2 share options

7ee4c50

anhappdev added 6 commits May 3, 2023 20:14

Fix dead link in env-setup-ios.md

3520378

Exclude java from sonar scanner

c7dee3d

Fix issue with multidex

97d775d

com.android.builder.dexing.DexArchiveMergerException: Error while merging dex archives: The number of method references in a .dex file cannot exceed 64K.

Fix UnsupportedError on Windows

1af3f03

Upload result after finishing integration test

d291bdc

Rename vars in first_test.dart

588bf67

anhappdev commented May 3, 2023

View reviewed changes

anhappdev added 2 commits May 4, 2023 00:24

Use file template instead of dart-define

b61e05f

Update UI for ShareButton

5c8cd6f

anhappdev marked this pull request as ready for review May 4, 2023 01:33

anhappdev requested review from mohitmundhragithub, freedomtan and AhmedTElthakeb May 4, 2023 01:34

Merge origin/master

ec030a7

anhappdev requested a review from a team as a code owner May 24, 2023 10:54

Merge remote-tracking branch 'origin/master' into 670/add-result-upload

2e482f8

anhappdev marked this pull request as ready for review June 20, 2023 08:56

Merge remote-tracking branch 'origin/master' into 670/add-result-upload

ff5d63c

anhappdev added 2 commits June 27, 2023 07:10

Add Firebase env vars

690beb1

Fix issue with new released python version

dea0452

mohitmundhragithub approved these changes Jul 6, 2023

View reviewed changes

Merge branch 'master' into 670/add-result-upload

612361b

anhappdev merged commit cd2af4b into master Jul 7, 2023

anhappdev deleted the 670/add-result-upload branch July 7, 2023 06:35

github-actions bot locked and limited conversation to collaborators Jul 7, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add result upload #703

feat: add result upload #703

anhappdev commented May 3, 2023 •

edited

Loading

github-actions bot commented May 3, 2023 •

edited

Loading

anhappdev May 3, 2023

freedomtan commented May 23, 2023

anhappdev commented May 24, 2023

freedomtan commented May 25, 2023

anhappdev commented May 25, 2023

freedomtan commented Jun 6, 2023 •

edited

Loading

freedomtan commented Jun 20, 2023

anhappdev commented Jun 20, 2023

AhmedTElthakeb commented Jun 21, 2023 •

edited

Loading

anhappdev commented Jun 21, 2023

mohitmundhragithub commented Jun 26, 2023

mohitmundhragithub commented Jun 26, 2023

anhappdev commented Jun 26, 2023 •

edited

Loading

freedomtan commented Jun 27, 2023

mohitmundhragithub commented Jun 27, 2023 •

edited

Loading

anhappdev commented Jun 27, 2023

anhappdev commented Jun 29, 2023

mohitmundhragithub commented Jul 3, 2023 •

edited

Loading

mohitmundhragithub commented Jul 3, 2023 •

edited

Loading

anhappdev commented Jul 3, 2023

mohitmundhragithub commented Jul 3, 2023 •

edited

Loading

anhappdev commented Jul 3, 2023

mohitmundhragithub commented Jul 3, 2023 •

edited

Loading

anhappdev commented Jul 5, 2023

mohitmundhragithub commented Jul 6, 2023

anhappdev commented Jul 7, 2023

sonarqubecloud bot commented Jul 7, 2023

feat: add result upload #703

feat: add result upload #703

Conversation

anhappdev commented May 3, 2023 • edited Loading

github-actions bot commented May 3, 2023 • edited Loading

anhappdev May 3, 2023

Choose a reason for hiding this comment

freedomtan commented May 23, 2023

anhappdev commented May 24, 2023

freedomtan commented May 25, 2023

anhappdev commented May 25, 2023

freedomtan commented Jun 6, 2023 • edited Loading

freedomtan commented Jun 20, 2023

anhappdev commented Jun 20, 2023

AhmedTElthakeb commented Jun 21, 2023 • edited Loading

anhappdev commented Jun 21, 2023

mohitmundhragithub commented Jun 26, 2023

mohitmundhragithub commented Jun 26, 2023

anhappdev commented Jun 26, 2023 • edited Loading

freedomtan commented Jun 27, 2023

mohitmundhragithub commented Jun 27, 2023 • edited Loading

anhappdev commented Jun 27, 2023

anhappdev commented Jun 29, 2023

mohitmundhragithub commented Jul 3, 2023 • edited Loading

mohitmundhragithub commented Jul 3, 2023 • edited Loading

anhappdev commented Jul 3, 2023

mohitmundhragithub commented Jul 3, 2023 • edited Loading

anhappdev commented Jul 3, 2023

mohitmundhragithub commented Jul 3, 2023 • edited Loading

anhappdev commented Jul 5, 2023

mohitmundhragithub commented Jul 6, 2023

anhappdev commented Jul 7, 2023

sonarqubecloud bot commented Jul 7, 2023

anhappdev commented May 3, 2023 •

edited

Loading

github-actions bot commented May 3, 2023 •

edited

Loading

freedomtan commented Jun 6, 2023 •

edited

Loading

AhmedTElthakeb commented Jun 21, 2023 •

edited

Loading

anhappdev commented Jun 26, 2023 •

edited

Loading

mohitmundhragithub commented Jun 27, 2023 •

edited

Loading

mohitmundhragithub commented Jul 3, 2023 •

edited

Loading

mohitmundhragithub commented Jul 3, 2023 •

edited

Loading

mohitmundhragithub commented Jul 3, 2023 •

edited

Loading

mohitmundhragithub commented Jul 3, 2023 •

edited

Loading