Emulator crashing?

[lz1asl]

Hello there!
First of all, congratulations on the work done on this great project.
I am facing the following problem:

• Trying to run Android connected tests with the following configuration https://github.com/lz1asl/CaveSurvey/blob/fixes/.github/workflows/android.yml

- name: Connected tests uses: reactivecircus/android-emulator-runner@v2 with: api-level: 29 script: ./gradlew connectedAndroidTest

• They fail (for example https://github.com/lz1asl/CaveSurvey/runs/739356848?check_suite_focus=true) with

Task :connectedDefaultFlavorDebugAndroidTest Starting 12 tests on test(AVD) - 10 com.astoev.cave.survey.test.service.data.ExcelExportTest > excelExportTest[test(AVD) - 10] FAILED Test failed to run to completion. Reason: 'Test run failed to complete. Expected 12 tests, received 6'. Check device logcat for details Tests on test(AVD) - 10 failed: Test run failed to complete. Expected 12 tests, received 6. onError: commandError=false message=INSTRUMENTATION_ABORTED: System has crashed. > Task :connectedDefaultFlavorDebugAndroidTest FAILED

• As mentioned in other issues added logcat output https://github.com/lz1asl/CaveSurvey/runs/751427443?check_suite_focus=true and I see the crash

--------- beginning of crash 06-08 20:23:57.017 1590 1590 F libc : Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 1590 (surfaceflinger), pid 1590 (surfaceflinger)

but am still not sure what is causing it.
Do you think you can have a look? Same tests work fine locally.

[ychescale9]

I haven't read through all the logs and your test code, but let's try to narrow down the issue a little:

• Have you tried with a lower API level e.g. API 23?
• Does it always crash or is it intermittent?
• Do the tests / code under test interact with Android hardware e.g. Camera?
• (If you're comfortable with the SDK command-line tools) would you be able to create a new AVD, start the emulator and run the tests from the cli locally with the API 29 system image?
• there are 12 tests in ExcelExportTest, would you be able to find 1 test that causes a crash and comment out the rest?

[lz1asl]

Hello ychescale9,

on API 29 the tests always crash. My application do requests camera permission but I don't use camera in the tests yet. I tested with API 23 and now the emulator didn't crash so each of the tests failed separately :) I will have to play with the emulator options and will come back to you.
Thank you

[ashutoshgngwr]

@lz1asl @ychescale9 I am facing a similar issue although it seems to be unrelated to this action. In my case, the tests execute fine on my local Linux machine but the Instrumentation process crashes in Github Runner (MacOS) for some reason. I also ran a few tests from android/testing-samples and these run fine with this action. Unfortunately I am not able to reproduce the problem locally since I don't own a Mac. My best guess is that problem is originating from upstream, potentially due to a recent emulator update or something.

[ychescale9]

@ashutoshgngwr you could try to pin the emulator binary version to a specific version by specifying the emulator-build. Last year they promised to provide a way to specify the emulator binary version used from the sdkmanager but it's still unavailable in the latest cmdline-tools.

Have you tried running on lower API-levels? Usually if it works on certain API levels it's easier to debug the issue. If it works on macos but not linux (or vice-versa), it's unlikely there's anything we can do.

[ashutoshgngwr]

@ychescale9 I am using Mockk which forces me to run my tests on Android 29 or higher. Besides, I did run the tests on macos-10.14 and there test fail normally without crashing the instrumentation process.

Let me see if I can get it working by pinning the emulator version to last successful build.

PS, I just realised that I confused Android Pie with 29. Mockk can work on Pie. Let me also check if that works.

[ashutoshgngwr]

And its gone. The error is gone. I did nothing except adding 28 to my build matrix and the error is gone. I have been at this for 3-4 hours now. 😐

[ashutoshgngwr]

Update: From my experience, I think instrumentation process is crashing with Google API emulator images. Prior to this I was running tests on the default images with x86 arch and everything was working smoothly. Due to recent changes in my project, I had to use Google API images with x86_64 arch and since then, I am occasionally seeing this issue along with Out of Memory errors.

[ychescale9]

Yeah I have similar experience with google API and Play store images 😞 . What's the lowest API level you can test with? I think lower API levels are more stable in general.

[ashutoshgngwr]

@ychescale9 I am testing with 28 and 29. Seems to work fine but sometimes instrumentation process crashes. And this time I can reproduce these random crashes on my local Linux machine.

[ychescale9]

Might be worth filing a bug on the issue tracker.

[ashutoshgngwr]

I am not so sure about that. Their standard solution for the issue is to use Android Test Orchestrator. Test orchestrator is capable of running each test in its own instrumentation process so if one of the test crashes, rest of the suit can still run but it makes everything very very slow. It also has a 'clearPackageData' option that removes the app's state from RAM and CPU before each test. This can be used to mitigate OOM kills in tests but it is not capable of generating coverage reports since that data is also cleared.

[ychescale9]

I don’t think the orchestrator is a solution to the crash caused by the system image. It just makes it easier to identify which test is causing a crash. In this case if the crash you’re getting is completely random and there’s nothing you can do in your code to fix it, you probably want to file a bug.

[ashutoshgngwr]

A bug is already filed. It's not on the Google's issue tracker though.

Edit: Apparently there's another report in the samples repo. Still no solutions though.

I think it's safe to close this issue here. When it was filed, the instrumentation process was crashing with consistency so that might have been a bug in the emulator that got resolved.

[ychescale9]

Firing the issue on the issuetracker will have a much better chance getting to the Emulator team who have been very responsive and helpful in my personal experience (sometimes they will even send you an unreleased emulator binary for you to try). If it's a bug in the emulator binary or system images, those GitHub repos are completely irrelevant and the best they can do is firing the issue for you.

Closing now as no action can be taken on our side.

[ashutoshgngwr]

Filed an issue after an in-depth analysis of the problem.

https://issuetracker.google.com/issues/159732638

[vivekweb2013]

In my case the emulator(api-29) was crashing while executing line containing adapter.clear() from espresso test

Removing such code temporarily resolved the crashing issue.

I think there is some issue while clearing the adapter (or the list used in adapter). Its not because of any exception. This works well in api 28 and below emulator but only crashing with api 29 emulator

[sidharthadas-tudip]

Hi All getting the same issue
Expected tests 80 and received 45