Flaky tests on `macos-14` GH runner (M1, iOS) #4415

kirillzyusko · 2024-03-20T11:14:24Z

What happened?

I've been running tests on macos-12 (Intel) and everything was working well.

After switching to macos-14 I see that test fails, because:

endless loop of messages like:

10:32:11.110 detox[18599] i ws-client:APP_STATUS The app is busy with the following tasks:
• Run loop "Main Run Loop" is awake.
• There are 1 work items pending on the dispatch queue: "Main Queue (<OS_dispatch_queue_main: com.apple.main-thread>)".
• The event "Gesture Recognizer" is taking place with object: "<UITapAndAHalfRecognizer: 0x1380dec70 (UITextInteractionNameTapAndAHalf); state = Possible; view = <RCTUITextView: 0x137914e00>; target= <(action=tapAndAHalf:, target=<UITextSelectionInteraction 0x600002a53aa0>)>>".

or here

11:52:59.776 detox[30968] i ws-client:APP_STATUS The app is busy with the following tasks:
• There are 1 work items pending on the dispatch queue: "Main Queue (<OS_dispatch_queue_main: com.apple.main-thread>)".
• Run loop "Main Run Loop" is awake.
• The event "Runloop Perform Block" is taking place with object: "Main Run Loop".

thrown: "Exceeded timeout of 240000 ms for a hook. (https://github.com/kirillzyusko/react-native-keyboard-controller/actions/runs/8355963767/job/22873098946)

I also debugged video output and it looks like simulator freezes - and I'm not sure what exactly causes this freeze (but cursor is not blinking in TextInputs and before a freeze the simulator is very laggy).

The issue is not reproducible on my local machine.

What was the expected behaviour?

Tests should be running without problems.

Was it tested on latest Detox?

I have tested this issue on the latest Detox release and it still reproduces.

Did your test throw out a timeout?

I have followed the instructions under Identifying which synchronization mechanism causes us to wait too much.

Help us reproduce this issue!

I think you can reproduce it via fork of https://github.com/kirillzyusko/react-native-keyboard-controller

And then run https://github.com/kirillzyusko/react-native-keyboard-controller/blob/main/.github/workflows/ios-e2e-test.yml this action (from my observation successful rate is 50/50).

In what environment did this happen?

Detox version: 20.19.3
React Native version: 0.73.5
Has Fabric (React Native's new rendering system) enabled: (yes/no) no
Node version: 18.x
Device model: iPhone 15 Pro (simulator)
iOS version: iOS 16.4, iOS 17.2
macOS version: 14
Xcode version: 14.3, 15.0, 15.2
Test-runner (select one): jest

Detox logs

yarn run v1.22.19
$ detox test --configuration example.ios.sim.release --loglevel verbose --take-screenshots failing --record-videos failing
09:25:34.823 detox[33493] i ws-server Detox server listening on localhost:49245...
09:25:34.835 detox[33493] B lifecycle jest --config kit/jest.config.js
09:25:35.373 detox[33493] B ws-server connection :49245<->:49247
09:25:35.554 detox[33493] i child-process:EXEC_CMD applesimutils --list --byType "iPhone 14 Pro"
09:25:36.528 detox[33493] i device settled on 6C3E50C2-B609-47A0-8988-3CC825FB8AB1
  data: {
    "id": "6C3E50C2-B609-47A0-8988-3CC825FB8AB1",
    "udid": "6C3E50C2-B609-47A0-8988-3CC825FB8AB1"
  }
09:25:36.528 detox[33493] i child-process:EXEC_CMD applesimutils --list --byId 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 --maxResults 1
09:25:36.742 detox[33493] i child-process:EXEC_CMD /usr/bin/xcrun simctl boot 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 
09:25:36.742 detox[33493] i child-process:EXEC_TRY Booting device 6C3E50C2-B609-47A0-8988-3CC825FB8AB1...
09:25:38.[12](https://github.com/kirillzyusko/react-native-keyboard-controller/actions/runs/8356749445/job/22874463631?pr=392#step:18:13)7 detox[33493] i child-process:EXEC_CMD /usr/bin/xcrun simctl bootstatus 6C3E50C2-B609-47A0-8988-3CC825FB8AB1
09:28:10.500 detox[33493] i child-process:EXEC_CMD open -a Simulator --args -CurrentDeviceUDID 6C3E50C2-B609-47A0-8988-3CC825FB8AB1
09:28:[13](https://github.com/kirillzyusko/react-native-keyboard-controller/actions/runs/8356749445/job/22874463631?pr=392#step:18:14).821 detox[33498] i child-process:EXEC_CMD /usr/bin/xcrun simctl uninstall 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller
09:28:13.836 detox[33498] i child-process:EXEC_TRY Uninstalling com.example.reactnativekeyboardcontroller...
09:32:04.002 detox[33498] i child-process:EXEC_SUCCESS com.example.reactnativekeyboardcontroller uninstalled
09:32:04.260 detox[33498] i child-process:EXEC_CMD /usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller
09:32:04.261 detox[33498] i child-process:EXEC_TRY Terminating com.example.reactnativekeyboardcontroller...
09:32:28.182 detox[33498] i child-process:EXEC_TRY Terminating com.example.reactnativekeyboardcontroller...
09:32:34.013 detox[33498] i child-process:EXEC_FAIL "/usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller" failed with error = ChildProcessError: Command failed: /usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller
An error was encountered processing the command (domain=NSPOSIXErrorDomain, code=3):
Simulator device returned an error for the requested operation.
found nothing to terminate
Underlying error (domain=NSPOSIXErrorDomain, code=3):
	The request to terminate "com.example.reactnativekeyboardcontroller" failed. found nothing to terminate
	found nothing to terminate
 `/usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller` (exited with error code 3) (code=3), stdout and stderr:

09:32:34.0[14](https://github.com/kirillzyusko/react-native-keyboard-controller/actions/runs/8356749445/job/22874463631?pr=392#step:18:15) detox[33498] i child-process:EXEC_FAIL 
09:32:34.028 detox[33498] i child-process:EXEC_FAIL An error was encountered processing the command (domain=NSPOSIXErrorDomain, code=3):
Simulator device returned an error for the requested operation.
found nothing to terminate
Underlying error (domain=NSPOSIXErrorDomain, code=3):
	The request to terminate "com.example.reactnativekeyboardcontroller" failed. found nothing to terminate
	found nothing to terminate

09:32:34.034 detox[33498] i child-process:EXEC_CMD /usr/bin/xcrun simctl install 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 "/Users/runner/work/react-native-keyboard-controller/react-native-keyboard-controller/example/ios/build/Build/Products/Release-iphonesimulator/KeyboardControllerExample.app"
09:32:34.035 detox[33498] i child-process:EXEC_TRY Installing /Users/runner/work/react-native-keyboard-controller/react-native-keyboard-controller/example/ios/build/Build/Products/Release-iphonesimulator/KeyboardControllerExample.app...
09:33:09.494 detox[33498] i child-process:EXEC_SUCCESS /Users/runner/work/react-native-keyboard-controller/react-native-keyboard-controller/example/ios/build/Build/Products/Release-iphonesimulator/KeyboardControllerExample.app installed
09:33:09.569 detox[33498] i child-process:EXEC_CMD /usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller
09:33:09.570 detox[33498] i child-process:EXEC_TRY Terminating com.example.reactnativekeyboardcontroller...
09:33:25.736 detox[33498] i child-process:EXEC_TRY Terminating com.example.reactnativekeyboardcontroller...
09:33:26.280 detox[33498] i child-process:EXEC_FAIL "/usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller" failed with error = ChildProcessError: Command failed: /usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller
An error was encountered processing the command (domain=NSPOSIXErrorDomain, code=3):
Simulator device returned an error for the requested operation.
found nothing to terminate
Underlying error (domain=NSPOSIXErrorDomain, code=3):
	The request to terminate "com.example.reactnativekeyboardcontroller" failed. found nothing to terminate
	found nothing to terminate
 `/usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller` (exited with error code 3) (code=3), stdout and stderr:

09:33:26.280 detox[33498] i child-process:EXEC_FAIL 
09:33:26.281 detox[33498] i child-process:EXEC_FAIL An error was encountered processing the command (domain=NSPOSIXErrorDomain, code=3):
Simulator device returned an error for the requested operation.
found nothing to terminate
Underlying error (domain=NSPOSIXErrorDomain, code=3):
	The request to terminate "com.example.reactnativekeyboardcontroller" failed. found nothing to terminate
	found nothing to terminate

09:33:33.985 detox[33498] i lifecycle 005-keyboard-toolbar.e2e.ts is assigned to 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 (iPhone 14 Pro)
09:33:33.993 detox[33498] i child-process:EXEC_CMD /usr/bin/xcrun simctl status_bar 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 override --time "12:00" --dataNetwork "wifi" --wifiMode "active" --wifiBars "3" --cellularMode "active" --cellularBars "4" --batteryLevel "100"
09:33:52.323 detox[33498] i child-process:EXEC_CMD /usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller
09:33:52.323 detox[33498] i child-process:EXEC_TRY Terminating com.example.reactnativekeyboardcontroller...
09:34:06.336 detox[33498] i child-process:EXEC_TRY Terminating com.example.reactnativekeyboardcontroller...
09:34:07.040 detox[33498] i child-process:EXEC_FAIL "/usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller" failed with error = ChildProcessError: Command failed: /usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller
An error was encountered processing the command (domain=NSPOSIXErrorDomain, code=3):
Simulator device returned an error for the requested operation.
found nothing to terminate
Underlying error (domain=NSPOSIXErrorDomain, code=3):
	The request to terminate "com.example.reactnativekeyboardcontroller" failed. found nothing to terminate
	found nothing to terminate
 `/usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller` (exited with error code 3) (code=3), stdout and stderr:

09:34:07.040 detox[33498] i child-process:EXEC_FAIL 
09:34:07.040 detox[33498] i child-process:EXEC_FAIL An error was encountered processing the command (domain=NSPOSIXErrorDomain, code=3):
Simulator device returned an error for the requested operation.
found nothing to terminate
Underlying error (domain=NSPOSIXErrorDomain, code=3):
	The request to terminate "com.example.reactnativekeyboardcontroller" failed. found nothing to terminate
	found nothing to terminate

09:34:07.044 detox[33498] i child-process:EXEC_CMD SIMCTL_CHILD_GULGeneratedClassDisposeDisabled=YES SIMCTL_CHILD_DYLD_INSERT_LIBRARIES="/Users/runner/Library/Detox/ios/99070475dd1fc273e5d617c755416087077e322d/Detox.framework/Detox" /usr/bin/xcrun simctl launch 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller --args -detoxServer ws://localhost:49245 -detoxSessionId 8af1db61-77d6-3c4f-d223-9cfb905afdeb -detoxDebugVisibility YES -detoxDisableHierarchyDump YES
09:34:07.045 detox[33498] i child-process:EXEC_TRY Launching com.example.reactnativekeyboardcontroller...
09:37:45.998 detox[33493] B ws-server connection :49245<->:56040
09:37:46.121 detox[33493] i ws-server The app has dispatched "ready" action too early.
09:37:46.125 detox[33498] i child-process:EXEC_CMD /usr/bin/xcrun simctl io 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 screenshot "/private/var/folders/c3/yyzr8vp96lg9ycnj632jvpnw0000gn/T/28464a83-c2ec-4a11-a241-92d9e5cb5495.detox.png"
09:37:46.392 detox[33498] i child-process:EXEC_CMD /usr/bin/xcrun simctl get_app_container 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller
09:38:48.074 detox[33498] i lifecycle `KeyboardToolbar` specification: should navigate to `Toolbar` screen
09:38:48.096 detox[33498] i lifecycle `KeyboardToolbar` specification: should navigate to `Toolbar` screen [FAIL]
09:38:48.227 detox[33498] i lifecycle `KeyboardToolbar` specification: should have expected UI when first input is focused
09:38:48.228 detox[33498] i lifecycle `KeyboardToolbar` specification: should have expected UI when first input is focused [FAIL]
09:38:48.230 detox[33498] i lifecycle `KeyboardToolbar` specification: should show bottom sheet when `AutoFill Contacts` is pressed
09:38:48.230 detox[33498] i lifecycle `KeyboardToolbar` specification: should show bottom sheet when `AutoFill Contacts` is pressed [FAIL]
09:38:48.232 detox[33498] i lifecycle `KeyboardToolbar` specification: should set focus back when modal closed
09:38:48.234 detox[33498] i lifecycle `KeyboardToolbar` specification: should set focus back when modal closed [FAIL]
09:38:48.235 detox[33498] i lifecycle `KeyboardToolbar` specification: should do correct actions when contact gets selected
09:38:48.238 detox[33498] i lifecycle `KeyboardToolbar` specification: should do correct actions when contact gets selected [FAIL]
09:38:48.239 detox[33498] i lifecycle `KeyboardToolbar` specification: should skip a disabled fields
09:38:48.242 detox[33498] i lifecycle `KeyboardToolbar` specification: should skip a disabled fields [FAIL]
09:38:48.242 detox[33498] i lifecycle `KeyboardToolbar` specification: should handle multiple clicks in row
09:38:48.242 detox[33498] i lifecycle `KeyboardToolbar` specification: should handle multiple clicks in row [FAIL]
09:38:48.243 detox[33498] i lifecycle `KeyboardToolbar` specification: should handle `previous` clicks correctly
09:38:48.243 detox[33498] i lifecycle `KeyboardToolbar` specification: should handle `previous` clicks correctly [FAIL]
09:38:48.243 detox[33498] i lifecycle `KeyboardToolbar` specification: should have expected UI state when end of form reached
09:38:48.244 detox[33498] i lifecycle `KeyboardToolbar` specification: should have expected UI state when end of form reached [FAIL]
09:38:48.244 detox[33498] i lifecycle `KeyboardToolbar` specification: should enable next button when go to previous field from the last one
09:38:48.244 detox[33498] i lifecycle `KeyboardToolbar` specification: should enable next button when go to previous field from the last one [FAIL]
09:38:48.245 detox[33498] i lifecycle `KeyboardToolbar` specification: should close keyboard when press `Done`
09:38:48.248 detox[33498] i lifecycle `KeyboardToolbar` specification: should close keyboard when press `Done` [FAIL]

09:38:48.304 detox[33498] i artifact:MOVE_FILE moving "/private/var/folders/c3/yyzr8vp96lg9ycnj632jvpnw0000gn/T/28464a83-c2ec-4a11-a241-92d9e5cb5495.detox.png" to artifacts/example.ios.sim.release.2024-03-20 09-25-34Z/beforeAllFailure.png
09:38:53.328 detox[33498] i ws-client:ERROR The pending request #-49642 ("cleanup") has been rejected due to the following error:

The tester has not received a response within 5000ms timeout to the message:

Cleanup {
  type: 'cleanup',
  params: [Object],
  messageId: -49642
}
09:38:53.367 detox[33493] E ws-server connection :49245<->:49247
FAIL kit/005-keyboard-toolbar.e2e.ts (793.056 s)
  `KeyboardToolbar` specification
    ✕ should navigate to `Toolbar` screen (28 ms)
    ✕ should have expected UI when first input is focused (1 ms)
    ✕ should show bottom sheet when `AutoFill Contacts` is pressed
    ✕ should set focus back when modal closed (2 ms)
    ✕ should do correct actions when contact gets selected (1 ms)
    ✕ should skip a disabled fields (3 ms)
    ✕ should handle multiple clicks in row
    ✕ should handle `previous` clicks correctly (1 ms)
    ✕ should have expected UI state when end of form reached
    ✕ should enable next button when go to previous field from the last one
    ✕ should close keyboard when press `Done` (4 ms)

  ● `KeyboardToolbar` specification › should navigate to `Toolbar` screen

    thrown: "Exceeded timeout of 240000 ms for a hook.
    Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."

      14 |
      [15](https://github.com/kirillzyusko/react-native-keyboard-controller/actions/runs/8356749445/job/22874463631?pr=392#step:18:16) | describe("`KeyboardToolbar` specification", () => {
    > [16](https://github.com/kirillzyusko/react-native-keyboard-controller/actions/runs/8356749445/job/22874463631?pr=392#step:18:17) |   beforeAll(async () => {
         |   ^
      [17](https://github.com/kirillzyusko/react-native-keyboard-controller/actions/runs/8356749445/job/22874463631?pr=392#step:18:18) |     await setDemoMode();
      [18](https://github.com/kirillzyusko/react-native-keyboard-controller/actions/runs/8356749445/job/22874463631?pr=392#step:18:19) |     await device.launchApp();
      19 |   });

      at 005-keyboard-toolbar.e2e.ts:16:3
      at Object.<anonymous> (005-keyboard-toolbar.e2e.ts:15:1)

  ● `KeyboardToolbar` specification › should have expected UI when first input is focused

    thrown: "Exceeded timeout of 240000 ms for a hook.
    Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."

      14 |
      15 | describe("`KeyboardToolbar` specification", () => {
    > 16 |   beforeAll(async () => {
         |   ^
      17 |     await setDemoMode();
      18 |     await device.launchApp();
      19 |   });

      at 005-keyboard-toolbar.e2e.ts:16:3
      at Object.<anonymous> (005-keyboard-toolbar.e2e.ts:15:1)

  ● `KeyboardToolbar` specification › should show bottom sheet when `AutoFill Contacts` is pressed

    thrown: "Exceeded timeout of 240000 ms for a hook.
    Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."

      14 |
      15 | describe("`KeyboardToolbar` specification", () => {
    > 16 |   beforeAll(async () => {
         |   ^
      17 |     await setDemoMode();
      18 |     await device.launchApp();
      19 |   });

      at 005-keyboard-toolbar.e2e.ts:16:3
      at Object.<anonymous> (005-keyboard-toolbar.e2e.ts:15:1)

  ● `KeyboardToolbar` specification › should set focus back when modal closed

    thrown: "Exceeded timeout of 240000 ms for a hook.
    Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."

      14 |
      15 | describe("`KeyboardToolbar` specification", () => {
    > 16 |   beforeAll(async () => {
         |   ^
      17 |     await setDemoMode();
      18 |     await device.launchApp();
      19 |   });

      at 005-keyboard-toolbar.e2e.ts:16:3
      at Object.<anonymous> (005-keyboard-toolbar.e2e.ts:15:1)

  ● `KeyboardToolbar` specification › should do correct actions when contact gets selected

    thrown: "Exceeded timeout of 240000 ms for a hook.
    Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."

      14 |
      15 | describe("`KeyboardToolbar` specification", () => {
    > 16 |   beforeAll(async () => {
         |   ^
      17 |     await setDemoMode();
      18 |     await device.launchApp();
      19 |   });

      at 005-keyboard-toolbar.e2e.ts:16:3
      at Object.<anonymous> (005-keyboard-toolbar.e2e.ts:15:1)

  ● `KeyboardToolbar` specification › should skip a disabled fields

    thrown: "Exceeded timeout of 240000 ms for a hook.
    Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."

      14 |
      15 | describe("`KeyboardToolbar` specification", () => {
    > 16 |   beforeAll(async () => {
         |   ^
      17 |     await setDemoMode();
      18 |     await device.launchApp();
      19 |   });

      at 005-keyboard-toolbar.e2e.ts:16:3
      at Object.<anonymous> (005-keyboard-toolbar.e2e.ts:15:1)

  ● `KeyboardToolbar` specification › should handle multiple clicks in row

    thrown: "Exceeded timeout of 240000 ms for a hook.
    Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."

      14 |
      15 | describe("`KeyboardToolbar` specification", () => {
    > 16 |   beforeAll(async () => {
         |   ^
      17 |     await setDemoMode();
      18 |     await device.launchApp();
      19 |   });

      at 005-keyboard-toolbar.e2e.ts:16:3
      at Object.<anonymous> (005-keyboard-toolbar.e2e.ts:15:1)

  ● `KeyboardToolbar` specification › should handle `previous` clicks correctly

    thrown: "Exceeded timeout of 240000 ms for a hook.
    Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."

      14 |
      15 | describe("`KeyboardToolbar` specification", () => {
    > 16 |   beforeAll(async () => {
         |   ^
      17 |     await setDemoMode();
      18 |     await device.launchApp();
      19 |   });

      at 005-keyboard-toolbar.e2e.ts:16:3
      at Object.<anonymous> (005-keyboard-toolbar.e2e.ts:15:1)

  ● `KeyboardToolbar` specification › should have expected UI state when end of form reached

    thrown: "Exceeded timeout of 240000 ms for a hook.
    Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."

      14 |
      15 | describe("`KeyboardToolbar` specification", () => {
    > 16 |   beforeAll(async () => {
         |   ^
      17 |     await setDemoMode();
      18 |     await device.launchApp();
      19 |   });

      at 005-keyboard-toolbar.e2e.ts:16:3
      at Object.<anonymous> (005-keyboard-toolbar.e2e.ts:15:1)

  ● `KeyboardToolbar` specification › should enable next button when go to previous field from the last one

    thrown: "Exceeded timeout of 240000 ms for a hook.
    Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."

      14 |
      15 | describe("`KeyboardToolbar` specification", () => {
    > 16 |   beforeAll(async () => {
         |   ^
      17 |     await setDemoMode();
      18 |     await device.launchApp();
      19 |   });

      at 005-keyboard-toolbar.e2e.ts:16:3
      at Object.<anonymous> (005-keyboard-toolbar.e2e.ts:15:1)

  ● `KeyboardToolbar` specification › should close keyboard when press `Done`

    thrown: "Exceeded timeout of 240000 ms for a hook.
    Add a timeout value to this test to increase the timeout, if this is a long-running test. See https://jestjs.io/docs/api#testname-fn-timeout."

      14 |
      15 | describe("`KeyboardToolbar` specification", () => {
    > 16 |   beforeAll(async () => {
         |   ^
      17 |     await setDemoMode();
      18 |     await device.launchApp();
      19 |   });

      at 005-keyboard-toolbar.e2e.ts:16:3
      at Object.<anonymous> (005-keyboard-toolbar.e2e.ts:15:1)

09:38:53.595 detox[33493] B ws-server connection :49245<->:57139
09:38:53.608 detox[33493] i child-process:EXEC_CMD applesimutils --list --byType "iPhone 14 Pro"
09:39:06.266 detox[33498] i ws-client 
  error: Unexpected error on an attempt to handle the response received over the web socket.
  
  HINT: Examine the inner error:
  
  Unexpected message received over the web socket: cleanupDone
  
  The payload was:
  
  {"messageId":-49642,"params":{},"type":"cleanupDone"}
   
09:39:09.801 detox[33498] i device com.example.reactnativekeyboardcontroller launched. To watch simulator logs, run:
        /usr/bin/xcrun simctl spawn 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 log stream --level debug --style compact --predicate 'process == "KeyboardControllerExample"'
09:39:10.245 detox[33493] i device settled on 6C3E50C2-B609-47A0-8988-3CC825FB8AB1
  data: {
    "id": "6C3E50C2-B609-47A0-8988-3CC825FB8AB1",
    "udid": "6C3E50C2-B609-47A0-8988-3CC825FB8AB1"
  }
09:39:10.246 detox[33493] i child-process:EXEC_CMD applesimutils --list --byId 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 --maxResults 1
09:39:11.183 detox[33498] i child-process:EXEC_CMD /usr/bin/xcrun simctl uninstall 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller
09:39:11.184 detox[33498] i child-process:EXEC_TRY Uninstalling com.example.reactnativekeyboardcontroller...
09:39:38.076 detox[33493] E ws-server connection :49245<->:56040
09:39:38.162 detox[33498] i child-process:EXEC_SUCCESS com.example.reactnativekeyboardcontroller uninstalled
09:39:38.[20](https://github.com/kirillzyusko/react-native-keyboard-controller/actions/runs/8356749445/job/22874463631?pr=392#step:18:21)8 detox[33498] i child-process:EXEC_CMD /usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller
09:39:38.209 detox[33498] i child-process:EXEC_TRY Terminating com.example.reactnativekeyboardcontroller...
09:39:55.453 detox[33498] i child-process:EXEC_TRY Terminating com.example.reactnativekeyboardcontroller...
09:39:56.057 detox[33498] i child-process:EXEC_FAIL "/usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller" failed with error = ChildProcessError: Command failed: /usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller
An error was encountered processing the command (domain=NSPOSIXErrorDomain, code=3):
Simulator device returned an error for the requested operation.
found nothing to terminate
Underlying error (domain=NSPOSIXErrorDomain, code=3):
	The request to terminate "com.example.reactnativekeyboardcontroller" failed. found nothing to terminate
	found nothing to terminate
 `/usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller` (exited with error code 3) (code=3), stdout and stderr:

09:39:56.058 detox[33498] i child-process:EXEC_FAIL 
09:39:56.058 detox[33498] i child-process:EXEC_FAIL An error was encountered processing the command (domain=NSPOSIXErrorDomain, code=3):
Simulator device returned an error for the requested operation.
found nothing to terminate
Underlying error (domain=NSPOSIXErrorDomain, code=3):
	The request to terminate "com.example.reactnativekeyboardcontroller" failed. found nothing to terminate
	found nothing to terminate

09:39:56.061 detox[33498] i child-process:EXEC_CMD /usr/bin/xcrun simctl install 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 "/Users/runner/work/react-native-keyboard-controller/react-native-keyboard-controller/example/ios/build/Build/Products/Release-iphonesimulator/KeyboardControllerExample.app"
09:39:56.061 detox[33498] i child-process:EXEC_TRY Installing /Users/runner/work/react-native-keyboard-controller/react-native-keyboard-controller/example/ios/build/Build/Products/Release-iphonesimulator/KeyboardControllerExample.app...
09:40:20.132 detox[33498] i child-process:EXEC_SUCCESS /Users/runner/work/react-native-keyboard-controller/react-native-keyboard-controller/example/ios/build/Build/Products/Release-iphonesimulator/KeyboardControllerExample.app installed
09:40:[21](https://github.com/kirillzyusko/react-native-keyboard-controller/actions/runs/8356749445/job/22874463631?pr=392#step:18:22).166 detox[33498] i child-process:EXEC_CMD /usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller
09:40:21.167 detox[33498] i child-process:EXEC_TRY Terminating com.example.reactnativekeyboardcontroller...
09:40:30.797 detox[33498] i child-process:EXEC_TRY Terminating com.example.reactnativekeyboardcontroller...
09:40:31.651 detox[33498] i child-process:EXEC_FAIL "/usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller" failed with error = ChildProcessError: Command failed: /usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller
An error was encountered processing the command (domain=NSPOSIXErrorDomain, code=3):
Simulator device returned an error for the requested operation.
found nothing to terminate
Underlying error (domain=NSPOSIXErrorDomain, code=3):
	The request to terminate "com.example.reactnativekeyboardcontroller" failed. found nothing to terminate
	found nothing to terminate
 `/usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller` (exited with error code 3) (code=3), stdout and stderr:

09:40:31.652 detox[33498] i child-process:EXEC_FAIL 
09:40:31.652 detox[33498] i child-process:EXEC_FAIL An error was encountered processing the command (domain=NSPOSIXErrorDomain, code=3):
Simulator device returned an error for the requested operation.
found nothing to terminate
Underlying error (domain=NSPOSIXErrorDomain, code=3):
	The request to terminate "com.example.reactnativekeyboardcontroller" failed. found nothing to terminate
	found nothing to terminate

09:40:33.280 detox[33498] i lifecycle 002-aware-scroll-view.e2e.ts is assigned to 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 (iPhone 14 Pro)
09:40:33.305 detox[33498] i child-process:EXEC_CMD /usr/bin/xcrun simctl status_bar 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 override --time "12:00" --dataNetwork "wifi" --wifiMode "active" --wifiBars "3" --cellularMode "active" --cellularBars "4" --batteryLevel "100"
09:40:33.946 detox[33498] i child-process:EXEC_CMD /usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller
09:40:33.946 detox[33498] i child-process:EXEC_TRY Terminating com.example.reactnativekeyboardcontroller...
09:40:35.242 detox[33498] i child-process:EXEC_TRY Terminating com.example.reactnativekeyboardcontroller...
09:40:35.951 detox[33498] i child-process:EXEC_FAIL "/usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller" failed with error = ChildProcessError: Command failed: /usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller
An error was encountered processing the command (domain=NSPOSIXErrorDomain, code=3):
Simulator device returned an error for the requested operation.
found nothing to terminate
Underlying error (domain=NSPOSIXErrorDomain, code=3):
	The request to terminate "com.example.reactnativekeyboardcontroller" failed. found nothing to terminate
	found nothing to terminate
 `/usr/bin/xcrun simctl terminate 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller` (exited with error code 3) (code=3), stdout and stderr:

09:40:35.952 detox[33498] i child-process:EXEC_FAIL 
09:40:35.952 detox[33498] i child-process:EXEC_FAIL An error was encountered processing the command (domain=NSPOSIXErrorDomain, code=3):
Simulator device returned an error for the requested operation.
found nothing to terminate
Underlying error (domain=NSPOSIXErrorDomain, code=3):
	The request to terminate "com.example.reactnativekeyboardcontroller" failed. found nothing to terminate
	found nothing to terminate

09:40:35.952 detox[33498] i child-process:EXEC_CMD SIMCTL_CHILD_GULGeneratedClassDisposeDisabled=YES SIMCTL_CHILD_DYLD_INSERT_LIBRARIES="/Users/runner/Library/Detox/ios/99070475dd1fc273e5d617c755416087077e3[22](https://github.com/kirillzyusko/react-native-keyboard-controller/actions/runs/8356749445/job/22874463631?pr=392#step:18:23)d/Detox.framework/Detox" /usr/bin/xcrun simctl launch 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller --args -detoxServer ws://localhost:49245 -detoxSessionId 8af1db61-77d6-3c4f-d2[23](https://github.com/kirillzyusko/react-native-keyboard-controller/actions/runs/8356749445/job/22874463631?pr=392#step:18:24)-9cfb905afdeb -detoxDebugVisibility YES -detoxDisableHierarchyDump YES
09:40:35.952 detox[33498] i child-process:EXEC_TRY Launching com.example.reactnativekeyboardcontroller...
09:41:38.672 detox[33498] i child-process:EXEC_CMD /usr/bin/xcrun simctl get_app_container 6C3E50C2-B609-47A0-8988-3CC825FB8AB1 com.example.reactnativekeyboardcontroller
09:41:38.683 detox[33493] B ws-server connection :49[24](https://github.com/kirillzyusko/react-native-keyboard-controller/actions/runs/8356749445/job/22874463631?pr=392#step:18:25)5<->:59[33](https://github.com/kirillzyusko/react-native-keyboard-controller/actions/runs/8356749445/job/22874463631?pr=392#step:18:34)4
Error: The operation was canceled.

Device logs

Can be found here: https://github.com/kirillzyusko/react-native-keyboard-controller/actions/runs/8358181451/artifacts/1342341074

But I don't understand why failed tests do not include these logs?..

Device logs

I don't have them at the moment but I'll paste later!

More data, please!

I'd be glad to help but I even don't know a direction for investigation. I assume that issue can be not because of detox but I haven't found any problems searching other macos-14 problems and UI tests over github.

The text was updated successfully, but these errors were encountered:

kirillzyusko · 2024-03-26T17:40:01Z

@asafkorem @noomorph sorry to tag you, but just wanted to bring a little bit more attention to this issue 😅

I'm keen to help you with the investigation, but it would be good to know at least a direction and which additional information I need to provide.

awinograd · 2024-03-27T14:27:55Z

Not 100% sure our issues are releated, but @kirillzyusko have you tried running your tests with Xcode 14.1? We've also started experiencing issues with synchronization timeouts on CircleCI (not locally) when bumping from Xcode 14.1 to Xcode 14.3.1

More details in discord: https://discord.com/channels/957617863550697482/1219819351109275648

kirillzyusko · 2024-03-27T14:34:50Z

@awinograd no, I tried only 15.0, 15.2 and 14.3 🤔 Thank you for a hint - I'll try to use XCode 14.1 and return back with results 👀

From what I observed - I switched from macos-12 to macos-14, and it has 14GB RAM vs 7 GB RAM (so it can also be a case). When failures happens CI runner has <1 GB of free RAM (and I think it also can be a root cause).

awinograd · 2024-03-27T14:52:43Z

@kirillzyusko the memory change is an interesting observation. Our machines went from 8GB to 6GB when switching from 14.1 to 14.3.

For both machines, we see memory utilization around 80-85% which is near the 1GB mark of free RAM

kirillzyusko · 2024-03-27T15:34:02Z

@awinograd yeah, I observed similar values. The discord link - I can not open it... Do I need to join to detox server first and only then press a link?

the memory change is an interesting observation.

Yeah, but I decided to came back to macos-13 (14 GB RAM) and tests are still flaky even though 5 GB are free 😔 So it looks like it's partially related 🤷‍♂️

P. S. I've tried to use XCode 14.1, but in this case my build fails 🙈

awinograd · 2024-03-27T15:59:31Z

@kirillzyusko hmm i'm not sure about discord... i found this issue from the official detox discord (i think you posted it in response to someone else (edit: it was ericswpark)).

Here's a link to the message (vs the thread which i linked above)
https://discord.com/channels/957617863550697482/957619870919442432/1219819351109275648

invite link to discord: https://discord.gg/CkD5QKheF5

and a link to the post contents as a txt file discordmessage.txt

kirillzyusko · 2024-03-28T14:21:33Z

@awinograd thank you! I joined a server 😊

When I debug this problem more I discovered that the problem happens on macos-12 and macos-13 (which have more RAM) as well.

Unfortunately I can not use XCode 14.1 again, because I use RN 0.73 and build stage fails 😔

kirillzyusko · 2024-04-02T13:07:33Z

I'm closing this issue. It looks like it's a problem with GH runners rather than Detox. Similar problem was reported in actions/runner-images#9591

kirillzyusko added platform: ios status: triage type: bug 🐞 labels Mar 20, 2024

kirillzyusko mentioned this issue Mar 20, 2024

fix: stable iOS e2e tests kirillzyusko/react-native-keyboard-controller#392

Merged

2 tasks

ericswpark mentioned this issue Mar 23, 2024

Automated mobile tests failing Purdue-CS307-tsundoku/tsundoku#110

Open

kirillzyusko closed this as completed Apr 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Flaky tests on `macos-14` GH runner (M1, iOS) #4415

Flaky tests on `macos-14` GH runner (M1, iOS) #4415

kirillzyusko commented Mar 20, 2024 •

edited

Loading

kirillzyusko commented Mar 26, 2024

awinograd commented Mar 27, 2024

kirillzyusko commented Mar 27, 2024

awinograd commented Mar 27, 2024

kirillzyusko commented Mar 27, 2024 •

edited

Loading

awinograd commented Mar 27, 2024 •

edited

Loading

kirillzyusko commented Mar 28, 2024

kirillzyusko commented Apr 2, 2024

Flaky tests on macos-14 GH runner (M1, iOS) #4415

Flaky tests on macos-14 GH runner (M1, iOS) #4415

Comments

kirillzyusko commented Mar 20, 2024 • edited Loading

What happened?

What was the expected behaviour?

Was it tested on latest Detox?

Did your test throw out a timeout?

Help us reproduce this issue!

In what environment did this happen?

Detox logs

Device logs

More data, please!

kirillzyusko commented Mar 26, 2024

awinograd commented Mar 27, 2024

kirillzyusko commented Mar 27, 2024

awinograd commented Mar 27, 2024

kirillzyusko commented Mar 27, 2024 • edited Loading

awinograd commented Mar 27, 2024 • edited Loading

kirillzyusko commented Mar 28, 2024

kirillzyusko commented Apr 2, 2024

Flaky tests on `macos-14` GH runner (M1, iOS) #4415

Flaky tests on `macos-14` GH runner (M1, iOS) #4415

kirillzyusko commented Mar 20, 2024 •

edited

Loading

kirillzyusko commented Mar 27, 2024 •

edited

Loading

awinograd commented Mar 27, 2024 •

edited

Loading