Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] test fix for #16693 #16729

Closed
wants to merge 1 commit into from
Closed

[WIP] test fix for #16693 #16729

wants to merge 1 commit into from

Conversation

Parveshdhull
Copy link
Member

@Parveshdhull Parveshdhull commented Jul 20, 2023

status: just small testing for #16693

@Parveshdhull Parveshdhull self-assigned this Jul 20, 2023
@Parveshdhull Parveshdhull marked this pull request as draft July 20, 2023 10:33
@Parveshdhull Parveshdhull changed the title test fix for #16693 [WIP] test fix for #16693 Jul 20, 2023
@status-im-auto
Copy link
Member

status-im-auto commented Jul 20, 2023

Jenkins Builds

Click to see older builds (4)
Commit #️⃣ Finished (UTC) Duration Platform Result
020e3ea #1 2023-07-20 10:34:39 ~3 min tests 📄log
✔️ 020e3ea #1 2023-07-20 10:37:09 ~5 min android-e2e 🤖apk 📲
✔️ 020e3ea #1 2023-07-20 10:37:37 ~5 min android 🤖apk 📲
✔️ 020e3ea #1 2023-07-20 10:39:21 ~7 min ios 📱ipa 📲
Commit #️⃣ Finished (UTC) Duration Platform Result
4861264 #2 2023-07-26 12:37:53 ~3 min tests 📄log
✔️ 4861264 #2 2023-07-26 12:40:50 ~6 min android 🤖apk 📲
✔️ 4861264 #2 2023-07-26 12:41:16 ~6 min android-e2e 🤖apk 📲
✔️ 4861264 #2 2023-07-26 12:43:58 ~9 min ios 📱ipa 📲
2a10df3 #3 2023-07-31 14:18:46 ~2 min tests 📄log
✔️ 2a10df3 #3 2023-07-31 14:21:31 ~5 min android-e2e 🤖apk 📲
✔️ 2a10df3 #3 2023-07-31 14:22:21 ~6 min android 🤖apk 📲
✔️ 2a10df3 #3 2023-07-31 14:39:27 ~23 min ios 📱ipa 📲

@qoqobolo qoqobolo self-assigned this Jul 31, 2023
@status-im-auto
Copy link
Member

90% of end-end tests have passed

Total executed tests: 40
Failed tests: 4
Passed tests: 36
IDs of failed tests: 702732,703133,703495,702731 

Failed tests (4)

Click to expand
  • Rerun failed tests

  • Class TestCommunityOneDeviceMerged:

    1. test_restore_multiaccount_with_waku_backup_remove_switch, id: 703133
    Test is not run, e2e blocker  
    

    [[reason: [NOTRUN] Restoring communities issue: 16787; restoring contacts issue: 15500]]

    Class TestGroupChatMultipleDeviceMergedNewUI:

    1. test_group_chat_pin_messages, id: 702732

    Test is not run, e2e blocker  
    

    [[reason: [NOTRUN] Pin feature is in development]]

    2. test_group_chat_mute_chat, id: 703495

    Device 2: Click until ChatMessageInput by accessibility id: chat-message-input will be presented
    Device 2: Looking for a message by text: Chat is unmuted now

    critical/chats/test_group_chat.py:607: in test_group_chat_mute_chat
        self.errors.verify_no_errors()
    base_test_case.py:183: in verify_no_errors
        pytest.fail('\n '.join([self.errors.pop(0) for _ in range(len(self.errors))]))
     New messages counter near chat name is not shown after unmute
    



    Device sessions

    Class TestOneToOneChatMultipleSharedDevicesNewUi:

    1. test_1_1_chat_pin_messages, id: 702731

    Test is not run, e2e blocker  
    

    [[reason: [NOTRUN] Pin feature is in development]]

    Passed tests (36)

    Click to expand

    Class TestCommunityOneDeviceMerged:

    1. test_community_copy_and_paste_message_in_chat_input, id: 702742
    Device sessions

    2. test_community_undo_delete_message, id: 702869
    Device sessions

    3. test_community_navigate_to_channel_when_relaunch, id: 702846
    Device sessions

    4. test_community_mute_community_and_channel, id: 703382
    Device sessions

    Class TestGroupChatMultipleDeviceMergedNewUI:

    1. test_group_chat_send_image_save_and_share, id: 703297
    Device sessions

    2. test_group_chat_reactions, id: 703202
    Device sessions

    3. test_group_chat_join_send_text_messages_push, id: 702807
    Device sessions

    4. test_group_chat_offline_pn, id: 702808
    Device sessions

    Class TestActivityCenterContactRequestMultipleDevicePR:

    1. test_activity_center_contact_request_accept_swipe_mark_all_as_read, id: 702851
    Device sessions

    2. test_activity_center_contact_request_decline, id: 702850
    Device sessions

    Class TestActivityMultipleDevicePR:

    1. test_activity_center_mentions, id: 702957
    Device sessions

    2. test_navigation_jump_to, id: 702936
    Device sessions

    3. test_activity_center_reply_read_unread_delete_filter_swipe, id: 702947
    Device sessions

    4. test_activity_center_admin_notification_accept_swipe, id: 702958
    Device sessions

    Class TestCommunityMultipleDeviceMerged:

    1. test_community_several_images_send_reply, id: 703194
    Device sessions

    2. test_community_one_image_send_reply, id: 702859
    Device sessions

    3. test_community_emoji_send_copy_paste_reply, id: 702840
    Device sessions

    4. test_community_mark_all_messages_as_read, id: 703086
    Device sessions

    5. test_community_contact_block_unblock_offline, id: 702894
    Device sessions

    6. test_community_mentions_push_notification, id: 702786
    Device sessions

    7. test_community_message_delete, id: 702839
    Device sessions

    8. test_community_message_send_check_timestamps_sender_username, id: 702838
    Device sessions

    9. test_community_links_with_previews_github_youtube_twitter_gif_send_enable, id: 702844
    Device sessions

    10. test_community_message_edit, id: 702843
    Device sessions

    11. test_community_leave, id: 702845
    Device sessions

    12. test_community_unread_messages_badge, id: 702841
    Device sessions

    Class TestOneToOneChatMultipleSharedDevicesNewUi:

    1. test_1_1_chat_emoji_send_reply_and_open_link, id: 702782
    Device sessions

    2. test_1_1_chat_text_message_delete_push_disappear, id: 702733
    Device sessions

    3. test_1_1_chat_delete_via_long_press_relogin, id: 702784
    Device sessions

    4. test_1_1_chat_push_emoji, id: 702813
    Device sessions

    5. test_1_1_chat_non_latin_messages_stack_update_profile_photo, id: 702745
    Device sessions

    6. test_1_1_chat_is_shown_message_sent_delivered_from_offline, id: 702783
    Device sessions

    7. test_1_1_chat_mute_chat, id: 703496
    Device sessions

    8. test_1_1_chat_edit_message, id: 702855
    Device sessions

    9. test_1_1_chat_send_image_save_and_share, id: 703391
    Device sessions

    10. test_1_1_chat_message_reaction, id: 702730
    Device sessions

    @qoqobolo
    Copy link
    Contributor

    qoqobolo commented Aug 1, 2023

    Hey @Parveshdhull, here is a short summary of what happened during this time with the issue:

    Yesterday, for some reason, the issue began to reproduce both in PR and nightly, manually on physical devices and e2e.
    I rebased your PR to compare with nightly, and in the PR I could not reproduce the bug even once.

    It was reproduced with the following steps in e2e and manually (some of the steps may of course be redundant, but I could not cut it off, since it was not reproduced every time again):

    1. Create a profile
    2. Copy your profile link
    3. Open the Jump to screen
    4. Go to your profile from the Jump to screen
    5. Log out
    6. Quickly press the Profiles button at the top right
    Screenshot 2023-08-01 at 11 45 03 Screenshot 2023-08-01 at 11 45 19

    But that's no longer reproducible today, so I cannot compare it with today's nightly.
    I have no idea what's going on, but maybe it will give you some insights (especially the fact that it was not reproducible in your PR yesterday).

    @status-im-auto
    Copy link
    Member

    88% of end-end tests have passed

    Total executed tests: 40
    Failed tests: 5
    Passed tests: 35
    
    IDs of failed tests: 702731,702851,703133,702732,703495 
    

    Failed tests (5)

    Click to expand
  • Rerun failed tests

  • Class TestOneToOneChatMultipleSharedDevicesNewUi:

    1. test_1_1_chat_pin_messages, id: 702731
    Test is not run, e2e blocker  
    

    [[reason: [NOTRUN] Pin feature is in development]]

    Class TestGroupChatMultipleDeviceMergedNewUI:

    1. test_group_chat_pin_messages, id: 702732

    Test is not run, e2e blocker  
    

    [[reason: [NOTRUN] Pin feature is in development]]

    2. test_group_chat_mute_chat, id: 703495

    Device 2: Click until ChatMessageInput by accessibility id: chat-message-input will be presented
    Device 2: Looking for a message by text: Chat is unmuted now

    critical/chats/test_group_chat.py:607: in test_group_chat_mute_chat
        self.errors.verify_no_errors()
    base_test_case.py:183: in verify_no_errors
        pytest.fail('\n '.join([self.errors.pop(0) for _ in range(len(self.errors))]))
     New messages counter near chat name is not shown after unmute
    



    Device sessions

    Class TestCommunityOneDeviceMerged:

    1. test_restore_multiaccount_with_waku_backup_remove_switch, id: 703133

    Test is not run, e2e blocker  
    

    [[reason: [NOTRUN] Restoring communities issue: 16787; restoring contacts issue: 15500]]

    Class TestActivityCenterContactRequestMultipleDevicePR:

    1. test_activity_center_contact_request_accept_swipe_mark_all_as_read, id: 702851

    # STEP: Check that can accept contact request from read notifications
    Device 1: Find Button by accessibility id: selector-filter

    /home/jenkins/.local/lib/python3.10/site-packages/urllib3/connectionpool.py:703: in urlopen
        httplib_response = self._make_request(
    /home/jenkins/.local/lib/python3.10/site-packages/urllib3/connectionpool.py:386: in _make_request
        self._validate_conn(conn)
    /home/jenkins/.local/lib/python3.10/site-packages/urllib3/connectionpool.py:1042: in _validate_conn
        conn.connect()
    /home/jenkins/.local/lib/python3.10/site-packages/urllib3/connection.py:414: in connect
        self.sock = ssl_wrap_socket(
    /home/jenkins/.local/lib/python3.10/site-packages/urllib3/util/ssl_.py:449: in ssl_wrap_socket
        ssl_sock = _ssl_wrap_socket_impl(
    /home/jenkins/.local/lib/python3.10/site-packages/urllib3/util/ssl_.py:493: in _ssl_wrap_socket_impl
        return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
    /usr/lib/python3.10/ssl.py:513: in wrap_socket
        return self.sslsocket_class._create(
    /usr/lib/python3.10/ssl.py:1071: in _create
        self.do_handshake()
    /usr/lib/python3.10/ssl.py:1342: in do_handshake
        self._sslobj.do_handshake()
     TLS/SSL connection has been closed (EOF) (_ssl.c:997)
    
    During handling of the above exception, another exception occurred:
    medium/test_activity_center.py:115: in test_activity_center_contact_request_accept_swipe_mark_all_as_read
        self.home_1.activity_unread_filter_button.click()
    ../views/base_element.py:91: in click
        self.find_element().click()
    /home/jenkins/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webelement.py:80: in click
        self._execute(Command.CLICK_ELEMENT)
    /home/jenkins/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webelement.py:633: in _execute
        return self._parent.execute(command, params)
    /home/jenkins/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py:319: in execute
        response = self.command_executor.execute(driver_command, params)
    /home/jenkins/.local/lib/python3.10/site-packages/selenium/webdriver/remote/remote_connection.py:376: in execute
        return self._request(command_info[0], url, body=data)
    /home/jenkins/.local/lib/python3.10/site-packages/selenium/webdriver/remote/remote_connection.py:399: in _request
        resp = self._conn.request(method, url, body=body, headers=headers)
    /home/jenkins/.local/lib/python3.10/site-packages/urllib3/request.py:78: in request
        return self.request_encode_body(
    /home/jenkins/.local/lib/python3.10/site-packages/urllib3/request.py:170: in request_encode_body
        return self.urlopen(method, url, **extra_kw)
    /home/jenkins/.local/lib/python3.10/site-packages/urllib3/poolmanager.py:376: in urlopen
        response = conn.urlopen(method, u.request_uri, **kw)
    /home/jenkins/.local/lib/python3.10/site-packages/urllib3/connectionpool.py:815: in urlopen
        return self.urlopen(
    /home/jenkins/.local/lib/python3.10/site-packages/urllib3/connectionpool.py:815: in urlopen
        return self.urlopen(
    /home/jenkins/.local/lib/python3.10/site-packages/urllib3/connectionpool.py:815: in urlopen
        return self.urlopen(
    /home/jenkins/.local/lib/python3.10/site-packages/urllib3/connectionpool.py:787: in urlopen
        retries = retries.increment(
    /home/jenkins/.local/lib/python3.10/site-packages/urllib3/util/retry.py:592: in increment
        raise MaxRetryError(_pool, url, error or ResponseError(cause))
    E   urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='ondemand.eu-central-1.saucelabs.com', port=443): Max retries exceeded with url: /wd/hub/session/af90cbc50be144a6926bf4f16352bbeb/element/a7761993-7ea6-4712-ad6a-3e63c6813fb3/click (Caused by SSLError(SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:997)')))
    



    Device sessions

    Passed tests (35)

    Click to expand

    Class TestOneToOneChatMultipleSharedDevicesNewUi:

    1. test_1_1_chat_edit_message, id: 702855
    Device sessions

    2. test_1_1_chat_message_reaction, id: 702730
    Device sessions

    3. test_1_1_chat_non_latin_messages_stack_update_profile_photo, id: 702745
    Device sessions

    4. test_1_1_chat_mute_chat, id: 703496
    Device sessions

    5. test_1_1_chat_text_message_delete_push_disappear, id: 702733
    Device sessions

    6. test_1_1_chat_push_emoji, id: 702813
    Device sessions

    7. test_1_1_chat_is_shown_message_sent_delivered_from_offline, id: 702783
    Device sessions

    8. test_1_1_chat_emoji_send_reply_and_open_link, id: 702782
    Device sessions

    9. test_1_1_chat_delete_via_long_press_relogin, id: 702784
    Device sessions

    10. test_1_1_chat_send_image_save_and_share, id: 703391
    Device sessions

    Class TestCommunityMultipleDeviceMerged:

    1. test_community_leave, id: 702845
    Device sessions

    2. test_community_emoji_send_copy_paste_reply, id: 702840
    Device sessions

    3. test_community_contact_block_unblock_offline, id: 702894
    Device sessions

    4. test_community_mark_all_messages_as_read, id: 703086
    Device sessions

    5. test_community_mentions_push_notification, id: 702786
    Device sessions

    6. test_community_links_with_previews_github_youtube_twitter_gif_send_enable, id: 702844
    Device sessions

    7. test_community_unread_messages_badge, id: 702841
    Device sessions

    8. test_community_message_delete, id: 702839
    Device sessions

    9. test_community_message_send_check_timestamps_sender_username, id: 702838
    Device sessions

    10. test_community_one_image_send_reply, id: 702859
    Device sessions

    11. test_community_message_edit, id: 702843
    Device sessions

    12. test_community_several_images_send_reply, id: 703194
    Device sessions

    Class TestActivityCenterContactRequestMultipleDevicePR:

    1. test_activity_center_contact_request_decline, id: 702850
    Device sessions

    Class TestGroupChatMultipleDeviceMergedNewUI:

    1. test_group_chat_reactions, id: 703202
    Device sessions

    2. test_group_chat_join_send_text_messages_push, id: 702807
    Device sessions

    3. test_group_chat_offline_pn, id: 702808
    Device sessions

    4. test_group_chat_send_image_save_and_share, id: 703297
    Device sessions

    Class TestActivityMultipleDevicePR:

    1. test_activity_center_reply_read_unread_delete_filter_swipe, id: 702947
    Device sessions

    2. test_activity_center_admin_notification_accept_swipe, id: 702958
    Device sessions

    3. test_activity_center_mentions, id: 702957
    Device sessions

    4. test_navigation_jump_to, id: 702936
    Device sessions

    Class TestCommunityOneDeviceMerged:

    1. test_community_copy_and_paste_message_in_chat_input, id: 702742
    Device sessions

    2. test_community_navigate_to_channel_when_relaunch, id: 702846
    Device sessions

    3. test_community_undo_delete_message, id: 702869
    Device sessions

    4. test_community_mute_community_and_channel, id: 703382
    Device sessions

    @Parveshdhull
    Copy link
    Member Author

    Hi @qoqobolo,
    Thank you very much for testing the PR.

    Yesterday, for some reason, the issue began to reproduce both in PR and nightly

    The fact that it was reproducible in the PR even before rebase, tells that the current fix is not working and layout animations are not the culprit.

    But that's no longer reproducible today, so I cannot compare it with today's nightly

    So, now the issue is also not reproducible in the previous nightly too? (or only today)

    Anyway, looks like we are not sure about the issue's cause, and the current fix is not working, so closing this PR.
    Thank you very much for your amazing work, and reproduction steps. I will keep an eye on this flow.

    @status-im-auto
    Copy link
    Member

    100% of end-end tests have passed

    Total executed tests: 1
    Failed tests: 0
    Passed tests: 1
    

    Passed tests (1)

    Click to expand

    Class TestActivityCenterContactRequestMultipleDevicePR:

    1. test_activity_center_contact_request_accept_swipe_mark_all_as_read, id: 702851
    Device sessions

    @qoqobolo
    Copy link
    Contributor

    qoqobolo commented Aug 1, 2023

    Yesterday, for some reason, the issue began to reproduce both in PR and nightly

    The fact that it was reproducible in the PR even before rebase, tells that the current fix is not working and layout animations are not the culprit.

    Oops, sorry, I wrote it a bit inaccurately. The issue was reproducible yesterday in other PRs and in nightly. In this PR, I could not reproduce it even once, before or after.
    So probably this fix still could work 🤔

    So, now the issue is also not reproducible in the previous nightly too? (or only today)

    So, today the issue is not reproducible anywhere, but yesterday I reproduced it in nightly several times.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Projects
    Archived in project
    Archived in project
    Development

    Successfully merging this pull request may close these issues.

    3 participants