Flaky tests Investigation in Q2 2024

[chloeYue]

What is this about?

This ticket is the result of detailed examination of current reports on test flakiness, during which I filtered out tests that have already been resolved or were duplicated within the reports.

For the remaining flaky tests, which have not yet been addressed, are grouped based on a likely common root cause of flakiness. For each of these groups, as well as for individual tests not part of any group, I have created GitHub tickets, these tickets include the CI link where the failure occurred, the specific error message, and a screenshot for clarity.

Furthermore, the flaky tests are ranked by pass rate, starting with the most flaky and moving to the less flaky. This ranking is designed to help us prioritize our efforts, focusing first on the most critical issues to enhance the stability and reliability of our testing suite significantly.

As a result, we have compiled a curated list of tickets specifically targeting test flakiness. Furthermore, we have organized the flaky tests by team. For tests that clearly fall within a particular team's domain, we are seeking that team's help in resolving the assigned flaky tests. For tests that do not have a clear boundary or involve functionalities across teams, the extension platform team takes responsibility. Up to this point, the extension platform team has already resolved several issues, and you will find that some of these tickets are already marked as closed in the list below.

• If you are part of a feature team and have flaky tests assigned to your team, please directly assign them to yourself and start working on them.
• If you are in a feature team without any flaky tests assigned but have the capacity and desire to contribute, please feel free to pick up tests that lack a clear boundary, assign them to yourself, and begin work.

Our ambition is to arrive 0 flaky test, thereby improving the developer experience with a more reliable and robust test suite. Collaboration is key to the success of this initiative, and we appreciate everyone's help.

Extension Platform team (Flaky tests that do not have a clear boundary or involve cross-team functionalities ):

• #24575
• #24574
• #24596
• #24597
• #24604
• #24601
• #24602
• #24603
• #24608
• #24615
• #24616
• #24621
• #24624
• #24628
• #24632
• #24634
• #24636
• #24637
• #24641
• #24642
• #24643
• #24645
• #24647
• #24651
• #24652
• #24653
• #24654
• #24655
• #24657
• #24658
• #24659
• #24660
• #24661
• #24668
• #24669
• #25240

Confirmation team:

• #24606
• #23851
• #24626
• #24627
• #24633

Snap team:

• #24613
• #24635
• #24638

Swap team:

• #24640
• #24656
• #24667
• #25546

Wallet API team:

• #24625

Scenario

No response

Design

No response

Technical Details

No response

Threat Modeling Framework

No response

Acceptance Criteria

No response

Stakeholder review needed before the work gets merged

• Engineering (needed in most cases)
• Design
• Product
• QA (automation tests are required to pass before merging PRs but not all changes are covered by automation tests - please review if QA is needed beyond automation tests)
• Security
• Legal
• Marketing
• Management (please specify)
• Other (please specify)

References

No response

[gauthierpetetin]

Closing this epic as it was used in Q2 and is now outdated