Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Flaky Tests #10348

Closed
7 tasks done
paul-marechal opened this issue Oct 28, 2021 · 1 comment
Closed
7 tasks done

Fix Flaky Tests #10348

paul-marechal opened this issue Oct 28, 2021 · 1 comment
Labels
bug bugs found in the application ci issues related to CI / tests quality issues related to code and application quality

Comments

@paul-marechal
Copy link
Member

paul-marechal commented Oct 28, 2021

These tests are flaky and should be fixed to make CI more reliable and more informative:

À propos the discussion here, this issue should be used to track tests that fail for no clear reason.

  • 'workbench.editor.closeOnFileDelete' should close the editor when set to 'true'
  • run reference code lens - this one is currently skipped, but its flakiness reflects a real problem with the display of code lenses: they often take a long time to appear for no very clear reason, and even when the editor leaves space between lines to render them.
  • Different parts of the suite in launch-preferences.spec.js occasionally fail or time out. E.g.

Lately some failures of comparison where undefined !== null

  • Recent PR's have seen some issues with Should toggle explorer and basic menu tests. Many logs about view containers having been disposed.
  • A few failures that break down at the step of loading Mocha with messages like this:
loading mocha...
Uncaught Exception:  Error: Evaluation failed: TypeError: Cannot read property 'logger' of undefined
   at eval (eval at waitForPredicatePageFunction (:2:21), <anonymous>:3:66)
   at eval (eval at waitForPredicatePageFunction (:2:21), <anonymous>:3:73)

#11069 should help.

@paul-marechal paul-marechal added bug bugs found in the application ci issues related to CI / tests quality issues related to code and application quality labels Oct 28, 2021
@colin-grant-work colin-grant-work changed the title fix TypeScript - run reference code lens api-test because it is flaky Fix Flaky Tests Mar 1, 2022
@vince-fugnitto
Copy link
Member

I believe we can close as tests have been pretty stable lately. We can open individual issues whenever we encounter future flakiness.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug bugs found in the application ci issues related to CI / tests quality issues related to code and application quality
Projects
None yet
Development

No branches or pull requests

2 participants