Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multiroot .env file update tests failing on CI. #4468

Closed
DonJayamanne opened this issue Feb 21, 2019 · 3 comments
Closed

Multiroot .env file update tests failing on CI. #4468

DonJayamanne opened this issue Feb 21, 2019 · 3 comments
Assignees
@DonJayamanne
Labels
bug Issue identified by VS Code Team member as probable bug debt Covers everything internal: CI, testing, refactoring of the codebase, etc. important Issue identified as high-priority
Milestone
2019, week 11 - M...

Comments

@DonJayamanne
Copy link

@d3r3kk /cc
@DonJayamanne DonJayamanne added bug Issue identified by VS Code Team member as probable bug important Issue identified as high-priority labels Feb 21, 2019
@d3r3kk d3r3kk changed the title Root cause analysis for failing venv tests on CI Multiroot .env file update tests failing on CI. Feb 21, 2019
@d3r3kk
Copy link

d3r3kk commented Feb 21, 2019 •
edited
Loading

Find out and fix tests that have begun failing intermittently on AzDO CI. The tests that are failing are the last two found in the file src/test/common/variables/envVarsProvider.multiroot.test.ts. The test is failing due to the timeout of 20s being exceeded.

One hypothesis is that the tests are failing and being re-run, exacerbating the trouble with async tests accessing the same file. (Rejected due to PR #4479). Another hypothesis is that the timeouts for the test just aren't long enough.

The tests are both set to having a total timeout of 20 seconds. There are 3 places in both tests where an imposed wait of 5 seconds is being processed. This means that the test must complete all other tasks in < 5 seconds. Perhaps the overall test timeout needs to be set to something a bit higher, like 25 seconds for each test.

If it is at all possible, and we could be notified when the configuration has been updated, we could remove the waits added to the tests altogether and reduce the overall time these tests are set to run for.

@d3r3kk d3r3kk mentioned this issue Feb 21, 2019
Closed
8 tasks
@DonJayamanne
Copy link
Author

Hmm, how about we consider re-writing the tests? Wouldn't that be a better approach?
Can get rid of 20s delays, and randomizing file names, etc.

d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Feb 24, 2019
@d3r3kk
Skip failing tests tracked by microsoft#4468
9aa95aa 
- only skip for Windows
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Feb 24, 2019
@d3r3kk
Skip failing tests tracked by microsoft#4468
4143d97 
- only skip for Windows
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Feb 26, 2019
@d3r3kk
Skip failing tests tracked by microsoft#4468
ae42aab 
- only skip for Windows
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Feb 26, 2019
@d3r3kk
Skip failing tests tracked by microsoft#4468
c5481f7 
- only skip for Windows
@DonJayamanne DonJayamanne added needs PR debt Covers everything internal: CI, testing, refactoring of the codebase, etc. labels Feb 28, 2019
@DonJayamanne
Copy link
Author

TODO: Refactor the tests.

d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Mar 1, 2019
@d3r3kk
Skip failing tests tracked by microsoft#4468
0ddb34c 
- only skip for Windows
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Mar 5, 2019
@d3r3kk
Skip failing tests tracked by microsoft#4468
2618e3d 
- only skip for Windows
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Mar 9, 2019
@d3r3kk
Skip failing tests tracked by microsoft#4468
cc54673 
- only skip for Windows
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Mar 12, 2019
@d3r3kk
Skip failing tests tracked by microsoft#4468
5d3fe2b 
- only skip for Windows
@luabud luabud added important Issue identified as high-priority and removed important Issue identified as high-priority labels Mar 12, 2019
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Mar 13, 2019
@d3r3kk
Skip failing tests tracked by microsoft#4468
d2b6a01 
- only skip for Windows
@d3r3kk d3r3kk removed their assignment Mar 14, 2019
d3r3kk added a commit to d3r3kk/vscode-python that referenced this issue Mar 14, 2019
@d3r3kk
Skip failing tests tracked by microsoft#4468
291989a 
- only skip for Windows
@DonJayamanne DonJayamanne self-assigned this Mar 21, 2019
@DonJayamanne DonJayamanne mentioned this issue Mar 21, 2019
Merged
4 tasks
@ghost ghost removed the needs PR label Mar 27, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Apr 24, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@DonJayamanne DonJayamanne

Labels
bug Issue identified by VS Code Team member as probable bug debt Covers everything internal: CI, testing, refactoring of the codebase, etc. important Issue identified as high-priority
Projects
None yet
Milestone
2019, week 11 - March Sprint 6
Development

No branches or pull requests
4 participants
@DonJayamanne @d3r3kk @qubitron @luabud