allow large scale testing #21269

eleanorjboyd · 2023-05-18T18:01:32Z

allows new testing rewrite to handle 500+ tests and load and run these tests. High limit tested was 10,000 tests.

eleanorjboyd · 2023-05-19T19:13:51Z

tests will be updated when new resultResolver class is created that will take most responsibility away from workspaceTestAdapter.

eleanorjboyd · 2023-05-22T17:47:59Z

src/client/testing/testController/pytest/pytestExecutionAdapter.ts

@@ -114,7 +115,7 @@ export class PytestTestExecutionAdapter implements ITestExecutionAdapter {
            if (debugBool && !testArgs.some((a) => a.startsWith('--capture') || a === '-s')) {
                testArgs.push('--capture', 'no');
            }
-            const pluginArgs = ['-p', 'vscode_pytest', '-v'].concat(testArgs).concat(testIds);


-v was not supposed to be included in the first place- this should be removed

src/client/common/process/types.ts

.github/workflows/build.yml

src/client/common/process/types.ts

karrtikr

Other than Karthik's comments LGTM.

…on release 2023.10.0 Merge pull request #122 from posit-dev/merge/2023.10.0 Merging upstream vscode-python release 2023.10.0 -------------------- Commit message for posit-dev/positron-python@fc0926f: Add typings to get updated EnvironmentVariableCollection API -------------------- Commit message for posit-dev/positron-python@8c11f29: Merge remote-tracking branch 'upstream/release/2023.10' -------------------- Commit message for posit-dev/positron-python@fea895f: Update version and readme for release (posit-dev/positron-python#21388) Co-authored-by: Soojin (Min) Choi <[email protected]> -------------------- Commit message for microsoft/vscode-python@ad9c899: Update version for release candidate (microsoft/vscode-python#21369) -------------------- Commit message for microsoft/vscode-python@a395e2e: fix bug so canceling debug works in rewrite (microsoft/vscode-python#21361) fixes microsoft/vscode-python#21336 -------------------- Commit message for microsoft/vscode-python@be829b3: Unittest for large workspaces (microsoft/vscode-python#21351) follows the same steps as making pytest compatible with large workspaces with many tests. Now test_ids are sent over a port as a json instead of in the exec function which can hit a cap on # of characters. Should fix microsoft/vscode-python#21339. -------------------- Commit message for microsoft/vscode-python@cd76ee1: add pythonTestAdapter to experiment enum (microsoft/vscode-python#21357) allow people to opt in and out of the pythonTestAdapter rewrite via the settings `python.experiment.optInto` or `python.experiment.optOutfrom` -------------------- Commit message for microsoft/vscode-python@dbd0b73: adding extra log messages for rewrite debugging (microsoft/vscode-python#21352) These logs print errors and other bits of information which will be helpful for debugging workflows of users where we need to get information such as args or which step in the process they got to. -------------------- Commit message for microsoft/vscode-python@d968b8c: Dont show command for button trigger in command pallet (microsoft/vscode-python#21350) Fixes microsoft/vscode-python#21322 -------------------- Commit message for microsoft/vscode-python@e9a8dd5: remove duplicates from test_ids array (microsoft/vscode-python#21347) this will partially remediate microsoft/vscode-python#21339 in regards to the duplicate IDs being run. -------------------- Commit message for microsoft/vscode-python@f148139: allow pytest tests to handle multiple payloads (microsoft/vscode-python#21301) As part of the switch to allow for dynamic run- the pytest discovery and execution tests are now switched to be take lists of dicts where the dicts are the payloads. -------------------- Commit message for microsoft/vscode-python@c213491: Apply environment variables after shell initialization scripts are run in `pythonTerminalEnvVarActivation` experiment (microsoft/vscode-python#21290) For microsoft/vscode-python#11039 microsoft/vscode-python#20822 Closes microsoft/vscode-python#21297 Update proposed APIs to be used in Terminal activation experiment. -------------------- Commit message for microsoft/vscode-python@72f7ef8: Set up testing rewrite experiment (microsoft/vscode-python#21258) is the beginning of this issue: microsoft/vscode-python#21150, in that it will start the process of implementing the setting in the extension -------------------- Commit message for microsoft/vscode-python@4109228: fix debugging with new pytest run script (microsoft/vscode-python#21299) fix debugging for run_pytest_script.py setup -------------------- Commit message for microsoft/vscode-python@b916981: remove unneeded multiroot code (microsoft/vscode-python#21295) removed extra steps to wrap data since this creates duplicate folders in the controller and only keeps the most recent instead of all the roots from different workspaces. -------------------- Commit message for microsoft/vscode-python@e2a9cec: allow large scale testing (microsoft/vscode-python#21269) allows new testing rewrite to handle 500+ tests and load and run these tests. High limit tested was 10,000 tests. -------------------- Commit message for microsoft/vscode-python@f2f5fe2: Check config type in the ChildProcessAttachEvents (microsoft/vscode-python#21272) -------------------- Commit message for microsoft/vscode-python@4b4e5b7: Update pyright version (microsoft/vscode-python#21296) Fix error in tests, updating pyright version -------------------- Commit message for microsoft/vscode-python@c9a7268: Revert "Remove hack to check the vscode version" (microsoft/vscode-python#21294) Reverts microsoft/vscode-python#21180 For microsoft/vscode-python#20769 -------------------- Commit message for microsoft/vscode-python@a74f1d1: Detect installed packages in the selected environment (microsoft/vscode-python#21231) Fixes microsoft/vscode-python#21140 -------------------- Commit message for microsoft/vscode-python@b0ebc9b: Enable debug pytest (microsoft/vscode-python#21228) fixes microsoft/vscode-python#21147 --------- Co-authored-by: Aidos Kanapyanov <[email protected]> Co-authored-by: Karthik Nadig <[email protected]> -------------------- Commit message for microsoft/vscode-python@be9662f: revert testing to using socket (microsoft/vscode-python#21242) switch back to using a socket instead of an output file for use in the plugin communication during testing. This should work now that we resolved the issue with python path for windows. -------------------- Commit message for microsoft/vscode-python@b4a47bb: Add reload flag on fastApi provider (microsoft/vscode-python#21241) -------------------- Commit message for microsoft/vscode-python@fcfc54c: Add option for pyenv interpreters when creating environments with venv (microsoft/vscode-python#21219) Resolves microsoft/vscode-python#20881 . Testing: Behaves as expected when testing with Extension Development Host: ![image](https://github.com/microsoft/vscode-python/assets/30149293/d114d9ab-f2d8-4273-877b-d7dd030cfe76) -------------------- Commit message for microsoft/vscode-python@b3d43e5: Do not open "save as" window when running existing Python files (microsoft/vscode-python#21232) Closes microsoft/vscode-python#21209 -------------------- Commit message for microsoft/vscode-python@b0da28c: Remove IS_WINDOWS constant in favor of PlatformService (microsoft/vscode-python#21157) Solves partially microsoft/vscode-python#8542 -------------------- Commit message for microsoft/vscode-python@0c4fa40: Change name of command to run Python files in separate terminals (microsoft/vscode-python#21229) Closes microsoft/vscode-python#14094 -------------------- Commit message for microsoft/vscode-python@1533818: Added option to run multiple Python files in separate terminals (microsoft/vscode-python#21223) Closes microsoft/vscode-python#21215 microsoft/vscode-python#14094 Added the option to assign a dedicated terminal for each Python file: ![image](https://github.com/microsoft/vscode-python/assets/13199757/b01248e4-c826-4de0-b15f-cde959965e68) -------------------- Commit message for microsoft/vscode-python@eb9fde3: Add `createEnvironment.contentButton` setting (microsoft/vscode-python#21212) Closes microsoft/vscode-python#20982 --------- Co-authored-by: Luciana Abud <[email protected]> -------------------- Commit message for microsoft/vscode-python@5eef525: Add logging when interpreter path changes (microsoft/vscode-python#21210) For microsoft/vscode-python#21208 -------------------- Commit message for microsoft/vscode-python@8d291f7: Disable "snippets" expansion in Jedi LSP (microsoft/vscode-python#21194) This brings the Jedi based completion experience in line with that provided by Pylance. Completions now insert only the current symbol rather than assuming that the user wants to e.g: call that symbol. This means for example that completing `max` will insert just `max` rather `max(arg1, arg2)`. While for this case this may be seen as less useful, it means that insertions in places where a call is not desired (such as imports and typing contexts) will not be forced to manually remove the parentheses and template arguments which might otherwise be inserted. Users can still use the signature support UI to explore signatures and of course insertion of an opening parenthesis will still insert a closing one. Hopefully this new configuration will be preferable to a majority of users. I've done some light testing to check that this disables the described additional completion, however I'm not massively familiar with JediLSP so I'm not sure what other behaviours this will disable. Fixes microsoft/vscode-python#15858 -------------------- Commit message for microsoft/vscode-python@a85eb3b: Fix startup telemetry issue (microsoft/vscode-python#21203) Could fix microsoft/vscode-python#20874 based on error trace. -------------------- Commit message for microsoft/vscode-python@17daae4: Open separate Python terminals when running different Python files (microsoft/vscode-python#21202) Closes microsoft/vscode-python#21097 Closes microsoft/vscode-python#14094 -------------------- Commit message for microsoft/vscode-python@f0253e5: Use actions from `vscode-github-triage-actions` in all Python automations (microsoft/vscode-python#21178) -------------------- Commit message for microsoft/vscode-python@678f70d: Remove hack to check the vscode version (microsoft/vscode-python#21180) Closed: microsoft/vscode-python#20769 -------------------- Commit message for microsoft/vscode-python@6bdada0: Use `saveEditor` proposed API for running untitled Python files (microsoft/vscode-python#21183) Closes microsoft/vscode-python#21182 Lead-authored-by: Eleanor Boyd <[email protected]> Co-authored-by: Peter Law <[email protected]> Co-authored-by: Carlos Piña Martinez <[email protected]> Co-authored-by: Jonathan Rayner <[email protected]> Co-authored-by: Pete Farland <[email protected]> Co-authored-by: paulacamargo25 <[email protected]> Co-authored-by: Karthik Nadig <[email protected]> Co-authored-by: Kartik Raj <[email protected]> Signed-off-by: GitHub <[email protected]>

eleanorjboyd added no-changelog No news entry required skip tests Updates to tests unnecessary labels May 19, 2023

eleanorjboyd commented May 22, 2023

View reviewed changes

eleanorjboyd self-assigned this May 23, 2023

eleanorjboyd marked this pull request as ready for review May 23, 2023 21:55

eleanorjboyd requested review from karthiknadig, karrtikr and paulacamargo25 and removed request for karthiknadig and karrtikr May 23, 2023 21:55

vscodenpa added this to the May 2023 milestone May 23, 2023

karthiknadig reviewed May 23, 2023

View reviewed changes

src/client/common/process/types.ts Outdated Show resolved Hide resolved

karthiknadig reviewed May 24, 2023

View reviewed changes

.github/workflows/build.yml Outdated Show resolved Hide resolved

karthiknadig reviewed May 24, 2023

View reviewed changes

src/client/common/process/types.ts Show resolved Hide resolved

karrtikr reviewed May 24, 2023

View reviewed changes

eleanorjboyd added 11 commits May 24, 2023 13:03

allow large scale testing

cc4b112

remove comment

972d708

pass test ids through stdin

e90deca

working but need to clean up

807fa21

working with process json

cf0cde1

clean up

4eba881

type edits

9730dcb

fix logging

9db9cd2

fix pyright for pr check

83256eb

fix typing

2090f23

switch to stdinStr

81c5971

eleanorjboyd force-pushed the large-workspaces branch from 4805e2f to 81c5971 Compare May 24, 2023 20:04

remove extra version pyright note

88b0389

eleanorjboyd requested a review from karrtikr May 24, 2023 20:07

paulacamargo25 approved these changes May 24, 2023

View reviewed changes

eleanorjboyd merged commit e2a9cec into microsoft:main May 24, 2023

eleanorjboyd deleted the large-workspaces branch May 24, 2023 23:42

eleanorjboyd restored the large-workspaces branch May 25, 2023 04:07

eleanorjboyd mentioned this pull request May 26, 2023

Support large testing repos #21309

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

allow large scale testing #21269

allow large scale testing #21269

eleanorjboyd commented May 18, 2023

eleanorjboyd commented May 19, 2023

eleanorjboyd May 22, 2023

karrtikr left a comment

allow large scale testing #21269

allow large scale testing #21269

Conversation

eleanorjboyd commented May 18, 2023

eleanorjboyd commented May 19, 2023

eleanorjboyd May 22, 2023

Choose a reason for hiding this comment

karrtikr left a comment

Choose a reason for hiding this comment