Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: make sure user's configuration or environment does not affect tests #43

Closed
stamblerre opened this issue May 14, 2020 · 5 comments
Closed
Assignees
Labels
chore/testing issues related to bugs in test, CI setup FrozenDueToAge

Comments

@stamblerre
Copy link
Contributor

stamblerre commented May 14, 2020

Some tests in the extension fail when the user running them has unexpected configurations or when they don't have the correct tools installed. We should make sure all test runs are independent of the user's environment.

microsoft/vscode-go#801 describes a specific fix that should be applied to one of the tests.

@hyangah
Copy link
Contributor

hyangah commented May 15, 2020

That's one of the motivation I had when adding the docker-based setup build/all.bash.

Another approach is to utilize --user-data-dir e.g.,
https://github.com/golang/vscode-go/blob/master/.vscode/launch.json#L66

(reference: https://vscode-dev-community.slack.com/archives/C74CB59NE/p1585956905048000)

@stamblerre stamblerre added the chore/testing issues related to bugs in test, CI setup label May 15, 2020
@hyangah
Copy link
Contributor

hyangah commented May 18, 2020

Seems like the --user-data-dir trick no longer works. Filed microsoft/vscode#97995

hyangah added a commit to hyangah/vscode-go-old that referenced this issue Jun 16, 2020
installation tests require multiple fs operations, and 2s may be
too short to complete everything on time. Allow more time.

And disable the gopls update check in gopls testing. That shortens
the time to bring up the gopls server.

Also remove the hack to start the test with a clean user profile
(--user-data-dir). That doesn't seem to work but interferes with
picking up the workspace settings, which I do not understand why.

Updates golang/vscode-go#169
Updates golang/vscode-go#43
Updates microsoft/vscode#97995

Change-Id: I5dbb33514a25a2071bf5bdf4ad248087e7e9b322
@gopherbot
Copy link
Collaborator

Change https://golang.org/cl/238001 mentions this issue: test: increase timeout and skip gopls update check in testing

gopherbot pushed a commit that referenced this issue Jun 16, 2020
installation tests require multiple fs operations, and 2s may be
too short to complete everything on time. Allow more time.

And disable the gopls update check in gopls testing. That shortens
the time to bring up the gopls server.

Also remove the hack to start the test with a clean user profile
(--user-data-dir). That doesn't seem to work but interferes with
picking up the workspace settings, which I do not understand why.

Updates #169
Updates #43
Updates microsoft/vscode#97995

Change-Id: I0e8b7d8f1f47f2e68696c6bd078c1ad2a0fe6780
GitHub-Last-Rev: f414011
GitHub-Pull-Request: #231
Reviewed-on: https://go-review.googlesource.com/c/vscode-go/+/238001
Reviewed-by: Rebecca Stambler <[email protected]>
gopherbot pushed a commit that referenced this issue Nov 10, 2020
…fects

The delveConfig user settings applied in settings.json, should be applied
to the debug configuration. Test that these are correctly applied.

Additionally, testFlags and buildFlags do not currently affect the debug
configuration, so we also test that to document that behavior. This behavior
is likely to change and the test should be updated when it does.

This change also reintroduces the --user-data-dir= argument to the launch
configuration for running the integration tests. This starts the test with
the default user settings, so running the tests should not be affected by
the user settings.

Update #43

Change-Id: I99c9d7f17892e6766fae731c9288880b109ba3fd
Reviewed-on: https://go-review.googlesource.com/c/vscode-go/+/267217
Trust: Suzy Mueller <[email protected]>
Trust: Hyang-Ah Hana Kim <[email protected]>
Reviewed-by: Hyang-Ah Hana Kim <[email protected]>
@gopherbot
Copy link
Collaborator

Change https://golang.org/cl/267217 mentions this issue: test/goDebugConfiguration.test.ts: test settings.json debug config effects

@hyangah
Copy link
Contributor

hyangah commented Dec 11, 2020

The adjustment made by https://go-review.googlesource.com/c/vscode-go/+/267217 fixed this issue. Thanks @suzmue !

@hyangah hyangah closed this as completed Dec 11, 2020
@golang golang locked and limited conversation to collaborators Dec 11, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
chore/testing issues related to bugs in test, CI setup FrozenDueToAge
Projects
None yet
Development

No branches or pull requests

4 participants